Public SourceMage Discussion List

[Benoit PAPILLAULT <benoit.papillault AT sourcemage.org>]

Seth Alan Woolley a écrit :
> > This definition of stable is not quite clear, even if you describe it 
> > later in more details.
>
>
> If you have a better idea, I'm open for a short, distinct definition.

Yep. I wrote a small paper on it. Here is a copy/paste. I don't think my 
ideas could be taken as such, however, you can use some ideas to better 
define "stable".

Benoit PAPILLAULT, ISO guru

=== DOCUMENT ===

* test scripts
 -------------

  - spell-lint

    - it is a program checking one spell syntax and outputs all results
      on stdout
    - exit code is 0 on success, 1 on failure
    - check the syntax of all spell files
    - warns about unknow files in a spell
    - check URL validity (depends on PREPARE?)
    - download source files
    - check their md5sum
    - check that no user input is required (except sorcery defined 
questions)
    - check that dependencies are valid names

  - spell-compilation

    - it is a program checking one spell compilation and outputs all 
results
      on stdout as well as in various files in the current directory
    - exit code is 0 on success, 1 on failure
    - all compilation are done on a "base" system
    - all dependency are simply cast'ed
    - spell is first tested with default options
    - spell is then tested with all option/optional_dependencies
    - aliens should be checked
    - file conflict should be checked
    - check if files from other spells are removed
    - prometheus can be used to make random test (like a monkey)
    - after casting a spell, the system is restored to its original state
      (like prometheus does)
    - patches are applied on devel (so they can be used later) and test
      (since there is no direct access to devel).
    - detect spell that install nothing (missing make install)

  - spell-integration

    - it is a program checking one spell integration and outputs all 
results
      on stdout as well as in various files in the current directory
    - exit code is 0 on success, 1 on failure
    - it checks that the iso-stable grimoire "base" + this spell + all
      dependencies produces a system that can be "sorcery rebuild" twice.

* iso-stable grimoire updates
 ----------------------------

    "iso-stable" grimoire is updated when a spell from devel, test or
    stable (in this order) have been checked successfully using
    "spell-integration". This later check is needed for at least
    "base" spells.

    If all checkings are OK => "iso-stable" grimoire is updated.
    If all checkings are KO => "iso-stable" grimoire is not updated, a bug
      is generated.

* Definition
 -----------

  - "base system" is a list of spells + sorcery that are needed for
    sorcery.  Since testing can occur in a chroot, spells needed to
    boot sorcery will not be in "base". All dependencies from "base"
    should be in base since they should be installed as well.

  - "iso-stable" system is made up from "iso-stable" grimoire and
  "iso-stable" sorcery.

  - "iso-stable" grimoire is a grimoire where all spells have been
    checked using the previous test.

  - "iso-stable" sorcery is a sorcery version which is part of the
    "iso-stable" system

SourceMage GNU/Linux QA process:
================================

  We would like to define a process (mainly automatic) that could lead 
to a stable grimoire from anything. stable grimoire is here a grimoire 
where any spells can be installed on any supported architecture and 
after installing such spell, doing a sorcery rebuild twice would succeed 
and would still produce a fully functionnal system.

  To achieve such results, we are going to various steps and various 
grimoires:
  iso-devel, iso-test and iso-stable

  iso-devel : no requirement on spells in iso-devel grimoire.

  iso-test: all spells in iso-test must compiles on all the supported
  architectures.

  iso-stable: all spells in iso-stable must compiles on all the supported
  architectures and the system can sucessfully pass a sorcery rebuild 
twice and
  still be fully fontional.

iso-devel -> iso-test move:

  we select a list of spells that we want to move from iso-devel to 
iso-test at
  the same time (one spell at a time is not always possible). We create a
  temporary grimoire based on iso-stable where the selected spells are 
replaced
  with their iso-devel version. If the compilation is successfull on 
all the
  supported architecture, the spell list can go to iso-test.

iso-test -> iso-stable move:

  we select a list of spells that we want to move from iso-test to 
iso-stable
  at the same time. We create a temporary grimoire based on iso-stable 
where
  the selected spells are replaced with their iso-test version. If the
  compilation is successfull on all supported architecture and if the new
  system can suffer a sorcery rebuild twice and still be functionnal, 
the spell
  list can go to iso-stable.

In practice, using CVS as an example:

  * iso-devel can be the MAIN branch
  * iso-test can be an iso-test tag
  * iso-stable can be an iso-stable tag

If testing on all architectures cannot be done at the same time, we can 
have a
CVS tag for each architecture using the $ARCHITECTURE variable from sorcery,
like:

  * iso-devel-sun4u64
  * iso-devel-g3
  * iso-devel-i486

Using CVS branches:

  * MAIN for the MAIN branch
  * stable for the official stable grimoire
  * test for the official test grimoire
  * devel for the official devel grimoire