Public SourceMage Discussion List

[Seth Alan Woolley <seth AT positivism.org>]

I'm not sure if you read the "short, distinct definition" part. :)

Furthermore, what am I supposed to do with this document?  It looks like 
corporate ramblings of a high priestess manager. :) id est lots of 
requirements but no actual means of attainment.

Secondly, we have scripts to do most of this already.  You seem to have 
a consistent nack for ignoring existing processes (prometheus, bugzilla, 
sorcery, etc.) and making up all your own.  The QA scripts you wrote are 
just one example and the iso binary install process is another.  This 
isn't a critique of your technical ability -- mostly a critique of how 
to work together with an existing project well.

My proposal contained small changes to how things are done, mostly that 
were additions.  The major changes were how the QA team would work 
(seeing as how there was not much of a team before, it's not so much a 
big change) in dealing with stable grimoire updates only.

The spell lint and spell compilation parts are done already by 
prometheus.  You have a few additions that could be integrated into 
prometheus.  If you expect us to use your cauldron scripts to generate 
ISOs, you should use existing scripts that the other teams have written, 
and if not, explain why.

Thirdly, the spell integration portion is covered by my proposal, except 
your proposal supposes an individual test.  People upgrade entire 
systems, so we need to upgrade entire systems for testing.  My stable 
grimoire proposal includes that.  What it does not include is individual 
test -> stable integration tests, which actually because of API
changes will most certainly break a lot of stuff that's not feasable to 
fix and support.

For example, if you have two spells, A and B, and B depends on A and B 
depends on a particular track version of A, then your proposal will 
break things where a scribe system-upgrade would work.  A particular 
example is gnome or xfce.  They should be upgraded all at once.  
Upgrading piecemeal at the individual spell-level for those is a asking 
for trouble.

Later you mention integration tests should work at least for basesystem.  
At the spell-level, yes, this would be ideal, and might be something to 
consider since the basesystem should be more rock-solid and able to 
interoperate between versions, however, you'll often find cases where 
somebody at the fsf has decided to deprecate or change a switch in some 
command and now you'll need to update all the rest of the stuff that 
uses it.  Making sure this operation works on a system integration is 
perhaps the best and only option.  A recent example where spell-based 
integration would fail is in the gcc/g++ split.  An upgrade of g++ 
before gcc will break, but there's no way around that -- none 
whatsoever.  They _require_ the same version.  This works gracefully in 
a normal upgrade, but if you go through randomly upgrading spells it 
will break eventually, but you'd have to go looking for corner cases to 
get this issue to come up.  I filed a bug on that issue, so we'll 
probably have some fix on the gcc/g++ split spells where it sanity 
checks the version and does a compile of gcc first (or fails through the 
g++ compile).  > 95% of people care about whole system upgrades in my 
opinion.  Perhaps gcc should be integrated back into one spell once we 
get sub-spell-depends working.  I'm really open to ideas (comment on 
bugzilla though).

Lastly, I'm unsure of what you mean by iso-devel iso-test and iso-stable 
grimoires.  Why are they totally different names at all?  Does the world 
revolve around the ISO?

So those are my four main responses.  Anything you'd like to say to 
them? :)

Seth

On Tue, May 03, 2005 at 07:16:54PM +0200, Benoit PAPILLAULT wrote:
> Seth Alan Woolley a écrit :
> >>This definition of stable is not quite clear, even if you describe it 
> >>later in more details.
> >
> >
> >If you have a better idea, I'm open for a short, distinct definition.
> 
> Yep. I wrote a small paper on it. Here is a copy/paste. I don't think my 
> ideas could be taken as such, however, you can use some ideas to better 
> define "stable".
> 
> Benoit PAPILLAULT, ISO guru
> 
> === DOCUMENT ===
> 
> * test scripts
>  -------------
> 
>   - spell-lint
> 
>     - it is a program checking one spell syntax and outputs all results
>       on stdout
>     - exit code is 0 on success, 1 on failure
>     - check the syntax of all spell files
>     - warns about unknow files in a spell
>     - check URL validity (depends on PREPARE?)
>     - download source files
>     - check their md5sum
>     - check that no user input is required (except sorcery defined 
> questions)
>     - check that dependencies are valid names
> 
>   - spell-compilation
> 
>     - it is a program checking one spell compilation and outputs all 
> results
>       on stdout as well as in various files in the current directory
>     - exit code is 0 on success, 1 on failure
>     - all compilation are done on a "base" system
>     - all dependency are simply cast'ed
>     - spell is first tested with default options
>     - spell is then tested with all option/optional_dependencies
>     - aliens should be checked
>     - file conflict should be checked
>     - check if files from other spells are removed
>     - prometheus can be used to make random test (like a monkey)
>     - after casting a spell, the system is restored to its original state
>       (like prometheus does)
>     - patches are applied on devel (so they can be used later) and test
>       (since there is no direct access to devel).
>     - detect spell that install nothing (missing make install)
> 
>   - spell-integration
> 
>     - it is a program checking one spell integration and outputs all 
> results
>       on stdout as well as in various files in the current directory
>     - exit code is 0 on success, 1 on failure
>     - it checks that the iso-stable grimoire "base" + this spell + all
>       dependencies produces a system that can be "sorcery rebuild" twice.
> 
> * iso-stable grimoire updates
>  ----------------------------
> 
>     "iso-stable" grimoire is updated when a spell from devel, test or
>     stable (in this order) have been checked successfully using
>     "spell-integration". This later check is needed for at least
>     "base" spells.
> 
>     If all checkings are OK => "iso-stable" grimoire is updated.
>     If all checkings are KO => "iso-stable" grimoire is not updated, a bug
>       is generated.
> 
> * Definition
>  -----------
> 
>   - "base system" is a list of spells + sorcery that are needed for
>     sorcery.  Since testing can occur in a chroot, spells needed to
>     boot sorcery will not be in "base". All dependencies from "base"
>     should be in base since they should be installed as well.
> 
>   - "iso-stable" system is made up from "iso-stable" grimoire and
>   "iso-stable" sorcery.
> 
>   - "iso-stable" grimoire is a grimoire where all spells have been
>     checked using the previous test.
> 
>   - "iso-stable" sorcery is a sorcery version which is part of the
>     "iso-stable" system
> 
> SourceMage GNU/Linux QA process:
> ================================
> 
>   We would like to define a process (mainly automatic) that could lead 
> to a stable grimoire from anything. stable grimoire is here a grimoire 
> where any spells can be installed on any supported architecture and 
> after installing such spell, doing a sorcery rebuild twice would succeed 
> and would still produce a fully functionnal system.
> 
>   To achieve such results, we are going to various steps and various 
> grimoires:
>   iso-devel, iso-test and iso-stable
> 
>   iso-devel : no requirement on spells in iso-devel grimoire.
> 
>   iso-test: all spells in iso-test must compiles on all the supported
>   architectures.
> 
>   iso-stable: all spells in iso-stable must compiles on all the supported
>   architectures and the system can sucessfully pass a sorcery rebuild 
> twice and
>   still be fully fontional.
> 
> iso-devel -> iso-test move:
> 
>   we select a list of spells that we want to move from iso-devel to 
> iso-test at
>   the same time (one spell at a time is not always possible). We create a
>   temporary grimoire based on iso-stable where the selected spells are 
> replaced
>   with their iso-devel version. If the compilation is successfull on 
> all the
>   supported architecture, the spell list can go to iso-test.
> 
> iso-test -> iso-stable move:
> 
>   we select a list of spells that we want to move from iso-test to 
> iso-stable
>   at the same time. We create a temporary grimoire based on iso-stable 
> where
>   the selected spells are replaced with their iso-test version. If the
>   compilation is successfull on all supported architecture and if the new
>   system can suffer a sorcery rebuild twice and still be functionnal, 
> the spell
>   list can go to iso-stable.
> 
> In practice, using CVS as an example:
> 
>   * iso-devel can be the MAIN branch
>   * iso-test can be an iso-test tag
>   * iso-stable can be an iso-stable tag
> 
> If testing on all architectures cannot be done at the same time, we can 
> have a
> CVS tag for each architecture using the $ARCHITECTURE variable from sorcery,
> like:
> 
>   * iso-devel-sun4u64
>   * iso-devel-g3
>   * iso-devel-i486
> 
> Using CVS branches:
> 
>   * MAIN for the MAIN branch
>   * stable for the official stable grimoire
>   * test for the official test grimoire
>   * devel for the official devel grimoire
> 
> _______________________________________________
> SM-Discuss mailing list
> SM-Discuss AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/sm-discuss
> 

-- 
Seth Alan Woolley [seth at positivism.org], SPAM/UCE is unauthorized
Key id 00BA3AF3 = 8BE0 A72E A47E A92A 0737  F2FF 7A3F 6D3C 00BA 3AF3
Quality Assurance Team Leader; Security Team Member, Leader Emeritus
Linux so advanced, it may as well be magic http://www.sourcemage.org
Elected Coordinating Committee Member, Secretary, and Finances Chair
Pacific Green Party of Oregon - Peace - http://www.pacificgreens.org

Attachment: pgpbYvo7g2i4V.pgp
Description: PGP signature