Public SourceMage Discussion List

[Bor Kraljič <pyrobor AT ver.si>]

On 31. MES 2011 00:52:45 Remko van der Vossen wrote:
> Okay guys, here we go, the long awaited mail on grimoire splitting. I
> have to warn you though, it'll be long. Sorry about that.
Well long mail but I managed to read it. Ideas are nice! Realisation is 
another thing. 

> 
> 
> 
> 
> First, let's look at some of the major problems we are having;
> 
>   1. We have a lot of bitrot, a lot spells which have not been touched
>      in ages, spells which are simply not updated, or spells of software
>      which has been abandoned and left to rot.
> 
>   2. We have a lot of spells which are updated haphazardly, with barely
>      any testing, breaking a lot of peoples' boxes.
> 
>   3. We have a lot of development going on in branches, which goes
>      largely untested as there is no simple way for people to test these
>      kinds of updates easily, leading to good development work wasted
>      and left to rot.
> 
> These are things that really hurt us hard. It hurts our users, which
> includes us developers, who are left with random breakages every now and
> again which consume time and cause frustration. It also hurts us as
> developers in another way, as it feels that our development efforts go
> to waste, either they are left to rot, or people either complain that
> nothing gets done, or they complain that you broke stuff. All of this
> leads, understanibly, to frustration with the distro, a feeling that we
> aren't going anywhere and a general apathy towards the distro.
> 
> My idea to address these issues is to split the grimoire along various,
> flexible dimensions. Firstly, to address the first two problems, I'd
> like to split the grimoire along a quality assurance dimension. I'd like
> for us to have a core grimoire, which contains the spells from
> basesystem and the spells we include on our ISO. These spells need to
> work *at all times*, we need to have these spells working properly to
> make sure that ISO generation can be maintained without cauldron having
> to fix stupid problems everywhere, they need to be able to concentrate
> on actually generating the ISO.
> 
> To guard this quality of spells in the core grimoire, I'd like to see us
> expand the spells in this core grimoire with proper QA scripting. This
> scripting should allow us to easily ensure that spells work properly, it
> should test casting of each of the spells from previous stable release
> as well as previous ISO release, although I'd like us to synchronise
> stable releases of this core grimoire and ISO, but that's something for
> later. But merely casting is not sufficient, we need to develop QA
> scripting to actually test the functionality of the installed software.
> Now, this is of course quite a bit of a task and I understand that this
> won't be done next week, however, if we split this core group of spells
> into a separate grimoire we have a clear scope of what needs to be done,
> we have a clear goal to reach, this should be something that we can
> accomplish.
> 
> The next level of our ring of QA grimoires should be a grimoire of
> spells with the same level of automated QA testing, however for this
> grimoire the contents aren't defined by what's on the ISO or any such
> thing, instead we just add spells to this grimoire as and when we add
> proper QA scripting to these spells, which we'll do after we add QA
> scripting to all the spells in the core grimoire.
> 
> Level three of our little ring of grimoires should be a grimoire of
> maintained spells, that is, spells which have been picked up by a
> specific maintainer who is responsible for keeping the given spell up to
> date and in good order. I propose we reinstate the UPDATED field in
> DETAILS and add a CHECKED field as well. The UPDATED field should be
> updated as the spell is updated. The CHECKED field should be updated
> when the maintainer has performed a proper check of the spell, that is,
> checked the spell can be properly cast, preferably from both stable as
> well as ISO, and that the software works properly. If both UPDATED and
> CHECKED are more than a year ago the software will be nominted to be
> bumped from the maintained grimoire in three months. If either UPDATED
> or CHECKED is more than three years ago, the software should also be
> nominated to be bumped in three months. Of course we should automate
> this checking and bumping.
> 
> That leaves the grimoire of unmaintained spells, which is simply
> everything else. This gives us four grimoires, the names of which we can
> make properly magical in due time;
> 
>   1. core
>   2. tested
>   3. maintained
>   4. unmaintaied
IMO we don't need maintained and tested. However that is just the matter of 
taste more or less. But first problem I see is:

How do users migrate users from one grimoire (test or stable) to 4 of them?

> 
> These categories would allow users to be able to choose what level of QA
> they want on the software installed on their system. Of course at the
> start this won't account for much as almost everything will be in
> unmaintained, but I hope we can get moving in the right direction
> quickly.
I am pretty sure that UPDATED can be added (or fixed) to a correct value with 
some script and using git log or something. That way we would get pretty 
realistic spread spell of maintained and unmaintained. 

But like you mentioned. Spells will move up and down in grimoires. The main 
problems that presents it self here for me is:

Will you update all the dependencies (in other spells) every time the spell 
moves from one grimoire to another ?

Also one spell in "tested" will have one optional or non optional dependency 
to spell in unmaintained. Is that still "tested" spell ?
> 
> Now, it'd be horribly annoying if we'd need to start moving spells
> between repositories all the time, not even to mention how annoying it
> would be to have to look for a spell in different repos all the time.
> Therefore I don't want to split the repo, I want to keep everything in
> one repo. To facilitate the different grimoires I'd like to add a field
> to DETAILS indicating the QA level of the spell. Then we'd need to
> update our grimoire tarball generation scripts to pick and choose spells
> from the repo depending on this field in DETAILS. Anything without a the
> field goes into unmaintained so that we only have to add the field as we
> move specific spells into the higher echelons of our grimoire
> architecture.
Have considered option same repo with different branches for different 
grimoires?

> 
> That is the QA dimension of splitting grimoires. I'd like to add the
> possibility to specify other dimensions. A topical dimension for
> instance in which we can indicate all spells that belong to say X, KDE
> or Gnome. Now, note that I don't actually want to split X, KDE, Gnome or
> any other such subdivision from the main grimoire, I'd like them to
> remain part of the QA subdivision described earlier. Instead I'd like to
> see that we can specify various different dimensions to split along in
> the spells. Then we extend the grimoire tarball generation in the following
> way. Per tarball we'd specify the branch, the grimoire dimension and the
> value in that dimension.
> 
> For example, the core devel grimoire would be specified by:
>   tarball: devel-core
>   branch: master
>   dimension: QA
>   value: core
> 
> This mechanism however would allow for another very interesting way to
> specify and use grimoires. Say for instance that we have a new KDE
> release in a devel branch, let's use devel-kde47 as example, then we can
> define the following grimoire tarball:
>   tarball: devel-kde47
>   branch: devel-kde47
>   dimension: topic
>   value: KDE
Making tarballs with content of devel branches is awesome idea. Even without 
any splitting of the grimoire (even if user gets all the spells with devel-
kde47.tar.gz). That would make testing of branches much easier (with only 
scribe add devel-kde47). However we should set a list or some other way to 
control that not all the branches generate tarballs. 


> 
> If we define all KDE spells to be in the KDE topical grimoire dimension,
> then this would generate a grimoire tarball of only the KDE spells from
> the devel-kde47 branch. Allowing users to easily overlay the grimoire
> over the standard stable or devel grimoires. Once the branch is properly
> tested in this way, we can merge it back into master and drop the
> grimoire.
> 
> Of course we could use the very same mechanism to test any collection of
> spells. Say for instance the devel-libpng branch. In this branch, not
> just the libpng spell is modified, but also some other spells not
> directly related to libpng, but which had compatibility issues. If in
> that branch we add an entry to all relevant spells to indicate that the
> spell belongs in the libpng part of the devel dimension, then we can
> also easily generate a test grimoire for libpng with the following
> definition:
>   tarball: devel-libpng
>   branch: devel-libpng
>   dimension: devel
>   value: libpng
> 
> All these posibilities should take care of the third point in our list
> of problems by providing an easy but powerful way for our users to test
> developments. Additionally we could split a branch of master just before
> we merge in a major new version of, for example, KDE. Then we could
> define a legacy grimoire for the users that wish to stick with the old
> version, e.g.:
>   tarball: legacy-kde3
>   branch: legacy-kde3
>   dimension: topic
>   value: KDE
> 
> 
> 
> 
> Now, what needs to be done to realize all of this. Firstly we need to
> determine how we want to add grimoire dimensions to the spells. I see
> two possibilities for this, the cleaner is bash associative arrays,
> however this is a bash 4 specific feature and requires a declare
> command, e.g.:
> 
>            SPELL=kdebase
>          VERSION=4.6.5
> declare -A GRIMOIRE
>     GRIMOIRE[QA]=maintained
>  GRIMOIRE[topic]=KDE
This solution is much nicer in my opinion. 
>              ...
>          UPDATED=20110711
>          CHECKED=20110806
> 
> Alternatively we could go for something like
> 
>            SPELL=kdebase
>          VERSION=4.6.5
>      GRIMOIRE_QA=maintained
>   GRIMOIRE_topic=KDE
>              ...
>          UPDATED=20110711
>          CHECKED=20110806
> 
> which would get rid of the ugly declare statement, but would cause us to
> need nasty string concats in eval statements in the scripting.
> 
> Next we would need to update or perhaps even rewrite the grimoire
> tarball generation scripting. This is something that needs some careful
> attention and some good testing, but should not be that much work. In my
> opinion we should be able to have this up and running quickly.
Just from curiosity. Where is that script? Is that script public?

> 
> This will already allow us to differentiate between maintained and
> unmaintained spells as well as generate grimoire tarballs for devel
> branches.
> 
> Then we need to start work on good support for QA in sorcery, we need to
> define what the QA scripting in the spells should look like and how
> sorcery should use these. Once we have this in place we should start
> working on the QA scripting of the spells in core.
> 
> 
> 
> 
> Well, I guess that's enough of my rambling, so over to you guys. The
> above is the way I think we should see our future, I hope you will be
> able to agree, at least to some degree. So please tell us all what you
> think. Think it's a good idea? Bad idea? See any caveats? Have any
> bright ideas? Please feel free to share.
Last and perhaps most important question. Lets say we decide to do this. What 
part of this massive change you could take care of?

> 
> 
> Remko.

Attachment: signature.asc
Description: This is a digitally signed message part.