Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] Grimoire splitting

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: David Kowis <dkowis AT shlrm.org>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] Grimoire splitting
  • Date: Wed, 31 Aug 2011 18:07:48 -0500

My summary: This is an excellent idea. I love it. I have a few
questions/comments that I will mix into the email.

On 8/30/2011 5:52 PM, Remko van der Vossen wrote:
>
> First, let's look at some of the major problems we are having;
>
> 1. We have a lot of bitrot, a lot spells which have not been touched
> in ages, spells which are simply not updated, or spells of software
> which has been abandoned and left to rot.
>
> 2. We have a lot of spells which are updated haphazardly, with barely
> any testing, breaking a lot of peoples' boxes.
>
> 3. We have a lot of development going on in branches, which goes
> largely untested as there is no simple way for people to test these
> kinds of updates easily, leading to good development work wasted
> and left to rot.
>
> These are things that really hurt us hard. It hurts our users, which
> includes us developers, who are left with random breakages every now and
> again which consume time and cause frustration. It also hurts us as
> developers in another way, as it feels that our development efforts go
> to waste, either they are left to rot, or people either complain that
> nothing gets done, or they complain that you broke stuff. All of this
> leads, understanibly, to frustration with the distro, a feeling that we
> aren't going anywhere and a general apathy towards the distro.

Agreed.

> To guard this quality of spells in the core grimoire, I'd like to see us
> expand the spells in this core grimoire with proper QA scripting. This
> scripting should allow us to easily ensure that spells work properly, it
> should test casting of each of the spells from previous stable release
> as well as previous ISO release, although I'd like us to synchronise
> stable releases of this core grimoire and ISO, but that's something for
> later. But merely casting is not sufficient, we need to develop QA
> scripting to actually test the functionality of the installed software.
> Now, this is of course quite a bit of a task and I understand that this
> won't be done next week, however, if we split this core group of spells
> into a separate grimoire we have a clear scope of what needs to be done,
> we have a clear goal to reach, this should be something that we can
> accomplish.

This is what I've been rattling around in my brain for quite a while.
Automated QA tests to go along with the spell to ensure that the spell
does indeed work.

>
> The next level of our ring of QA grimoires should be a grimoire of
> spells with the same level of automated QA testing, however for this
> grimoire the contents aren't defined by what's on the ISO or any such
> thing, instead we just add spells to this grimoire as and when we add
> proper QA scripting to these spells, which we'll do after we add QA
> scripting to all the spells in the core grimoire.
>
> Level three of our little ring of grimoires should be a grimoire of
> maintained spells, that is, spells which have been picked up by a
> specific maintainer who is responsible for keeping the given spell up to
> date and in good order. I propose we reinstate the UPDATED field in
> DETAILS and add a CHECKED field as well. The UPDATED field should be
> updated as the spell is updated. The CHECKED field should be updated
> when the maintainer has performed a proper check of the spell, that is,
> checked the spell can be properly cast, preferably from both stable as
> well as ISO, and that the software works properly. If both UPDATED and
> CHECKED are more than a year ago the software will be nominted to be
> bumped from the maintained grimoire in three months. If either UPDATED
> or CHECKED is more than three years ago, the software should also be
> nominated to be bumped in three months. Of course we should automate
> this checking and bumping.

Question: What makes something qualify as CHECKED? Do we rely on a
single developer to say "it works for me"? Do we have levels of
verification, kind of how Fedora does it, where sufficient "karma" must
be reached in order for it to really be CHECKED? This is assuming it's
not part of the automated QA tests.

>
> That leaves the grimoire of unmaintained spells, which is simply
> everything else. This gives us four grimoires, the names of which we can
> make properly magical in due time;
>
> 1. core
> 2. tested
> 3. maintained
> 4. unmaintaied
>
> These categories would allow users to be able to choose what level of QA
> they want on the software installed on their system. Of course at the
> start this won't account for much as almost everything will be in
> unmaintained, but I hope we can get moving in the right direction
> quickly.
>
> Now, it'd be horribly annoying if we'd need to start moving spells
> between repositories all the time, not even to mention how annoying it
> would be to have to look for a spell in different repos all the time.
> Therefore I don't want to split the repo, I want to keep everything in
> one repo. To facilitate the different grimoires I'd like to add a field
> to DETAILS indicating the QA level of the spell. Then we'd need to
> update our grimoire tarball generation scripts to pick and choose spells
> from the repo depending on this field in DETAILS. Anything without a the
> field goes into unmaintained so that we only have to add the field as we
> move specific spells into the higher echelons of our grimoire
> architecture.
>
> That is the QA dimension of splitting grimoires. I'd like to add the
> possibility to specify other dimensions. A topical dimension for
> instance in which we can indicate all spells that belong to say X, KDE
> or Gnome. Now, note that I don't actually want to split X, KDE, Gnome or
> any other such subdivision from the main grimoire, I'd like them to
> remain part of the QA subdivision described earlier. Instead I'd like to
> see that we can specify various different dimensions to split along in
> the spells. Then we extend the grimoire tarball generation in the following
> way. Per tarball we'd specify the branch, the grimoire dimension and the
> value in that dimension.
>
> For example, the core devel grimoire would be specified by:
> tarball: devel-core
> branch: master
> dimension: QA
> value: core


So as I understand it, we're adding a bit more metadata to the spell.
Kind of like an additional SECTION?


> Now, what needs to be done to realize all of this. Firstly we need to
> determine how we want to add grimoire dimensions to the spells. I see
> two possibilities for this, the cleaner is bash associative arrays,
> however this is a bash 4 specific feature and requires a declare
> command, e.g.:
>
> SPELL=kdebase
> VERSION=4.6.5
> declare -A GRIMOIRE
> GRIMOIRE[QA]=maintained
> GRIMOIRE[topic]=KDE
> ...
> UPDATED=20110711
> CHECKED=20110806
>
> Alternatively we could go for something like
>
> SPELL=kdebase
> VERSION=4.6.5
> GRIMOIRE_QA=maintained
> GRIMOIRE_topic=KDE
> ...
> UPDATED=20110711
> CHECKED=20110806
>
> which would get rid of the ugly declare statement, but would cause us to
> need nasty string concats in eval statements in the scripting.

Could the declare -A be handled before the DETAILS file is loaded? Or
must it be declared in the same file?

>
> Next we would need to update or perhaps even rewrite the grimoire
> tarball generation scripting. This is something that needs some careful
> attention and some good testing, but should not be that much work. In my
> opinion we should be able to have this up and running quickly.
>
> This will already allow us to differentiate between maintained and
> unmaintained spells as well as generate grimoire tarballs for devel
> branches.
>
> Then we need to start work on good support for QA in sorcery, we need to
> define what the QA scripting in the spells should look like and how
> sorcery should use these. Once we have this in place we should start
> working on the QA scripting of the spells in core.
>

I think this is a great idea. I'd be willing to get started on this
immediately. I think it's a wonderful idea.

I think we need the dimension support before we can take advantage of
the QA scripting stuff. Then we can have automated testing that looks at
the dimension, and verifies the automated tests, or spams people if the
automated tests don't exist.

In order of things to get done:
1. Dimensions
* Core -- start with just this, and get some automated QA tests in there.
2. Actual execution and validation of the automated tests
3. Magic tarball script

I think this gets us to the end state of having a much higher quality
grimoire, and much better software :)

Excellent work, Remko.

David

Attachment: signature.asc
Description: OpenPGP digital signature




Archive powered by MHonArc 2.6.24.

Top of Page