Public SourceMage Discussion List

[Andrew <afrayedknot AT thefrayedknot.armory.com>]

On Thu, Jan 04, 2007 at 09:32:36PM +0400, George Sherwood wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Thu, 4 Jan 2007 08:54:56 -0800
> Andrew <afrayedknot AT thefrayedknot.armory.com> wrote:
> 
> > 
> > I dont see the advantage in dropping stable-rc entirely. I do see a
> > point in treating stable-rc as a -rc. That is something we have not
> > yet tried, but is core to the release process. Stable-rc absolutely
> > can not sit around and become stale. its no longer an -rc at that
> > point, it becomes what "test" was, when we had devel, test, and
> > stable.
> 
> It becomes and is much worse then test ever was.  It gets very selected
> updates and becomes stale as the release drags out and out which is
> something that test never did.

Again, this isnt the intended behavior. If we pick a realistic number
of spells to fix (say basesystem) for a release then the process can
move forward. What stops it is picking more spells than we can fix.

> 
> > 
> > How do we treat stable-rc as a -rc? Following our release process we
> > branch from test, figure out what spells need to work, and focus our
> > efforts on fixing bugs related to those spells. The breakdown occured
> > because we picked too many spells, and usually had only one person
> > fixing them.
> > 
> > Howerver the process is designed to work with any number of spells,
> > even just one. Whatever our resources can accomplish. Over time the
> > number of spells is supposed to grow, but the process helps make that
> > happen through the policy of no regressions (this is also fundamental
> > to the process). The work remains the same between release cycles but
> > the number of spells increases, and thus so does the stability of the
> > grimoire as a whole. Or, if resources diminish, to a point, the
> > over-all stability remains constant. Beyond that point the number of
> > spells must be scaled back again. But the system can handle those
> > situations.
> > 
> > The release cycles we've tried, the only person helping fix spells was
> > usually seth. Nobody else. That made it difficult to have the bugs
> > fixed in all the spells we wanted. Also, I cant speak for seth, but I
> > imagine its frustrating trying to fix tons of bugs in the grimoire
> > with no one helping and most people version bumping spells left and
> > right.
> 
> I have to disagree here.  I have fixed close to every one of the bugs in
> my section and many bugs in other sections during the release process.
> I also know other section maintainers that were working diligently on
> their bugs also. The frustrating part of our release process was getting
> them into stable-rc after fixing the bugs.  The decision was made not to
> integrate some because they were not important enough and after fixing
> many bugs like this, I just quit.  Bugzilla is such a mess now, you
> can't tell what has been fixed, what has been integrated and what
> hasn't been touched.  IMHO, pretty much everything there needs to just
> be closed out and we start clean.  It is really ugly.

Theres lots of ugliness in bugzilla because the process broke-down and a
backlog started. The rules applied for integrations assumed the release
machinery was running, and it was not. If I can use an analogy, its silly
to say that your car is too heavy to move and to fix it you should get rid
of all that crap under the hood.

Again, I think the core problem was we picked too many spells to fix,
we thought that we could fix all the spells and we couldn't. That was
an unrealistic goal. Lets try a more realistic one.

Certainly the bugzilla machinery is less than ideal in its current form,
but thats no reason to ditch the process entirely. We should either
fix bugzilla, use bugzilla differently, or use an entirely different
defect tracker, or we can leave it, and try the process out and see if
the problems become less annoying.  What we shouldn't do is confuse a
HUGE backlog of bugs and integration requests the reason the process
failed. I dont think the backlog of bugs is the reason, its just a very
painful and obvious side-effect.

> 
> The process broke down by running prometheus on every spell in the
> grimoire without us ever having nearly enough manpower to fix those
> bugs.  It was this idea that we are making the distribution better by
> finding all the corner cases of every spell instead of concentrating on
> fixing the bugs that were being reported by actual users using the
> distribution.  Automated testing is great when you have the manpower to
> deal with the output of the testing.  We didn't have then, we probably
> have less now and I don't see that changing anytime soon.

First, users dont like reporting bugs. So we cant reliably expect them
to file bugs for every problem. Thats where automated testing comes in,
it lets us find the bugs more quickly and easily, before users try things,
thats generally considered good.

There was obviously a mis-understanding about the implications of what
prometheus runs mean. Running it was intended to find the bugs, then we
were supposed to fix the bugs in a *subset* of spells. Certainly we can
run prometheus until we're blue in the face and it wont fix any bugs. The
intent is NOT to find all the bugs and fix all the bugs. The intent is to
find the bugs so we can tell realistically how many there are, and what
fixing more spells would means in terms of work requirements. The bugs
are there with or without prometheus finding them. We could arbitrarily
pick spells to fix, but if we dont know how many bugs that is, its kind
of meaningless. Furthermore, Im not talking about finding all the corner
cases in every obscure spell and fixing them, im talking about real bugs
in spells people use. If its an obscure corner case, lower the priority
and dont worry about. Bug priorities exist for a reason. The process
even had some figures for what types of bugs were important enough to
fix and not. We need to pick a realistic subset of spells for a release 
cycle, oh, and actually fix the bugs in them.

Over time the process expands, and you (a grimoire dev) can pick the
spells that are important to you to support in the coming release
cycle. The process is supposed to work with you.

> 
> > 
> > So, if we try the process again with a few more volunteers, and a more
> > realistic list of spells, we can get a new stable out in short order.
> > 
> > I suggest for the next round, basesystem spells only. We can make
> > sure all those spells work pretty easily. You can build all of them
> > in a few hours. I modified all of them for install-root a year or so
> > ago in an evening. I think its do-able. Of course if init.d isnt
> > working, we need to either fix it, or back it out.
> > 
> > The process will work, but we have to give it a chance, theres only
> > one way to do that, which is to do the work.
> 
> No doubt about that, but will it get done?  I have pretty much been
> around here as long as any one and believe the list of bugs I have
> fixed compares with almost any other devel and truthfully I see great
> motion at times like this, but in the end run very little results.

Yes, this group is plagued by all talk and no action. Im trying to find
a course of action thats realistic and that other people agree might work.

Perhaps the problem is a lack of enthusiasm, but I dont know how to fix
that.  Maybe if we made it so everyone is participating then those driving
things forward wouldnt feel discouraged. Instead of just basesystem, maybe
all the grimoire devs pick A spell, just one spell, to be responsible
for in the release process. Then everyone can see the process working,
we can find the real bottlenecks, streamline them, and repeat. How does
that sound? All of us can probably help work on *one* spell.

-Andrew

-- 
_________________________________________________________________________
| Andrew D. Stitt  | acedit at armory.com    | astitt at sourcemage.org | 
| irc: afrayedknot | Sorcery Team Lead       |      ftp://t.armory.com/ |
| 1024D/D39B096C   | 76E4 728A 04EE 62B2 A09A 96D7 4D9E 239B D39B 096C  |
-------------------------------------------------------------------------