Public SourceMage Discussion List

[dave AT sourcemage.org]

On Fri, Feb 25, 2005 at 08:11:45PM -0800, Andrew wrote:

<snip>

> Like I said, given our resources its an intractable problem to ensure
> one spell update didnt break anything else, hence the need for a release
> cycle between the "B" and "D" codebases. Like I said before, it just
> seems a lot like you're trying to move B closer to C and D closer to C,
> then say we dont need C. We need the C phase in some way shape or form.

<snip>
 
> Well, I can tell you right now some people want bleeding edge, and some
> people want stable, and theres no way we're gonna get both at exactly the
> same time. The old devel/test/stable idea was basically to have a scale
> between stability and bleeding edgeness, but it just dont work that way,
> stable ends up just out of date.

So.. please don't take this the wrong way, it seems to me that we're
effectively saying that it is hard to keep track of making a large number
of parallel changes to a flat-structured codebase. So in order to
maintain our ability to make a large number of parallel changes to a
flat-structured codebase, we're going to impose release schedules, which
will force us to promote lots of buggy code all at once, and then
backport bugfixes in order to obtain stability. I agree that changing
this structure in this manner is a difficult problem, but I'm not sure I
understand why we're insisting that doing development this way is
necessary, or at least desireable.

fwiw (which isn't much I know), this problem is functionally solved.
Projects larger than ours achieve stability, and quick incremental
releases on a daily basis by having developers branch off sections of
the codebase they're working on, working on them, and then merging them
back into "test". It's a fairly widespread development model and in my
experience works well. It allows developers to isolate themselves from
each other, maintains a stable root (but not at the expense of fast
development), and puts the bugfixing where it belongs (in the merge back
into test or devel, not in staging or stable). 

Maintaining the concept of 4 flat branches (with any naming scheme)
doesn't imo really address the problem, which is developers can't
control all the variables they need to ensure their changes will work
when they're integrated into the next tier. Enforcing a release schedule
on top of that will just make it so we know when to expect a whole bunch
of new bug reports. (and I mean all of that in the nicest possible way
hehe). 

<snip>

> But theres also long since been a desire to not have to think about
> individual upgrades and wanting to just say "system-update" and have it
> all work. Its impossible to guarentee that everything will work in B,
> but in D we want to at least try and be able to guarentee that. We want
> to have a stable grimoire that as a whole is solid. The only way to do
> that is to test it as a whole, and that takes a non trivial amount of
> time, hence C.

Agreed system-update just working is a good thing. But users doing
system-update is a separate issue from what Jer is talking about imo.
Giving developers the ability to branch off sections for the purpose of
development doesn't affect the stable codebase until those changes are
merged back and promoted up through staging. Nobody has to worry about
individual upgrades except for the individuals developing those
upgrades. Stable can still be tested as a whole anytime you want to, and
if you want to snapshot stable, and call that 1.0, more power to you,
none of the small development branches going on will have any effect on
you. 

<snip>

> > 1) We shouldn't do changes in the last pre-stable environment, we should
> >    have a way to do those changes in devel, with per-spell branching if
> >    necessary (note this would also facilitate users running older versions
> >    of spells, which there will likely be significant and increasing demand
> >    for).
> 
> Okay, so we have a branch off of the codebase C where we make changes,
> then we backport them to A then B, or B then A, if they still apply, or if
> the fix is B push them directly into C, then sync the C sub-branch(es)
> up, note that C sub-branches can just be people's uncommitted sandboxes.
> 
> I think the point needs to be made that C and B (and A with it) are
> effectively disjoint tracks of development once C is pulled off of B
> (in my model). B can continue without needing to depend on C, if they
> dont diverge then great, but it shouldnt be a restriction. Also, per
> spell branching I think provides too much overhead and complication,
> we end up basically with the same result I think, just we put different
> weights on spells, i say just branch the whole thing and use what you
> need, you say branch individual things from the head-of-line.

I think you guys are missing each other, and it's mostly terminology.
We'd be isolating individual spells, or groups of spells from the rest
of the codebase for the purpose of making incremental changes to them
and then merging those changes back, and only where necessary. We
wouldn't be maintaining permanent branches of individual spells or
grimoire sections, in the way we maintain devel, test and stable
branches now.  Nobody need be aware that the sections have been branched
off until those changes are merged back. It's elegant, because it's
modular.  It allows customized, and stable development environments, and
comes with built-in break-fix during merge. It's done all the time, and
really has the effect of reducing confusion compared to lots of parallel
development on a flat codebase.  

> > 2) We shouldn't introduce release cycles into stable unless we absolutely
> >    can't avoid them.  I don't think it's that hard to avoid them, even
> >    without (1).
> 
> After nearly three years with the project and knowing how long it takes to
> run Prometheus on the entire grimoire and the corner cases it'll find,
> we cant avoid a "release cycle" with what we're trying to accomplish (a
> stable grimoire). We've tried to avoid it for three years and its just
> not gonna work.

Again, I agree that continuing the way we're going, maintaining a ...
well ... stable stable, is difficult. But IMO enforcing Q&A with release
cycles is a solution looking for a problem, .  I certainly don't want to
question anyone's experience with the project, I'm very new here. I just
think Jar's suggestion is that we change our development model to match
what we want, which is quick, stable modifications. I agree with him
that we can get them, but I don't think we can get them going about it
with 30 developers making parallel changes to 4 permanent branches.  To
get there we're going to need better staging practices at least, and
ideally the ability to make better use of branching.  Does that make any
sense?

> 
> -Andrew
>

Thanks!

-dave.

Attachment: pgpfYfaHQdiHb.pgp
Description: PGP signature