Public SourceMage Discussion List

[Ryan Abrams <rabrams AT sourcemage.org>]

Hey Mark,

> RA> 1) It will get fragmented and duplicated across multiple spells.
>
> Only where appropriate. As I thought I'd explained, the cpudetect
> stuff is actually just generating the appropriate gcc flags. So if
> someone wanted to use say IBMs compiler, then this routine is no use
> to them. The IBM compiler spell would have it's own version of the
> cpudetect routines to produce IBM compiler flags. As mine is gcc
> specific then it seems logical to stick it in the gcc spell. Why do
> you see this as wrong?

GCC flags, as currently supplied via sorcery's config menus? It 
automates that, correct? So what is really being started here is an 
abstraction of the compiler optimization flags out of sorcery and into 
the compiler's spell, to allow for IBM compilers or Intel compilers, or 
whatever, based on what is installed.

This is virtually guaranteed to NOT work in sorcery 2.0 at all, since 
one of the primary goals of sorcery 2.0 is to make all sorcery code 
independant of the spells and vice versa. In other words, to make it a 
package management system, with independant packages.

> RA> 2) It will not work in sorcery 2.0 at all.
>
> It's a bit premature to say that don't you think?

Possibly. But I would be willing to go as far as to say nearly 
everything we have will not work "out of the box" in sorcery 2.0 - 
There are a LOT of roadblocks encountered in the 1.0 codebase that we 
have had to work around. I fully expect that in 2.0 we will be 
rewriting to remove those roadblocks.. which will change basic formats 
around a bit. Obviously this wont happen overnight, but instead will 
happen over time. The 1.0 base will have some legs.

One of my personal priorities in sorcery 2.0 is to get the api 
independent enough that we remove all the spaghetti code and dirty 
hacks that are in the grimoire atm. I have discussed this with the 
sorcery team many times, as well as with many in the grimoire team on 
various occasions. It seems like a reasonable goal.. and despite 
differences in how we believe the api's should work, the end goal seems 
like a mutually agreed upon one.

> RA> Sorcery 2.0 will be defining a more rigid structure for spells, and
> RA> generally separating them from sorcery more, via a cleaner api.
>
> I thought my libvariables idea was a cleaner API that seperated
> spells, or at least their variables, from sorcery more. Do you
> disagree and if so can you explain why?

I don't disagree with the concept of separating spells and spell 
configs from sorcery more. I have been after the sorcery team for 
something like that for a long time (at least 2 months at this point). 
And I believe that some work has happened on that front... or at least 
some planning. It has pretty much been postponed to after 1.0 though.

> > > So to recap, next cpudetect will contain:
> > >
> > > libvariables  :  lookup and save routines
> > > cpu2gcc       :  code to scan cpuinfo and produce TARGET file
> > > cpu2gcc refs  :  lookup tables for auto target generation
> > > autocpu       :  reads local TARGET and sets sorcery bash vars
>
> The only bit of this that is tied to Sorcery is autocpu. In fact by
> replacing autocpu with different code, this system could integrate
> with ANY source based distro. I designed it this way so that it could
> promoted seperately to:

Thats well and good. I am just saying, be careful if you tie any code 
into a gcc or compiler spell that would be required by sorcery, as one 
of my goals for 2.0 is to separate sorcery from spells entirely.

> But if you want to summarily dismiss the work as not sourcemage 2.0
> compatible I'm sure I can find a suitable alternative distro to work
> with. Your loss not mine. cpudetect might be my first bash program,
> but I do have 20 years of programming experience across a lot of other
> languages.

Mark, I did not summarily dismiss your work in any way shape or form. 
In fact, I quite like it. What I did do was reply to your email stating 
(for the first time) that you will integrate autocpu into spells and 
such with a comment that touches on many discussions we have had over 
the past months regarding sorcery and where to take it. And all I said 
was that the closer you tie it to the current system, the more it will 
need a complete rewrite. This is a FACT.

Regardless of how many years of experience you have coding, you 
probably don't know /everything/ that has been discussed regarding 
sorcery, and I would hazard a guess that you dont even know the entire 
current codebase. Even if you DO, I wont have someone running through 
the codebase changing code that is being maintained by another. For 
example, you would be stepping on the toes of the gcc maintainer, the 
sorcery team, and just about everyone who has done any thought about 
where to take things post 1.0.

I am trying to save you a lot of work in the future. You yourself have 
claimed that this is probably not a 1.0 thing.. if you are aiming for 
1.x, keep going. Maybe it will happen in a few months. If you would 
rather take it slow, and have input on the 2.0 process (making this a 
part of it), then maybe you shouldn't just post "this is how its going 
to be" and get upset if someone points out an incompatibility in your 
idea.

> Be interesting to see how the community reacts to this discussion. Is
> SourceMage development driven by user needs or by someone at the top's
> idea of where it should go? Hate to say it, but didn't we have this
> problem before?

I don't think it will be all that interesting. I have been through 
things like this before with developers who are core members of the 
team. We almost always work it out, and life always goes on.

SourceMage development is driven by developer consensus, based on user 
needs AND development ideas/goals. I dictate nearly nothing (and have 
actually been asked to give MORE "this is how it is" type of direction 
by some). That said, I comment on almost everything.. challenging 
people, trying to share information, and working to get people to look 
at the big picture and what others are doing before they just 
"announce" (as you did) that something will be the way they want it... 
because that leads to their work not being used, or being used only 
partially.

The problem we had before was a single owner of the entire project 
summarily rejecting or discarding code based on his whims. What you 
have now is a project leader who takes an active interest in what is 
happening in the distro, and tries to steer development along a common 
path. This prevents summary rejection of work at the end, and hopefully 
gets everything and everyone working together.

My email may have seemed harsh, and may not have provided an 
alternative or direct technical argument against your plans. That 
wasn't it's goal. It also didn't question the need for auto hardware 
detection. What it DID do is point out that your current plan has flaws 
when tied into the rest of sorcery, as well as when tied into the 
current thinking on the 2.0 direction.

You can take that two ways:
1) As a hint that you should probably get more involved in what the 
rest of the team is doing before making summary choices about 
architecture.

or

2) As a summary rejection of your work completely requiring you to 
threaten to leave the distro or imply nasty things about the distros 
leadership.

It was meant as #1, and after having reread it a few times... it IS #1, 
though without any padding and handholding.  How you choose to take it 
is up to you. I hope you can see it as what it was.

Your work is valuable, and quality. I am sure I am not the only one who 
would love to see it continue. But you can't develop changes to the gcc 
spell and sorcery libraries without understanding what the various 
current developers are doing and planning. It doesn't work that way in 
a team environment.

If you can take this as constructive criticism, I recommend that you 
contact:

Julian v. Bock - about the gcc spell (julian AT openit.de)
Nick Jennings - about sorcery (njennings AT sourcemage.org)
Dufflebunk - he and i both have pretty strong views on 2.0.. he may be 
able to find a way that it could best fit. (pmahon AT sourcemage.org)

These people will help you get your idea further developed in the 1.0 
tree. Or just ask in IRC or on the lists, and we can start some sort of 
2.0 developmental planning group, which you would be more than welcome 
to join.

Thanks,
-Ryan