Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] SourceMage Binary Grimoire Proposal

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Casey Harkins <charkins AT upl.cs.wisc.edu>
  • To: Seth Woolley <seth AT tautology.org>
  • Cc: Source Mage Discuss <sm-discuss AT lists.ibiblio.org>, Eric Sandall <eric AT sandall.us>
  • Subject: Re: [SM-Discuss] SourceMage Binary Grimoire Proposal
  • Date: Sun, 3 Aug 2003 16:52:49 -0500 (CDT)



On Sat, 2 Aug 2003, Seth Woolley wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> The tool idea seems pointless if we can do it in sorcery, and the plan I
> was thinking was that it would be PREFERED (for emphasis, not
> loudness) rather than binary ONLY. That way, things that don't archive,
> like alsa-driver, will get compiled, but everything that you could install
> from cache would be resurrected. It would just be a preference, and I
> would want GCC on the machine just for this reason and in case you really
> do want to recompile it. Of course, we could make the compilation tools
> optionally sustained depending on the content of PREFER_BIN. PREFER_BIN
> would be like "RESURRECT_PREFER" in your nomenclature.

I guess my only issue with this, is trusting that things compile correctly
if it does end up compiling. Also, I would never want the larger packages
to get recompiled on any of the client machines.

> All the packages/depends, etc, should be updated on resurrect already, and
> if not, it's a bug.

I checked at one time and didn't see anything in the cache that could
provide this information. So either it isn't being removed, or it isn't
being restored. The required dependencies in /var/state/sorcery/depends
could be determined automatically on resurrection (from the spell
DEPENDS), but I don't see how sorcery could tell which option depends were
on/off.

> Also, I like it in sorcery for reasons of orthogonality. Having a
> separate tool seems kind of like a gentoo approach, doesn't it (from what
> I hear)?

Agreed. I just didn't want to push features into sorcery that others
wouldn't use, and didn't want to continually maintain patches to sorcery
if they weren't accepted.

> Also, NFS is one way, but I thought that if PREFER_BIN was enabled and
> there was no local cache existing, and there was a CACHE_FALLBACK_URL
> given, it could download it via http/ftp/rsync/cvs, whatever lib_url can
> do... Failing a non-existent cache entry on the remote site, go ahead and
> compile it locally perhaps even using distcc. But if the cache were on
> NFS, this would be even better because caches could be saved back upstream
> for another computer to use in the farm. We could also provide upload
> extensions via liburl to cvs or http/dav or ftp and publish them back up
> that way.

I also don't like the idea of the client machines having write access to
the cache. If someone happened to get access to NFS they could drop a
hacked ssh binary in for example. With a binary grimoire, the binaries
would be MD5 checked (and gpg checked once that is available in sorcery).

> Are these ideas helping?

Definitely! I'm going to spend tomorrow touring the sorcery source again
(its been a while) and determine how much work that approach would be.
Perhaps a PREFER_PKG flag could be added with a few options:

ONLY_SOURCE
never resurrect, effectively the same as disabling
cache; this however could be used on a build machine
to produce the cache files

SOURCE
the current behaviour of sorcery

BINARY
preferring binary as you described

ONLY_BINARY
never build from source; not sure if this will be possible
because some spells don't archive (a point I overlooked
previously)

Just some ideas. I guess I would like to avoid compiling at all costs
on the client machines to save them the build time and avoid any possible
compilation problems. Perhaps using ONLY_BINARY would be possible so long
as spells which don't archive were not used.

Keep the ideas coming.

-casey



>
> Seth
>
> On Sat, 2 Aug 2003, Casey Harkins wrote:
>
> > On Sat, 2 Aug 2003, Seth Woolley wrote:
> > > Just modify sorcery to override the -c switch to mean "resurrect", or
> > > where sorcery cast -c's spells for dependency resolution and things like
> > > that; have it not use -c and pull from compile cache instead if
> > > PREFER_BIN
> > > is enabled in sorcery config.
> >
> > I thought about something like this, but I wanted to come up with
> > something without modifying sorcery. If anything at all would need to be
> > done to sorcery, it would be to allow the gcc check (I think there is one)
> > to be disabled so gcc wouldn't be necessary on the client machines.
> >
> > Now if I did go the sorcery patching route, I was thinking something like
> > a RESURRECT_ONLY option that would force all casts (cast, sorcery
> > system-update and triggers) to only resurrect, probably similar to what
> > you're describing. However, I'm not sure (and haven't looked) at if/how
> > state information is maintained when dispelling and resurrecting.
> > Particularly, do the entries in /var/state/sorcery/[packages|depends] get
> > removed when dispelling? Do they get added when resurrecting? If I were on
> > my machine right now I'd check!
> >
> > > I read Casey's document, and it seemed a bit overkill if we can use NFS
> > > as
> > > many of us do here for managing multiple boxes or even http to publish
> > > sorcery caches. Personally, I'd rather this go into sorcery as an
> > > option
> > > than to have an entire new grimoire that needs publishing and
> > > maintaining.
> > > It seems it really doesn't need that much modified in sorcery to just do
> > > it without a new grimoire... probably the same amount as with a whole
> > > new
> > > grimoire.
> >
> > I wasn't proposing that we create or maintain a binary grimoire (I may
> > have not made this clear enough). What I am proposing, and will hopefully
> > be implementing in the next two weeks, is the "tool" to create a binary
> > grimoire. It would be intended only to be used locally.
> >
> > The NFS/http route will not work for all of our computers, as we have some
> > stand-alone machines with dial-up access only (people working from their
> > homes in the middle of nowhere). I could treat these as a special case,
> > but I'd prefer a solution that works the same on these remote computers as
> > well as our network workstations. Our networks would still use the NFS
> > approach to distribute the binary grimoire and packages.
> >
> > Unless someone convinces me that it'd be easy to implement a
> > RESURRECT_ONLY feature in sorcery, I'll start tackling the binary grimoire
> > tool Monday.
> >
> > A couple other questions (that I could probably answer myself if I were on
> > my computer):
> >
> > Does BASE_URL just get prepended to the SOURCE field for spells?
> > Does BASE_URL effect the url for downloading a grimoire? for sorcery?
> >
> >
> > Thanks for everyone's advice so far, it all keeps by brain moving.
> >
> > -casey
> >
> >
>
> - --
> Seth Alan Woolley <seth at tautology.org>, SPAM/UCE is unauthorized
> Key id 7BEACC7D = 2978 0BD1 BA48 B671 C1EB 93F7 EDF4 3CDF 7BEA CC7D
> Full Key at seth.tautology.org and pgp.mit.edu. info: www.gnupg.org
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.2 (FreeBSD)
>
> iD8DBQE/LIXb7fQ833vqzH0RAhCwAKCzprpYeVReCYEY02nvCGEPofYE0gCdE2Pk
> p39DQFe2K4JIRa6sbHHHKM0=
> =zgon
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> SM-Discuss mailing list
> SM-Discuss AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/sm-discuss
>





Archive powered by MHonArc 2.6.24.

Top of Page