Skip to Content.
Sympa Menu

sm-discuss - [SM-Discuss] SourceMage Binary Grimoire Proposal

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Casey Harkins <charkins AT upl.cs.wisc.edu>
  • To: Source Mage Discuss <sm-discuss AT lists.ibiblio.org>
  • Subject: [SM-Discuss] SourceMage Binary Grimoire Proposal
  • Date: Sat, 2 Aug 2003 14:47:01 -0500 (CDT)


I'm not sure if anyone is interested, but I wrote up a quick
proposal/outline on how binary packages could be used locally to maintain
a whole network of SourceMage based machines without having to worry about
builds on all of the machines. I plan on implementing and testing this
over the next two weeks. I've attached the proposal if anyone is
interested. Feel free to reply if you have comments/suggestions. If it
works out, I'll create a spell for the utility described in the proposal.

-casey

SourceMage Binary Grimoire Proposal
August 2nd, 2003
Casey Harkins <charkins AT pobox.com>

Abstract:
========
This document proposes a binary packaging system for the SourceMage
GNU/Linux distribution. At first this may seem to conflict with the
goals of the SourceMage distribution, there are situtuations where
binary packages could be used without compromising those goals. The
approach described fits into the existing SourceMage packaging scheme
with little or no modifications to the sorcery tools.

Purpose:
=======
There are situations where binary packages would be extremely useful
to cut down on unnecessary compile times, improve stability of
production machines and simplify installations on multiple similar
machines. The most obvious example is a cluster or network of idential
machines. Rather than compiling the same set of software on multiple
machines with identical hardware, a single "reference machine" could do
the compiling and the binary packages could be distributed to the
"client machines". This could be extended to cover clusters or networks
with non-identical hardware by compiling for the lowest common
denominator architecture. Another example is for development and
production servers. The complexity of a source based distribution can
cause problems when major updates are made (gcc, glibc, get-text,
kernels). Crossing your fingers when performing a 'sorcery
system-update' on a production server is far from an ideal situation. A
development server could be used as the "reference machine", allowing
pre-tested binary packages to be used on the production server.

While this system could likely be used to produce a full binary
SourceMage distribution, the intent of this proposal is to use this
system for local use by making use of a standard source based SourceMage
machine as the "reference machine."

Goals:
=====
(1) Provide binary packages within the existing SourceMage packaging
system, with little or no modifications necessary to the existing
sorcery tools.

(2) Allow (mostly) seamless transition between a source and binary
based system.

(3) Allow updates to source packages to propagate to the binary
packages in an efficient manner.

Approach:
========
One possible approach is to make use of a binary grimoire with
spells that function much like the binary spells found frequently in the
z-rejected grimoire, satisfying the first goal above. By ensuring that
binary spells match the names of their source based counterparts, the
binary grimoire could essentially be swapped at anytime with a source
grimoire satisfying the second goal above. Most of the work required for
creating and maintaining the binary grimoire should be handled with a
single tool. This tool should be able to incrementally update the binary
grimoire from the reference machine to satisfy the third goal above. The
binary package tool should perform at least the following functions:


(1) create an initial grimoire directory with corresponding metadata

(2) add binary spells based on a currently installed spell on the
reference machine, preserving dependency information (includes
creating the binary package)

(3) remove binary spells when no longer wanted

(4) incrementally update spells in the binary grimoire that have been
updated on the reference machine


The bulk of the work is done in the second function listed above.
The binary spell should have a DETAILS that closely resembled that of the
source spell it created from, the "reference spell." All other files
would likely be identical between all binary spells and could simply be
symlinks to a single set of scripts located somewhere in the grimoire.
The PRE_BUILD should not unpack the SOURCE, only verify the MD5. The
BUILD should extract the binary package to the root directory,
effectively installing it. The POST_INSTALL should exist to prevent the
cleanup of a source directory (since none was used) and could be
extended to perform spell specific post install tasks. The incremental
update function should leave a modified POST_INSTALL in the binary
grimoire alone. The DEPENDS file should have all optional dependencies
from the reference spell that were enabled converted to required
dependencies, all other optional dependencies should be removed (or
commented out).

The binary packages themselves could either be created by the tool
itself, or simply pulled from the /var/cache/sorcery. In either case,
the /var/log/sorcery files for the spell should be excluded when the
package is extracted from BUILD as these files will be recreated by
sorcery on the client machine.

To facilitate the second goal above the depends and config options
in /etc/sorcery/local/depends and config_options_cache would need to be
preserved from the reference machine and set automatically in the
CONFIGURE script of the binary spell.



Archive powered by MHonArc 2.6.24.

Top of Page