Public SourceMage Discussion List

[Andrew <afrayedknot AT thefrayedknot.armory.com>]

On Wed, Jun 01, 2005 at 02:50:02PM -0400, Sergey A. Lipnevich wrote:
> Hi All,
> 
> Don't kill me just yet, I'm not trying to revive the old INSTALL_ROOT and
> TRACK_ROOT thread. Also, my apologies if this was suggested and ruled out
> already.

Actually its a good idea and it exists currently, just with a slightly
different implementation.

> 
> I had an idea on how to make both of these variables unnecessary. What if we
> follow this logic during cast:
> 1) build the spell the regular way;
> 2) if INSTALL_ROOT is empty, do nothing;
> 3) if INSTALL_ROOT is not empty, go into "chroot $INSTALL_ROOT" mode, where
> /usr/src/<spell> is mounted in the same place path
> ($INSTALL_ROOT/usr/src/<spell> would be a full path);
> 4) do the install without using $INSTALL_ROOT, in chroot jail;
> 5) after POST_INSTALL, get out of chroot and continue as usual.
> 
> The chroot area can be pre-created when a system builder chooses non-empty 
> value
> for INSTALL_ROOT. The problem here is that spells that need to access
> dependencies /inside/ $INSTALL_ROOT on step #1 will not be able to do so. 
> But I
> believe those would be very few spells, and none of the spells required for 
> the
> ISO fall into this group. Maybe in those cases chroot could be set up before
> step 1, as an exception.
> 
> There would be two benefits of doing it this way: (a) no mention of 
> INSTALL_ROOT
> or TRACK_ROOT anywhere in spells; (b) simplified ISO creation (?).
> 
> I'm sure it's not as simple as that, but would it work in principle?
> Thanks,
> 

First, lets try and get away from thinking that by doing complex
gymnastics around installing we can make INSTALL_ROOT and TRACK_ROOT
unnecessary. They dont just go away, we just change the implementation
which hides them from certain pieces of code (and makes other code more
aware). I'll ellaborate on this later.

In order to do what you want (build inside a chroot) you have to have an
entire functioning system, including all the dependencies of the spell
you're building, inside the chroot. You cant simply build something that
links against libraries without that stuff installed. You cant build
subversion if apr isnt installed. So the chroot for installing would
have to be pre-generated. Is this an idea worth considering? Definitly.
Will it eliminate INSTALL/TRACK_ROOT variables? I disagree, it just
changes (some of) the implementation. Will it make for less work for
spell maintainers? Depends. (more on this after some background on how
things work now)


Throughout the ages people have had many ideas on how installing to
somewhere other than / works. There are two degrees of freedom from which
to work. Great turmoil seems to arise because people think theres only
one degree of freedom involved in this discussion, and assign different
meanings to that one degree. This is a two dimensional plane. Lets take
some case studies of things people want to do. We'll leave the various
sorcery variables out of the discussion for the moment.


Case 1:
I want to generate a seperate chroot. I want to be able to chroot into
this directory and have it feel exactly as if it was a native system. In
other words, no reference to things outside the chroot on the building
system should exist, the path to the chroot from outside should not be
mentioned anywhere inside the chroot.


Case 2:
I want to install firefox, but not directly over /. I already have one
version installed onto /, but want to have other versions availible
to try and I dont want them getting in each other's way. I want the
subordinate versions installed on /opt/firefox-<version>.  Each version
of firefox should understand that all of its libraries and related data
live in its /opt/firefox-<version>/ directory and not in / or anywhere else.


So, we have two distinct things we may want to do, in the first, files
installed should *not* refer to themselves using a full path because
they are expected to be run from inside a chroot, in the second, files
should refer to themselves since they are not intended to be run from
inside a chroot root.

We can hopefully agree that there are two degrees of freedom shown
above. A variable should denote *where* files go. And another variable
should describe from where they are to be run. In the autoconf world there
are HOST and BUILD variables to describe the system doing the compiling
and the system thats going to run the binaries.  In sorcery someone choose
INSTALL_ROOT to deonte where files go. TRACK_ROOT describes where files
prefer to be tracked from. These are arbitrary names for the two degrees
of freedom.

So, regardless of implementation, in order to support both usage patterns
these variables must exist in some for or another. They are atomic,
one cannot imply the other.

Now, inside sorcery what happens? Currently sorcery trusts spells
to respect these values. So, for now assume a spell builds files
with TRACK_ROOT burned in as paths but actually installs them to
INSTALL_ROOT. installwatch see's files installed with INSTALL_ROOT on
their path. Sorcery then strips INSTALL_ROOT off and replaces it with
TRACK_ROOT for storage in the logs. The idea is, in case 1, you're
storing your sorcery data inside the chroot as well, and thus, you can
run sorcery commands inside the chroot with empty paths.

Now, one could reasonably argue that trusting spells to install properly
is a fallacy and that spell writers should only be concerned with
installing to /. Which I believe is the motivating factor behind this
email. You would like spells to install to wherever they're going to
install and sorcery should take care of the rest. In terms of
implementation, spells are built inside a chroot for protecting the main
system using TRACK_ROOT as a prefix (to satisfy usage case 2). That
way everything is built with TRACK_ROOT burned in as we want, and its
installed in some sane and consistent manner. Probably prefixed by
TRACK_ROOT (note this is what many non-basesystem spells do, but with
INSTALL_ROOT instead).  Once this is done, sorcery would then politely
copy all the files to their INSTALL_ROOTified location. Another source
distro has a chroot'less variation of this (eg, everything builds against
/ but then installs onto a seperate path and is copied post-mortem).

In your solution spells basically all substitute INSTALL_ROOT for
TRACK_ROOT. This solution solves both case 1 and case 2 (the other
distro's solution only solves 1 case).

That aside, your solution requires on the fly or pre-generation of
a chroot to build in that includes all the spell dependencies for
implementing non-empty INSTALL_ROOT. So, what is currently done? If
I want to build a chrootable system, I build a minimum subset of spells
(basesystem) into the INSTALL_ROOT, then chroot there, then build
whatever else I want.  This is basically what you suggested if we go
with the pre-generated quantifier. The only difference is sorcery is
somehow responsible for the chroot in slightly different ways (i guess).

Using this method one can effectively get all spells to build into a
chroot of the same architecture by simply bootstrapping a base install.
Bug 8853 focuses on making basesystem spells do this properly.

So in other words, spells outside basesystem dont need to worry as much
about supporting both usage cases.

Cross compiling is a different story altogether though since we have
to break some of the assumptions we make, such as the chroot being
executable, in that model you have to install to somewhere other than
/ since otherwise you'd stomp on the host system, but you may want to
burn / in as the location in which things expect to be built...So in
that regard its almost unavoidable having to make spells respect only
one of the two degrees of freedom.

Sorry for the long post, hopefully the analysis will clear things up.

-Andrew

-- 
__________________________________________________________________________
|Andrew D. Stitt                           | astitt at sourcemage.org    |
|irc: afrayedknot                          | afrayedknot at t.armory.com |
|aim: thefrayedknot or iteratorplusplus    | acedit at armory.com        |
|Sorcery Team Lead                         | ftp://t.armory.com/         |
--------------------------------------------------------------------------

Attachment: pgp0QJvV_GQfi.pgp
Description: PGP signature