Source Mage code commit list

[Justin Boffemmyer <scm AT sourcemage.org>]

GIT changes to test cauldron by Justin Boffemmyer <flux AT sourcemage.org>:

 data/README.spells     |   30 +++++
 data/ispells.x86       |   84 +++++++++++++++
 data/ospells.x86       |   43 +------
 data/rspells.x86       |   21 ---
 scripts/spellcaster.sh |  268 
++++++++++++++++++++++++++++++++-----------------
 5 files changed, 302 insertions(+), 144 deletions(-)

New commits:
commit ba4167a81b61e7e111a411ffb4f4a9e1e9a63913
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    scripts/spellcaster.sh: big overhaul
    
    This script got a big overhaul. Most of the functionality the script was
    providing has put into proper functions. This makes the code more
    separable and easier to read. The script also supports optional spell
    installation methods. Also, the script will no longer attempt to set up
    everything "in-house", but will rather create spell caches in the build
    directory and then copy/unpack those caches to separate directories
    (system and iso) accordingly. This is a much cleaner method of
    generation.

commit dddb32fed9b8fd8536f4147555842198b101991d
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/tspells.x86 -> data/ospells.x86
    
    Renamed tspells to ospells. I originally planned to use the tspells file
    differently than it is being used, so I named it after "target". Now it
    is named after "optional" because it makes more sense for how it's
    actually used.

commit dd46f4e71a263a5c0528c08e033db704b50ff9f6
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    Properly separate spell listings
    
    Updated the spell listing files to really be separate. This allows for
    cleaner setups of the individual components. As a result, the only
    interdependency is that tspells expects the caches of spells in ispells
    to also be present, so the list of all available caches on a given ISO
    will be caches(ispells) + caches(tspells).

commit 322d940b564c722c3f6394e0080cacc5ba338a66
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/README.spells: new, explains spell listings
    
    Documents the contents of the different spell listing files and the
    reasoning behind those contents.

commit 3389348cccb2e521757dd9350ee16db7726d9029
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/tspells.x86: new, spells only for opt install
    
    This is a listing of those spells that are _only_ for optional install.
    In other words, they should only exist as cache files on the ISO, and
    not be installed either into system.tar.bz2 or onto the ISO runtime
    filesystem.

commit 52d5fa4001b4510ef4b6efe28b077bfb4a9a3bf9
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/ispells.x86: updated for missing spells
    
    Updated ispells.x86 to contain some useful/needed spells that were
    missing.

commit ad2ac966137845dbd1de433ace994f850c624f28
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/rspells.x86: added glibc hard dependencies
    
    Added libtool and libidn, which are hard dependencies for stable glibc
    and therefore required for a functioning system.

commit 2e9a3d89220ecfdf305f31ce17ec3effe97fad71
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/ospells.x86: renamed to ispells.x86
    
    The listing in ospells is really all of the spells that are in addition
    to the system.tar.bz2 spells, and this includes what is needed for the
    ISO to function as well as real "optional" spells. Therefore, this was
    renamed to more accurately reflect the fact that these spells are needed
    on the ISO as opposed to the system.

commit 9a6c30e9e1d0e68906ceca68ddc3c10dab617bb2
Author: Justin Boffemmyer <flux AT sourcemage.org>
Commit: Justin Boffemmyer <flux AT sourcemage.org>

    data/rspells.x86: cleaned out irrelevant spells
    
    Removed any spells that didn't really belong in the list. This will be
    used to build a bare minimal system.tar.bz2, so the list should be as
    minimal as possible.

diff --git a/data/README.spells b/data/README.spells
new file mode 100644
index 0000000..55c9858
--- /dev/null
+++ b/data/README.spells
@@ -0,0 +1,30 @@
+spell listings:
+       rspells.*  spells that should be in system.tar.bz2
+       ispells.*  spells that are needed on the ISO or for optional install
+       ospells.*  spells that should only exist as caches (not actually be
+                       "installed" in either system.tar.bz2 or the ISO 
runtime
+                       environment)
+
+system.tar.bz2 consists entirely (and only) of those spells explicitly listed
+in rspells.*
+
+       system.tar.bz2 = rspells
+
+The ISO runtime environment/filesystem will consist of those spells listed in
+ispells.*, but with the spells in ospells.* dispelled so that only their 
caches
+remain for installation.
+
+       ISO = ispells + ospells(caches)
+
+Anything which is meant _only_ for optional installation to the target system
+should be listed in ospells.*
+
+The listing in spells.x86 is the older leftover listing from before optional
+spells were re-implemented, and serves as a reference point for the newer 
spell
+listing files. It contains some cruft due to some of the spells listed in it
+having been obsoleted, and a few were left over as unweeded dependencies. 
These
+dependencies should have been weeded out in the newer listing files (i.e.,
+{rit}spells.x86), but it's possible there are still some "bugs" in this
+respect. This is why spells.x86 is being kept as a reference file, so that 
any
+spell/dependency bugs that crop up can be compared with known working ISO 
spell
+listings.
diff --git a/data/ispells.x86 b/data/ispells.x86
new file mode 100644
index 0000000..8312c49
--- /dev/null
+++ b/data/ispells.x86
@@ -0,0 +1,84 @@
+bash
+bzip2
+coreutils
+cpio
+device-mapper
+dhcpcd
+dialog
+e2fsprogs
+ed
+eject
+elvis
+file
+findutils
+g++
+gawk
+gcc
+gettext
+glibc
+gnupg
+gpm
+grep
+groff
+grub
+gzip
+hdparm
+iana-etc
+init.d
+ifupdown
+kbd
+jed
+jfsutils
+less
+libaal
+libidn
+libpcap
+libtool
+lilo
+locale
+lsscsi
+lvm
+lynx
+madwifi
+man
+mdadm
+memtest86+
+module-init-tools
+nano
+ncurses
+net-tools
+ndiswrapper
+nfs-utils
+ntfsprogs
+openssl
+parted
+pciutils
+pcmciautils
+portmap
+ppp
+procps
+readline
+reiser4progs
+reiserfsprogs
+rp-pppoe
+rpmunpack
+run-parts
+sed
+shadow
+simpleinit-msb
+slang
+smgl-fhs
+sysfsutils
+tar
+tcp_wrappers
+texinfo
+udev
+unzip
+usbutils
+util-linux
+uudeview
+wget
+which
+wireless_tools
+xfsprogs
+zlib
diff --git a/data/ospells.x86 b/data/ospells.x86
index 01e9636..ff9b9e9 100644
--- a/data/ospells.x86
+++ b/data/ospells.x86
@@ -1,39 +1,10 @@
-device-mapper
-dhcpcd
-e2fsprogs
-ed
-eject
-elvis
-g++
-gpm
+autoconf
+automake
+bin86
+bison
+flex
 grub
-hdparm
-ifupdown
-jed
-jfsutils
-libaal
 lilo
-lsscsi
-lvm
-lynx
-madwifi
-mdadm
-memtest86+
-ndiswrapper
-nfs-utils
-ntfsprogs
-openssl
-parted
-pciutils
-pcmciautils
-ppp
-reiser4progs
-reiserfsprogs
-rp-pppoe
-rpmunpack
-sysfsutils
+m4
+perl
 tcp_wrappers
-udev
-usbutils
-wireless_tools
-xfsprogs
diff --git a/data/rspells.x86 b/data/rspells.x86
index f39987d..07b8124 100644
--- a/data/rspells.x86
+++ b/data/rspells.x86
@@ -1,10 +1,5 @@
-autoconf
-automake
-basesystem
 bash
-bin86
 binutils
-bison
 bzip2
 coreutils
 cpio
@@ -12,52 +7,38 @@ dialog
 diffutils
 file
 findutils
-flex
 gawk
 gcc
 gettext
 glibc
 gnupg
 grep
-groff
 gzip
 iana-etc
 init.d
+ifupdown
 installwatch
 kbd
-less
 libidn
-libpcap
 libtool
 locale
-m4
 make
-man
 module-init-tools
 nano
 ncurses
 net-tools
-netkit-base
-netkit-ping
 patch
-perl
-popt
-portmap
 procps
 readline
 run-parts
 sed
 shadow
 simpleinit-msb
-slang
 smgl-archspecs
 smgl-fhs
 sorcery-pubkeys
 tar
-texinfo
 unzip
 util-linux
-uudeview
 wget
-which
 zlib
diff --git a/scripts/spellcaster.sh b/scripts/spellcaster.sh
index 062d3b0..79505f9 100755
--- a/scripts/spellcaster.sh
+++ b/scripts/spellcaster.sh
@@ -4,42 +4,17 @@
 # chroot, and also the casting and subsequent dispelling (with caches 
enabled)
 # of the optional spells.
 
-if [[ -n $CAULDRON_CHROOT && $# -eq 0 ]]
-then
-       if [[ $CAULDRON_CAST = y ]]
-       then
-               # cast all spells
-               /usr/sbin/cast $(tr '\n' ' ' </"$rspells") &&
-               /usr/sbin/cast $(tr '\n' ' ' </"$ospells")
-
-       elif [[ $CAULDRON_DISPEL = y ]]
-       then
-               # dispel the optional spells, so that we have only their 
cache files
-               # available
-               /usr/sbin/dispel $(tr '\n' ' ' </"$ospells")
-       fi
-
-       exit $?
-fi
-
 MYDIR="$(dirname $0)"
 CAULDRONDIR="$MYDIR"/../data
 
 function usage() {
   cat << EndUsage
-Usage: $(basename $0) [-h] -c | -d /path/to/target ARCHITECTURE
+Usage: $(basename $0) [-h] [-i ISO] [-s SYS] /path/to/target ARCHITECTURE
 
 Casts the required and optional spells onto the ISO.
 This script requires superuser privileges.
 
 Required:
-       -c
-           Cast the required and optional spells in the target directory. 
This
-           is the default action.
-
-       -d
-           Dispel the optional spells only from the target directory.
-
        /path/to/target
            The location of the chroot directory you would like to have the
            casting done in.
@@ -52,35 +27,165 @@ Required:
 
 Options:
        -h  Shows this help information
+
+       -i  Path to iso build directory (ISO). Defaults to /tmp/cauldron/iso.
+
+       -s  Path to system build directory (SYS). Defaults to
+           /tmp/cauldron/system.
 EndUsage
   exit 1
 } >&2
 
-export CAULDRON_CAST=y
+function parse_options() {
+       while getopts ":cdh" Option
+       do
+               case $Option in
+                       i ) ISODIR="$OPTARG" ;;
+                       s ) SYSDIR="$OPTARG" ;;
+                       h ) usage ;;
+                       * ) echo "Unrecognized option." >&2 && usage ;;
+               esac
+       done
+       shift $(($OPTIND - 1))
+}
+
+function priv_check() {
+       local SELF="$0"
+
+       if [[ $UID -ne 0 ]]
+       then
+               if [[ -x $(which sudo > /dev/null 2>&1) ]]
+               then
+                       exec sudo "$SELF $*"
+               else
+                       echo "Please enter the root password."
+                       exec su -c "$SELF $*" root
+               fi
+       fi
+}
 
-while getopts ":cdh" Option
-do
-  case $Option in
-    c ) export CAULDRON_DISPEL=n ; export CAULDRON_CAST=y ;;
-    d ) export CAULDRON_CAST=n ; export CAULDRON_DISPEL=y ;;
-    h ) usage ;;
-    * ) echo "Unrecognized option." >&2 && usage ;;
-  esac
-done
-shift $(($OPTIND - 1))
+# check to make sure that the chroot has sorcery set to do caches
+function sanity_check() {
+       local config="$TARGET/etc/sorcery/local/config"
+       local arch=
+       local choice=
+
+       if [[ -e "$config" ]]
+       then
+               arch=$(grep 'ARCHIVE=' "$config" | cut -d= -f2 | sed 's/"//g')
+               if [[ -n $arch && $arch != "on" ]]
+               then
+                       echo "Error! TARGET sorcery does not archive!" >&2
+                       echo -n "Set TARGET sorcery to archive? [yn]" >&2
+                       read -n1 choice
+
+                       if [[ $choice == 'y' ]]
+                       then
+                               sed -i 's/ARCHIVE=.*/ARCHIVE="on"/' "$config"
+                       else
+                               exit 2
+                       fi
+               fi
+       fi
+}
+
+function prepare_target() {
+       export rspells="rspells.$TYPE"
+       export ispells="ispells.$TYPE"
+       export ospells="ospells.$TYPE"
+
+       # If using the linux spell copy the kernel config to TARGET sorcery
+       grep -q '^linux$' "$CAULDRONDIR/$rspells" "$CAULDRONDIR/$ospells" &&
+       cp "$CAULDRONDIR/config-2.6" "$TARGET/etc/sorcery/local/kernel.config"
+
+       # Copy the list of spells needed for casting into the TARGET if 
casting
+       cp "$CAULDRONDIR/rspells.$TYPE" "$TARGET"/rspells
+       cp "$CAULDRONDIR/ispells.$TYPE" "$TARGET"/ispells
+       cp "$CAULDRONDIR/ospells.$TYPE" "$TARGET"/ospells
+
+       # generate basesystem casting script inside of TARGET
+       cat > "$TARGET"/build_base.sh <<-'BASE'
+
+       # If console-tools is found in TARGET, get rid of it to make way for 
kbd
+       [[ $(gaze -q installed console-tools) != "not installed" ]] &&
+               dispel --orphan always console-tools
 
-SELF="$0"
+       if [[ -n $CAULDRON_CHROOT && $# -eq 0 ]]
+       then
+               # cast basic/required spells
+               /usr/sbin/cast $(tr '\n' ' ' </rspells)
+               exit $?
+
+               # make a list of the caches to unpack for system
+               for spell in $(</rspells)
+               do
+                       echo $spell-$(gaze -q installed $spell) >> /sys-list
+               done
+               [[ $? -eq 0 ]] || exit 42
+       fi
+BASE
+
+       # generate iso spell casting script inside of TARGET
+       cat > "$TARGET"/build_iso.sh <<-'ISO'
+
+       if [[ -n $CAULDRON_CHROOT && $# -eq 0 ]]
+       then
+               # cast optional spells
+               /usr/sbin/cast $(tr '\n' ' ' </ispells)
+               exit $?
+
+               # make a list of the caches to unpack for iso
+               for spell in $(</ispells)
+               do
+                       echo $spell-$(gaze -q installed $spell) >> /iso-list
+               done
+               [[ $? -eq 0 ]] || exit 42
+       fi
+ISO
+
+       # generate optional spell casting script inside of TARGET
+       cat > "$TARGET"/build_optional.sh <<-'OPTIONAL'
+
+       if [[ -n $CAULDRON_CHROOT && $# -eq 0 ]]
+       then
+               # cast optional spells
+               /usr/sbin/cast $(tr '\n' ' ' </ospells)
+               exit $?
+
+               # make a list of the caches to unpack for iso
+               for spell in $(</ospells)
+               do
+                       echo $spell-$(gaze -q installed $spell) >> /opt-list
+               done
+               [[ $? -eq 0 ]] || exit 42
+
+               # dispel the optional spells, so that we have only their 
cache files
+               # available
+               /usr/sbin/dispel $(tr '\n' ' ' </ospells)
+               exit $?
+       fi
+OPTIONAL
+
+       chmod a+x "$TARGET"/build_base.sh
+       chmod a+x "$TARGET"/build_optional.sh
+}
+
+function clean_target() {
+       local config="etc/sorcery/local/kernel.config"
 
-if [[ $UID -ne 0 ]]
-then
-  if [[ -x $(which sudo > /dev/null 2>&1) ]]
-  then
-    exec sudo "$SELF $*"
-  else
-    echo "Please enter the root password."
-    exec su -c "$SELF $*" root
-  fi
-fi
+       # Clean up the target
+       rm -f "$TARGET/rspells" \
+               "$TARGET/ispells" \
+               "$TARGET/ospells" \
+               "$TARGET/$config" \
+               "$TARGET/build_base.sh" \
+               "$TARGET/build_optional.sh"
+}
+
+# main()
+parse_options
+
+priv_check
 
 [[ $# -lt 1 ]] && usage
 TARGET="$1"
@@ -89,54 +194,41 @@ shift
 [[ $# -gt 0 ]] && TYPE="$1"
 TYPE="${TYPE:-x86}"
 
-export rspells="rspells.$TYPE"
-export ospells="ospells.$TYPE"
+ISODIR="${$ISODIR:-/tmp/cauldron/iso}"
+SYSDIR="${$SYSDIR:-/tmp/cauldron/sys}"
 
-# check to make sure that the chroot has sorcery set to do caches
-if [[ -e "$TARGET"/etc/sorcery/local/config ]]
-then
-       CONFIG_ARCHIVE=$(grep 'ARCHIVE=' "$TARGET"/etc/sorcery/local/config | 
cut -d = -f 2 | sed 's/"//g')
-       if [[ -n $CONFIG_ARCHIVE && $CONFIG_ARCHIVE != "on" ]]
-       then
-               echo "Error! TARGET sorcery configured to not archive!" >&2
-               echo -n "Set the TARGET sorcery to archive? [yn]" >&2
-               read -n1 CHOICE
+sanity_check
 
-               if [[ $CHOICE == 'y' ]]
-               then
-                       sed -i 's/ARCHIVE=.*/ARCHIVE="on"/' 
"$TARGET"/etc/sorcery/local/config
-               else
-                       exit 2
-               fi
-       fi
-fi
+prepare_target
 
-# Copy necessary files to the target and chroot
-grep -q '^linux$' "$CAULDRONDIR/$rspells" "$CAULDRONDIR/$ospells" &&
-       cp "$CAULDRONDIR/config-2.6" "$TARGET/etc/sorcery/local/kernel.config"
+# chroot and build the basesystem inside the TARGET
+"$MYDIR/cauldronchr.sh" -d "$TARGET" /build_base.sh
 
-# Copy the list of spells needed for casting into the TARGET if casting
-[[ $CAULDRON_CAST = y ]] && cp "$CAULDRONDIR/$rspells" "$TARGET"/
+for cache in $(<"$TARGET"/sys-list)
+do
+       tar xjf "$TARGET"/var/cache/sorcery/$cache*.tar.bz2 -C "$SYSDIR"/
+done
 
-# ospells needs to be there whether casting or dispelling
-cp "$CAULDRONDIR/$ospells" "$TARGET"/
+# chroot and build the iso inside the TARGET
+"$MYDIR/cauldronchr.sh" -d "$TARGET" /build_iso.sh
 
-# copy the script into the TARGET so it can do the casting/dispelling
-cp "$0" "$TARGET"/
+# copy the caches over and unpack their contents
+for cache in $(<"$TARGET"/iso-list)
+do
+       tar xjf "$TARGET"/var/cache/sorcery/$cache*.tar.bz2 -C "$ISODIR"/
+       cp "$TARGET"/var/cache/sorcery/$cache*.tar.bz2 
"$ISODIR"/var/cache/sorcery/
+done
 
-# chroot and run the script inside the TARGET
-"$MYDIR/cauldronchr.sh" -d "$TARGET" /"$(basename $0)"
+# chroot and build the optional spells inside the TARGET
+"$MYDIR/cauldronchr.sh" -d "$TARGET" /build_optional.sh
 
-# Clean up the target
-[[ $CAULDRON_CAST = y ]] && rm "$TARGET/$rspells"
-rm "$TARGET/$ospells"
-[[ -e "$TARGET/etc/sorcery/local/kernel.config" ]] &&
-       rm "$TARGET/etc/sorcery/local/kernel.config"
-rm "$TARGET/$(basename $0)"
+# copy the caches over (only!)
+for cache in $(<"$TARGET"/opt-list)
+do
+       cp "$TARGET"/var/cache/sorcery/$cache*.tar.bz2 
"$ISODIR"/var/cache/sorcery/
+done
 
-unset rspells
-unset ospells
-unset CAULDRON_CAST
-unset CAULDRON_DISPEL
+# Keep a clean kitchen, wipes up the leftovers from the preparation step
+clean_target
 
 exit 0