Skip to Content.
Sympa Menu

sm-discuss - [SM-Discuss] SMGL and crisis management - coming out the other side

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: jack AT jbertram.net
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: [SM-Discuss] SMGL and crisis management - coming out the other side
  • Date: Sun, 20 Oct 2002 13:40:08 +0100

Hi all

Just thought I'd share my most recent experience of "sorcery update"
with you all in case anyone else comes across some of the same problems.

System: test grimoire, stable sorcery

Yesterday 18:00. Decided to sorcery update - first time since approx 20
Sep. Whirrs happily into action. List of about 8 spells to update,
including glibc. Give it no further thought.

Yesterday 23:30. Dinner party finishes. Head in to check out website
someone has mentioned. Opera crashes. Hmmm. Reload Opera. Doesn't
work. Try and bring up aterm. No joy. Flick over to other virtual
desktop and type 'opera' into an open aterm. Lots of glibc errors. Try
to look at sorcery log. Less doesn't work. Swear. Pull up mutt to
check out sm-discuss archives for help. Mutt doesn't work. Flick onto
virtual console 2. Can't log in. Can't su to root from aterms. Swear
louder. Decide to reboot.

Yesterday 23:50. Computer boots up. Lots of errors about /proc not
mounting. Eventually computer hangs pre-login prompt with last thing to
be loaded postgresql - which didn't work anyway, because /root not
mounted. Still assuming glibc problems.

Yesterday 00:00. Can't find sgl iso - lent it to a friend a while back.
No rescue CD. Bum.

This morning 00:10. Find SuSE 7.0 install cd. Boot rescue system.
Mount SGL partitions and chroot. Cast glibc. Compiles smoothly.

This morning 01:00. Glibc finishes compiling. Reboot with sigh of
relief.

This morning 01:05. See Yesterday 23:50. Now assuming that something
has got rid of my mount scripts. Reboot

This morning 01:10. Boot SuSE rescue system and chroot to SGL. Can't
find mount scripts. Create basic mount script to mount /proc, /dev/pts,
/root (what else is there, after all...) on boot. Reboot

This morning 01:15. Computer boots and no more /proc errors. But hangs
after launching (successfully) postgresql. Complete blank. Hit
Ctrl-Alt-Del to shut down smoothly. Reboot

This morning 01:20. Boot SuSE rescue system and mount /dev/hda3 on
/mnt... oh sh*t. Superblock corrupted. Realise had forgotten to
include unmounting instructions in basic mount script. Swear loudly
(waking up girlfriend). Break out in cold sweat. Launch fsck on
/dev/hda3. fsck can't find superblock. (/ is ext3, so should be
compatible with ext2 fsck). Feel sick. Try fsck -b 8193 /dev/hda3. No
joy. Try various other random numbers. No good. Try dumpe2fs.
Dumpe2fs laughs loudly in my face and tells me that the superblock is
corrupted. Thanks. Connect ADSL line to other Windows box (need
Internet). Boot and install drivers. Realise that user name and
password are in mail file on corrupted partition. Backups on xfs
partition which Suse rescue can't read. Feel sick again. Email has now
been bouncing for 5 hours. Boot up work laptop. First three cables to
phone line don't work. Eventually dial up at 14.4 kbps. In meantime,
launch "i=0; while true; do e2fsck -b $i /dev/hda3; i=$(($i+1)); done"
on SGL box in panic. Screen fills with unsuccessful messages. Have
worked out in meantime what problem is - had not mounted devfs on
/devices hence no virtual consoles to run agetty on.

This morning 01:35. Internet searching reveals that dumpe2fs is the
thing to use. Great.

This morning 01:40. Spare superblock turns out to be on block 32768.
How predictable. Fsck takes 15 minutes. Really panicked now.

This morning 01:55. Fsck finishes. Cross all fingers, toes and bodily
parts and mount /dev/hda3. Success!!!! Wipe brow. Edit
/etc/init.d/mount.sh to mount /devices. Reboot while praying to voodoo
gods near and far.

This morning 02:00. SGL prompt! Success. Lots of errors. Don't dare
to reboot in case filesystems bugger up again. Try to cast init.d.
Prompt hangs. Go to bed with email still bouncing. Girlfriend is
sympathetic but doesn't really understand what is going on.

This morning 11:00. Get up. Manage to coax ADSL into life. Email
starts flooding in. Update sorcery again. Sorcery hangs. Cast doesn't
work. Dispel sorcery. Doesn't work. Wget sorcery-stable and
/tmp/sorcery/install. Fine, but no change to cast/dispel problems.
Keep on hanging with "sleep 2" in the process table.

This morning 11:45. Decide to switch to sorcery-devel. Wget. Hangs.
Cast doesn't work. Lots of handle-depends messages. Mv
/etc/sorcery/local/depends somewhere. Frantically grepping sorcery
scripts for answers. Eventually manually delete lots of sorcery scripts
from all over the place. /tmp/sorcery/install again. Cast begins
working. Cheer (silently, so as not to wake girlfriend up) Cast
init.d. Don't understand networking menu. Guess and manually edit
init.d scripts. sorcery system-update for completeness. Beginning to
feel that the worst has passed.

This afternoon 13:00. Reboot. No errors. Done. Begin breathing
again.


Lessons learned:

- Mount scripts are important
- Devfs is important
- Glibc is important
- Don't sorcery update unattended
- Backup more stuff from / onto other hard disks
- Have sgl CD handy with xfs modules loaded.


Cheers all

jack




Archive powered by MHonArc 2.6.24.

Top of Page