Public SourceMage Discussion List

[Thomas Orgis <thomas-forum AT orgis.org>]

Am Sun, 5 Mar 2017 03:37:46 -0300
schrieb Ismael Luceno <ismael.luceno AT gmail.com>: 

> Our Achilles heel is our reliance on git-receive-pack for
> serialization, until we fix that there's no improvement.

I'm kindof missing the the issue. What is the goal here? Have the git
mirrors interchangeable, p.ex. behind a load balancer for pulling, or
also for pushing? A grid of git?

> IIRC, no. Thomas had some script, and I prototyped a parsing engine in
> AWK to process our grimoires faster:
> http://git.iodev.co.uk/ismael/smgl/experiments/spell-parser.git/

I am using brute force

for g in $grimoire_list
  do
  for s in $grimoire_spells
    summon -g $g --print raw $s 
  done
done

to focus on realiability over speed. A separate parser like Ismael's
might miss constructs that sorcery evaluates with out non-declarative
spells. But then, that parser might actually get more URLs extracted,
as it could analyse all branches in DETAILS and print out all possible
sources. Actually, I might just switch to that one, as it relieves the
mirror from running a SMGL chroot with the grimoires configured.

Anyhow, my method is a little perl script essentially running the above
and sorting URLs into sections. The result is

        http://mirror.sobukus.de/urls/grimoire/
and
        http://mirror.sobukus.de/files/grimoire/

. I do no verification of downloads yet. I intend to add an object
interface to the mirror at some point, along with the possibility to
store differing files with the same name. For now I based it on the
assumption that source archive file names are unique, ignoring corner
cases with possible conflict (patch for patch and patch for linux).
Hash checking comes naturally, then. Upstream signatures are more
tricky and actually annoying to deal with automatically. Well, we do
have the keyrings for them in the grimoire, but it has more context
than simply some file properties like size and hash sums.

> [Ismael's parser]
> While far from perfect, it does a pretty good job already. Could be
> rewritten in C for more speed.

My only gripe is that it will miss some things as it is no full
implementation of the bash language. But it is actually a shortcoming
of our package management that those basic spell properties are
constructed in bash. Perhaps that should be ignored for now, as for the
vast majority of spells, the parser should be fine (hm, is glibc
handled properly?).

> > project. For now I called it 'Wand', but if you want to replace it with
> > something better -- I'd love to hear about your suggestions!  
> <...>
> 
> +1.

So, what is called a Wizard elsewhere … stuff that magically makes all
the little things happen. Wave your wand to configure $stuff. Looks
fine. 

> Suggestion: recovery system and procedures could be called arround
> the topic of "necromancy" (since "resurrect" is already used), and

… or things about alternate dimensions … worlds where your system was
not broken … building interdimensional gates …

> mirroring around "reflection".

Hm. I guess we need an Olde English native speaker to tell how more
fitting the term is;-)


Alrighty then,

Thomas

Attachment: pgpO0eFF0Sakw.pgp
Description: Digitale Signatur von OpenPGP