Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] Source Mage mirroring and infrastructure status

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Vlad Glagolev <stealth AT tiberian.ru>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] Source Mage mirroring and infrastructure status
  • Date: Sun, 5 Mar 2017 09:30:07 -0500

On Sun, 5 Mar 2017 15:14:41 +0100
Thomas Orgis <thomas-forum AT orgis.org> wrote:

> Am Sun, 5 Mar 2017 03:37:46 -0300
> schrieb Ismael Luceno <ismael.luceno AT gmail.com>:
>
> > Our Achilles heel is our reliance on git-receive-pack for
> > serialization, until we fix that there's no improvement.
>
> I'm kindof missing the the issue. What is the goal here? Have the git
> mirrors interchangeable, p.ex. behind a load balancer for pulling, or
> also for pushing? A grid of git?
>
> > IIRC, no. Thomas had some script, and I prototyped a parsing engine in
> > AWK to process our grimoires faster:
> > http://git.iodev.co.uk/ismael/smgl/experiments/spell-parser.git/
>
> I am using brute force
>
> for g in $grimoire_list
> do
> for s in $grimoire_spells
> summon -g $g --print raw $s
> done
> done
>
> to focus on realiability over speed. A separate parser like Ismael's
> might miss constructs that sorcery evaluates with out non-declarative
> spells. But then, that parser might actually get more URLs extracted,
> as it could analyse all branches in DETAILS and print out all possible
> sources. Actually, I might just switch to that one, as it relieves the
> mirror from running a SMGL chroot with the grimoires configured.
>
> Anyhow, my method is a little perl script essentially running the above
> and sorting URLs into sections. The result is
>
> http://mirror.sobukus.de/urls/grimoire/
> and
> http://mirror.sobukus.de/files/grimoire/
>
> . I do no verification of downloads yet. I intend to add an object
> interface to the mirror at some point, along with the possibility to
> store differing files with the same name. For now I based it on the
> assumption that source archive file names are unique, ignoring corner
> cases with possible conflict (patch for patch and patch for linux).
> Hash checking comes naturally, then. Upstream signatures are more
> tricky and actually annoying to deal with automatically. Well, we do
> have the keyrings for them in the grimoire, but it has more context
> than simply some file properties like size and hash sums.
>
> > [Ismael's parser]
> > While far from perfect, it does a pretty good job already. Could be
> > rewritten in C for more speed.
>
> My only gripe is that it will miss some things as it is no full
> implementation of the bash language. But it is actually a shortcoming
> of our package management that those basic spell properties are
> constructed in bash. Perhaps that should be ignored for now, as for the
> vast majority of spells, the parser should be fine (hm, is glibc
> handled properly?).
>
> > > project. For now I called it 'Wand', but if you want to replace it with
> > > something better -- I'd love to hear about your suggestions!
> > <...>
> >
> > +1.
>
> So, what is called a Wizard elsewhere … stuff that magically makes all
> the little things happen. Wave your wand to configure $stuff. Looks
> fine.

Yeah, just to mention that wand is for us, mages, to keep our
infrastructure in declared (as part of the Infrastructure as Code) and
healthy state.

>
> > Suggestion: recovery system and procedures could be called arround
> > the topic of "necromancy" (since "resurrect" is already used), and
>
> … or things about alternate dimensions … worlds where your system was
> not broken … building interdimensional gates …
>
> > mirroring around "reflection".
>
> Hm. I guess we need an Olde English native speaker to tell how more
> fitting the term is;-)
>
>
> Alrighty then,
>
> Thomas


--
Vlad Glagolev <stealth AT tiberian.ru>

Attachment: pgpjMC5Qn53Nt.pgp
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page