Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] Source Mage mirroring and infrastructure status

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Pavel Vinogradov <vin.public AT gmail.com>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] Source Mage mirroring and infrastructure status
  • Date: Mon, 6 Mar 2017 16:21:27 -0500

What do you guys think of these as a backbone for mirroring:
1) https://ipfs.io
2) https://syncthing.net

On Sat, Mar 04, 2017 at 09:47:18PM -0500, Vlad Glagolev wrote:
Hi mages!

First of all, I want to give some updates on repository mirroring:

2 weeks ago I started to look into the tool[0] to auto-mirror our repos to several main SCM providers. I've been testing it for 2 weeks, and, well -- so far so good: it takes every repository we have configured on our primary server and mirrors them to GitHub and Bitbucket. It should be very easy to extend it to support other types of mirrors.

From now on there are 2 mirrors available, syncing every 6 hours:

- https://github.com/sourcemage - https://bitbucket.org/sourcemage/

Tech specs: Python, multi-processing, uses clean YAML config and 'requests' module to perform API calls against GitHub and Bitbucket.

By the way, recently GitHub has updated theirs ToS[1], so while communities have discussions on whether to remove the code from GitHub or not, we might want to re-think of what we'd better prefer on the reliability side.

Regarding Bitbucket: there's a clean view of projects, which you can find here:

https://bitbucket.org/sourcemage/profile/projects

I also try to keep our OpenHub page in sync to get proper metrics sync-ed back to the website:

https://www.openhub.net/p/sourcemage

Mirroring part is quite important. We need to resolve our fallback consistency once and for all. Recently I also updated 'axel' and 'netselect' spells, so maybe we could re-use them to keep our fallback mirrors up-to-date. Quick question: do we have any scripts responsible for auto-summon and hash-verification for all spells in the grimoire hanging somewhere? If yes, I'd like to revive them.

The last, but not least: I'd like to introduce a new component for our project. For now I called it 'Wand', but if you want to replace it with something better -- I'd love to hear about your suggestions!

It was initialized with the tool I mentioned above, but it'll get some expansion beyond this year. So the component will be about infra which we never properly declared, and it's a good year to fix it: mirroring, config management, user management, infrastructure testing, access management automation, server scaling/fault-tolerance, DNS-management, SSL certificate management, e-mail configuration, backups, monitoring, etc. So this would be a collection of tools and playbooks required to properly build and manage the whole infrastructure and keep it reliable and resilient.

Please let me know what you think.

[0] http://scmweb.sourcemage.org/?p=smgl/wand.git;a=tree;f=remirror [1] https://www.mirbsd.org/permalinks/wlog-10_e20170301-tg.htm

Attachment: signature.asc
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page