Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] Some Server Migration Progress!

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: David Kowis <dkowis AT shlrm.org>
  • To: Source Mage Discussion <sm-discuss AT lists.ibiblio.org>
  • Subject: Re: [SM-Discuss] Some Server Migration Progress!
  • Date: Wed, 20 Jan 2016 10:22:17 -0600

On 2016-01-20 02:57, Thomas Orgis wrote:
Am Tue, 19 Jan 2016 18:40:45 -0600
schrieb David Kowis <dkowis AT shlrm.org>:

On 01/19/2016 02:23 AM, Thomas Orgis wrote:
> Hi,
>
> about that new cheaper server … that kind of hardware/setup is it? I was
> wondering why mediawiki.sourcemage.org is that darn slow. It's also
> around a second after the initial loading of content that the style
> sheet seems to be arriving and then the whole page changes. Stealth's
> site is so much more snappy:-/ Wikipedia itself also is faster and I
> don't see the late style arrival. What's the difference? Is
> mediawiki.sourcemage.org under some DoS attack?

The hardware isn't bad, although I don't know the exact specifications.
I am running the pgsql, mediawiki, and bugzilla all on the same server.
We can probably get more ram for this VM if this is the way we want to
run it, or we can move the database server to a different VM. There's
some flexibility there.

Since there is not much data in there (I suppose), I doubt that a huge
amount of RAM would change things. Or is the VM at its RAM limit? No
disk cache and thus slow access times? I am not trying to diss your
setup, I am truly wondering why …

Perhaps emrys could speak more to that, also I'm not using the exact same technology that Stealth is using on the beta.sourcemage.ru site. I chose simpler, more popular tools (to the best of my knowledge) so that they'd be easy to maintain, and easy to use. This is also the same host hosting the primary sourcemage download location, so it's probably not ideal for hosting the site. The other VM I know of that is available is still waiting for the beta.sourcemage.ru transfer. We can decide what to do about that now, or later.

model name : Intel(R) Xeon(R) CPU E5405 @ 2.00GHz

total used free shared buffers cached
Mem: 2048 1825 223 0 760 593

We could probably stand for a bit more ram, especially if I were to fire up memcached to help bugzilla and mediawiki out. I'm sure PHP will want to consume more ram when we get the caching bits in there too.

<snip>
… this one needs 700 ms for transfer (1430 ms total). When I'm in the
browser, there is another second until the style arrives and the page
changes layout. There is some huge initial delay before anything is
delivered. There is both a moment of hesitation before the "200 OK" and
slower transfer of the data after that, even accounting for the
differing size.

That's possible that I'm doing the styling wrong for mediawiki, but I do not see the same behavior, so maybe it's something else. Perhaps your internet pip is happier to get to beta.sourcemage.ru than it is to get to mediawiki.sourcemage.org:
https://gist.github.com/dkowis/86dcdd272bac6e4ce4cd

It could also be the lack of PHP caching that's causing this, because it certainly had a short delay before it delivered the index.html. I know we're not at peak performance, and that is a thing that we'll work on.


However, as I mentioned in the previous email, our PHP spell is out of
date, and does not include the useful caching stuff that would make
mediawiki perform substantially faster.

Point taken, that is something we should fix, then.

get something done, and out there first, as a test run to see how well
it worked putting some content in there.

Understood. As long as we got some routes to optimise things. And I
hope one can really skin this cat so that it looks more like Stealth's
site and not like Wikipedia.

There's plenty of documentation on skinning mediawiki on mediawiki's page. I don't have time to do everything: https://www.mediawiki.org/wiki/Manual:Skinning_Part_1

This distro only goes as far as the volunteers that do things. I'd be more than happy to import Stealth's site over to the distro's VM and start using it, as well. Much good work has been done there.


It's extremely fast for anonymous users

Not as it is now. I didn't try to login yet. Are accounts migrated?

No accounts have been migrated, I saw no reason to migrate accounts from redmine to mediawiki. There are people who haven't been party to SourceMage for a while in there, and with how trivial it is to set up a new account, one can just sign up.

If we want to have some kind of centralized authentication mechanism, more work will need to be done around an LDAP server or something so that bugzilla and mediawiki can use the same authentication backend and the accounts are truly the same. It would be cool to have some kind of nice integration maybe a standalone CAS server that could do 2fa for all our things, but now I'm kinda dreaming, since we don't even have HTTPS anyway, heh.

https://letsencrypt.org/ is in public beta, so that can go on the backlog of things to use, and we can set up the automated https certs from those guys! I think that work is certainly worth doing at some point as well :)



appreciate some help with the php spell though). I think it probably
wouldn't hurt to go all the way to php 7 and get that working.

The spell should updated in any case (not that I am that much
interested in PHP myself, though). So that work is not wasted even if
Stealth reappears.

Since there hasn't been any objection to my earlier proposal, I'm
going to start the wheels in motion to move to the things that I've
set up on the distro hardware, so at the least, we don't have a
"Sorry it's broke" page.

OK, any page is better than that.

With regards to bugzilla and importing bugs,
I'm still not fully convinced that ancient,

I am growing to accept that the old bugs are not of much use. I'd like
to have an archive for referencing old fixed bugs, since those are
mentioned in commits and HISTORY files, so we know why we did something
there. But the live instance of bugzilla does not really need those.

But can we start counting at a point > 1 so that bug numbers don't get
repeated?

I think we can tell bugzilla to start with a newer bug ID, but I'm not sure. It's easy enough to install your own standalone bugzilla and play with the api, or even the database, and perhaps we can start at bug 9000 just for giggles ;) Feel free to experiment with the existing bugzilla, right now we can still clobber it, and start all over again without consequence. Go ahead and break stuff!

As for an archive, I have the bugzilla database, and I suppose I could put together either another old bugzilla with the old database and old structure put together, or we can write some code(direct sql queries, or API calls against the old one) to pull them out with the proper bug IDs and inject them into the new bugzilla (modifying components for compatibility with the newer organization). I think that's important, but more important is getting the site up and running well.


I do not wish to keep
the same bugzilla structure as before, as that doesn't make any sense
to do at all with the type of organization we are right now.

Agreed. Simpler bugzilla setup (just make sure we got those latest
security fixes in there … CGI.pm and all) is good.

I am currently running the latest version of bugzilla, with the intent on keeping it up to date. I let it automatically install the perl modules it wants, as it installs them into it's local directory, so I'm offloading the bugzilla perl module expertise to the bugzilla developers themselves :) We should be good on this front. I feel pretty confident in it. I was eyeing mod_security as well, but apparently that and mediawiki don't get along so well, and since the spell for mod_security is super out of date, I'm not going to worry about that either. I'll just be sure to keep mediawiki up to date, and we'll run with it.

--
David Kowis




Archive powered by MHonArc 2.6.24.

Top of Page