Public SourceMage Discussion List

[David Kowis <dkowis AT shlrm.org>]

On 2016-01-20 02:57, Thomas Orgis wrote:
> Am Tue, 19 Jan 2016 18:40:45 -0600
> schrieb David Kowis <dkowis AT shlrm.org>:
>
> > On 01/19/2016 02:23 AM, Thomas Orgis wrote:
> > > Hi,
> > >
> > > about that new cheaper server … that kind of hardware/setup is it? I was
> > > wondering why mediawiki.sourcemage.org is that darn slow. It's also
> > > around a second after the initial loading of content that the style
> > > sheet seems to be arriving and then the whole page changes. Stealth's
> > > site is so much more snappy:-/ Wikipedia itself also is faster and I
> > > don't see the late style arrival. What's the difference? Is
> > > mediawiki.sourcemage.org under some DoS attack?
> >
> > The hardware isn't bad, although I don't know the exact 
> > specifications.
> > I am running the pgsql, mediawiki, and bugzilla all on the same 
> > server.
> > We can probably get more ram for this VM if this is the way we want to
> > run it, or we can move the database server to a different VM. There's
> > some flexibility there.
>
> Since there is not much data in there (I suppose), I doubt that a huge
> amount of RAM would change things. Or is the VM at its RAM limit? No
> disk cache and thus slow access times? I am not trying to diss your
> setup, I am truly wondering why …

Perhaps emrys could speak more to that, also I'm not using the exact 
same technology that Stealth is using on the beta.sourcemage.ru site. I 
chose simpler, more popular tools (to the best of my knowledge) so that 
they'd be easy to maintain, and easy to use. This is also the same host 
hosting the primary sourcemage download location, so it's probably not 
ideal for hosting the site. The other VM I know of that is available is 
still waiting for the beta.sourcemage.ru transfer. We can decide what to 
do about that now, or later.

model name      : Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz

             total       used       free     shared    buffers     
cached
Mem:          2048       1825        223          0        760        
593

We could probably stand for a bit more ram, especially if I were to fire 
up memcached to help bugzilla and mediawiki out. I'm sure PHP will want 
to consume more ram when we get the caching bits in there too.

<snip>
> … this one needs 700 ms for transfer (1430 ms total). When I'm in the
> browser, there is another second until the style arrives and the page
> changes layout. There is some huge initial delay before anything is
> delivered. There is both a moment of hesitation before the "200 OK" and
> slower transfer of the data after that, even accounting for the
> differing size.

That's possible that I'm doing the styling wrong for mediawiki, but I do 
not see the same behavior, so maybe it's something else. Perhaps your 
internet pip is happier to get to beta.sourcemage.ru than it is to get 
to mediawiki.sourcemage.org:
https://gist.github.com/dkowis/86dcdd272bac6e4ce4cd

It could also be the lack of PHP caching that's causing this, because it 
certainly had a short delay before it delivered the index.html. I know 
we're not at peak performance, and that is a thing that we'll work on.

> > However, as I mentioned in the previous email, our PHP spell is out of
> > date, and does not include the useful caching stuff that would make
> > mediawiki perform substantially faster.
>
> Point taken, that is something we should fix, then.
>
> > get something done, and out there first, as a test run to see how well
> > it worked putting some content in there.
>
> Understood. As long as we got some routes to optimise things. And I
> hope one can really skin this cat so that it looks more like Stealth's
> site and not like Wikipedia.

There's plenty of documentation on skinning mediawiki on mediawiki's 
page. I don't have time to do everything: 
https://www.mediawiki.org/wiki/Manual:Skinning_Part_1

This distro only goes as far as the volunteers that do things. I'd be 
more than happy to import Stealth's site over to the distro's VM and 
start using it, as well. Much good work has been done there.

> > It's extremely fast for anonymous users
>
> Not as it is now. I didn't try to login yet. Are accounts migrated?

No accounts have been migrated, I saw no reason to migrate accounts from 
redmine to mediawiki. There are people who haven't been party to 
SourceMage for a while in there, and with how trivial it is to set up a 
new account, one can just sign up.

If we want to have some kind of centralized authentication mechanism, 
more work will need to be done around an LDAP server or something so 
that bugzilla and mediawiki can use the same authentication backend and 
the accounts are truly the same. It would be cool to have some kind of 
nice integration maybe a standalone CAS server that could do 2fa for all 
our things, but now I'm kinda dreaming, since we don't even have HTTPS 
anyway, heh.

https://letsencrypt.org/ is in public beta, so that can go on the 
backlog of things to use, and we can set up the automated https certs 
from those guys! I think that work is certainly worth doing at some 
point as well :)


> > appreciate some help with the php spell though). I think it probably
> > wouldn't hurt to go all the way to php 7 and get that working.
>
> The spell should updated in any case (not that I am that much
> interested in PHP myself, though). So that work is not wasted even if
> Stealth reappears.
>
> > Since there hasn't been any objection to my earlier proposal, I'm
> > going to start the wheels in motion to move to the things that I've
> > set up on the distro hardware, so at the least, we don't have a
> > "Sorry it's broke" page.
>
> OK, any page is better than that.
>
> > With regards to bugzilla and importing bugs,
> > I'm still not fully convinced that ancient,
>
> I am growing to accept that the old bugs are not of much use. I'd like
> to have an archive for referencing old fixed bugs, since those are
> mentioned in commits and HISTORY files, so we know why we did something
> there. But the live instance of bugzilla does not really need those.
>
> But can we start counting at a point > 1 so that bug numbers don't get
> repeated?

I think we can tell bugzilla to start with a newer bug ID, but I'm not 
sure. It's easy enough to install your own standalone bugzilla and play 
with the api, or even the database, and perhaps we can start at bug 9000 
just for giggles ;) Feel free to experiment with the existing bugzilla, 
right now we can still clobber it, and start all over again without 
consequence. Go ahead and break stuff!

As for an archive, I have the bugzilla database, and I suppose I could 
put together either another old bugzilla with the old database and old 
structure put together, or we can write some code(direct sql queries, or 
API calls against the old one) to pull them out with the proper bug IDs 
and inject them into the new bugzilla (modifying components for 
compatibility with the newer organization). I think that's important, 
but more important is getting the site up and running well.

> > I do not wish to keep
> > the same bugzilla structure as before, as that doesn't make any sense
> > to do at all with the type of organization we are right now.
>
> Agreed. Simpler bugzilla setup (just make sure we got those latest
> security fixes in there … CGI.pm and all) is good.

I am currently running the latest version of bugzilla, with the intent 
on keeping it up to date. I let it automatically install the perl 
modules it wants, as it installs them into it's local directory, so I'm 
offloading the bugzilla perl module expertise to the bugzilla developers 
themselves :) We should be good on this front. I feel pretty confident 
in it. I was eyeing mod_security as well, but apparently that and 
mediawiki don't get along so well, and since the spell for mod_security 
is super out of date, I'm not going to worry about that either. I'll 
just be sure to keep mediawiki up to date, and we'll run with it.

--
David Kowis