Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] scm next steps

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Jeremy Blosser <jblosser-smgl AT firinn.org>
  • To: SM-Discuss <sm-discuss AT lists.ibiblio.org>
  • Subject: Re: [SM-Discuss] scm next steps
  • Date: Tue, 23 May 2006 17:09:40 -0500

On May 20, Jeremy Blosser [jblosser-smgl AT firinn.org] wrote:
> Ok, I'll set up hg if you want it. I may try to get git cvs working first,
> I'm really curious to see how that performs.

hg is imported, but I'm waiting on the upstream devs to provide some more
authentication functionality before I can let you guys use it. It seems
that having an access-controlled push repository is a new idea to them.
The hook framework to do it is there the same was it is with git or svk,
and I could probably implement it myself, but since they offered I'm going
to let them. We can see how responsive they really are to compare them to
the others.

While I'm waiting I have been running some benchmarks with hg vs. git.
While doing the import I noticed hg was running a good bit slower, and this
frankly surprised me. hg's big pitch is their speed, and from all I'd
heard I expected them to compare favorably with git. It turns out this
isn't the case for our repository. I asked the devs about this, and after
some back and forth and letting them look at our repos, the main dev came
back with this:

>>For what it's worth, your test repository represents a worst case
>>performance and space usage scenario for Mercurial. You have tens of
>>thousands of tiny files, each containing only one or two revisions.
>>Internal file fragmentation increases by 3x the amount of data that
>>Mercurial has to read. git will often see better compression, as in
>>this case, simply because a single gzip compression context can span an
>>entire pack file, while Mercurial's are necessarily per-file.
>>
>>To compare, .git is 25MB, .hg is 125MB, and "tar cf hg.tar .hg" is 40MB.
>>
>>Also, I suspect that the performance difference on "status" is due
>>solely to git being written in C.
>>
>>What you will find is that as your project history grows, the
>>performance and space consumption gap between the two closes. Also,
>>Mercurial's more extensive use of file indices will make many operations
>>faster. You should try some of the same operations on Linux kernel
>>repositories in git and hg as an example of what to expect once you have
>>deeper history. You'll find there that in many cases, hg is faster than
>>git.

The reference to "only one or two revisions" has to do with the way these
trees were imported for the eval. They aren't full imports, they're just
imports of the four main repositories in a way that maintains ancestry, as
I've described before.

I do have the git repo with the last couple years of history for devel and
test imported from p4, so I did some comparisons with that against the
history-less hg as well to test the claim that the gap would close with
more history. git's still running faster *with* history than hg does
without history; however, the operations I've tried so far aren't really
history-intensive, so the file size benefits of git are still primarily at
work. The thing is, I'm just not sure how much of what we do is really
going to be history-intensive beyond the raw data that git does so well
with. The overriding performance factor is still going to be the size and
composition of our tree.

Full benchmark data so far is attached. One more thing to note is that git
is imported as 4 branches in one repository, while hg is set up as one
repository for branch. hg supports multiple heads (branches) in one repo,
but they recommend doing a repo per branch, and I found the user interface
for the multiple head support to be confusing. Given those and that this
is an eval and git and hg can both do it either way I decided to set hg up
this way so we can try multiple workflows as well. Note, git's direct
cherry-pick is only available between local branches of one repo, while hg
has no direct cherry-pick command regardless. It uses an export/import
method (git can also do it this way).
git:
local clone (all 4 branches):
1: real 0m17.959s user 0m8.009s sys 0m1.660s
2: real 0m17.903s user 0m8.005s sys 0m1.404s
3: real 0m19.780s user 0m7.876s sys 0m1.632s
remote clone (ssh)
1: real 3m6.587s user 0m9.000s sys 0m5.050s
2: real 2m41.592s user 0m9.320s sys 0m4.970s
3: real 2m41.106s user 0m9.290s sys 0m4.430s
remote clone (http)
1: real 3m11.079s user 0m31.530s sys 0m4.960s
2: real 3m0.892s user 0m31.510s sys 0m5.530s
3: real 3m1.111s user 0m31.960s sys 0m4.970s
status (on server, test branch only):
1: real 0m1.015s user 0m0.552s sys 0m0.180s
2: real 0m0.717s user 0m0.496s sys 0m0.220s
3: real 0m0.733s user 0m0.536s sys 0m0.184s
status (on workstation)
1: real 0m2.187s user 0m1.470s sys 0m0.480s
2: real 0m1.869s user 0m1.470s sys 0m0.400s
3: real 0m1.871s user 0m1.490s sys 0m0.370s
pull/merge stable to test (on server)
1: real 3m6.284s user 1m56.059s sys 0m49.435s
pull/merge test to stable (on workstation)
1: real 9m50.296s user 6m0.680s sys 1m59.070s


hg:
local clone (1 branch per repo; devel, test, stable, stable-rc)
1: real 2m7.367s user 0m53.343s sys 0m11.905s
real 2m9.653s user 0m51.943s sys 0m12.517s
real 0m55.240s user 0m27.610s sys 0m8.053s
real 1m56.197s user 0m41.623s sys 0m10.069s
2: real 2m12.840s user 0m53.023s sys 0m11.749s
real 1m47.684s user 0m51.071s sys 0m11.337s
real 0m54.851s user 0m27.754s sys 0m7.568s
real 1m59.149s user 0m41.359s sys 0m10.477s
3: real 2m11.827s user 0m53.367s sys 0m11.569s
real 2m1.117s user 0m51.103s sys 0m11.269s
real 1m0.341s user 0m27.310s sys 0m7.572s
real 1m53.526s user 0m41.135s sys 0m10.669s
remote clone (ssh)
1: real 6m11.956s user 1m30.080s sys 0m42.570s
real 5m48.154s user 1m23.580s sys 0m38.950s
real 4m6.526s user 0m48.820s sys 0m22.160s
real 5m14.175s user 1m6.920s sys 0m30.730s
2: real 6m13.889s user 1m30.620s sys 0m41.210s
real 6m0.840s user 1m28.610s sys 0m38.950s
real 4m33.292s user 0m54.000s sys 0m24.350s
real 5m32.342s user 1m8.520s sys 0m31.560s
3: real 6m11.815s user 1m33.130s sys 0m42.810s
real 5m47.702s user 1m28.070s sys 0m39.040s
real 3m45.966s user 0m49.550s sys 0m22.500s
real 5m28.377s user 1m17.560s sys 0m34.330s
remote clone (http)
1: real 4m7.723s user 1m32.990s sys 0m32.730s
real 5m10.567s user 1m35.470s sys 0m34.670s
real 3m29.762s user 0m49.060s sys 0m18.820s
real 4m20.337s user 1m19.650s sys 0m30.280s
2: real 4m45.754s user 1m36.710s sys 0m34.250s
real 3m55.035s user 1m38.710s sys 0m37.090s
real 2m37.379s user 0m50.150s sys 0m19.950s
real 4m0.448s user 1m16.240s sys 0m28.250s
3: real 4m17.481s user 1m35.870s sys 0m35.550s
real 4m22.205s user 1m32.320s sys 0m33.700s
real 2m52.628s user 0m52.440s sys 0m22.320s
real 3m48.892s user 1m14.790s sys 0m29.650s
status (test branch only):
1: real 0m2.493s user 0m1.612s sys 0m0.324s
2: real 0m1.849s user 0m1.556s sys 0m0.256s
3: real 0m1.835s user 0m1.544s sys 0m0.280s
status
1: real 0m29.988s user 0m2.500s sys 0m0.440s
2: real 0m4.802s user 0m4.320s sys 0m0.480s
3: real 0m4.774s user 0m4.290s sys 0m0.470s
hg log
1: real 0m13.398s user 0m5.760s sys 0m0.260s
hg log -p (stopped)
1: real 44m37.792s user 39m33.496s sys 0m37.190s
pull/merge test to stable (on server)
1: real 4m9.073s user 0m40.831s sys 0m8.161s
pull/merge test to stable (on workstation)
1: real 6m50.038s user 1m25.340s sys 0m22.810s


git_p4:
local clone (2 branches with ~1.5 years history, including tags per p4
changeset)
1: real 2m22.608s user 0m17.957s sys 0m25.342s
2: real 2m19.403s user 0m17.533s sys 0m18.813s
3: real 1m0.110s user 0m17.401s sys 0m18.617s
local clone (2 branches with ~1.5 years history, no tags)
1: real 0m31.722s user 0m12.021s sys 0m1.588s
1: real 0m30.161s user 0m11.957s sys 0m1.600s
1: real 0m29.401s user 0m12.037s sys 0m1.848s
git log
1: real 0m12.933s user 0m0.304s sys 0m0.076s
git log -p (1314598 lines)
1: real 7m7.708s user 0m17.389s sys 0m2.616s
pull/merge devel to test (note, this left a ton of conflicts and probably
isn't that useful of a comparison)
real 0m13.041s user 0m5.128s sys 0m3.912s

Attachment: pgpgJg82EXGgf.pgp
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page