Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] scm next steps

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: "Jeremy Blosser (emrys)" <jblosser-smgl AT firinn.org>
  • To: sm-discuss AT lists.ibiblio.org
  • Subject: Re: [SM-Discuss] scm next steps
  • Date: Thu, 27 Apr 2006 00:47:14 -0500

On Apr 12, Jeremy Blosser (emrys) [jblosser-smgl AT firinn.org] wrote:
> The current identified SCMs to try are git, svk, and svn. git is most
> similar to bzr (which the other teams are already using) and has a very
> robust feature set. svk is most similar to p4. svn is probably in the
> widest general community use. Is this the right list? Do we have some
> volunteer gurus/sections for each?

Ok, I think git is ready for people to try. Sorry it's taking so long, git
is really rather flexible and David Brown and I had to do some research and
tests on the right way to approach setting it up for grimoire work. I
really wanted to have all 3 working by now, with a script to generate the
tarballs from all, but I don't want to make people wait for that anymore.
I still need to get git pushing to the commits ML and get gitweb up but it
should be useable now. If someone wants to take this and dump it onto the
wiki I'd appreciate it, else I'll try to find time. I need to work on svn
next.

If you want to work on a section in git, let me know and I'll get you
access.

Sizes
=====
First, some numbers for those who are worried about disk space of having
the repository history on your disk:

~/smgl/scm/p4/sgl/grimoires $ du -chs devel test stable-rc/0.4 stable/0.3
113M devel
110M test
110M stable-rc/0.4
98M stable/0.3
430M total

~/smgl/scm/git/smgl/grimoire $ du -chs .
127M grimoire
~/smgl/scm/git/smgl/grimoire $ git branch
devel
* master
origin
stable-0.3
stable-rc-0.4

So that means the git grimoire repository, with all branches, is around
1/4th - 1/3rd the size of what you get checked out from p4 right now to get
the same. Yes, that doesn't include history yet, but you can fit a fair
amount of history in that much space. More on those branches later...

Notes
=====
- The server is using git 1.3.0 from current test grimoire, you probably
want that version too.

- git has a ton of commands because it's a unix-style app with a util for
everything and then aggregate commands that call things the ways that
make sense for day to day work. Other frontends like cogito are
replacements for the aggregation layer. git has docs for every command,
and they're worth reading. Or just follow the basic list below.

- Some of this is going to look very different to what you're used to.
Some of it I didn't like at first either. I tried to forget my
assumptions and be open-minded to internalizing the 'git way' and found
that once I broke through, it made a lot of sense and is a very unix
approach to SCM. git does revisions and it does branching really, really
well. Those tools let you do about anything you can think of, but the
answer to your difficult use case / question is quite possibly going to
involve 'make a [temporary] branch'. Try to give it a fair chance, I
promise it is not so bad. :-)

- There's a ton of stuff I could get into, I will try to keep this to the
basic things and FAQs, that will be long enough for now.

Repository
==========
I built the repository for this trial as follows:
1) init a git repository off of stable/0.3
2) make a stable-0.3 branch from this
3) bring in the stable-rc/0.4 changes and commit them
4) branch stable-rc-0.4 from this
5) bring in the test changes and commit them
6) make a devel branch and change to it
7) bring in the devel changes and commit them

- We said we weren't going to keep a standing devel except for when we had
WIP stuff, but branches in git are repository-wide, so making it a
regular branch now (but not the trunk) made sense.

- This ancestry isn't perfect, since stable-0.3 should be a child of
stable-rc-0.3 instead of directly from test... some examples below show
merging to 0.3 from -rc/0.4, which isn't correct, either. But during the
trial this works.

- I did the updates by copying and committing per-spell, so we have a
commit per spell per branch. This isn't a perfect match for the changes
made in p4, but hopefully it's enough for a trial. It lets you guys do
per-spell integrations for most existing changes. If we decide on a full
trial for git and we can get diff-level history out of p4 we can probably
script a full replay conversion.

- This sync is a couple of days old now, you'll want to check p4 to make
sure you have the latest before you work on things.

- Sync back to p4 is definitely not implemented right now, I'll leave it to
Arwed to call how that needs to work out.

Commands
========
Get a copy of the grimoire:
$ git clone http://scm.sourcemage.org/smgl/grimoire.git [<dest dir>]
or
$ git clone ssh://scm.sourcemage.org/smgl/grimoire.git [<dest dir>]

- This gets you a full copy of the upstream grimoire repository and all of
its branches. As noted above, this is currently around 130MB total.
- The first is unauthenticated, the second will go over ssh. I'm going to
put the http one up for now and invite anyone to check it out, but I need
to reserve the right to shut it down if we get a lot of non-devs hitting
it excessively to pull stuff. We don't have infinite bandwidth on this.
- Also note it's named grimoire.git for now because we will also have a
grimoire.svn and .svk under that smgl dir, once we pick something we'll
just have a "grimoire" path.

$ cd grimoire

See what you got:
$ git branch
devel
* master
origin
stable-0.3
stable-rc-0.4

- Without an argument git-branch lists the defined branches.
- The '*' shows that 'master' is the active checkout.
- 'master' is the default name of the local repository's trunk; this is our
'test' (I think we can rename it 'test' in git if we really want to, but
I don't see the point).
- 'origin' is a special branch on cloned repositories that is the pristine
state of 'master' on upstream. You should treat this branch as
read-only.

Tell git who you are:
$ git repo-config user.name "Foo Bar"
$ git repo-config user.email "foobar AT sourcemage.org"

- The name defaults to your local GECOS entry or $GIT_AUTHOR_NAME.
- The email defaults to the local login@hostname or $GIT_AUTHOR_EMAIL.
- You can also edit .git/config directly.

Switch the working copy to stable:
$ time git checkout stable-0.3
real 0m8.790s
user 0m4.670s
sys 0m2.950s
And back:
$ time git checkout master
real 0m23.256s
user 0m4.610s
sys 0m2.550s

- The way we ended up building this, the branches are local branches of one
repository instead of their own branched repositories. This is necessary
to allow cherry-picking; that works between local branches but not remote
branches. We could have used separate repos and required temporary local
branches for any cherry-pick, but that would have probably gotten
annoying. See notes above about being open-minded. :-)

Decide you want a permanent checkout of devel and test for local system
grimoires:
$ git clone --shared . /var/lib/sorcery/codex/devel.git
$ pushd /var/lib/sorcery/codex/devel.git
$ git checkout devel
$ popd
$ git clone --shared . /var/lib/sorcery/codex/test.git
$ pushd /var/lib/sorcery/codex/test.git
$ git checkout master
$ popd

- These clones are 'shared', which will make them just point to the primary
local copy for their repository meta-information. In other words, on
disk they're just the spell files for that branch of the grimoire and a
pointer to the primary local copy's git data.

Keep the above updated periodically:
$ pushd /var/lib/sorcery/codex/devel.git; git pull; popd
$ pushd /var/lib/sorcery/codex/test.git; git pull; popd

- This updates those clones and their working copies to the latest state of
the local repository they were cloned from.

Now let's do some real work. A new version of spell foo is out, and we
need to put it in test:
$ git pull
$ cd section/foo
$ <edit/test/edit/test>
$ git status (optional)
$ git diff (optional)
$ git add . (if there were new files)
$ git commit .
$ git push ssh://scm.sourcemage.org/smgl/grimoire.git master:master

- This is a basic example that should make sense to most of you as-is.
More on some of this is below.

apache2 needs updating in stable, stable-rc has the version we want:
$ gaze versions apache2
Grimoire Section Spell Grimoire Version Installed Version
-------- ------- ----- ---------------- -----------------
stable http apache2 2.0.54 -
stable-rc http apache2 2.0.55 -

Make sure we're in stable-rc:
$ git checkout stable-rc-0.4

Find the commit that updated apache2:
$ git log http/apache2
commit ec9304c6ecc767d7dc646c0fb5e674fae94a7479 <- this is what we want
Author: Jeremy Blosser <jblosser AT fawkes.sourcemage.org>
Date: Mon Apr 24 10:31:31 2006 -0500

sync ./http/apache2 from stable-rc 0.4

commit 84d8b510ac289af0a43bfa9e110844af27a90196
Author: root <root AT fawkes.sourcemage.org>
Date: Mon Apr 24 09:03:08 2006 -0500

initial commit from stable 0.3

- git commits are referenced by the sha1 of the compressed repository data
after that commit.

Optionally view the full data on that commit:
$ git show ec9304c6ecc767d7dc646c0fb5e674fae94a7479
[ snip commit log and diff output ]

Apply that commit only:
$ git cherry-pick ec9304c6ecc767d7dc646c0fb5e674fae94a7479
First trying simple merge strategy to cherry-pick.
Simple cherry-pick fails; trying Automatic cherry-pick.
Removing http/apache2/04_all_ssl_makefile.patch
Removing http/apache2/05_httpd-2.0.54_openssl-0.9.8.patch
Removing http/apache2/06_CAN-2005-2700.patch
Removing http/apache2/byterange.patch
Finished one cherry-pick.

- When the simple merge fails, it moves to trying /usr/bin/merge. This
normally come from RCS. You probably want it.
- If that second merge failed, you'd have standard conflict markers to
clean up and re-commit.

View results of what you just did:
$ git log http/apache2
commit 347bca6cd9122e5003fc8d6d0409798092d335d0
Author: Jeremy Blosser <jblosser AT fawkes.sourcemage.org>
Date: Mon Apr 24 10:31:31 2006 -0500

sync ./http/apache2 from stable-rc 0.4
(cherry picked from ec9304c6ecc767d7dc646c0fb5e674fae94a7479 commit)

- By default the original commit author, date, and message are retained, as
is a log pointer to the original commit. The actual committer is also
logged separately from the original author. You can opt to edit this
message or not have cherry-pick do an automatic commit at all; man
git-cherry-pick.

That just updated our local copy of the repository, push this back
upstream:
$ git push ssh://scm.sourcemage.org/smgl/grimoire.git
347bca6cd9122e5003fc8d6d0409798092d335d0:stable-0.3

- You need to understand this one: This will connect to the remote
repository listed and update its stable-0.3 branch to the same state as
the local branch as of the specified sha1 spec. This is a point-in-time
full fast-forward, not a cherry-pick; if you did other committed updates
locally before the specified one that you hadn't pushed, you're pushing
them now. If you did any after the one you pushed, you aren't pushing
those. If you have locally committed work you don't want to push, just
create a local branch off of the 'origin' local branch, cherry-pick just
the change you want to push, and then push from that branch. Note, that
will work for local changes to "test", since that's what origin points to
on the remote repo; for other branches you may want to keep eg an
'origin-stable' branch of the stable branch. Or depending on what you do
in the project you may find it makes sense to keep a 'to-push' branch
around or something for things like this. There are tons of options and
I'm not going to try to suggest what will work best for you.

To revert something, use the same syntax as cherry-pick:
$ git revert 347bca6cd9122e5003fc8d6d0409798092d335d0
First trying simple merge strategy to revert.
Finished one revert.

- 'revert' and 'cherry-pick' are the same command invoked different ways.

Attachment: pgpLSbo1bWCdD.pgp
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page