Mixed git and svn working copy

Since I am working on project where the code is:

  • currently in an SVN repository
  • will be available on GitHub to the community
  • will stay in SVN for use in internal projects

As I was trying to figure out a way to have this work with people making changes on the public version on GitHub, others making changes in the private version in SVN and in most cases propagating all changes in both directions. One idea is to have some working copies which are SVN only, some which are Git only and some which are both a Git and an SVN Working Copy and can be used to propagate changes.

Setup

The structure would looks like this:

playground git svn mix

So first I created the directories:

cd ~
mkdir playground
cd ~/playground
rm -rf ~/playground/svn ~/playground/git ~/playground/mix
mkdir ~/playground/svn ~/playground/git ~/playground/mix

Then the working copies:

git clone https://github.com/benohead/my_product_name.git ~/playground/git/my_product_name

svn co --username my_user --password my_password --non-interactive http://my_private_server/svn/my_product_name ~/playground/svn/my_product_name

git clone https://github.com/benohead/my_product_name.git ~/playground/mix/my_product_name
svn co --username my_user --password my_password --non-interactive http://my_private_server/svn/my_product_name ~/playground/mix/my_product_name

It can be done in almost any order but for the “mix” directory, you need to first create the git clone and then checkout from svn as git will complain if the directory is not empty.

Since the files were already there when we checked out, you’ll probably have many conflicts. It’s the reason I used the –non-interactive option. Like this the checkout happens without having to handle the conflicts right now. But later on, you’ll need to handle them in order to be able to check in. So we have two solutions either keep the GitHub files or use the SVN files (i.e. revert to the repository version).

Keeping the GitHub files

If you want to keep the GitHub files i.e. have the GitHub version of the files in both the GitHub and the SVN repositories, the first idea is probably to resolve the conflict saying we want to keep the working copy e.g.:

cd ~/playground/mix/my_product_name
svn resolve --accept working -R *

Unfortunately after you do this, all files will be marked for deletion. This is because the files we have in there were there before the first checkout so they are considered unversioned files. So since there are no locally modified versioned files, svn will consider that when you say that you want to keep the local copy, you actually mean you want to delete the files. So after running the previous command, if you then commit, you’ll just delete all files.

So what do we do ? Can we only keep the svn version and not the GitHub version ? Of course not. The solution is actually pretty easy.

First, revert all files in the svn working copy.

svn revert -R *

After that you have the version from SVN (not what you want) but the conflicts are all gone. Then you reset in git to get back to the versions from GitHub:

git reset --hard

Now you have no conflicts and have the version from GitHub, you only need to check it in (in SVN):

svn ci -m "Sync with GitHub"

Now both repositories are in sync and the version we kept is the one from GitHub.

Keeping the SVN files

Basically, it starts the same way:

cd ~/playground/mix/my_product_name
svn revert -R *

Now, you have the version from SVN. Now if you do a diff in GitHub, you’ll see the delta between svn and GitHub:

git diff

So all you have to do is to commit the changes and push them to GitHub:

git commit -a -m "Sync with SVN"
git push origin

After that you also have a conflict-free state with identical versions of the files in both repository. The only difference is that now GitHub has the same files as SVN (instead of the opposite in the previous case).

Propagating changes

Then I tested the following:

  1. Make a change in a Git working copy
  2. Commit and push to GitHub
  3. Pull it from GitHub into the mixed working copy
  4. Commit to the private SVN repo from the mixed working copy
  5. Update from the SVN repo to a pure svn working copy

The first step… Well, use vim, nano, notepad or anything else (guess you know how to edit a file).

Then commit and push to GitHub:

cd ~/playground/git/my_product_name
git commit -a -m "test"
git push origin

Then pull it from GitHub into the mixed working copy:

cd ~/playground/mix/my_product_name/
git pull origin

Then commit to the private SVN repo from the mixed working copy:

svn ci -m "test"

And finally, update from the SVN repo to a pure svn working copy:

cd ~/playground/svn/my_product_name
svn up
diff  ~/playground/git/my_product_name/my_modified_file ~/playground/svn/my_product_name/my_modified_file

You will then see that the file in the Git clone and in the SVN working copy are identical.

GitHub Unicorn

GitHub was working perfectly fine today. But tonight, loading any page is just a pain. I keep getting either an HTTP 500 page or the GitHub unicorn. Actually I had never seen the unicorn before. I’m not exactly into pink unicorns but this one doesn’t look too girly even though it’s pink:

github unicorn

I suppose showing an angry unicorn is supposed to calm down people who are angry because they cannot do what they wanted to do and see that the unicorn is even angrier…

The HTTP 500 page was also nice but they seem to have less problems now and I didn’t manage to get it displayed anymore in order to take a screenshot. What a pity !

Anyway, I’m happy I can finish what I wanted to do on GitHub. Not sure whether they did anything to fix the problem or it just works because most people gave up for today. But if they did fix something, their reaction time is impressive !

Update: I just got what the unicorn is about. Here’s what GitHub says about unicorn:

Slow Actions

The Unicorn master process knows exactly how long each worker has been processing a request. If a worker takes longer than 30s (we lowered it from mongrel’s default of 60s) to respond, the master immediately kills the worker and forks a new one. The new worker is instantly able to serve a new request – no multi-second startup penalty.

When this happens the client is sent a 502 error page. You may have seen ours and wondered what it meant. Usually it means your request was killed before it completed.

What’s strange is that the unicorn was displayed pretty fast. Way too fast to be some kind of time out happening. I guess the unicorn knows how to show itself when the request will lead to a timeout (maybe because no workers are available or because others are currently getting a timeout for this resource or whatever reason…).

GitHub: Cloning your own project

Today I released the first version of MySQLBuddy. It is basically a fork of SybaseBuddy where I’ve removed a few things (for Sybase) and made a few changes for MySQL. So I wanted to fork SybaseBuddy in GitHub to a new project. Unfortunately it looks like GitHub doesn’t support this. It seems to assume that the owner of a project will not fork his own project.

Here is what I did to fork the project:

I first created a new GitHub repository: MySQLBuddy.
Then I created a clone of SybaseBuddy on my machine:

$ git clone https://github.com/benohead/SybaseBuddy.git MySQLBuddy
Cloning into 'MySQLBuddy'...
remote: Counting objects: 105, done.
remote: Compressing objects: 100% (78/78), done.
remote: Total 105 (delta 33), reused 74 (delta 7)
Receiving objects: 100% (105/105), 6.48 MiB | 1.38 MiB/s, done.
Resolving deltas: 100% (33/33), done.

Now I had a clone but needed to teach it that it’s not linked to the SybaseBuddy GitHub repository anymore but to the MySQLBuddy repository. For this you need to update the .git/config in your clone:

Under Linux:

$ sed -i 's/SybaseBuddy/MySQLBuddy/g' .git/config

Under Mac OS X:

$ sed -i .tmp 's/SybaseBuddy/MySQLBuddy/g' .git/config

Or simply edit the file and update the URL to the repository.

Now you need to push this clone to the new repository:

$ git push -u origin master

You might get the following error message:

$ git push -u origin master
To https://github.com/benohead/MySQLBuddy.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'https://github.com/benohead/MySQLBuddy.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes (e.g. 'git pull') before pushing again.  See the
'Note about fast-forwards' section of 'git push --help' for details.

What you need to do is first pull from the repository before pushing:

$ git pull

You might also get there the following error message:

$ git pull
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From https://github.com/benohead/MySQLBuddy
 + c1c2949...820a952 master     -> origin/master  (forced update)
Auto-merging README.md
CONFLICT (add/add): Merge conflict in README.md
Automatic merge failed; fix conflicts and then commit the result.

So you’ll need to handle the merge manually. Just update the file (here README.md) and pull again.

For some reason, I did the following before modifying the file:

$ git pull
U	README.md
Pull is not possible because you have unmerged files.
Please, fix them up in the work tree, and then use 'git add/rm <file>'
as appropriate to mark resolution, or use 'git commit -a'.
$ git commit -a
[master 00dd552] Merge branch 'master' of https://github.com/benohead/MySQLBuddy

But you don’t need to do it like this and can directly modify the file and pull.

After that you’ll be able to push:

$ git push -u origin master
To https://github.com/benohead/MySQLBuddy.git
   820a952..00dd552  master -> master
Branch master set up to track remote branch master from origin.

And you’re done !