On unfortunate side effect of how PostGIS is implemented is that you end up with absolute filesystem path references in your databases. This makes moving a database between machines a pain.

I often run into the need to move databases between machines as I prefer to mirror the production stack as closely as reasonably possible on my local machine. Making a copy the database on the remote cluster and tunneling is an option but can be slow if you aren’t on a nice pipe.

My solution is essentially a selective restore of the dump file. If we only restore the items that have been added to the database beyond the initial PostGIS items then we can load the dump into another fresh postgis database.

pg_restore has two flags that will be useful:

• -l: List the contents of the archive. The output of this operation can be used with the -L option to restrict and reorder the items that are restored.
• -L list-file: Restore elements in list-file only, and in the order they appear in the file.

How handy.

Note: These options are only available if you have a Postgres formatted dump, you can create one of these with the -Fc option to pg_dump. Also be aware that crossing Postgres versions with these can sometimes break.

Here’s a little script I hacked together that will give you the list of items in one Postgres dump but not the other. Given the template_postgis dump file and the dump of your populated database you can generate the list of items that have been added to the populated database beyond the initial PostGIS items. Once you have this then loading the dump should proceed smoothly on any fresh PostGIS database. Awesome.

See help text inline for more info:

hg-git is a fantastic project by GitHub’s Scott Chacon that allows bidirectional communication between Mercurial and Git.

Most of the documentation is angled towards hg users contributing to git projects but being more in the git camp I have the reverse use for it.

Here are the basic steps I follow to get set up to work with git on an hg project.

First, we need a mercurial checkout. I’m going to use pyglet – an awesome little python windowing and graphics library. Some little play code I have uses it and the cocoa backend needs some love (diving head first into PyObjC has been.. interesting).

I’ll assume you have hg-git installed and enabled in your ~/.hgrc
$ hg clone https://pyglet.googlecode.com/hg/ pyglet

The basic operation to generate a git repository and convert hg commits to git commits is gexport. This will create refs in the git repo for each hg bookmark that exists. Hg bookmarks are essentially equivalent to git branches — named pointers to commits that move when new child commits are created. See http://mercurial.selenic.com/wiki/BookmarksExtension for more info.

Before we create the git repo let’s create bookmarks for the hg branches we would like to interact with in git (this can be done after the initial gexport as well).
$ hg bookmark hg/default -r default
$ hg bookmark hg/cocoa-port -r cocoa-port

I’m prefixing the refs with hg so we have slight namespace separation between our git branches and the refs that are updated with gexport

Ok. Let’s go ahead and create the git repo:
$ hg gexport

This can take some time with a large repo (and pyglet’s repo isn’t tiny) but almost all of the cost is one-time. This takes a little under 10 minutes on my machine.

By default gexport will create the git repo at .hg/git. I prefer my .git and .hg directories to live side-by-side. Just simply symlink the .git repo into the right place:

$ ln -s .hg/git .git

(You can make this the default behavior, see [1] below.)

From here let’s create a branch corresponding to the “hg” branch:
$ git branch cocoa-port hg/cocoa-port

And make our master the same as hg/default:
$ git reset hg/default

From here you can make commits on your git branches and pull them into your hg repo with:
$ hg gimport

Subsequent pulls/fetches and gexport calls will push new commits to the hg/ refs in git.

There you go! A push from there would get your git commits to your remote hg repo. You’re set up to work with git and publish to hg!

One annoyance you’ll likely notice is the issue of the .hg and .git directories showing up in git status and hg status repectively.

To hide .hg from git simply:
$ echo ".hg" >> .git/info/exclude

Unfortunately Mercurial doesn’t have quite as flexible of a local ignore file but we can get part of the way there with:
$ echo "[ui]
ignore = `pwd`/.hg/hgignore" >> .hg/hgrc
and then:
$ echo ".git" >> .hg/hgignore
The absolute path is required here for the ignore to work throughout the repository, unfortunately.

Here’s that again with less jabber:

$ hg clone https://pyglet.googlecode.com/hg/ pyglet
$ hg bookmark hg/default -r default
$ hg bookmark hg/cocoa-port -r cocoa-port
$ hg gexport
(wait)
$ ln -s .hg/git .git
$ git branch cocoa-port hg/cocoa-port
$ git reset hg/default
(hack, commit)
$ hg gimport
$ hg push

To set up ignores:
$ echo ".hg" >> .git/info/exclude
$ echo "[ui]
ignore = `pwd`/.hg/hgignore" >> .hg/hgrc
$ echo ".git" >> .hg/hgignore

Addendum

1. You can change hg-git’s default behavior by adding this section to your ~/.hgrc:
   [git]
intree=1


update 09/04/2009: Leaving shortly for Seattle and then to Portland for Djangocon, woo!

DjangoSnippet 1002

I added optional message type passing so you can display them differently.