Lightweight shelving of your work-in-progress, with Mercurial

Before we switched to Mercurial in January 2011, we were a TFS shop. I hated TFS with some level of passion after using it for 4 years. It was slow, cranky and, well… merging was frequently a nightmare or sometimes actually impossible.

But one neat thing that TFS did provide was the concept of “shelve-sets”. For those that don’t know: these are basically uncommitted (or unchecked-in) changes, effectively your work-in-progress.

This feature was great because it let me:

• develop on multiple PCs, and be able to easily migrate unfinished code changes between those PCs.
• an immediate requirement came in that perhaps evaporated before the changes could be finished – but I suspected the requirement might spark up again a week or month later.
• perform zero-risk code reviews without getting stuck in all the mud of the wider TFS system (this point clearly doesn’t apply to Mercurial nor any of the popular DVCS).

I’m not a big fan of the “Continuous Check-ins” movement that somebody seems to have started. It feels wrong to be committing stuff in unlogically grouped and unatomic units. And hell, the work-in-progress that you want to “shelve” may not even compile yet! Do I really want to be committing stuff that I might throw away later anyway? Maybe, if I was using Git which allows history rewriting far more easily. But Mercurial has a stricter policy around history and maintains a level of discipline to ensure an immutable history.

There is already an extension for Mercurial that performs a type of shelving. But this was not suitable for me. It maintains the storage folder locally on the PC and doesn’t provide any obviously supported means of copying that between PCs.

Mercurial provides (almost) everything you need out-of-the-box to be able to do a lightweight form of shelving, and you don’t even need to enable any extensions. What you need are the hg diff and hg import commands. Coupled with two shell scripts, and you will have a complete solution.

First setup your Mercurial.ini (or .hgrc) with the following useful aliases:

[alias]
gitdiff = diff --git
exportwip = gitdiff
importwip = import --no-commit --force

The reason we use the extended diff format from Git is that, primarily, this supports binary files whereas a vanilla diff does not. And the reason for the --no-commit option on the importwip alias is to prevent the imported work-in-progress from being immediately committed to the repository. You definitely don’t want that!

Now from a shell prompt you can do:

$ hg exportwip > my-work-in-progress-for-xyz-customer-22-02-2011.diff

Then, at a later date, or immediately from another PC (once you’ve copied the diff file across or e-mailed it to yourself):

$ hg importwip my-work-in-progress-for-xyz-customer-22-02-2011.diff

Note: There is no need for piping the file input on the importwip alias because the real command already requires that as an input argument.

Not-so powerful, uh, PowerShell…

If you’re a PowerShell user (like myself) there is a rather large annoyance in the way it performs binary file output when piping. If you use the straight “>” output piping operator in PowerShell then it will default to use Unicode encoding. This sounds fine on the surface, except that Mercurial’s diff engine does not support Unicode. It outputs in UTF8 and expects any input it is given to also be in UTF8. And when I say it does not support Unicode – I mean it really doesn’t. It will give a vague error of:

abort: no diffs found

Additionally, if you try to workaround this by changing the encoding used by PowerShell, as follows:

PS $ hg exportwip | out-file -filepath "my-work-in-progress-for-xyz-customer-22-02-2011.diff" -encoding OEM -force

Then what you get is a file that looks suitable. And in fact it should be suitable. The only difference appears to be that the line endings have been normalised from \n (Mercurial’s raw output when producing diffs) to \r\n (Windows environment style). Unfortunately there is a bug, as of at least Mercurial 1.7.5, where it cannot handle diffs that use the \r\n style of line endings (despite having the eol extension enabled). What you will get is an ugly exception like the following:

** unknown exception encountered, please report by visiting
**  http://mercurial.selenic.com/wiki/BugTracker
** Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
** Mercurial Distributed SCM (version 1.7.5)
** Extensions loaded: fixfrozenexts, graphlog, eol, fetch, transplant, rebase, purge, churn, mq
Traceback (most recent call last):
  File "hg", line 36, in
  File "mercurial\dispatch.pyo", line 16, in run
  File "mercurial\dispatch.pyo", line 36, in dispatch
  File "mercurial\dispatch.pyo", line 58, in _runcatch
  File "mercurial\dispatch.pyo", line 590, in _dispatch
  File "mercurial\dispatch.pyo", line 401, in runcommand
  File "mercurial\dispatch.pyo", line 641, in _runcommand
  File "mercurial\dispatch.pyo", line 595, in checkargs
  File "mercurial\dispatch.pyo", line 588, in
  File "mercurial\util.pyo", line 426, in check
  File "mercurial\dispatch.pyo", line 297, in __call__
  File "mercurial\util.pyo", line 426, in check
  File "mercurial\extensions.pyo", line 130, in wrap
  File "mercurial\util.pyo", line 426, in check
  File "hgext\mq.pyo", line 2988, in mqcommand
  File "mercurial\util.pyo", line 426, in check
  File "mercurial\extensions.pyo", line 130, in wrap
  File "mercurial\util.pyo", line 426, in check
  File "hgext\mq.pyo", line 2960, in mqimport
  File "mercurial\util.pyo", line 426, in check
  File "mercurial\commands.pyo", line 2371, in import_
  File "mercurial\commands.pyo", line 2328, in tryone
  File "mercurial\patch.pyo", line 1259, in patch
  File "mercurial\patch.pyo", line 1231, in internalpatch
  File "mercurial\patch.pyo", line 1110, in applydiff
  File "mercurial\patch.pyo", line 1129, in _applydiff
  File "mercurial\patch.pyo", line 1028, in iterhunks
KeyError: 'mycompany.snk\r'

The guys in the Mercurial chat room are aware of this issue as we were all scratching our collective heads on it for an hour or two the other day. Here’s hoping for a bug fix soon! Alternatively, the whole issue could be bypassed if the hg diff command had an option such as “--out <filename>” to write the diff to a file directly. That would remove the dependency upon the shell for file output piping.