[null, {"document_url": "https://longair.net/blog/page/2/", "unformatted_src": "https://farm8.staticflickr.com/7072/6995433154_749a4200a9_n.jpg", "src": "https://farm8.staticflickr.com/7072/6995433154_749a4200a9_n.jpg", "formatted_filename": "a a n", "rendered_width": 320, "rendered_height": 213}, null, {"document_url": "https://longair.net/blog/page/2/", "unformatted_src": "https://farm7.static.flickr.com/6191/6084458443_c9dd82957b.jpg", "src": "https://farm7.static.flickr.com/6191/6084458443_c9dd82957b.jpg", "formatted_filename": "c dd b", "rendered_width": 299, "rendered_height": 500}, null, {"document_url": "https://longair.net/blog/page/2/", "unformatted_src": "https://farm7.static.flickr.com/6201/6049508689_f072da923f_m.jpg", "src": "https://farm7.static.flickr.com/6201/6049508689_f072da923f_m.jpg", "formatted_filename": "f da f m", "rendered_width": 240, "rendered_height": 160}, null, {"document_url": "https://longair.net/blog/page/2/", "unformatted_src": "https://farm7.static.flickr.com/6142/5926773830_dfab915cc5_m.jpg", "src": "https://farm7.static.flickr.com/6142/5926773830_dfab915cc5_m.jpg", "formatted_filename": "dfab cc m", "rendered_width": 171, "rendered_height": 240}, null]
This post was originally intended for the mySociety blog, but because of its length and technical content people suggested to me it might be more suitable for my own blog. This about the development of the YourNextRepresentative project (GitHub link), and in particular why and how we migrated its storage system from using the PopIt web service to instead use PostgreSQL via django-popolo. Hopefully this might be of interest to those who have been following the development of YourNextRepresentative (previously YourNextMP).


YourNextMP for 2015

For the 2015 General Election in the UK, we developed a new codebase to crowd-source the candidates who were standing in every constituency; this site was YourNextMP. (Edmund von der Burg kindly let us use the yournextmp.com domain name, which he’d used for his site with the same intent for the 2010 election.) We were developing this software to help out Democracy Club, who ran the site and built up the community of enthusiastic contributors that made YourNextMP so successful.

At the time, we saw this crowd-sourcing challenge as a natural fit for another technology we had been developing called PopIt, which was a data store and HTTP-based API for information about people and the organizations they hold positions in. PopIt used the Popolo data model, which is a very carefully thought-out specification for storing open government data. The Popolo standard helps you avoid common mistakes in modelling data about people and organisations, and PopIt provided interfaces for helping people to create structured data that conformed to that data model, and also made it easily available.

The original intention was that YourNextMP would be quite a thin front-end for PopIt but, as development progressed, but it became clear that this would provide a very poor user experience for editors. So, the YourNextMP wrapper, even in the very early versions, had to provide a lot of features that weren’t available in PopIt, such as:

Later on in development we also added more advanced features such as a photo moderation queue. To help support all of this, the YourNextMP front-end used a traditional RDBMS (in our case usually PostgreSQL), but effectively PopIt was the primary data store, which had all the information about people and the posts they were standing for.

This system worked fine for the general election, and I think everyone considered the YourNextMP project to be a great success  – we had over a million unique users, and the data was extensively reused, including by Google for their special election information widget. (You can see some more about the project in this presentation.)

Turning YourNextMP into YourNextRepresentative

There had been a considerable demand to reuse the code from YourNextMP for other elections internationally, so our development efforts then focussed on making the code reusable for other elections. The key parts of this were:

We worked on this with the target of supporting a similar candidate crowd-sourcing effort for the general election in Argentina in 2015, which was being run by Congreso Interactivo, Open Knowledge Argentina and the Fundación Legislativo. This new version of the code was deployed as part of their Yo Quiero Saber site.

Since the name “YourNextMP” implies that the project is specific to electoral systems with  Members of Parliament, we also changed the name to YourNextRepresentative. This name change wasn’t just about international re-use of the code – it’s going to be supporting the 2016 elections in the UK as well, where the candidates won’t be aspiring to be MPs, but rather MSPs, MLAs, AMs, mayors, PCCs and local councillors.

Problems with PopIt

It had become apparent to us throughout this development history that PopIt was creating more problems than it was solving for us. In particular:

At the same time, we were coming to the more broad conclusion that the PopIt project hadn’t been achieving the goals that we’d hoped it would, despite having putting a lot of development time into it, and we decided to stop all development on PopIt. (For many applications of PopIt we now felt the user need was better served by the new EveryPolitician project, but in YourNextRepresentative’s case that didn’t apply, since EveryPolitician only tracks politicians after they’re elected, not when they’re candidates.)

As developers we wanted to be able to use a traditional RDBMS again (through the Django ORM) while still getting the benefits of the carefully thought-out Popolo data model. And, happily, there was a project that would help use to do exactly that – the django-popolo package developed by openpolis.

Migrating YourNextRepresentative to django-popolo

django-popolo provides Django models which correspond to the Popolo Person, Organisation, Membership, Post and Area classes (and their associated models like names and contact details).

We had used django-popolo previously for SayIt, and have been maintaining a fork of the project which is very close to the upstream codebase, except for removing the requirement that one uses django-autoslug for managing the id field of its models. We opted to use the mySociety fork for related reasons:

Extending django-popolo’s models

Perhaps the biggest technical decision about how to use the django-popolo models (the main ones being Person, Organization, Post and Membership) is how we extended those models to add the various extra fields we needed for YourNextRepresentative’s use case. (With PopIt you could simply post a JSON object with extra attributes.) The kinds of extra-Popolo data we recorded were:

Perhaps the most natural way of adding this data would be through multi-table inheritance; indeed, that is how SayIt uses django-popolo. However, we were wary of this approach because of the warnings in Two Scoops of Django and elsewhere that using multi-table inheritance can land you with difficult performance problems because queries on the parent model will use OUTER JOINs whether you need them or not. We decided instead to follow the Two Scoops of Django suggestion and make the one-to-one relationship between parent and child table explicit by creating new models called PersonExtra, PostExtra, etc. with a `base` attribute which is a OneToOneField to Person, Post, etc., respectively. This means that the code that uses these models is slightly less clear than it would be otherwise (since sometimes you use person, sometimes person.extra) but we do have control over when joins between these tables are done by the ORM.

Data migration

Once the Extra models were created, we wrote the main data migration. The idea of this was that if your installation of YourNextRepresentative was configured as one of the known existing installations at the time (i.e. the ELECTION_APP setting specified the St Paul, Burkina Faso, Argentina or the UK site) this data migration would download a JSON export of the data from the corresponding PopIt instance and load it into the django-popolo models and the *Extra models that extend them.

As the basis for this migration, we contributed a PopIt importer class and management command upstream to django-popolo. This should make it easier for any project that used to use PopIt to migrate to django-popolo, if it makes sense for them to do so. Then the data migration in YourNextRepresentative subclassed the django-popolo PopItImporter class, extending it to also handle the extra-Popolo data we needed to import.

(A perhaps surprising ramification of this approach is that once an installation has been migrated to django-popolo we should change the name of that country’s ELECTION_APP, or otherwise someone setting up a new site with that ELECTION_APP configured will have to wait for a long time for out-of-date data to be imported on running the initial migrations. So we will shortly be renaming the “uk_general_election_2015” application to just “uk”. To support people who want that feature (cloning an existing site to a development instance) we’ve added a new “candidates_import_from_live_site” management command that uses its new API to mirror the current version of an existing instance.)

Another issue that came up in working on this data migration is that we needed to preserve any identifiers that were used in URLs on the site previously so that after upgrading each site every bookmarked or search-engine-indexed URL would still show the same content. In the case of the Person model, this was easy because it used an integer ID previously. Parties and posts, however, used strings as their IDs. We migrated these IDs to fields called ‘slug’ (perhaps a slightly misleading name) on the OrganisationExtra and PostExtra models.

This turns out to be quite a slow migration to run – as well as importing the core data, it also downloads every image of people and parties, which is pretty slow even on a fast connection.

Updating views, tests and management commands

The next part of the migration was updating all the code that previously used PopIt’s API to instead use the new Django models. This was a significant amount of work, which left very little code in the project unchanged by the end. In general we tried to update a test at a time and then change the core code such that the test passed, but we knew there was quite a bit of code that wasn’t exercised by the tests. (One nice side-effect of this work is that we greatly improved the coverage of the test suite.)

We did think about whether we could avoid doing this update of the code essentially in one go – it felt rather like a “stop-the-world refactoring”; was there an incremental approach that would have worked better? Maybe so, but we didn’t come up with one that might reasonably have saved time. If the old code that interacted with the PopIt API had been better encapsulated, perhaps it would have made sense to use proxy models which in the migration period updated both PopIt’s API and the database, but this seemed like it would be at least as much work, and it was lucky that we did have a period of time we could set aside for this work during which the sites weren’t being actively updated.

Moving other code and configuration to the database

We also took the opportunity of this migration to introduce some new Django models for data that had previously been defined in code or as Django settings. In particular:

These all had the effect of simplifying the setup of a new site – more of the initial setup can now be done in the Django admin interface, rather than needing to be done by someone who’s happy to edit source code.

Replacing PopIt’s API

One of the nice aspects of using PopIt as the data store for the site was that it supplied a RESTful API for the core data of the site, so we previously hadn’t had to put much thought into the API other than adding some basic documentation of what was there already. However, after the migration away from PopIt we still wanted to provide some HTTP-based API for users of the site. We chose to use Django REST framework to provide this; it seems to be the most widely recommended tool at the moment for providing a RESTful API for a Django site (and lots of people and talks at djangocon EU in Cardiff had independently recommended it too). Their recommendations certainly weren’t misplaced – it was remarkably quick to add the basic read-only API to YourNextRepresentative. We’re missing the more sophisticated search API and various other features that the old PopIt API provided, but there’s already enough information available via the API to completely mirror an existing site, and Django REST framework provides a nice framework for extending the API in response to developers’ needs.

The end result

As is probably apparent from the above, this migration was a lot of work, but we’re seeing the benefits every day:

I think it’s also worth adding a note that if you’re starting a new site that deals with data about people and their memberships of organizations, then using the Popolo standard (and if you’re a Django developer, django-popolo in particular) can save you time – it’s easy to make mistakes in data modelling in this domain (e.g. even just related to names). It should also help with interoperability with other projects that use the same standard (although it’s a bit more complicated than that – this post is long enough already, though :))

The UK instance of YourNextRepresentative (at edit.yournextmp.com) has been using the new codebase for some time now, and that will be relaunched shortly to collect data on the 2016 elections in the UK.

If you’re interested in starting up an candidate crowd-sourcing site using YourNextRepresentative, please get in touch with our international team or email email@example.com for any other issues.

On one of the projects I’ve been working on at mySociety had a large number of open issues, most of which hadn’t been looked at for some time, and didn’t have estimates of difficulty. To address this we tried a variation of an exercise that’s suggested in “The Scrum Field Guide” in the chapter called “Prioritizing and Estimating Large Backlogs”, which involves printing out all the stories in the product backlog onto index cards and arranging them on a wall roughly in order of difficulty from left to right. (A second stage is to let the product owner adjust the height of each story on the wall to reflect its priority.) It seemed as if this might be a helpful way of estimating our large number of open issues, even though they’re not written as users stories.

An obvious problem here was that we didn’t have an easy way of GitHub tickets en masse, so I wrote a script that would generate a PDF for each open issue (excluding those that are pull requests) in a repository’s issue tracker using Pandoc. That script is here in case it’s of use to anyone:

As it turned out, this session was pretty successful – as well as the value of generating lots of difficulty estimates, it was a good way of getting a picture of the history of the project, and starting lots of discussions of issues that only one member of the team knew about.

This is a cryptic crossword that Jenny and I wrote for my sister for her birthday, and I thought it might be worth putting up here (with her permission) in case anyone wants to give it a try. Many of the answers were chosen because the word might have some particular meaning for her, but I think they should all be generally accessible words. 7a relies on a reference which may be unfamiliar to some, however. The intention was to make the clues pretty easy, since she’s a relatively new solver, and the grid we ended up with to fit in all the themed words is quite a tough one.

Here’s a link to a PDF of the crossword.

This is a rough list of the books I read in 2012 with some brief comments – I’ve seen other people do this on their blogs and enjoyed reading their summaries, so thought that I would have a go. (Originally I added a mention of each person who recommended one of these to me, but that turned out to be problematic, both in privacy and completeness terms – let me just say that I’m always grateful for the wonderful recommendations I get from friends and colleagues.)

Fear and Loathing in Las Vegas – Hunter S. Thompson

I’ve read it at least twice before – this time I didn’t feel the momentum of the writing bowling me along quite as it did when I first read it, but that’s probably to be expected.

Bury the Chains – Adam Hochschild

An excellent account of the movement in Britain to abolish slavery. It’s still very relevant today if you’re interested in activism and campaigning, and, as the author points out, it’s the story of the first mass movement where a group campaigned for the rights of people other than themselves.

Mort – Terry Pratchett

This is the first time I’d re-read it in many years, although I usually suggest it (as do many people) as the first Terry Pratchett novel to try if you’ve never read any of his Discworld novels before.  I was glad to find still every bit as enjoyable as the first time. (At the end of the previous year I’d finished the Discworld series, after not reading any of them roughly between the ages of 18 and 32.)

Mockingjay – Suzanne Collins

I’d listened to the first two of the Hunger Games novels as audiobooks when I still had Emusic‘s “one audiobook a month” deal. It’s worth finishing the trilogy if you’ve read the first two, which I also enjoyed despite finding the character of Katniss terribly frustrating (as is probably intended).  The nature of the resistance movement is interesting, and I found the conclusion satisfying.

Knots and Crosses – Ian Rankin

Hide And Seek – Ian Rankin

Tooth and Nail – Ian Rankin

Strip Jack – Ian Rankin

The Black Book – Ian Rankin

Mortal Causes – Ian Rankin

Let It Bleed – Ian Rankin

Black and Blue – Ian Rankin

The Hanging Garden – Ian Rankin

Dead Souls – Ian Rankin

Set in Darkness – Ian Rankin

The Falls – Ian Rankin

These are the first 12 of Ian Rankin’s Rebus series of detective novels, and I’ll probably read the last few of them in 2013. I found them quick to read, and I loved the portrait of Edinburgh – it’s much more true to my experiences of the city both as a child and an adult than virtually any other books I’ve read that are set there.  I found the character of Rebus compelling as well.  The first two novels (Knots and Crosses and Hide and Seek) have some points where the writing really jarred for me, but from the third onwards they’re consistently well-written.

The Player of Games – Iain M. Banks

I seem to re-read this about once a year – it’s one of my favourite of Iain M. Banks’ novels (along with Consider Phlebas and Excession). I’ve actually stopped reading his new books, the last straw being The Steep Approach to Garbadale, where I spent most of the book thinking “oh, please don’t let the twist be the one that I think is coming, [a disturbing theme from several previous novels]”, and of course it was…

The Open-Focus Brain – Les Fehmi

This was a kind present from a student in Zürich who was concerned about my style of concentration. The idea of the book is that one can switch from a very intense “fight or flight” style of concentration to something the author describes as “open focus” by using various techniques, in particular thinking about volumes of space. It comes with a CD with some example exercises. When I’ve tried this it’s certainly been relaxing – I hadn’t tried any form of meditation before. It’s also interesting that thinking about the space around you is also something that’s used in the Alexander technique and (I’m told) various meditation techniques. Unfortunately, the book is written in a self-help style that makes me rather suspicious of it, particularly the wide-ranging claims for its health benefits sometimes based on single cases or unpublished data.

Moonwalking with Einstein – Joshua Foer

A book about memory champions and memory techniques of the “a journalist tries to become really good at something relatively obscure” genre (see also “Word Freak”, “Born To Run”, etc.) It’s a quick read, and if you’re interested in the subject of memory, or building expertise Anders Ericsson-style, worth a look.  In a strange coincidence, 6 months after I read this, my partner started work at a company run by one of the memory champions who appears in the book.

Why Have Kids? – Jessica Valenti

This probably isn’t what it sounds like from the title – it’s an excellent discussion of the absurd pressures on mothers (particularly in US culture) to meet impossible standards. It was written by Jessica Valenti after having her first child.

The Revolution will be Digitised – Heather Brooke

An interesting account of the Wikileaks saga from Heather Brooke, who is a fantastic campaigner on freedom of information issues. The most extraordinary sections deal with Julian Assange’s interactions with her, including his advances wherein he identifies himself with Jesus…

Born to Run – Christopher McDougall

An entertaining book about ultra-runners, barefoot-style running and the Tarahumara people, who run huge distances without apparently being susceptible to the injuries that plague runners elsewhere. It’s great in terms of adventure and storytelling, with lovely portraits of the runners the author meets. I wouldn’t read it if you’re looking for rigorous science and anthropology, but it’s excellent fun.

Rocket Surgery Made Easy – Steve Krug

I generally haven’t added work-related books to this list, but since I read this one straight through, it seemed as if it fitted better than then others which I tended to dip into and out-of more. This is by Steve Krug as a follow-up to “Don’t Make Me Think”, and aims to give you step-by-step guidance for running DIY usability testing sessions of websites.  I found the style somewhat irritating (as so often where technical books try for a light touch) but in practice it was very helpful for running some usability testing last year.  I’m hoping I’ll get to do that again next year.

The House of Silk – Anthony Horowitz

A new Sherlock Holmes novel by Anthony Horowitz. It’s a disturbing book, but brilliantly matches the style of the Holmes stories, I thought.

Plan of Attack – Bob Woodward

Bob Woodward’s book about the decisions that led to the invasion of Iraq in 2003 – I found this frankly terrifying.

The Sense of an Ending – Julian Barnes

The 2011 Booker prize winner – consummately well-written and an excellent read.

The Narrative of John Smith – Arthur Conan Doyle

The unpublished first novel of Arthur Conan Doyle. I found it tiresome, and not terribly interesting – I wouldn’t recommend it unless, I suppose, you’re a Conan Doyle completist.

Catherine of Aragon: Henry’s Spanish Queen – Giles Tremlett

A sympathetic biography of Catherine of Aragon – embarrassingly, I know virtually nothing of Tudor history, so this wasn’t just interesting, but almost entirely novel for me…

I’d forgotten how short this story is, in fact, but it’s still hugely enjoyable. It’s tense throughout, and I’d forgotten about the interesting character of the narrator, a quiet man who’s happy sitting in silence by the fire with his old friends.

Kidnapped – Robert Louis Stevenson

I don’t think that I’d ever read the original text of Kidnapped before, just a children’s edition and later a graphic novel version. Anyway, it’s still a great adventure and Balfour’s journey crosses lots of parts of Scotland that I know from past walking holidays.

The Ascent of Rum Doodle –  W. E. Bowman

An excellent comic novel from the 1950s about a group of mountaineers attempting to climb the fictional mountain “Rum Doodle” with the help of thousands of porters and a supply of Champagne for “medicinal purposes”. Very silly, and highly recommended.

Care of Wooden Floors – Will Wiles

This is the first novel by a friend I knew at college. I enjoyed this very much – I’ve been accused of giving away too much when discussing it before, so I’ll just say that it’s hilarious and agonising in equal parts, and definitely worth reading.

Wolf Hall – Hilary Mantel

Another Booker prize winner; my ignorance of history probably meant that I missed quite a lot of pleasure in associating the wonderfully drawn characters with their actual stories, but I enjoyed it nonetheless. The (much discussed) style of the writing, in particular the use of the pronoun “he”, initially irritated me a great deal, but I got used to it after a hundred pages or so. I really did think that Wolf Hall was good, but I don’t really see why so many people are quite so passionate about it.

The Afterparty – Leo Benedictus

Another novel written by someone that I used to know, and happily another great read. I found this story of a naive journalist caught up in events around a celebrity party completely gripping, and I enjoyed the meta-narrative too – it’s a smart and very well-written novel.

How To Be A Woman – Caitlin Moran

An often hilarious book – there are several sections that had both of us laughing out loud. The subject matter’s also great – I think it’s essentially aimed at younger people who would say that they’re in favour of equal rights for women, but bizarrely would also say that they’re not feminists.

The Blank Wall – Elisabeth Sanxay Holding

Elisabeth Sanxay Holding was an author that Raymond Chandler regarded as “the top suspense writer of them all”, but who is relatively unread. (You can get it from Persephone Books.) This is the story of a housewife during WWII who is trying to protect her family from events that are going out of control, a bit like Brief Encounter crossed with hardboiled detective fiction – I enjoyed it very much.

Quiet – Susan Cain

This is probably the most personally and professionally relevant book I read this year, as someone who’s introverted but works hard to not give that impression.  It’s about the ways in which society increasingly values the qualities we associate with extroversion rather than introversion, and why that might be a mistake. I found it encouraging and thought-provoking – I’d recommend it to anyone who’s quiet, introverted or sensitive (the book deals with several associated personality traits). There’s also a little material towards the end about how these traits can affect relationships which struck quite a few chords with me.

To add my usual disclaimer to the start of these blog posts, I should say that I love git; I think it’s a beautiful and elegant system, and it saves me huge amounts of time in my daily work. However, I think it’s a fair criticism of the system that its terminology is very confusing for newcomers, and in particular those who have come from using CVS or Subversion.

This is a personal list of some of my “favourite” points of confusion, which I’ve seen arise time and time again, both in real life and when answering questions on Stack Overflow. To be fair to all the excellent people who have contributed to git’s development, in most cases it’s clear that they are well aware that these terms can be problematic, and are trying to improve the situation subject to compatibility constraints. The problems that seem most bizarre are those that reuse CVS and Subversion terms for completely different concepts – I speculate a bit about that at the bottom.


If you’ve used Subversion or CVS, you’re probably used to “update” being a command that goes to the remote repository, and incorporates changes from the remote version into your local copy – this is (very broadly) analogous to “git pull”.  So, when you see the following error message when using git:

You might imagine that this means you need to run “git pull”. However, that’s wrong.  In fact, what “needs update” means is approximately: “there are local modifications to this file, which you should probably commit or stash”.

“track” and “tracking”

The word “track” is used in git in three senses that I’m aware of. This ambiguity is particularly nasty, because the latter two collide at a point in learning the system where newcomers to git are likely to be baffled anyway. Fortunately, this seems to have been recognized by git’s developers (see below).

1. “track” as in “untracked files”

To say that a file is tracked in the repository appears to mean that it is either present in the index or exists in the commit pointed to by HEAD.  You see this usage most often in the output of “git status”, where it will list “untracked files”:

This sense is relatively intuitive, I think – it was only after complaining for a while about the next two senses of “track” that I even remembered that there was also this one :)

2. “track” as in “remote-tracking branch”

As a bit of background, you can think of a remote-tracking branch as a local cache of the state of a branch in a remote repository.  The most commonly seen example is origin/master, or, to name that ref in full, refs/remotes/origin/master.  Such branches are usually updated by git fetch (and thus also potentially by git pull).  They are also updated by a successful push to the branch in the remote repository that they correspond to.   You can merge from them, examine their history, etc. but you can’t work directly on them.

The sense of “track” in the phrase “remote-tracking branch” is indicating that the remote-tracking branch is tracking the state of the branch in the remote repository the last time that remote-tracking branch was updated.  So, you might say that refs/remotes/origin/master is tracking the state of the branch master in origin.

The “tracking” here is defined by the refspec in the config variable remote.<remote-name>.fetch and the URL in the config variable remote.<remote-name>.url.

3. “track” as in “git branch –track foo origin/bar” and “Branch foo set up to track remote branch bar from origin”

Again, if you want to do some work on a branch from a remote repository, but want to keep your work separate from everything else in your repository, you’ll typically use a command like the following (or one of its many “Do What I Mean” equivalents):

… which will result in the following messages:

The sense of “track” both in the command and the output is distinct from the previous sense – it means that config options have been set that associate your new local branch with another branch in the remote repository. The documentation sometimes refers to this relationship as making bar in origin “upstream” of foo. This “upstream” association is very useful, in fact: it enables nice features like being able to just type git pull while you’re on branch foo in order to fetch from origin and then merge from origin/bar. It’s also how you get helpful messages about the state of your branch relative to the remote-tracking branch, like “Your branch foo is 24 commits ahead of origin/bar and can be fast-forwarded”.

The tracking here is defined by config variables branch.<branch-name>.remote and branch.<branch-name>.merge.

“tracking” Summary

Fortunately, the third sense of “tracking” seems to be being carefully deprecated – for example, one of the possible options for push.default used to be tracking, but this is now deprecated in favour of the option name upstream. The commit message for 53c403116 says:

push.default: Rename ‘tracking’ to ‘upstream’

Users are sometimes confused with two different types of “tracking” behavior in Git: “remote-tracking” branches (e.g. refs/remotes/*/*) versus the merge/rebase relationship between a local branch and its @{upstream} (controlled by branch.foo.remote and branch.foo.merge config settings).

When the push.default is set to ‘tracking’, it specifies that a branch should be pushed to its @{upstream} branch. In other words, setting push.default to ‘tracking’ applies only to the latter of the above two types of “tracking” behavior.

In order to make this more understandable to the user, we rename the push.default == ‘tracking’ option to push.default == ‘upstream’.

push.default == ‘tracking’ is left as a deprecated synonym for ‘upstream’.


In CVS and Subversion, “commit” means to send your changes to the remote repository. In git the action of committing (with “git commit”) is entirely local; the closest equivalent of “cvs commit” is “git push”. In addition, the word “commit” in git has different verb and noun senses (although frankly I’ve never found this confusing myself). To quote from the helpful git glossary:


As a noun: A single point in the git history; the entire history of a project is represented as a set of interrelated commits. The word “commit” is often used by git in the same places other revision control systems use the words “revision” or “version”. Also used as a short hand for commit object.

As a verb: The action of storing a new snapshot of the project’s state in the git history, by creating a new commit representing the current state of the index and advancing HEAD to point at the new commit.

Strictly speaking that makes two different noun senses, but as a user I’ve rarely found that confusing.


In CVS and Subversion “checkout” creates a new local copy of the source code that is linked to that repository. The closest command in git is “git clone”. However, in git, “git checkout” is used for something completely distinct. In fact, it has two largely distinct modes of operation:

(git checkout is also frequently used with -b, to create a new branch, but that’s really a sub-case of usage 1.)

In my ideal world, these two modes of operation would have different verbs, and neither of them would be “checkout”.

“HEAD” and “head”

There are usually many “heads” (lower-case) in a git repository – the tip of each branch is a head. However, there is only one HEAD (upper-case) which is a symbolic ref which points to the current branch or commit.

“fetch” and “pull”

I wasn’t aware of this until Roy Badami pointed it out, but it seems that git and Mercurial have opposite meanings for “fetch” and “pull” – see the top two lines in this table of git / hg equivalences. I think it’s understandable that since git’s and Mercurial’s development were more or less concurrent, such unfortunate clashes in terminology might occur.

“push” and “pull”

“git pull” is not the opposite of “git push”; the closest there is to an opposite of “git push” is “git fetch”.

“hash”, “SHA1”, “SHA1sum”, “object name” and “object identifier”

These terms are often used synonymously to mean the 40 characters hexadecimal strings that uniquely identify objects in git. “object name” seems to be the most official, but the least used in general. Referring to an object name as a SHA1sum is potentially confusing, since the object name for a blob is not the same as the SHA1sum of the file.

“remote branch”

This term is only used occasionally in the git documentation, but it’s one that I would always try to avoid because it tends to be unclear whether you mean “a branch in a remote repository” or “a remote-tracking branch”. Whenever a git beginner uses this phrase, I think it’s worth clarifying this, since it can avoid later confusion.

“index”, “staging area” and “cache”

As nouns, these are all synonyms, which all exist for historical reasons. Personally, I like “staging area” the best since it seems to be the easiest concept to understand for git beginners, but the other two are used more commonly in the documentation.

When used as command options, --index and --cached have distinct and consistent meanings, as explained by Junio C. Hamano in this useful blog post.

Why are there so many of these points of confusion?

I would speculate that the most significant effect that contributed to these terminology confusions is that git was being actively used by an enthusiastic community from very early in its development, which means that early names for concepts have tended to persist for the sake of compatibility and consistency. That doesn’t necessarily account for the many conflicts with CVS / Subversion usage, however.

To be fair to git, thinking up verbs for particular commands in any software is tough, and there have been enough version control systems written that to completely avoid clashes would lead to some convoluted choices. However, it’s hard to see git’s use of CVS / Subversion terminology for completely different concepts as anything but perverse. Linus has made it very clear many times that he hated CVS, and joked that a design principle for git was WWCVSND (What Would CVS Never Do); I’m sympathetic to that, as I’m sure most are, especially after having switched to the DVCS mindset. However, could that attitude have extended to deliberately disregarding concerns about terminology that might make it actively harder for people to migrate to git from CVS / Subversion? I don’t know nearly enough about the early development of git to know. However, it wouldn’t have been tough to find better choices for commit, checkout and update in each of their various senses.

Update: While the device I discuss below initially seemed to work promisingly as a repeater, we’ve had lots of problems with it – it seems to work fine for a few hours, but then will stop working for no clear reason. At some point I’m going to try reflashing it as suggested in the link below, but I haven’t been in the same house as the router for any appreciable amount of time in the last 6 months. The post below is as I originally wrote it, but please be aware that it hasn’t worked nearly as well as I thought at the time of writing. Update 2:  having revisited the house recently, it seemed to be better having changed the default address of the TP Link device to be different from the BT Home Hub (duh) and outside the latter’s DHCP range, so I’d recommend checking that.

I’m posting this because we found it surprisingly hard to find convincing instructions for setting up a device to extend the range of a wireless router, and I suspect many other people might be in the same position.  In the end, the device we got was very cheap, easy to set up, and it’s a great improvement.

Please bear in mind that I’m just reporting what worked for us – I can’t guarantee the same will work in your situation or provide any technical support.  (I’m pretty ignorant about WiFi, it turns out.)  I imagine that this will work fine with other wireless routers and the BT Home Hub 3, but of course I can’t promise that.

The Problem

We found that one end of my parents’ house had very poor wireless LAN coverage, since for various reasons the BT Home Hub 2 has to be placed at the opposite end of the house and there are plenty of walls and other obstructions.  (For those outside the UK, the BT Home Hub is just an ADSL modem and wireless access point which BT give you when you order their broadband service – it’s quite a basic model and I assume the method described here would work with any wireless access point.)

We first tried a couple of commonly suggested solutions which didn’t involve buying new hardware: The Solution

Some searching turned up this suggestion from Keith Beddoe from the BT community forums.  His second suggested option is to set up a particular TP Link wireless access point as a “Universal Repeater”, but suggests reflashing the access point with some files he supplies.  I’m always slightly wary of reflashing devices unless it’s absolutely necessary, and it’s not clear why it is in this case – the manual for the TL-WA701ND says that it can act both as a Repeater and a Universal Repeater.  (The distinction between those two for this device seems to be that the existing router must support WDS in order to use the Repeater option – since the BT Home Hub 2 doesn’t, we want the Universal Repeater option.)

The only steps I needed to take to set up the access point as a range extender were as follows:

The effect of this is that there are now two access points with the same SSID as the BT Home Hub listed in the output of “iwlist scan”.  The network manager on my laptop only lists it once, though, as does my smartphone.  If I walk with my laptop from one end of the house to the other, it seems to reconnect seamlessly when moving into an area that is only covered by the range extender, and our phones behave similarly.  So, that seems to be ideal and we get a good signal all over the house now.

The price of the TP Link TL-WA701ND is only £23, and this seems like a reasonable price to pay to get wireless LAN in the garden :)

I’ve had a long-standing affection for Star Trek: The Next Generation, since it used to be shown on TV right after I got home from school and I would watch it almost every day.  I must have seen every episode several times when I was younger, but it was only recently that I went back to rewatch them on DVD, partly out of curiosity to see how they would stand up.  I think it’s fair to say that a lot of these episodes haven’t aged well (or were dreadful even at the time), but there are still plenty of gems there. As much for my own reference as anything else, the table below marks which episodes I still thought were good after a gap of many years.

In the table, I’ve also added links to two series of reviews.  The first are Zack Handlen’s excellent reviews of every ST:TNG episode (linked from the initials “ZH” below), and I’ve also added his grade for each episode.  I’ve found myself agreeing with his reviews more often than not, and in a some cases the reviews are significantly more enjoyable than the episodes themselves.  The second set of links are to reviews by Wil Wheaton (who played Wesley Crusher) of some of the episodes from Season 1 – it’s interesting to hear his take on these very variable episodes, particularly since the writers were not good to him at all.

A few other random observations:

These other links might also be of interest if you enjoyed ST:TNG:

Summary Table of Episodes

The source for this table is the spreadsheet here and the HTML was generated from that with this script.

The key to this colours in this table is:

Season 1

Season 2

Season 3

Season 4

Season 5

Season 6

Season 7

I’ve seen many people link to a news story claiming that “cassette tape” has been removed from the Concise OED — this story appears to be complete nonsense,  and I’m genuinely intrigued as to how it got started.  Of course, none of the news stories that repeat this claim cite any source.  This is a minor example of a serious problem that Francis has blogged about.

TIME magazine is perhaps the highest profile publication to  make the claim, saying that “cassette tape” is being removed from the Oxford English Dictionary and referencing The Huffington Post as its only source.  Firstly, the TIME story is obviously nonsense taken literally, since the OED is a historical dictionary and words generally aren’t removed from its 20 volumes at all.  Other stories clarify that this is supposedly a scandal with the new (12th) edition of the Concise Oxford English Dictionary, which was released in August 2011 to a flurry of stories about the various entertaining new words that have been added to it.  The news stories about those words seem to be largely based on two blog posts (here and here) from Oxford University Press that discuss in general terms what has defined the COED over its 100 year history.  Neither of those posts mention anything about “cassette tape” being removed.  (I haven’t found any more official press release about this new edition online.)

Anyway, suspecting that this story was nonsense, I did the obvious thing and looked in the new 12th edition of the COED in a local bookshop.  Of course, cassette tape is still here: So, my real question is: “Who is it that just invents this stuff?” tl;dr: This post describes how well Ubuntu GNU/Linux works on the Sony Vaio VPCYB2M1E since there’s otherwise not much information on the web about the hardware support for this model on Linux.  In summary, so far it seems that this is a good choice for running Ubuntu on so long as you use the 32-bit version, but these are still early days.

At the moment, looking for a new computer in the netbook space which runs Linux well is a rather frustrating exercise.  This is chiefly because:

Anyway, if you’re looking for one of these slightly-better-than-a-netbook machines to run Ubuntu on, you might consider the VPCYB2M1E – the Sony Vaio Y series that is available in the UK at the moment (summer 2011).  We believe that the “/S” at the end of the model number just means that it’s the silver rather than pink version.  This is based around the AMD E-350, so we initially chose to use the 64-bit version of Natty – however, due to some freezes that don’t seem to occur with the 32-bit flavour, we reinstalled with the 32-bit version.  (See below for more details.)

The specifications of this laptop can be found on Sony’s website.  The weight (at 1.46kg) is slightly more than I’d like, but the 1366 x 768 screen is very nice.

First we checked by booting from a live USB stick that enough of the hardware seemed to work (wireless, sound, graphics) that we were willing to risk wiping all the existing partitions and claiming all the disk space for Ubuntu.  On the second laptop I installed Natty alongside Windows 7, which worked fine.  (I note that the installation of Natty is very slick now.)

At the end of this post, we’ve include the output of “cat /proc/cpuinfo” and “lspci -v”, for those who are interested in the details.  There’s a summary of what we’ve tested so far below.  However, first a description of the only big problem we’ve seen so far:

I/O Related Freezes

When using rsync to copy large files onto this system, we found that the interface would freeze to a greater or lesser extent.  This seemed to be reproducible with any command that wrote a lot of data to disk (e.g. even dd from /dev/zero) but was only present on 64-bit installations, not 32-bit.  This problem is described here:


… and linked to a bug that might be related.  However, since this didn’t seem to be a problem with the 32-bit installation, we didn’t pursue it further.  (However, it should be still reproducible with the 64-bit Live USB stick.)


Works fine out-of-the-box


Works out-of-the-box

Power Management


Works out-of-the-box.  (When suspended, the power light glows bright orange, though, which is somewhat annoying.)

Hibernate (Suspend-to-Disk)


Special Keys

Work out-of-the-box



Work out-of-the-box – pretty loud.  I’ve found on one occasion that the sound stopped working after a while and I had to reboot, but I need to investigate that more.


Works out-of-the-box, a bit noisy, but we haven’t tried to change any of the mixer settings yet.


With the default driver, some of the desktop effects flickered a great deal, and the desktop background showed some flickering. Installing the closed source ATI drivers (via the “Additional Drivers” in the System Settings) seems to have fixed this.

External monitor

– Not tested yet.  In particular we need to check how large a resolution it can drive.

Card Readers

The SD card reader and the slot for Sony’s memory sticks both work fine.

HDMI output

– Not tested yet.  (No HDMI monitor available.)




Works fine out-of-the-box.  Changing the scrolling behaviour (from dragging on the right hand side to two-fingered scrolling) can be easily done with the mouse properties.

Hardware Details 

The Guardian and The Observer [Kindle Edition]


(occasional miscellanea)

Tweets by mhl20