Rough guide to building UK election maps for D3.js

on

During my time at CITY A.M. I helped to build an interactive British General Election Map. This blog outlines how I created it.

Let's see the map then

Here's a simplified version of the map, hosted at Codepen.

See the Pen D3 Election Map by Stephen Scott (@sassquad) on CodePen.


Here, the map looks rather small, as it has been resized to fit the page. You can use your mouse wheel (or double-click/double-tap) to zoom in or out of the map, and click constituencies to view more detailed results on the left hand side. Each constituency is coloured according to party livery.

This blog post gives a very general guide as to how that map was created. I'll be adding further detail about the creation of the map in subsequent posts.

Acknowledgements

This project was the work of several people, in addition to myself. Here are the other former colleagues who collectively made the election map possible:
  • Michael King (back end programming of data capture system)
  • Billy Ehrenberg (for D3.js support and data journalism expertise)
  • Emma Haslett (who toiled through the night gathering the results)
  • Chris Parameter (design and feel of interactive visuals) 
  • Ben Barden (project management) 
Thank you all!

Disclaimer

There's a couple of things I need to mention beforehand:
  • A fair amount of time has passed since this project was completed, so forgive me if I appear to have skipped over the details. 
  • Early in March 2016, the CITY A.M. website was relaunched, so the election map doesn't display as well in the new layout. This is partly why I created the Codepen version.

Introduction

Many news organisations - such as BBC News or The Guardian - put a lot of resources into their election coverage, both offline as well as online. Way back in 2010, Adobe Flash was primarily used to produce interactive mapping solutions. Now we have at our disposal a wider range of tools with which to produce compelling content, without resorting to plug-ins.

To produce our map, we needed the following:
  1. a suitable back end with which to assimilate results data
  2. a map of all the 650 parliamentary constituencies which comprise England, Wales, Scotland and Northern Ireland.
  3. a way to bind the data to the map
  4. make the election map usable on mobile devices.
That's no mean feat. There's a lot of moving parts. This article will be a general overview. I'll then go into further detail in subsequent posts.

In the end the following technologies were used, in no order of importance:

D3.js

D3 - or Data Driven Documents - is a JavaScript library that binds data to elements of the Document Object Model (DOM) that are present - or yet to be created - on your web page. It's a piece of magic. You give it data, and inform it as to how you want the data to be displayed visually. D3.js then does the rest.

First released back in 2011 by Mike Bostock, it has grown exponentially in its popularity. The website alone is worth a visit, to see the myriad visualisations that are possible. It practically invented the recent career of data journalism, and lead to renewed interest in data science. The gradual open sourcing of governmental data and statistics has made the visualisation of previously impenetrable data far more compelling, and can reveal trends that would never have been seen previously.

D3.js follows web standards, and no proprietary solutions, so is extremely adaptable. It offers a deceptively simple syntax, but is more than capable of producing extremely compelling and beautiful data visualisations.

Building a map

For mapping, D3 supports various projections, in data formats such as GeoJSON or TopoJSON. If you care about file size or topology, then use TopoJSON. Otherwise, use GeoJSON.

Playing with map data is actually rather involved. Fortunately, Mike Bostock's D3.js article "Let's make a Map" is well worth a read. It happens to use the UK and Ireland as an example map, so some of the technical detail described within is very useful! My only problem was that I was using Windows to develop the map. Therefore, references to Homebrew had to be replaced with Windows equivalents, which proved to be a bit of a minefield.

However, most available mapping data comes in ESRI Shapefile format, for use in geographical mapping software, such as QGIS.  So, we need to get hold of this data. As Bostock's article attests, Natural Earth provide some great geographical data provided entirely by voluntary contributors.

The primary source of mapping data for parliamentary constituencies is the Ordnance Survey. Under their OS OpenData resource, you can download their datasets for free, under an open source licence. For authoritative data such as elections, it's important to get data from an official source.

However, the first thing to realise is that Northern Ireland mapping data is owned by a separate service - the Ordnance Survey of Northern Ireland. We therefore need to obtain maps from both services, then combine them. 

This is where the QGIS software comes into its own. QGIS is a hefty piece of software, and is well beyond the scope of this article, but is available for all major platforms as a free download. With QGIS, we can import both Shape files. We then export a new Shape file of the combined data. Once that is done, we can finally import the file into the Mapshaper.org site, and export a more simplified GeoJSON file, ready for our purposes.

The building of the map is the part of the journey I've had to skip through for the time being. I'll get into the nitty gritty of this in a future blog post.

Data binding

In order to marry the constituencies to the election result data, an unique identifier was required. In this case, the common element was that used by the Press Association. This id is a simple integer value, which enabled d3 to bind the data more quickly.

The main nub of the code is this:

queue()
    .defer(d3.json, "map file")
    .defer(d3.json, "election data")
    .await(ready);
...
function ready(error, uk, boundaries) {
    mapFeatures = topojson.feature(uk, uk.objects.subunits).features;
    var map = mapContainer
      .append("g")
      .attr("class","subunits")
      .selectAll("path")
      .data(mapFeatures),
      constituency = boundaries.data,...

The queue command allows for the loading of large datasets asynchronously. The await handler has a single argument, the function that builds the map, called ready(). The ready() function then maps the election result data to the constituency subunit features. 

The function call has three arguments, error, uk and boundaries. The last two correlate to the data sources referred to in the defer() handlers of the queue() command. In this instance, uk maps to the map file, while boundaries maps to the election data. The map variable is an instance of the SVG file which will contain all of the map data and election data combined.

Mobile and Tablet

As the election map had to be usable on mobile devices, the map file used on the webpage had to be simplified further. SVG files render very poorly on tablets and mobile devices, so the mapfile had to be reducd in size and compleity, a function the Mapshaper.org site offers. CSS breakpoints were then used to serve mobile and tablet specific versions which would work accordingly. D3.js supports touch and swipe events as standard.

Again, this will need to be the subject of another post, as it requires deeper explanation than can be provided for the time being.

Conclusion

This election map solution is adequate, but is merely one approach. There are sure to be other ways of producing a data visualization such as this. Much of the code was put together fairly quickly, with various iterations performed in initial research and discovery phase. Added complexity came in the form of mobile and tablet support. Much throttling had to be performed to prevent slow performance. SVG files don't work too well if they are too complex in structure and require features such as pan and zoom.

The D3.js library, and it's accompanying Queue library are available as Node NPM modules, which can be beneficial in preparing server side mapping solutions. The JavaScript itself is very messy in this implementation, and can be clearly improved.

As mentioned, I'll write further, more comprehensive accounts on some of the above in due course. Thanks for reading.

Comments

Post a Comment