Week Beginning 7th January 2019

This was my first week back after an enjoyable but all too rapid Christmas holiday.  I spent most of Monday catching up with emails, writing a new and up to date ‘to do’ list and upgrading all of the WordPress sites I manage to the new Version 5 of WordPress, which has introduced a horrible new editor in place of the perfectly good one that was there before.  I decided to do a full backup of all of the sites before I upgraded them, which took some time, but it’s good to have a local backup of everything, just in case the main backups go wrong.  Rather annoyingly when I upgraded WordPress on Monday it was version 5.0.2 any by the end of the week 5.0.3 had been released, so I’ll have to upgrade all the sites again sometime.

I also spent a bit of time this week making further updates to the RNSN song stories, based on feedback received from the extended project team over the past few weeks.  Thankfully these were mostly minor tweaks to content rather than big requests for major changes.  I also made some updates to the Data Management Plan for Thomas Clancy’s Iona project and met briefly with him to discuss some related matters.  I also gave some advice to Meg MacDonald and Bryony Randall about the proposal Bryony is putting together and had a chat to Luca about one of Graeme Cannon’s old projects that Luca is helping to update.

Other than these issues I spent the rest of my week working on the SCOSYA project.  I had been hoping to get started on the public interface for the Atlas in the run-up to Christmas, but had been unable to find the time to work on it due to other work commitments.  Somewhat amazingly, this week proved to be rather quiet (not having anything to do for the Historical Thesaurus for once certainly made a big difference) and it was great to have a nice big block of time to devote to the project.

Previously we had agreed that several guided ‘stories’ through views of the data would be good to have, and E, the new RA on the project sent me a draft version of a story based on the data for ‘Gonnae’.  I’d suggested using the ‘Storymap’ interface (see https://storymap.knightlab.com/) as a means of navigating through different views of map-based data, and E and I had agreed that I’d also investigate ‘Voronoi’ maps as a means of extrapolating point-based data into broader areas.

I had found a nice example of a Voronoi map that was created using the d3.js library, which I’ve had quite a lot of experience with.  It’s a map of supermarkets in the UK, and you can view it here: https://chriszetter.com/voronoi-map/examples/uk-supermarkets/.

I decided to try and get a Voronoi interface working with our data as a first step, using the source code for the ‘supermarkets’ map as a starting point.  Below is a screenshot of my first attempt:

This version uses the ‘Gonnae’ data (an adapted version of a CSV file generated by the project’s API) and our base map.  You can see in the above screenshot how the ‘cells’ are generated based on our questionnaire locations, but no colours are used to differentiate rating levels, the checkboxes in the top right don’t actually do anything and the text in the bottom left still says ‘explore supermarkets in the UK’ until you click on a cell, at which point the location name is displayed.

For the next version I wanted to properly integrate the map with our website interface, investigate how to shade the cells, and replace the ‘supermarket’ text in the bottom left with something more useful.  Below is a screenshot of this version:

This version also pulls its data directly from our API whereas the previous version used a stripped-down CSV file I’d manually made from the API data.  This is why this version has slightly different cell layouts (e.g. it includes cells for where there is no data for this question).  Cells are given different colours depending on the average rating (the same colours as on the main point-based map), with grey used for locations where there is no data for this question.  Information about what the map shows (‘Gonnae’) is found in the bottom left, and clicking on a cell highlights it and displays the location name, average score and what this score means in the bottom left too.  The cell colours are possibly not different enough to easily distinguish them, and the opacity level does make it rather difficult to make out the underlying map, but these are things that can be tweaked.  I also included points to show where the actual locations are.  I personally find these quite helpful, but they can always be removed.

With this version in place I then decided to try and get the storymap interface working with our base map.  This is where things started going wrong.  The Voronoi script uses a rather old version of the MapBox library, which in turn uses an older version of the Leaflet mapping library.  I tried switching to a more recent version but unfortunately this breaks the code.  I could (and I still might) go through the code line by line to figure out what function calls need to be updated for the script to work with a more recent version, but this is likely to be a tricky and time-consuming process, and one that might not actually be successful.  The issue is that storymap requires a more up to date version of the MapBox library to function, and even getting it to work with our basemap in a way that would be compatible with the Voronoi code took a lot of hacking about with the storymap source code.  In the end I managed to get storymap to work with our base map, but failed to get the Voronoi cells to actually display.  Here is a screenshot of this version, to get an example of how the story would work:

As you can see, the information from the ‘Gonnae’ document is there, as is our base map.  A Youtube clip from Chewin the Fat is embedded in the first slide and as you navigate between slides the map pans and zooms.  It’s obviously not that great because the map doesn’t display any data.  But in addition I realised that the storymap interface might not be all that suitable anyway.  The ‘slide’ part takes up half the width of the browser window, which obscures far too much of the map.  Also, the map icons for the slides and the dotted lines aren’t really much use to us either.

I therefore decided I’d write my own version of the storymap library from scratch – a version that would work with the Voronoi script, would take up less room on the map, and would only include features we actually needed, plus new features that storymap didn’t include that I would otherwise have had to have added in (e.g. triggers to change the data displayed on the underlying map when navigating from slide to slide).  I say ‘from scratch’ but it wasn’t really as bad as all that as I could reuse a lot of code I’d written for previous projects.

It’s taken quite a while to implement all of the features, but I managed to complete a working version by Friday.  It’s still just a proof of concept and will need further work, but here is a screenshot of it:

And here is some information about how it works:

  1. There are (currently) two different slide types: ‘Full’ and ‘Side’.  Full takes up the entire width of the map while side hovers in the top right of the map.
  2. The YouTube video is embedded in the first slide.  Any content could be added to a slide as required.
  3. There are ‘next’ and ‘previous’ buttons at the edges of the map (so long as there is a next or previous slide).  Pressing on one of these loads the appropriate slide, repositions / zooms the map (if necessary) and loads new map data (if necessary).
  4. When navigating to a different slide, the slide contents animate smoothly, swishing off to the left or right depending on which direction you’ve pressed.  The map panning and zooming and the Voronoi data layer are unfortunately a bit clunkier but without upgrading the mapping library (and rewriting the Voronoi script) I’m not sure I’ll be able to make this any smoother.
  5. No matter what slide you’re on you can zoom and pan around the map as much as you want.  You can also click on a cell to view its details.
  6. When a cell is clicked on you can now click it a second time to deselect it.
  7. The cell shading is slightly less intense in this version as opposed to the previous version, making the background map a bit easier to see.
  8. When you navigate to the ‘gonnae you leave us alone’ slide this triggers the loading of a different data view (H2 rather than H1).  The information in the bottom left box updates to reflect this.  I might temporarily highlight this box when the data changes to make it clearer that a change has been made.  If you navigate back to the previous slide the ‘H1’ data loads in again.  For other stories we can change the view of the data once per slide, and any searches can be incorporated (e.g. limits by age range or ratings).  If we don’t change the position and zoom between slides but just change the data users will be able to very easily compare two datasets by navigating between the slides, which could be useful.
  9. The data for the ‘storymap’ is found in a JSON file that I created manually.  Each slide contains the following fields:
    1. ID: a unique numerical identifier for the slide.  The script expects the first slide to have an ID of 1
    2. Type: This is either ‘full’ for a full-screen slide or ‘side’ for a side-based slide.  We can add more types if required.
    3. Title: An optional field for when a slide has a title.  Only the first and last slides currently have titles.
    4. Body: The body of the slide, which is entered as HTML.  Note that double quotes need to be escaped (\” not “) otherwise the JSON file breaks.  Any required HTML content can go in here.
    5. mapData: The URL of the API search that will be used by the Voronoi script.  Must be JSON data not CSV data.  Can be left blank if the data is the same as the previous slide but bear in mind that when data changes the slide before the new data needs to feature the URL of the previous data for this data to be reloaded when a user presses the ‘previous’ button.
    6. dataDescription: The information about the data that gets displayed in the bottom left box.  As with the mapData field this can be left blank if it’s the same as for other slides, but the same ‘previous slide’ issue needs to be considered.
    7. Lat: The latitude where the map should focus for this slide
    8. Lng: The longitude where the map should focus for this slide
    9. Zoom: The zoom level of the map for this slide
    10. Prev: The ID of the previous slide (can be left blank if no previous slide)
    11. Next: The ID of the next slide (can be left blank if no next slide)
  10. To create a new story all I’d need to do is create a new JSON file following the same layout as above, which should be fairly straightforward.

That’s as far as I’ve got for now.  I have sent the URLs off to E and Jennifer Smith for feedback, and will probably not do any further work on this until I hear back from them.  There are other features we might also want to implement, e.g. an option to switch from Voronoi cells to points, or a slider to dynamically change the opacity of the cells, or options to allow us to highlight a particular group of cells for a slide.  I also still need to make this interface work better on mobile phones (tablets should already be ok, though) and try to add in a ‘full screen’ option, which might prove to be tricky.  But all of that is for another week.