This was the first full week of the Coronavirus lockdown and as such I was working from home and also having to look after my nine year-old son who is also at home on lockdown. My wife and I have arranged to split the days into morning and afternoon shifts, with one of us home-schooling our son while the other works during each shift and extra work squeezed in before and after these shifts. The arrangement has worked pretty well for all of us this week and I’ve managed to get a fair amount of work done.
This included spotting and requesting fixes for a number of other sites that had started to display scary warnings about their SSL certificates, working on an updated version of the Data Management Plan for the SCOSYA follow-on proposal, fixing some log-in and account related issues for the DSL people and helping Carolyn Jess-Cooke in English Literature with some technical issues relating to a WordPress blog she has set up for a ‘Stay at home’ literary festival (https://stayathomefest.wordpress.com/). I also had a conference call with Katie Halsey and Matt Sangster about the Books and Borrowers project, which is due to start at the beginning of June. It was my first time using the Zoom videoconferencing software and it worked very well, other than my cat trying to participate several times. We had a good call and made some plans for the coming weeks and months. I’m going to try and get an initial version of the content management system and database for the project in place before the official start of the project so that the RAs will be able to use this straight away. This is of even greater importance now as they are likely to be limited in the kinds of research activities they can do at the start of the project because of travel restrictions and will need to work with digital materials.
Other than these issues I divided my time between three projects. The first was the Burns Supper map for Paul Malgrati in Scottish Literature. Paul had sent me some images that are to be used in the map and I spent some time integrating these. The image appears as a thumbnail with credit text (if available) appearing underneath. If there is a link to the place the image was taken from the credit text appears as a link. Clicking on the image thumbnail opens the full image in a new tab. I also added links to the videos where applicable too, but I decided not to embed the videos in the page as I think these would be too small and there would be just too much going on for locations that have both videos and an image. Paul also wanted clusters to be limited by areas (e.g. a cluster for Scotland rather than these just being amalgamated with a big cluster for Europe when zooming out) and I investigated this. I discovered that it is possible to create groups of locations. E.g. have a new column in the spreadsheet named ‘cluster’ or something like that and all the ‘Scotland’ ones could have ‘Scotland’ here, or all the South American ones could have ‘South America’ here. These will then be the top level clusters and they will not be further amalgamated on zoom out. Once Paul gets back to me with the clusters he would like for the data I’ll update things further. Below is an image of the map with the photos embedded:
The second major project I worked on was the interactive map for Gerry McKeever’s Regional Romanticism project. Gerry had got back to me with a new version of the data he’d been working on and some feedback from other people he’d sent the map to. I created a new version of the map featuring the new data and incorporated some changes to how the map worked based on feedback, namely I moved the navigation buttons to the top of the story pane and have made them bigger, with a new white dividing line between the buttons and the rest of the pane. This hopefully makes them more obvious to people and means the buttons are immediately visible rather than people potentially having to scroll to see them. I’ve also replaced the directional arrows with thicker chevron icons and have changed the ‘Return to start’ button to ‘Restart’. I’ve also made the ‘Next’ button on both the overview and the first slide blink every few seconds, at Gerry’s request. Hopefully this won’t be too annoying for people. Finally I made the slide number bigger too. Here’s a screenshot of how things currently look:
My final project of the week was the Historical Thesaurus. Marc had previously come up with a very good idea of making a nice interactive ‘guess the category’ quiz that would present a word and four possible categories. The user would have to select the correct category. I decided to make a start on this during the week. The first task was to made a script that grabbed a random category from the database (ensuring it was one that contained at least one non-Old English word). The script then checked the part of speech of this category and grabbed a further three random categories of the same part of speech. These were all then exported as a JSON file. I then worked on the quiz page itself, with all of the logic handled in JavaScript. The page connects to the script to grab the JSON file then extracts the word and displays it. It then randomised the order of the four returned categories and displayed these as buttons that a user could click on. I worked through several iterations of the quiz, but eventually I made it so that upon clicking on a choice the script automatically gives a tick or cross and styles the background red or green. If the guess was incorrect the user can guess again until they get the correct answer, as you can see below:
I then decided to chain several questions together to make the quiz more fun. Once the correct answer is given a ‘Next’ button appears, leading to a new question. I set up a ‘max questions’ variable that controls how many questions there are (e.g. 3, 5 or 10) and the questions keep coming until this number is reached. When the number is reached the user can then view a summary that tells them which words and (correct) categories were included, provides links to the categories and gives the user an overall score. I decided that if the user guesses correctly the first time they should get one star. If they guess correctly a second time they get half a star and any more guesses get no stars. The summary and star ratings for each question are also displayed as the following screenshot shows:
It’s shaping up pretty nicely, but I still need to work on the script that exports data from the database. Identifying random categories that contain at least one non-OE word and are of the same part of speech as the first randomly chosen category currently means hundreds or even thousands of database calls before a suitable category is returned. This is inefficient and occasionally the script was getting caught in a loop and timing out before it found a suitable category. I managed to catch this by having some sample data that loads if a suitable category isn’t found after 1000 attempts, but it’s not ideal. I’ll need to work on this some more over the next few weeks as time allows.
Last week was a full five-day strike and the end of the current period of UCU strike action. This week I returned to work, but the Coronavirus situation, which has been gradually getting worse over the past few weeks ramped up considerably, with the University closed for teaching and many staff working from home. I came into work from Monday to Wednesday but the West End was deserted and there didn’t seem much point in me using public transport to come into my office when there was no-one else around so from Thursday onwards I began to work from home, as I will be doing for the foreseeable future.
Despite all of these upheavals and also suffering from a pretty horrible cold I managed to get a lot done this week. Some of Monday was spend catching up with emails that had come in whilst I had been on strike last week, including a request from Rhona Alcorn of SLD to send her the data and sound files from the Scots School Dictionary and responding to Alan Riach from Scottish Literature about some web pages he wanted updated (these were on the main University site and this is not something I am involved with updating). I also noticed that the version of this site that was being served up was the version on the old server, meaning my most recent blog posts were not appearing. Thankfully Raymond Brasas in Arts IT Support was able to sort this out. Raymond had also emailed me about some WordPress sites I mange that had out of date versions of the software installed. There were a couple of sites that I’d forgotten about, a couple that were no longer operational and a couple that had legitimate reasons for being out of date, so I got back to him about those, and also updated my spreadsheet of WordPress sites I manage to ensure the ones I’d forgotten about would not be overlooked again. I also became aware of SSL certificate errors on a couple of websites that were causing the sites to display scary warning messages before anyone could reach the sites, so asked Raymond to fix these. Finally, Fraser Dallachy, who is working on a pilot for a new Scots Thesaurus, contacted me to see if he could get access to the files that were used to put together the first version of the Concise Scots Dictionary. We had previously established that any electronic files relating to the printed Scots Thesaurus have been lost and he was hoping that these old dictionary files may contain data that was used in this old thesaurus. I managed to track the files down, but alas there appeared to be no semantic data in the entries found therein. I also had a chat with Marc Alexander about a little quiz he would like to develop for the Historical Thesaurus.
I spoke to Jennifer Smith on Monday about the follow-on funding application for her SCOSYA project and spent a bit of time during the week writing a first draft of a Data Management Plan for the application, after reviewing all of the proposal materials she had sent me. Writing the plan raised some questions and I will no doubt have to revise the plan before the proposal is finalised, but it was good to get a first version completed and sent off.
I also finished work on the interactive map for Gerry McKeever’s Regional Romanticism project this week. Previously I’d started to use a new plugin to get nice curved lines between markers and all appeared to be working well. This week I began to integrate the plugin with my map, but unfortunately I’m still encountering unusable slowdown with the new plugin. Everything works fine to begin with, but after a bit of scrolling and zooming, especially round an area with lots of lines, the page becomes unresponsive. I wondered whether the issue might be related to the midpoint of the curve being dynamically generated from a function I took from another plugin so instead made a version that generated and then saved these midpoints that could then be used without needing to be calculated each time. This would also have meant that we could have manually tweaked the curves to position them as desired, which would have been great as some lines were not ideally positioned (e.g. from Scotland to the US via the North Pole), but even this seems to have made little impact on the performance issues. I even tried turning everything else off (e.g. icons, popups, the NLS map) to see if I could identify another cause of the slowdown but nothing has worked. I unfortunately had to admit defeat and resort to using straight lines after all. These are somewhat less visually appealing, but they result in no performance issues. Here’s a screenshot of this new version:
With these updates in place I made a version of the map that would run directly on the desktop and sent Gerry some instructions on how to update the data, meaning he can continue to work on it and see how it looks. But my work on this is now complete for the time being.
I was supposed to meet with Paul Malgrati from Scottish Literature on Wednesday to discuss an interactive map of Burns Suppers he would like me to create. We decided to cancel our meeting due to the Coronavirus, but continued to communicate via email. Paul had sent me a spreadsheet containing data relating to the Burns Suppers and I spent some time working on some initial versions of the map, reusing some of the code from the Regional Romanticism map, which in turn used code from the SCOSYA map.
I migrated the spreadsheet to an online database and then wrote a script that exports this data in the JSON format that can be easily read into the map. The initial version uses OpenStreetMap.HOT as a basemap rather than the .DE one that Paul had selected as the latter displays all place-names in German where these are available (e.g. Großbritannien). The .HOT map is fairly similar, although for some reason parts of South America look like they’re underwater. We can easily change to an alternative basemap in future if required. In my initial version all locations are marked with red icons displaying a knife and fork. We can use other colours or icons to differentiate types if or when these are available. The map is full screen with an introductory panel in the top right. Hovering over an icon displays the title of the event while clicking on it replaces the introductory panel with a panel containing the information about the supper. The content is generated dynamically and only displays fields that contain data (e.g. very few include ‘Dress Code’). You can always return to the intro by clicking on the ‘Introduction’ button at the top.
I spotted a few issues with the latitude and longitude of some locations that will need fixed. E.g. St Petersburg has Russia as the country but it is positioned in St Petersburg in Florida while Bogota Burns night in Colombia is positioned in South Sudan. I also realised that we might want to think about grouping icons as when zoomed out it’s difficult to tell where there are multiple closely positioned icons – e.g. the two in Reykjavik and the two in Glasgow. However, grouping may be tricky if different locations are assigned different icons / types.
After further email discussions with Paul (and being sent a new version of the spreadsheet) I created an updated version of my initial map. This version incorporates the data from the spreadsheet and incorporates the new ‘Attendance’ field into the pop-up where applicable. It is also now possible to zoom further out, and also scroll past the international dateline and still see the data (in the previous version if you did this the data would not appear). I also integrated the Leaflet Plugin MarkerCluster (see https://github.com/Leaflet/Leaflet.markercluster) that very nicely handles clustering of markers. In this new version of my map markers are now grouped into clusters that split apart as you zoom in. I also added in an option to hide and show the pop-up area as on small screens (e.g. mobile phones) the area takes up a lot of space, and if you click on a marker that is already highlighted this now deselects the marker and closes the popup. Finally, I added a new ‘Filters’ section in the introduction that you can show or hide. This contains options to filter the data by period. The three periods are listed (all ‘on’ be default’) and you can deselect or select any of them. Doing so automatically updates the map to limit the markers to those that meet the criteria. This is ‘remembered’ as you click on other markers and you can update your criteria by returning to the introduction. I did wonder about adding a summary of the selected filters to the popup of every marker, but I think this will just add too much clutter, especially when viewing the map on smaller screens (these days most people access websites on tablets or phones). Here is an example of the map as it currently looks:
The main things left to do are adding more filters and adding in images and videos, but I’ll wait until Paul sends me more data before I do anything further. That’s all for this week. I’ll just need to see how work progresses over the next few weeks as with the schools now shut I’ll need to spent time looking after my son in addition to tackling my usual work.
This was another strike week, and I only worked on Friday. Next week will be a full five-day strike. On Friday I caught up with a few emails about future projects from Paul Malgrati and Sourit Bhattacharya, read through some of the documentation for the SCOSYA follow-on funding proposal that is currently in development and had a further email conversation with Heather Pagan regarding the redevelopment of the Anglo-Norman Dictionary. Other than that I focussed on the interactive map for Gerry McKeever’s Regional Romanticism project. Last week I’d imported the new data and made a number of changes to the map, but had discovered that the plugin I was using to give nice Bezier curve lines between markers was not scaling well, and was making the map unusably slow. This week I experimented a bit more with the plugin, trying to find a more efficient means of making the lines appear, but although the alternative I came up with was slightly less inefficient it was still pretty much unusable. I therefore began looking into alternative libraries. There is another called leaflet.curve (https://github.com/elfalem/Leaflet.curve) that is looking promising, but the only downside is you need to specify a latitude and longitude for the midpoint of the curve in addition to the start and end points. I didn’t want to do this for all 80-odd locations without seeing if the result would be usable so instead I created a test map that uses the same midpoint for all lines, which you can view below:
Even with all of the lines displayed the map loads without any noticeable lag, so I’d say this is looking promising. After doing this I looked at the first plugin I’d used again and realised that it included a function that automatically calculates the latitude and longitude of the midpoint of a curve between two lat/lon pairings that are passed to it. I wondered whether I could just rip this function out of the first plugin and incorporate it into the second to avoid having to manually work out where the midpoint of each line should go. I wasn’t sure if this would work but it looks like it has, as you can see here:
This second map does seem to have the occasional performance issues and I’ll need to test it out more fully, but it is a considerable improvement on the original map. If it does prove to be too laggy once I update the actual interface I can still use the function to output the midpoint values and save them as static lat/lon values. I’m still going to have to change quite a bit of the map code to replace the first plugin with this new version, and I didn’t have time to do so this week, but I’d say things are looking very promising.
It was another strike week, and this time I only worked on Thursday and Friday. I spent a lot of Thursday catching up with emails and thinking about the technical implications of potential projects that people had contacted me about. I responded to a PhD student who had asked me for advice about a Wellcome Trust application and I spent quite a bit of time going through the existing Anglo-Norman Dictionary website to get a better understanding of how it works, the features it offers and how it could be improved. I’d also been contacted by two members of Critical Studies staff who wanted some technical work doing. The first was Paul Malgrati, who is wanting to put together an interactive map of Burns Suppers across the world. He’d sent me a spreadsheet containing the data he’s so far compiled and I spent some time going through this and replying to his email with some suggestions. The second was Sourit Bhattacharya, who is submitting a Carnegie application to develop an online bibliography. I considered his requirements and replied to him with some ideas.
I spent most of Friday working on the interactive map that plots important locations in a novel for Gerry McKeever’s Regional Romanticism project. I created an initial version of this back in January and since then Gerry has been working on his data, extending it to cover all three volumes of the novel and locations across the globe. He’d also made some changes to the structure of the data (e.g. page numbers had been separated out from the extracts to a new column in the spreadsheet) and had greatly enhanced the annotations made for each item. He had also written some new introductory text. The new data consisted of 120 entries across 88 distinct locations, and I needed to convert this from an Excel spreadsheet into a JSON file, with locations separated out and linked to multiple entries to enable map markers to be associated with multiple items. It took several hours to do this. I did consider writing a script to handle to conversion but that would have taken some time in itself and this is the last time the entire dataset will be migrated so the script would never be needed again. Plus undertaking the task manually gave me the opportunity to check the data and to ensure that the same locations mentioned in different entries all matched up.
I also created a new version of the interactive map that used the new data and incorporated some other updates too. This version has the new intro slide and hopefully has all of the updates to the existing data plus all of the new data for all three volumes. I changed the display slightly to include page numbers as a separate field (the information now appears in the ‘volume’ section) and also to include the notes. To differentiate these from the extracts I’ve enclosed the extracts in large speech marks.
One issue I encountered is that with all of the new data in place the loading of new items and panning to them slowed to a crawl. The map unfortunately became absolutely unusable. After some investigation I realised that the plugin that makes the nice curved lines between locations was to blame. It appears to not be at all scalable and I had to disable the grey dotted lines on the map as with all of the markers visible the map was unusable. I may be able to fix this, or I may have to switch to an alternative plugin for the lines as the one that’s currently used appears to be horribly inefficient and unscalable. However, the yellow line connecting the current marker with the previous one is still visible as you scroll around the locations and I think this is the most important thing. Below is a screenshot of the map as it currently stands. Next week I will be continuing with the UCU strike action and will therefore only be working on the Friday.
