This week I completed work on a first version of the regions and dialect areas map for the Dictionaries of the Scots Language. Last week I’d begun work on the final aspect of the map: ensuring that whatever the user is viewing on the map is reflected in the hash to enable specific views of the map to be bookmarked, shared and cited. I’d begun updating the scripts to process details held in the hash and I completed this by adding the areas selected in from the info pane this week. This was slightly more complicated than I’d imagined as I needed to consider how to handle areas that are selected by default (e.g. all regions selected when the user clicks on the map) in addition to areas specifically turned on or off by the user from the checkboxes in the info pane. I decided to ensure that the default selected areas were also present in the hash to keep things simple.
With the hash processing in place I then needed to update the code to actually add the necessary information to the hash as the user interacts with the map. This was a fairly big job, involving updating much of the code, but I got there in the end. After a fair amount of testing and tweaking I was then able to send the URL for the map to the DSL people for them to test out and give feedback on – I’ll just need to see what changes may be required and also when we might go live with the map – probably not for a while yet I’d imagine.
Also for the DSL this week I had an online meeting to discuss the new entry XML structure. We went through the list of questions I’d come up with whilst working through the new XML examples a few weeks ago and had a good discussion about what would be included in the update and when I’ll start work on this. I should receive a test export of the dataset in the next week or so and once I have this I’ll be able to start implementing the necessary changes to the systems. I also investigated an issue with email lists for the DSL that Rhona contacted me about and fixed a minor layout issue with the live site.
Another sizeable task that I was able to tick off this week was finally launching the new Books and Borrowing versioning system. I’d encountered a strange issue with the online database last week that had prevented me from continuing with the rollout, but thankfully my contact in Stirling’s IT services was back at work on Tuesday and managed to identify the issue. The situation is that the online database’s data is replicated across three nodes. But this only works with database tables created in the ‘InnoDB’ format. I’d created the new tables on my laptop, which had defaulted to the ‘MyISAM’ format, something I’d not spotted. When I updated the online database these new tables were then stored in the ‘MyISAM’ format and were therefore not getting replicated. I wasn’t aware of this distinction between database tables and the implications this had for how the online data was stored – these formats refer to how the data is stored on disk and there is no difference when you view or query the tables and data through the database.
Thankfully once the issue had been identified I was then able to re-create the tables in the ‘InnoDB’ format and regenerate the data, after which all data magically appeared in both the PHPMyAdmin interface and through my scripts. With this in place I then completed the rollout of the new versioning system and the new ‘v2’ version of the API, which as far as I can tell has all been successful. As of yet there’s not really much different as we still only have version 1.0 of the data available. What you will see in the footer of every page is some text about the database version and a link to the ‘versions and changes’ page (https://borrowing.stir.ac.uk/versions-and-changes/). Also, when you perform a search and choose to cite the page the data version now appears in the citation and also the URL (so that when we do publish new versions the citation remains accurate). E.g.
“Quick search for ‘rome’. 2025. In Books and Borrowing: An Analysis of Scottish Borrowers’ Registers, 1750-1830, data version 1.0. University of Stirling. Retrieved 16 September 2025, from https://borrowing.stir.ac.uk/search/version-1.0/p-1/0/0/quick/rome”.
The URL with ‘version-1.0’ in it works in exactly the same way as the general URL for now. When we release ‘version-1.1’ the URL above will load the version 1 data with a warning that the user isn’t looking at the most up to date version. The new API is now available here: https://borrowing.stir.ac.uk/api/v2/ and I’ve updated the ‘About the database’ and ‘FAQs’ pages to point to this. I also sorted the encoded apostrophe issue and book works now appear in the filter options and ESTC has been removed. I spotted an issue with book works that had ampersands in them breaking the site when selected as a filter but I’ve managed to fix this, for example:
So after many months of work that’s the new versioning system complete. Hopefully when we come to launch version 1.1 the process will be a little more straightforward!
Also this week I had a discussion with Geert of the Anglo-Norman Dictionary about processing the data in preparation for adding in a new part of speech option to the searches, and I also fixed an issue with the management system that was preventing certain cross-reference entries from being uploaded.
I also gave some feedback to Matthew Creasy about a proposal he’s submitting soon and had a discussion with Jennifer Smith about sharing the SCOSYA data with the Mozilla Foundation. I also made a bit of progress with the arrangements for the new Place-names of Armagh project.
On Friday I had a bit of spare time that I devoted to the migration of the Place-names of Fife project to our standardised system. I realised that many of the historical forms had not been successfully processed by the script that had extracted data from the original Word files, resulting in several hundred forms that had no data in the historical form name field and everything in the ‘source’ field. I sorted these by a combination of batch processing and manual updating, and also sorted an issue with notes not appearing in the correct field either. I think I’m just about ready to migrate the historical forms now, although much work will still need to be done to extract sources.