Week Beginning 2nd September 2024

I was off work sick on Monday and Tuesday this week, but thankfully I was feeling well again by Wednesday.  On Wednesday I had a meeting with Katie Halsey and Matt Sangster about an article we’re going to write about the Books and Borrowing online resource.  We now have an overview of what we’re going to write about and will try to have a first draft written by the end of the month.  I also had to spend a bit of time investigating some speed issues with the Books and Borrowing website.  One thing that may have been contributing to this was a call to the API that isn’t used in the front-end but was being submitted by a web crawler.  The call was returning details for all library registers in the system, resulting in a query that was taking around 13 seconds to execute.  The endpoint is only ever used in the front-end with a library specified, which is a much quicker query and I therefore blocked the endpoint from running if a library isn’t specified.

On Thursday I had a  meeting with the Rangar’s Islands project (https://blogs.nottingham.ac.uk/ragnasislands/).  The project is based at Nottingham and they’re putting together a place-name database for three of the Orkney Islands.  They’d like to use the system I’ve created for other place-name projects such as Iona and we met to discuss how this might come about.  The project will be applying for funding to develop the front-end and all being well I’ll be involved in setting this up.

Also this week I made some further tweaks to the new date search facilities of the Dictionaries of the Scots Language.  The team wanted the date range that is visible to the left of the sparklines in the search results to default to the limited range reflected in the sparkline text rather than the actual first and last dates of attestation.  So for example a DOST entry with dates from 1175-1780 would not show these dates but would instead display ‘<1375-1700+’.  Implementing this wasn’t as straightforward as I’d hoped it might be.  The Solr index contains the first and last dates of attestation, and it’s these that are used to display the dates.  However, I couldn’t update them to something like ‘<1375’ or ‘1700+’ as these dates are used in the filter options and need to be valid integers.  I did consider updating the Solr index to store the amended first and last dates as new fields, but this would have meant updating the structure of the Solr indexes, plus another regeneration of the Solr data.  Instead the results script picks out the first and last dates from the sparkline text on the fly, taking into consideration any ‘<’ in the first date, ‘+’ in the last date and ensuring only one date is displayed if ‘from’ and ‘to’ are the same.

I’d also spotted a bug in the search when a phrase was searched for that wasn’t surrounded by double quotes.  This was bringing back some very garbled results.  To fix this I needed to update the search results page to ensure such searches were analysed and surrounded by double quotes if these were needed prior to sending the request to the API.  Implementing this was also rather trickier to fix than I’d hoped.  I couldn’t just add quotes if the query had a space as I also needed to check for Booleans.  If these are present then we definitely don’t want quotes to be added.  Also the query string includes all of the other bits that appear after the phrase (i.e. /quotes/full/both/full) so I couldn’t just surround the entire thing in quotes but instead needed to extract just the phrase, add quotes then add it back to the whole string.  However, I managed to get it working and the results returned now make more sense.

Also this week I had email conversations with Joanna Kopaczyk and Gerry Carruthers about new (separate) proposals they are putting together that I will be a part of and arranged to meet Garrick Allen to discuss a proposal he is developing.  I also replaced a couple of videos on the various Speech Star websites at the request of Eleanor Lawson.

I spent the remainder of the week working on the Burns Correspondence map, mostly on the API and queries and planning on how the map interface will function, specifically the lines joining locations.  Displaying all of the lines swamps the map and is just too much information to display at once, so I’m going to have to think about how best to proceed.  The screenshot below demonstrates just how busy the map gets with all of the lines on:

I’ll continue to think about this next week.