Week Beginning 20th November 2023

I spent most of this week working towards adding genre to the Books and Borrowing front-end, working on a version running on my laptop.  My initial task was to update the Solr index to add in additional fields for genre.  With the new fields added I then had to update my script that generates the data for Solr to incorporate the fields.  The Solr index is of borrowing records so as with authors, I needed to extract all genre associations at all book levels (work, edition, holding, item) for each book that was associated with a borrowing record, ensuring lower level associations replaced any higher level associations and removing any duplicates.  This is all academic for now as all genre associations are at Work level, but this may not always be the case.  It took a few attempts to get the data just right (e.g. after one export I realised it would be good to have genre IDs in the index as well as their names) and each run-through took about an hour or so to process, but all is looking good now.  I’ll need to ask Stirling IT to create a new Solr core and ingest the new data on the server at Stirling as this is not something I have the access to do myself, and I’ll do this next week.  The screenshot below shows one of the records in Solr with the new genre fields present.

With Solr updated I then began updating the front-end, in a version of the site running on my laptop.  This required making significant updates to the API that generates all of the data for the front-end by connecting to both Solr and the database as well as updating the actual output to ensure genre is displayed.   I updated the Advanced Search forms (simple and advanced) to add in a list of genres from which you can select any you’re interested in (see the following two screenshots) and updated the search facilities to enable the selected genres to be searched, either on their own or in combination with the other search options.

On the search results page any genres associated with a matching record are displayed, with associations at higher book levels cascading down to lower book levels (unless the lower book level has its own genre records).  Genres appear in the records as clickable items, allowing you to perform a search for a genre you’re interested in by clicking on it.  I’ve also added in genre as a filter option down the left of the results page.  Any genres present in the results are listed, together with a count of the number of associated records, and you can filter the results by pressing on a genre, as you can see in the following screenshot, which shows the results of a quick search for ‘Egypt’, displaying the genre filter options and showing the appearance of genre in the records.

Genre is displayed in a similar way wherever book records appear elsewhere in the site, for example the lists of books for a library, the top-level ‘book editions’ page and when viewing a specific page in a library register.

There is still more to be done with genre, which I’ll continue with next week.  This includes adding in new visualisations for genre, adding in new ‘facts and figures’ relating to genre and adding in facilities to limit the ‘browse books’ pages to specific genres.  I’ll keep you posted next week.

I also spent some time going through the API and front-end fixing any notifications and warnings given by the PHP scripting language.  These are not errors as such, just messages that PHP logs when it thinks there might be an issue, for example if a variable is referenced without it being explicitly instantiated first.  These messages get added to a log file and are never publicly displayed (unless the server is set to display them) but it’s better to address them to avoid cluttering up the log files so I’ve (hopefully) sorted them all now. Also for the project this week I generated a list of all book editions that currently have no associated book work.  There are currently 2474 of these and they will need to be investigated by the team.

I also met with Luca Guariento and Stevie Barret to have a catch-up and also to compile a list of key responsibilities for a server administrator who would manage the Arts servers.  We discovered this week that Arts IT Support is no longer continuing, with all support being moved to central IT Services.  We still have our own servers and require someone to manage them so hopefully our list will be taken into consideration and we will be kept informed of any future developments.

Also this week I created a new blog for a project Gavin Miller is setting up, fixed an issue that took down every dictionary entry in the Anglo-Norman Dictionary (caused by one of the project staff adding an invalid ID to the system) and completed the migration of the old Arts server to our third-party supplier.

I also investigated an issue with the Place-names of Mull and Ulva CMS that was causing source details to be wiped.  The script that populates the source fields when an existing source is selected from the autocomplete list was failing to load in data.  This meant that all other fields for the source were left blank, so when the ‘Add’ button was pressed the script assumed the user wanted all of the other fields to be blank and therefore wiped them.  This situation was only happening very infrequently and what I reckon happened is that the data for the source that failed included a character that is not permitted in JSON data (maybe a double quote or a tab), meaning when the script tried to grab the data it failed to parse it and silently failed to populate the required fields.  I therefore updated the script that returns the source fields so that double quotes and tab characters are stripped out of the fields before the data is returned.  I also created a script based on this that outputs all sources as JSON data to check for errors and thankfully the output is valid JSON.

I also made a couple of minor tweaks to the Dictionaries of the Scots Language site, fixing an issue with the display of the advanced search results that had been introduced when I updated the code prior to the site’s recent migration to a new server and updating the wording of the ‘About this entry’ box.  I also had an email conversation with Craig Lamont about a potential new project and spoke to Clara Cohen about a project she’s putting together.