Week Beginning 13th November 2023

This week I began the major task of integrating book genre with the Books and Borrowing dataset.  The team had been working on a spreadsheet that enabled them to assign top-level Book Work records to more than 13,000 Book Edition records and also assign up to three genres to each Work.  I had to write a script to parse this data, which involved extracting and storing the distinct genres, creating Book Work records, assigning Book Work authors, adding in associations to Book Edition records, deleting any author associations at Edition level and creating associations between Works and genres.  It took the best part of two days to create and test the script, running it on a local version of the data stored on my laptop.  After final testing the number of active Book Works increased from 75 to 9808 and the number of active Book Editions that have a Work association grew from 72 to 13,099.  The number of genre connections for Works stood at 11,536 and the number of active Book Works that have at least one author association stood at 9,808, up from 70, while the number of active Book Editions with at least one direct author association decreased to 2,191 from 14,384, due to the author association being shifted up to Work (and it will cascade from there).

With the data import sorted I then moved onto updating the project’s content management system to incorporate all facilities to add, edit, browse and delete genres.  This included creating facilities for associating genres with book records at any level (from Work down to Item) wherever books can be edited in the CMS.  The ‘Browse Genres’ page works in a similar way to ‘Browse Authors’, giving you a list of genres and a count of the number of each book at each level that has an association, as the following screenshot shows:

Pressing on a number opens a pop-up containing a list of the associated books and you can connect through to each book record from this.  As with authors, genre will cascade down from whichever level of book it is associated with to all lower levels.  You only need to make an association at a lower level if it differs from the genre at a higher level.  The counts in the ‘browse’ page show only the direct associations, so for now there are no editions or lower with any numbers listed.  Wherever a book at any level can be edited in the CMS a new ‘Genre’ section has been added to the edit form.  This consists of a list of genres with checkboxes beside them, as the following screenshot demonstrates:

You can tick as many checkboxes as are required and when updating the record the changes will be made.  I tested out the new genre features in the CMS and all seem to be working well.  I also imported all of the genre data so hopefully everything is now in place.  Next week I will move onto the front-end, where there is much to do – not only making genre visible wherever books are viewed but updating the search facilities and adding in a number of new visualisations for genre as well.  I also fixed a few issues with images of Registers from the Royal High School – a few that were missing I added in and the order of others needed to be updated.

Also this week I finalised the new project website for Rhona Brown’s new project.  It’s not live yet, but my work on it is now complete.  I was also involved in the migration of a number of my sites to a new server.  As always seems to be the case, the DSL website migration did not go very smoothly, with the DNS update taking a many hours to propagate and in the meantime the domain was serving up the Anglo-Norman Dictionary, which was not good at all.  This wasn’t something I had any direct control over, unfortunately, but thankfully the situation rectified itself the following day.

I also had to make a number of tweaks to the data in the Child speech error database for Speech Star, after many transcriptions were revised.  I also updated the Mull / Ulva place-names CMS to add in a facility to export place-names for publication limited by one or more selected islands.  In addition I began creating a new website for a project Gavin Miller is running and I created some new flat spreadsheet exports of the Historical Thesaurus for Fraser Dallachy and Marc Alexander to work with.