Week Beginning 22nd January 2024

I’d hoped that we’d be able to go live with the full Books and Borrowing site this week, but unfortunately there are still some final tweaks to be made so we’re not live yet.  I did manage to get everything ready to go, but then some final checking by the team uncovered some additional issues that needed sorting.

Migrating all of the pages in the development site to their final URLs proved to be quite challenging as it was not simply a matter of copying the pages.  Instead I needed to amalgamate three different sets of scripts and stylesheets (the WordPress site, the dev site and the Chambers map), which involved some rewriting and quite a lot of checking.  However, by the end of Monday I had completed the process and I sent the URLs to the team for a final round of checking.

The team uncovered a few issues that I managed to address, including some problems with the encoding of ampersands and books and borrowers without associated dates not getting returned in the site-wide browse options.  I needed to regenerate the cache files after sorting some of these, which also took a bit of time.

I also realised that the default view of the ‘Browse books’ page was taking fat too long to load, with load times of more than 20 seconds.  I therefore decided to create a cache for the standard view of books, with separate cache files for books by first letter of their title and their author.  These caches would then be used whenever such lists were requested rather than querying everything each time.  When filters are applied (e.g. date ranges, genres) the cache would be ignored, but such filtered views generally bring back less books and are returned quicker anyway.  It took a while to write a script to generate the cache, to update the API to use the cache and then add the cache generation script to the documentation, but once all was in place the page was much quicker to load.  It does still take a few seconds to load in as there are almost 3000 books beginning with the letter ‘A’ in our system and even processing a static file containing this much data takes time.  But it’s a marked improvement.

On Friday afternoon the project PI Katie Halsey got back to me with a list of further updates that I need to make before the launch, one of which will require the Solr index structure to be updated and all of the Solr data to be regenerated, which I’ll tackle next week.  Hopefully after that we’ll finally be able to launch the site.

Also this week I fixed a small bug that had been introduced to the DSL sites when we migrated to a new server before Christmas.  This was causing an error when the ‘clear results’ option was selected when viewing the entry page and I’ve sorted it now.  I also investigated an issue then including URLs in the description of place-names on the Ayrshire place-names site.  It turned out that relative URLs were getting blocked as a security risk, but a switch to absolute URLs sorted this.

I also made some fixes to some of the data stored in the Speech Star databases that had been uploaded with certain features missing and provided some CV-like information about the projects I’ve worked on for Clara Cohen in English Language to include in a proposal I’m involved with.

I then investigated a couple of issues with the Anglo-Norman Dictionary.  The first was that the Editor Geert needed a new ‘usage label’ to the XML for the site, which would then need to be searchable in the front-end.  After spending some time refamiliarising myself with how things worked I realised it should be very easy to add new labels.  All I needed to do was to add the label to the database and then it was automatically picked up by the DTD and given as an option in Oxygen.  Then when an entry is uploaded to our dictionary management system it will be processed and should (hopefully) automatically appear in the advanced search label options (once it has at least one active association to an entry).  Geert has begun using the new label and we’ll see if the process works as I think it will.

The second issue was one that Geert had noticed in the Dictionary Management System when manually editing the citations.  The ‘Edit citations’ feature allows you to select a source and then view every time this source is cited in all of the entries.  The list then lets you manually edit each of these.  The issue was cropping up when an entry featured more than one citation for a source and more than one of these needed to be edited.  In such cases some of the edits were being lost.  The problem was that when multiple citations were being edited at the same time the code was extracting the XML from the database and editing it afresh for each one, meaning only the changes for the last were being stored.  I’ve sorted this now.

I also spent some time continuing to develop the new place-names map for the Iona project.  I updated the elements glossary to add a new hidden column to the data that stores the plain alphanumeric value of the name (i.e. all weird characters removed).  This is now used for ordering purposes, meaning any elements beginning with a character such as an asterisk or a bracket now appear in the correct place.

I also began working on the ‘Table view’ and for now I’ve set this to appear in a pop-up, as with the place-name record.  Pressing on the ‘Table view’ button brings up the table, which contains all of the data currently displayed on the map.  You can reorder the table by any of the headings by pressing on them (press a second time to reverse the order).  Below is a screenshot demonstrating how this currently looks:

You can also press on a place-name to close the table and open the place-name’s full record.  The map also pans and zooms to the relevant position.  What I haven’t done yet is managed to get the marker pop-up to also appear, so it’s currently still rather difficult to tell which marker is the one you’re interested in.  I’m working on this, but it’s surprisingly difficult to achieve due to the leaflet mapping library not allowing you to assign IDs to markers.

Using the pop-up for the table is good in some ways as it keeps everything within the map interface, but it does have some issues.  I’ve had to make the pop-up wider to accommodate the table, so it’s not going to work great on mobile phones.  Also when the table is very long and you try to use the browser’s scrollbar the pop-up closes, which is pretty annoying.  I think I might have found a solution to this (adding a scrollbar to the pop-up content itself), but I haven’t had time to implement it yet.  An alternative may be to just have the table opening in a new page, but it seems a shame to not have everything contained in the map.

Finally this week I spent some time working through the poems on the old ‘Anthology of 16th and Early 17th Century Scots Poetry’ site.  I created a new interface for this last week (which is not yet live) and decided it would make sense to migrate the poems from ancient HTML to TEI XML.  This week I began this process, manually converting the firs 13 poems (out of about 130) to XML.  With these in place I then worked on an XSLT stylesheet that would transform this XML into the required HTML.  It was here that I ran into an issue.  I absolutely hate working with XSLT – it’s really old fashioned and brittle and cumbersome to use.  Things I can literally achieve in seconds for HTML using jQuery can take half a day of struggling with XSLT.  This time I couldn’t even get the XSLT script to output anything.  I was using similar code to that which I’d created for the DSL and AND, but nothing was happening.  I spent hours trying to get the XSLT to match any element in the XML but nothing I tried worked.  Eventually I asked Luca, who has had more experience with XML than I have had, for advice and thankfully he managed to identify the issue.  I hadn’t included a namespace for TEI in the XSLT file.  I needed to include the following reference:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:tei="http://www.tei-c.org/ns/1.0">

And then each time I wanted to match an XML element such as <text> I couldn’t just use ‘text’ but instead needed to add a tei prefix ‘tei:text’.  Without this the XSLT wouldn’t do anything.  Yep, XSLT is brittle and cumbersome and an absolute pain to use.  But with Luca’s help I managed to get it working and other than transcribing the remaining texts (a big job I’ll tackle bit by bit over many weeks) all is pretty much in place.


Week Beginning 15th January 2024

I received an email from Theo Van Heijnsberg in Scottish Literature over the weekend alerting me to the fact that one of the old STELLA resources that is still used for teaching was no longer available.  At the end of last year we’d shut down the server that was hosting the old STELLA resources, and at the time we thought we’d migrated all of the resources that were still being used but I guess this one was overlooked.  We had of course taken backups of the server before we shut it down so it was relatively easy to find and restore the resource, especially as it was an old plain HTML website that was last updated way back in 1999.  The website (An Anthology of 16th and Early 17th Century Scots Poetry) can now be found here: https://stella.glasgow.ac.uk/anthol/title.html.

As the website needs to continue to be hosted I decided that I should probably update the interface to bring it into the 21st century.  I therefore spent a bit of time creating a Bootstrap based interface, which is fairly unadorned and retains the logo.  I’ve added in a footer with links that University websites are now required to feature.  There is also a ‘Top’ button that appears as you scroll down the page, allowing you to quickly return to the top.  The homepage features two columns, one with the contents and the other containing the ‘Editors’ text.  So far I’ve only added content to the ‘Alexander Scott’ page.  This currently features the list of poems and the first two poems.  Pressing on the title of one of these in the list scrolls the page to the relevant point (you can use the ‘Top’ button to return to the list).  There is a button for returning to the contents page at the top (pressing on the site title or logo does the same).  Here’s a screenshot of how the new site currently looks:

The biggest issue is that the poems are all marked up in very old HTML using tables for layout and tags that were find in 1999 but are not great to use now.  In order to extract the text I need to copy and paste each individual line, line number and gloss into a new HTML structure.  After doing this for two poems I decided that instead of doing this it would be better to migrate the text to TEI XML and to store the data separately from the presentation.  That way it will be much easier to reuse the poems in future.  I would then write a script to transform the XML into the required HTML for display on the site.  However, after deciding this Theo got back to me to say that another project at St. Andrews is going to use the data from the resource and incorporate it into a larger online resource.  This project is still just at the planning stages, but it’s likely the Anthology website will not be needed for much longer.  I’ll probably still explore the XML pathway, though, as it’s a useful exercise to hone my XML skills.  It will take me some time to go through more than 100 poems, though, and I’ll need to do this between my commitments to funded research projects.

Also this week I did some further work for the Books and Borrowing project.  I fixed an image that wasn’t working in one of the Royal High School registers and reassigned images to one of the Orkney registers due to a duplicate image throwing everything from that point onwards off by one.  I also updated the website interface to ensure that tables work better on narrow screens.  I applied a minimum width of 800px to tables (e.g. the tabular list of libraries and the lists or registers), meaning on very narrow screens the tables no longer get squashed into spaces that make them entirely unusable but instead extend beyond the width of the screen and you need to scroll to view their full contents.  It seems to work ok on my rather ancient phone.

On Wednesday we were finally ready to generate the final pre-publication cache files, so I completed this task – updating all of the caches and generating a new Solr index file.  Once this had been set up for us by Stirling’s IT people I then updated the connection details in the API.  Also in this update I fixed an issue with the place of publication in the advanced search.  Previously if you typed in ‘london’ (lower case) and didn’t select ‘London’ form the drop-down list and then pressed the ‘search’ button you found no results as publication place was case sensitive.  The correct results are now returned.  I wasn’t able to ‘go live’ with the new site this week, however, due to other commitments.  I’ll start on this first thing next week.

On Thursday this week I attended a staff meeting in Edinburgh for the DSL.  Several new members of staff have joined the DSL recently and a meeting was arranged so everyone could get together, which is quite a rare occurrence, especially as the DSL do not have offices since the Lockdown happened.  I gave a little talk about the DSL’s website and my role in the DSL, and gave a brief history of the development of the DSL’s website.  It will be ten years this September since we launched the modern DSL website, which is quite a milestone.  It was a really great opportunity to meet everyone and we had a lovely meal out afterwards in a private dining room at the Scottish Arts Club.

I also continued with the new map of Iona place-names this week, tiocking some more items of my ‘to do’ list.  I made some further updates to the map legend.  When categorising markers by code or element language the marker colours are now fixed.  I’ve gone with shades of purple for ‘Gaelic’ and shades of blue for ‘SSE’ in the languages while in the classification codes coastal features are sandy colours, water is blue, vegetation is green etc.

I also worked on the ordering of the items in the legends for classification codes and languages.  Previously the order was pretty arbitrary and depended on the order the items were found in the results.  Now the groupings are ordered alphabetically, except for ‘Gaelic’ groups, which appear first.  I’m afraid ‘en’ appears last, though, as ordering is case sensitive.  I did attempt to negate this, but this caused other problems so I’ve kept it as it is.  I also ensured that ‘Gaelic’ and ‘SSE’ always come first in the order of languages (e.g. ‘Gaelic & en’ and not ‘en & Gaelic’) and I added tooltips for language groupings that need them.  Here’s a screenshot showing the new legend and colours:

I also added in the ‘Cite’ text to the record, so when opening a full record if you select the ‘cite this record’ tab you’ll see the citation options.  I also added in the citation option for the map as a whole (the ‘cite’ button in the left-hand menu).  This has taken some time to implement as the citation text needed to include descriptive text to reflect the options that have been selected, including the base map, the classification type and whether the map is displaying a browse, a quick search or an advanced search, plus which exact options have been selected.

So for example when browsing for all current names starting with ‘S’ using the modern OS map, categorised by language and you press on the ‘Cite’ button the APA style of citation is formatted to display the following:

Map of Iona’s place-names: browse for place-names starting with ‘S’, classified by language, base map: modern OS. 2024. In Iona’s Namescape. Glasgow: University of Glasgow. Retrieved 19 January 2024, from [Map URL here].

Next week I’ll begin on the tabular view of the data, as well as going live with the Books and Borrowing site.

Week Beginning 8th January 2024

This was my first week back after the Christmas holidays and after catching up with emails I spent the best part of two days fixing the content management system of one of the resources that had been migrated during the end of last year.   The Saints Places resource (https://saintsplaces.gla.ac.uk/) is not one I created but I’ve taken on responsibility for it due to my involvement with other place-names resources.  The front-end was migrated by Luca and was working perfectly, but he hadn’t touched the CMS, which is understandable given that the project launched more than ten years ago.  However, I was contacted during the holidays by one of the project team who said that the resource is still regularly updated and therefore I needed to get the CMS up and running again.  This required updates to database query calls and session management and it took quite some time to update and test everything.  I also lost an hour or so with a script that was failing to initiate a session, even though the session start code looked identical to other scripts that worked.  It turned out that this was due to the character encoding of the script, which had been set to UTF-8 BOM, which meant that hidden characters were being outputted to the browser by PHP before the session was instantiated, which then made the session fail.  Thankfully once I realised this it was straightforward to convert the script from UTF-8 BOM to regular UTF-8, which solved the problem.

With this unexpected task out of the way I then returned to my work on the new map interface for the Place-names of Iona project, working through the ‘to do’ list I’d created after our last project meeting just before Christmas.  I updated the map legend filter list to add in a ‘select all’ option.  This took some time to implement but I think it will be really useful.  You can now deselect the ‘select all’ to be left with an empty map, allowing you to start adding in the data you’re interested in rather than having to manually remove all of the uninteresting categories.  You can also reselect ‘select all’ to add everything back in again.

I did a bit of work on the altitude search, making it possible to search for an altitude of zero (either on its own or with a range starting at zero such as ‘0-10’).  This was not previously working as zero was being treated as empty, meaning the search didn’t run.  I’ve also fixed an issue with the display of place-names with a zero altitude – previously these displayed an altitude of ‘nullm’ but they now display ‘0m’.  I also updated the altitude filter groups to make them more fine-grained and updated the colours to make them more varied rather than the shades of green we previously had.  Now 0-24m is a sandy yellow, 25-49, is light green, 50-74m is dark green, 75-99 is brown and anything over 99 is dark grey (currently no matching data).

I also made the satellite view the default map tileset, with the previous default moved to third in the list and labelled ‘Relief’.  This proved to be trickier to update than I thought it would be (e.g. pressing the ‘reset map’ button was still loading the old default even though it shouldn’t have) but I managed to get it sorted.  I also updated the map popups so they have a white background and a blue header to match the look of the full record and removed all references to Landranger maps in the popup as these were not relevant.  Below is a screenshot showing these changes:

I then moved onto the development of the elements glossary, which I completed this week.  This can now be accessed from the ‘Element glossary’ menu item and opens in a pop-up the same as the advanced search and the record.  By default elements across all languages are loaded but you can select a specific language from the drop-down list.  It’s also possible to cite or bookmark a specific view of the glossary, which will load the map with the glossary open at the required place.

I’ve tried to make better use of space than similar pages on the old place-names sites by using three columns.  The place-name elements are links and pressing on one performs a search for the element in question.  I also updated the full record popup to link the elements listed in it to the search results.  I had intended to link to the glossary rather than the search results, which is what happens in the other place-names sites, but I thought it would be more useful and less confusing to link directly to the search results instead.  Below is a screenshot showing the glossary open and displaying elements in Scottish Standard English:

I also think I’ve sorted out the issue with in-record links not working as they should in Chrome and other issues involving bar characters.  I’ve done quite a bit of testing with Chrome and all seems fine to me, but I’ll need to wait an see if other members of the team encounter any issues.  I also added in the ‘translation’ field to the popup and full record, although there are only a few records that currently have this field populated, relabelled the historical OS maps and fixed a bug in the CMS that was resulting in multiple ampersands being generated when an ampersand was used in certain fields.

My final update for the project this week was to change the historical forms in the full record to hide the source information by default.  You now need to press a ‘show sources’ checkbox above the historical forms to turn these on.  I think having the sources turned off really helps to make the historical forms easier to understand.

I also spent a bit of time this week on the Books and Borrowing project, including participating in a project team Zoom call on Monday.  I had thought that we’d be ready for a final cache generation and the launch of the full website this week, but the team are still making final tweaks to the data and this had therefore been pushed back to Wednesday next week.  But this week I updated the ‘genre through time’ visualisation as it turned out that the query that returned the number of borrowing records per genre per year wasn’t quite right and this was giving somewhat inflated figures, which I managed to resolve.  I also created records for the first volume of the Leighton Library Minute Books.  There will be three such volumes in total, all of which will feature digitised images only (no transcriptions).  I processed the images and generated page records for the first volume and will tackle the other two once the images are ready.

Also this week I made a few visual tweaks to the Erskine project website (https://erskine.glasgow.ac.uk/) and I fixed a misplaced map marker in the Place-names of Berwickshire resource (https://berwickshire-placenames.glasgow.ac.uk/).  For some reason the longitude was incorrect for the place-name, even though the latitude was fine, which resulted in the marker displaying in Wales.  I also fixed a couple of issues with the Old English Thesaurus for Jane Roberts and responded to a query from Jennifer Smith regarding the Speak For Yersel resource.

Finally, I investigated an issue with the Anglo-Norman Dictionary.  An entry was displaying what appeared to be an erroneous first date so I investigated what was going on.  The earliest date for the entry was being generated from this attestation:

<attestation id="C-e055cdb1"><dateInfo>

<text_date post="1390" pre="1314" cert="">1390-1412</text_date>

<ms_date post="1400" pre="1449" cert="">s.xv<sup>1</sup></ms_date>


<quotation>luy donantz aussi congié et eleccion d’estudier en divinitee ou en loy canoun a son plesir, et ce le plus favorablement a cause de nous</quotation>

<reference><source siglum="Lett_and_Pet" target=""><loc>412.19</loc></source></reference>


Specifically the text date:

<text_date post="1390" pre="1314" cert="">1390-1412</text_date>

This particular attestation was being picked as the earliest due to a typo in the ‘pre’ date which is 1314 when it should be 1412.  Where there is a range of dates the code generates a single year at the midpoint that is used as a hidden first date for ordering purposes (this was agreed upon back when we were first adding in first dates of attestation).  The code to do this subtracts the ‘post’ date from the ‘pre’ date, divides this in two and then adds it to the ‘post’ date, which finds the middle point.  With the typo the code therefore subtracts 1390 from 1314, giving -76.  This is divided in two giving -38.  This is then added onto the ‘post’ date of 1390, which gives 1352.  1352 is the earliest date for any of the entry’s attestations and therefore the earliest display date is set to ‘1390-1412’.  Fixing the typo in the XML and processing the file would therefore rectify the issue.