I worked on many different projects this week, but the one I spent the most time on was the Dictionaries of the Scots Language. I’ve not done much work for the DSL since the intensive period I spent developing the new website interface and deploying it on our test server ahead of the face-to-face meeting in mid-November. I had a list of further updates I needed to make following on from this meeting, but I needed to work on other projects since then and hadn’t got around to it. I’d also received a number of emails about changes to the presentation of entries reflecting the structural changes to the entry XML that I’d put to one side.
On Wednesday I had an online call scheduled with the DSL team to discuss the new front-end and it seemed like a good opportunity to get back to grips with all of my outstanding DSL tasks. This mainly involved making updates to the XSLT on our test server to tweak the layout of various items in the new entry XML structure, such as adding commas between tags when they are rendered, ensuring certain tags or attributes that weren’t getting rendered before appeared in the generated HTML, updating the styles of certain elements like the content warning labels, fixing a few bugs such as the ‘sticky’ heading not displaying in certain circumstances. There were at least 20 such items that needed investigating, fixing and testing, so this took quite some time to work through, but I managed to complete it all during the course of the week.
The meeting itself was very useful and as always it was good to catch up with some of the other DSL team members. There’s going to be a big push towards getting the new website interface ready for publication this year, and I’m obviously going to be involved in this process. I already have a number of items I need to sort out with the new interface and I’ll try and get started on these over the coming weeks.
I also spent a bit of time this week working for the Anglo-Norman Dictionary, investigating a strange occurrence with the publication of updates to entries, which turned out to be a user rather than a system issue, reinstating the links out from entries to the DMF dictionary, as their website is now properly back online again, and tweaking the wording of the quick search and ‘jump to entry’ text throughout the site.
I also did small amounts of work for several other projects, such as updating the licensing statements across the Seeing Speech and Star sites, fixing an issue with the ‘download song’ facilities on the Editing Robert Burns site, sorting an issue with the HiMuJe Malabar site, submitting my expenses from the Zurich workshop, exporting some SCOSYA data for Jennifer Smith, helping to sort out an issue with the Helsinki Corpus, and having a conversation with Clara Cohen about a new proposal she’s putting together. I also made some further updates to the VARICS look-up system, adding in some introductory text, some further references, and reworking the measurement processing so that when a red or amber result is given new textual sections about what this means and what the next steps should be appear underneath in collapsible accordion sections.
Also this week I had a meeting with the Burns Supper Map team to discuss the data that is now coming in and how and when I should start working on a new interactive map to visualise it. I also put in a request for a new subdomain for the project that was set up by the end of the week. Next week I’ll probably write a brief specification document for the front-end.
My final project of the week was the Place-names of Armagh project, for which I started working with some existing place-name data for the area. There are around 230 place-names and several thousand historical forms and I spent quite some time researching how the data was structured and how it might be mapped onto the Glasgow place-names system. This included analysing the geospatial data, including shapefiles for Townlands and what I though was Parishes (but actually turned out to be the same as for Townlands). I had hoped to be able to import the data by the end of the week, but my analysis of the data raised a lot of questions that still need to be addressed, and I’ll need to continue with this next week.
My main task for this week was to continue with the creation of a new, unified API for the Historical Thesaurus. This has been on my ‘to do’ list for a long time, and I began work on it back in February, but was only able to spend a day or so on it. I was able to devote a few further days to the task this week and have made good progress. The Historical Thesaurus’s existing website was created over several years, with new features added as and when required, resulting in a structure that’s got a bit messy over time. For a while now I’ve wanted to bring this into line with the other resources I’ve developed and have a single API through which all data requests are processed. Back in early February I made a start on this, but only managed to get as far as creating the endpoint to generate a random category. This week I made a lot more progress, and now have all of the endpoint needed to provide data for the category browser and other parts of the site such as the quizzes. This has also included the endpoints necessary to generate the timeline visualisations. I’m also taking the opportunity to tweak the structure of the ancillary pages slightly, stripping out unnecessary function calls.
I still have a fair amount of work ahead of me – the entire search facilities still need to be tackled, as do the endpoints for other visualisations such as the heatmaps and other search options that were developed for the Linguistic DNA project. As I’m fitting this in around other commitments I’ll just need to see when I’m able to finish work on the API, but I’m pretty pleased with the progress I’ve made this week.
Also this week I spent some further time on the Anglo-Norman Dictionary. We haven’t gone live with the new cognate reference search yet, but I implemented a new feature for the dictionary’s content management system that the editor Geert suggested recently: a list of entries that were last updated in each month. Here’s a screenshot of how this looks:
By default the page lists all of the entries that were last updated in the current month, but you can also select a different month and year to view the entries that were last updated then. As the note in bold on the page states, this is just a list of when entries were last updated, it’s not a log of every time an entry has been updated. The table lists some details about each entry and includes a link to view the entry in the CMS. You can also reorder the table by pressing on the column headings (e.g. ordering by last editor). Pressing the heading a second time reverses the order. Hopefully the feature will prove useful to the team.
Also this week I had to fix a couple of issues with the Books and Borrowing website. On Monday I noticed that the website was offline, and this was caused by the database server failing to restart following an update. I liaised with Stirling’s IT people to get this working again. A user of the website had also spotted that the CSV download of the search results was failing to work and I had to spend a bit of time investigating this. Initially I thought the problem was due to the number of records, but on further investigation it turned out to be caused by the presence of one or more borrowing records that didn’t have an associated book holding record.
For example, on this page https://borrowing.stir.ac.uk/search/p-1/0/0/advanced/brids|4473 the borrowing record with ID 48238 with the transcription ‘Prices Alexero on Sab. Chur: [?]’ doesn’t have an associated book holding. The CSV output was expecting all borrowing records to have a book holding, and when it reached one that didn’t the script stopped working. Once I’d identified this I was able to fix the issue and the output now works as expected.
Also this week I liaised with my colleague Luca to get the Helsinki Corpus up and running again. We host the site, but I’d been alerted to the fact that it was offline on Monday and correctly surmised that it was because our hosting provider had silently upgraded the version of Python that was running on the server, and some aspects of the Helsinki Corpus code were incompatible with the new version. Thankfully Luca was able to install an alternative to the library that had been deprecated and we got the site back online again. If I can find the time I’d like to redevelop the Helsinki Corpus site, migrating it to eXist and building a new site around this. Something to add to my ‘to do’ list.
Also this week the Solr server that powers the Dictionaries of the Scots Language went offline and I needed to speak to our IT people about getting this back online again, which thankfully they were able to do pretty speedily. I also made a few updates to the help text on the advanced search page of our test site and replaced ‘dates of attestation’ with ‘quotation dates’. On Friday I received a new batch of data that will hopefully be the last batch that needs processed before the changes made on our test site over the past 18 months can finally go live. I’ll begin working on this next week.
I returned to a more normal working week this week, after having spent the previous one at a conference and the one before that on holiday. I probably spent about a day catching up with emails, submitting my expenses claim and writing last week’s rather extensive conference report / blog post. I also decided it was about time that I gathered all of my outstanding tasks together into one long ‘to do’ list as I seem to have a lot going on at the moment. The list currently has 47 items on it split across more than 12 different projects, not including other projects that will be starting up in the next month or two. There’s rather a lot going on at the moment and it is good to have everything written down in one place so I don’t forget anything. I also had some AHRC review duties to perform this week as well, which took up some further time.
With these tasks out of the way I could get stuck into working on some of my outstanding projects again. I met with Hannah Tweed on Tuesday to go through the Medical Humanities Network website with her. She had begun to populate the content management system with projects and people now and had encountered a few bugs and areas of confusion so we went through the system and I made a note of things that needed fixed. These were all thankfully small issues and all easily fixable, such as supressing the display of fields when the information isn’t available and it was good to get things working properly. I also returned to the SciFiMedHums bibliographical database. I updated the layout of the ‘associated information’ section of the ‘view item’ page to make it look nicer and I created the ‘advanced search’ form, that enables users to search for things like themes, mediums, dates, people and places. I also reworked the search results page to add in pagination, with results currently getting split over multiple pages when more than 10 items are returned. I’ve pretty much finished all I can do on this project now until I get some feedback from Gavin. I also helped Zanne to get some videos reformatted and uploaded to the Academic Publishing website, which will probably be my final task for this project.
Wendy contacted me this week to say that she’d spotted some slightly odd behaviour with the Scots Corpus website. The advanced search was saying that there were 1317 documents in the system but a search returning all of them was saying that it matched 99.92% of the corpus. The regular search stated that there were 1316 documents. We figured out that this was being caused by a request we had earlier this year to remove a document from the corpus. I had figured out a way to delete it but evidently there was some data somewhere that hadn’t been successfully updated. I managed to track this down: it turned out that the number of documents and the total number of words was being stored statically in a database table, and the advanced search was referencing this. Having discovered this I updated the static table and everything was sorted. Wendy also asked me about further updates to the Corpus that she would like to see in place before a new edition of a book goes to the printers in January. We agreed that it would be good to rework the advanced search criteria selection as the options are just too confusing as they stand. There is also a slight issue with the concordance ordering that I need to get sorted too.
At the conference last week Marc, Fraser and I met with Terttu Nevalainen and Matti Rissanen to discuss Glasgow hosting the Helsinki Corpus, which is currently only available on CD. This week I spent some time looking through the source code and getting a bit of server space set aside for hosting the resource. The scripts that power the corpus are Python based and I’ve not had a massive amount of experience with Python, but looking through the source code it all seemed fairly easy to understand. I managed to get the necessary scripts and the data (mostly XML and some plain text) uploaded to the server and the scripts executing. The only change I have so far made to the code is to remove the ‘Exit’ tab as this is no longer applicable. We will need to update some of the site text and also add in a ‘hosted by Glasgow’ link somewhere. The corpus all seems to work online in the same way as it does on the CD now, which is great. The only problem is the speed of the search facilities. The search is very slow, and can take up to 30 seconds to run. Without delving into the code I can’t say why this is the case, but I would suspect it is because the script has to run through every XML file in the system each time the search runs. There doesn’t appear to be any caching or indexing of the data (e.g. using an XML database) and I would imagine that without using such facilities we won’t be able to do much to improve the speed. The test site isn’t publicly accessible yet as I need to speak to Marc about it before we take things further.