Week Beginning 7th September 2026
I spent most of my time this week working on the redevelopment of the Dictionaries of the Scots Language website. We’re intending to launch a new version of the resource with a new interface and lots of additional functionality in November and I have a huge amount to do before then.
Whilst working on my gigantic list of things to do I realised that the fulltext search excluding quotations on our test site was not working as intended. Rather than excluding quotations it was in fact searching the entire text of entries and giving the same results as the ‘include quotations’ search. It turned out that this was due to the major changes to the entry structure that were introduced a while back.
My script that processed the data to generate the ‘no quotes’ field in our Solr index should have stripped all the <cit> content from the XML and storing the rest in the ‘withoutquotes’ field that is then indexed by Solr, but it was not stripping any content because the structure of the files has changed (e.g. <cits>, <citGrp>) and it’s only looking for <cit> directly within <sense>. At the time the structural changes were made I updated the <cit> processing elsewhere in my scripts (e.g. when generating data for the quotation search and generating date information) but unfortunately overlooked this part.
I therefore updated my processing script so it could deal with the new structure and re-ran the Solr index generation, which thankfully worked. Once the online Solr index was updated with the new version the search then worked as intended. Thank goodness I spotted this before the relaunch!
I then moved onto implementing the part of speech search, which we eventually decided should only be offered for headword searches. I can’t share any screenshots of this for the moment, but I updated the Advanced Search form to include a list of parts of speech as checkboxes, allowing multiple POS to be selected.
Processing this and dealing with the POS passed in a URL from the search form to the search results page required quite a bit of reworking of the search results page, as I needed to shift the elements found in the URL around to make space for the passed POS (and also region, which I added in later in the week). This required a major reworking of the results page and how it processed URLs, which took quite some time to implement. I also needed to make a lot of changes to the API in order to incorporate the POS search, which took a lot of time and effort too.
Thankfully with all of these changes in place the POS search was operational, and I also updated the Advanced Search form so that any previously selected POS were remembered when refining a search.
With this in place I then moved onto implementing the region search. This only appears when a ‘quotation’ search is selected, and only appears when the coverage is set to ‘both’ or ‘after 1700 (as regions are not found in the DOST data). Initially I tried to list the regions as checkboxes, as with POS, but there were just too many of them for this option to be suitable. Instead I used a multiple select box widget (https://select2.org/getting-started/basic-usage/) that Vasilis had suggested to me which allows multiple regions to be selected and managed in a compact and easy to use manner.
As with POS, I needed to make lots of changes to the results page and the API to incorporate the region search options, and I needed to ensure that any selected regions are remembered and are preselected when returning to the search results form. It took a lot of time and effort to get all of this working, but by the end of the week I had both the POS and region searches working on our test instance, which I reckon is a pretty major achievement.
There’s still a lot to do, such as integrating the map into the region search and ensuring that selected POS and regions are listed in the results header. I also still need to work on results filtering for POS and region, which I’m hoping to get started on next week.
I also spent a fair amount of time on Monday writing up my notes from the conference, submitting my expenses and other such administrative tasks. I also helped Eleanor Lawson with a query about Google Analytics for the SpeechStar website, gave some feedback on a new version of the place-names AHRC proposal for Jayne Carroll at Nottingham and spoke to Faidon about the migration of websites to our external provider.
On Thursday I also had an online meeting with Tom Bartlett about migrating his podcast site https://languagepowerpodcast.org/ from its current host SquareSpace to hosting provided by the University. The current subscription period expires in November, and I don’t have the time to migrate the site before then so we’re exploring some options.