I continued to work on the DOST Auld Laws project on Monday and Tuesday this week. My first task was to implement the tag selection for the Advanced Search. As discussed last week, this section allows users to either include or exclude tags from their searches, or limit their search to only look at the contents of one of more specified tags. Luca has already created the XML search options that would allow this, and my job was therefore to process the user’s selections, format the query and connect to the XML. I managed to complete this on Monday and it’s now possible to construct a query such as “find all occurrences of words beginning ‘ȝe’ in ‘Edzell Doouments’ excluding any text found in ‘Expanded Forms’, Aitken Notes and Other Notes”. It’s also possible to press on the ‘Refine your search’ button to return to the search form and the search criteria will be remembered in the form. I also added in a ‘Clear’ button.
I continued working on the site on Tuesday. I added a ‘Cite’ option to the search result page, which allows users to share or cite a particular page of a search, including the ordering. I also added a help popup about the tag selection to the search form and I integrated the ‘show all Aitken notes’ query with the Advanced Search. This is a special case that returns the full contents of all notes by Aitken across all or a selection of documents without the need to supply any search text. Information about how to do this appears in the help popup as follows:
“To retrieve all Aitken Notes enter a dash (-) into the search box, select documents (if required), set Aitken Notes to Limit to content in this tag and press the search button. This will display the full contents of all Aitken notes in your chosen documents.”
The Aitken Note results are a bit different to the regular results as they return the full contents of each tag, not a KWIC. Therefore all of the text is highlighted in yellow and the sorting by left and right of the term doesn’t do anything, as there is nothing left or right.
Whilst working on the Advanced Search I’d spotted that the advanced search KWIC (KeyWord In Context) was not crossing line boundaries, meaning that when a work is found at the beginning of a line no contextual text is returned before this. This is different to the Quick Search, which ignores line boundaries and is not ideal, as it means the search results are inconsistent and the quick search actually gives better results than the advanced search. Thankfully Luca was able to find a solution to this and to make the advanced search results consistent with the quick search.
I’ve now pretty much done all I can do for this project until I hear back from Joanna, who is going to supply ancillary content and edited XML files.
I spent a lot of the rest of the week working on the HiMuJe Malabar project, continuing to develop the map. On Thursday I attended a meeting with Ophira, Renu and project Co-I Ines, who is in Glasgow from Germany for a while to work on the project. We had a very productive meeting and we now have a much clearer idea about how the travel routes and other aspects of the map will function.
In terms of actual development of the map, this week I fixed an issue with the travel routes whereby when multiple travel routes are selected, deselecting one would remove all locations, even those that were associated with another active route. This took quite some time to sort out, unfortunately, but it’s fixed now and the routes work a lot better.
I then set about making it possible to share or cite the selected travel routes, also noting whether unrelated places have been hidden or not. This required some major reworking of the code to ensure that these options are added to the URL and when the page loads that the options are taken from the URL and processed. The citation text also needed to include information about the selections too, and any location selected when the route is active can also be cited, e.g.
“HiMuJe Malabar Digital Map: Satellite map, travel routes menu, showing the travel route ‘The migration of Jews and Christians in the Qissa’, with unrelated places hidden, placenames categorised by type / subtype, placename record for Mecca. 2026. In HiMuJe Malabar. Glasgow: University of Glasgow. Retrieved 19 August 2026”
At the meeting we decided what icons to use for the location categorisation so on Friday I returned to the map legend to rework this. I started off by slightly reworking the data to ensure that all locations had both a type and a subtype, as a few locations only had the former. This meant creating a new ‘Area’ type and making ‘Region’ a subtype of this, whereas previously ‘Region’ was the type. The reason for doing this is so that the legend could be grouped by type, giving each type a count of the total number of locations contained across all subtypes and to include a checkbox for each type that when pressed on would select and deselect all subtypes.
That was the plan, anyway, but unfortunately I had a very frustrating day and didn’t manage to get this working by the end of it. The difficulty is that Leaflet generates and processes the legend internally based on the map layers and trying to hook into this and change the default behaviour is very difficult. I managed to get the ‘Type’ checkboxes to select and deselect all corresponding ‘Subtype’ checkboxes but doing so was not actually triggering any changes on the map – the corresponding map layers were not being affected when the checkboxes changed even though manually pressing on the checkboxes did trigger the layer change. I’m afraid I ran out of time with this and didn’t manage to find a solution. For now the types are just headings in the legend without associated checkboxes, which is a bit of a shame but I just don’t have the time at the moment to look into this further.
Also this week I made a few more tweaks to the data for the Burns Supper Map, including adding in a new batch of images, discussed migrating a couple of sites to our third-party hosting supplier, read a document ahead of next week’s DSL meeting and read some information Deven Parker had sent me about a funding bid.
I was back in Glasgow and back to a five-day working week this week, as the summer holiday period drew to a close. I spent a fair amount of time this week continuing to work on the DOST Auld Laws project. Luca has been working on the XML queries I’d specified for the advanced search and I was able to test them out and give feedback on them. These are mostly all working as I’d hoped, which is really great, and I was able to begin working on the front-end aspects of the advanced search, with the aim of connecting all of this into the queries Luca had created.
However, I also had to spend quite a bit of time working with the source files, as the project PI Joanna Kopaczyk-McPherson had realised that two of the eight documents should really be split up into smaller sections. This was no straightforward task, as not only did it require the XML documents to be split into smaller sections, but many other aspects needed to be updated. The document IDs needed to be changed, which mean IDs used for image filenames throughout the documents also needed to be changed, with the filenames of the actual images also then needing to be updated too. For page navigation in the site data is stored in a database and this also needed to be updated, and the XML files stored in eXist for search purposes also needed to be updated. There were twelve steps I needed to follow for each required split, which took some time, but thankfully the process went pretty smoothly and we ended up with 13 documents in the site instead of the original 8.
There were further issues to come, as Joanna has now noticed that the documents need further edits and tweaks, and not just minor changes to text but structural issues such as the insertion of omitted lines. This is going to be very difficult to do as lines are linked to coordinates in the images, which was all handled via the Transkribus tool. We exported the XML files from Transkribus month ago and many major changes have been made to the files since the export so it’s not going to be possible to re-import them. Manually creating new lines in the XML files as they are now will not have the connections through to coordinates in the corresponding image files, so we’ll end up with inconsistent data. It’s not a great situation to be in, and ideally all editing of the documents should have been completed in Transkribus before we exported the files, something I’d mentioned back when I undertook the export process months ago. We haven’t reached a decision on how best to handle this yet and we’ll continue to discuss the options next week.
Despite all of this I did manage to work on the advanced search, creating the advanced search form which, as specified, features a textbox where you can enter some text, a list of documents from which you have the option of selecting the ones you’re interested in and a list of tags that you can either include, exclude or limit your search to, as you can see in the following screenshot:
As of yet I have not implemented the tag limits, but the limit by documents is operational. This connects through to Luca’s new eXist-db queries to perform a search limited by documents and as with the quick search, you can also sort the results by words to the left and right of the term, although I’ll need to get Luca to look into how the KWIC is generated for the advanced search results as they don’t seem to cross line boundaries, unlike the quick search. The advanced search results display information about the documents you’ve selected and if you press on a search result to load the corresponding page the link back to the search results takes you to the right place. There’s still lots to do. The limit by tags is the biggest thing and will probably take quite some time. I also still need to add in an option to cite a specific search result page and add in an option to refine your search, which will remember the options you previously filled in when you return to the search form. An option to clear the search also needs to be added. I’ll continue with this next week.
Also this week I participated in two meetings about the place-names AHRC proposal I’m involved with, and this is coming together very well. We now have an outline proposal completed and pretty much ready for submission.
I also spent a bit of time continuing with the travel routes for the HiMuJe Malabar project, adding in a few more travel routes that had been prepared and made a few tweaks to the XSLT that generates the entry HTML for the new DSL interface, fixing some issues with the layout of the new ‘combs’ sections.
I worked a total of four days over the past two weeks, and was on holiday for the remainder. During this time I had a meeting to further discuss a place-names related AHRC proposal I’m involved with. I can’t really say much more about it at this stage, but the proposal is coming together. I also had to spend some time working with IT Support and Luca to figure out why our local server kept going offline repeatedly. It looks like this was caused by the server getting swamped by requests from one particular source (almost certainly bot or AI) and thankfully IT Support were able to block this, after which the server was stable again. It’s something we’re going to have to keep looking out for in future. I also spent a bit of time working with Luca to get automatic WordPress updates working on our local server, as the way sites had been set up meant that the setting was not working. Luca managed to find a solution to this, which is really great.
I also spent a bit more time working on the Burns Supper Map, creating a record for it on this site (see https://digital-humanities.glasgow.ac.uk/project/?id=156), adding more suppers that had been submitted via the survey and making some requested edits to existing suppers. I also set up access to Google Analytics for the other two members of the project team.
In addition, I investigated an issue with the Scots Syntax Atlas after someone suggested that the linguists’ atlas was looking somewhat blurry. I managed to figure out why this might be the case, although I’m not entirely sure whether this is a new issue or if the markers always looked like that. I contacted the project PI and suggested a couple of updates, but I haven’t heard back yet so will need to wait and see what she says.
I spent most of the remainder of my time working on updates to our new test interface for Dictionaries of the Scots Language and working through the list of outstanding items for the DOST Auld Laws project. For the DSL I completed the updates to the bibliography page that I began working on a couple of weeks ago. I implemented pagination of the entries associated with bibliographical items, with navigation bars appearing above and below the entries, with 20 appearing per page and ‘jump to page’ buttons also appearing, just like with the search results. This works pretty well, but is somewhat cumbersome for someone like Sir Walter Scott, who is referenced in 2861 entries, split over 144 pages. We don’t have this issue with the search results are these are capped at 500 (25 pages) so we might need to think of other ways of handling this.
I also ensured that headword searches that don’t yield any results automatically perform a fulltext search for the term supplied. This works on the live site, but only when both dictionaries are selected. With the new site there are separate quick searches for SND and DOST so the additional search wasn’t being triggered. It is now, as is the advanced headword search for both dictionaries.
I also made tweaks to the DSL’s new regional map based on feedback I’d received – adding in some content where we previously had placeholder text and ensuring the ‘About’ popup didn’t disappear off the bottom of smaller screens and a few other small updates. I then began to look at the ancillary pages and how we can make them look a bit nicer. I spent a bit of time on the ‘Word of the Week’ page and liaised with William Ashford, who is responsible for such content about this, and further updates that were going to make to the ancillary content closer to the launch date of the new site (which will hopefully be in November).
For the DOST Auld Laws project I added the top navigation bar that will link the site in with the SCOTS Corpus and CMSW. I added in copyright information and added facilities to download page images and the XML files for each document. These being up a pop-up asking for users to abide by the license before leading to the actual content, which hopefully won’t be too annoying. I also added in the ‘cite’ popup to all document pages, which took a little time to implement, and added in Google Analytics. I also made the image thumbnails on the document overview pages smaller and placed them in a collapsible section that is closed by default, plus I removed the introduction to the documents page, as this will be covered by the homepage.
I also removed some pages that didn’t have content (e.g. blank pages) from the beginning and end of some of the documents and I added a feature to turn off and on the line highlighting feature. The highlighting feature allows the user to click on a line of text in the image or text for a page and for that line to be highlighted in both the image and the text, which is pretty nice. Unfortunately the line highlighting gets in the way of the image viewer’s zoom and pan functionality on touchscreens, making it a somewhat unreliable and frustrating experience. This new feature removes the option to ‘click’ on a line, meaning pointer events are not intercepted and make their way reliably through to the image viewer, which works much more smoothly.
Also this week I had an email conversation about user feedback and walkthough videos for the STAR resources and booked my accommodation for the DHC conference in Sheffield. Next week I’m back in Glasgow and back working a full week, with summer holidays all over.
I returned to work on Monday this week after a lovely holiday. Most of my time this week was spent working for the Dictionaries of the Scots Language, processing a new dataset exported from their editing system and integrating it into the website. The reason this process took considerably longer than usual is that the data had a major structural difference: Entry parts of speech had been moved, rationalised and restructured. Previously parts of speech appeared as elements within <meta> but also appeared embedded in the main entry where they were only tagged with HTML italic tags. There were more than 500 different combinations of parts of speech and things like homonym numbers were mixed in with them too.
The DSL editors have spent a huge amount of time working on the parts of speech to separate them out, standardise them and ensure other data such as homonym numbers are stored in a separate but related manner. The new structure also ensures that entry parts of speech are only stored once in the entry XML using a structure that makes sense semantically rather than for display only.
As the new entry XML in the data export now differed markedly from the earlier structure I then had to rewrite my data processing scripts, update the database and Solr cores, and the API and front-end to deal with the new structure. This was a lot of work. My first step was to create a new table to hold the individual POS data, consisting of a part pf speech and associated ‘hom’, ‘syn’ and ‘infl’ values. I then updated my extraction script so that the existing pos field in the entry table now gets its content from the ‘origpos’ attribute (so we continue to have a record of the original part of speech) and then to populate the new POS table to store each individual pos, including the value, hom, syn and infl data (where applicable) for each entry.
With this update in place I could then (after running a few smaller-scale tests) process every entry in the SND and DOST export files, which resulted in 35,655 parts of speech records being generated for SND entries and 49,828 being generated for DOST.
My next step was to update the entry browse order, which is used to decide in which order the entries appear in the browse pane when viewing entries. This previously used the old POS system to decide in which order entries with the same headword appeared (e.g. so that nouns appeared first). I had to update this to use the new POS system, and also ensure that the POS labels, which are used in the browse pane, the search results and the entry header.
With this update in place I then ran my scripts to process the citations and bibliographies. After that I worked on the Solr cores we use for search purposes. These consist of an ‘entry’ core used for headword and fulltext searching, and a ‘quotation’ core used for searching the individual quotations. Both of these needed to be updated to include new POS fields, which will eventually allow me to add parts of speech filters to the search facilities. I created new Solr cores for each, both featuring new POS fields, after which I would update my script that generates the data to populate the cores to ensure the POS data was included.
With the new cores successfully populated with the new data I then needed to update the API to work with the new POS fields (e.g. so that the labels returned for use in the search results and browse pane use the new POS data). After that I then needed to update the XSLT scripts that process the entry XML files for display to ensure that the new POS structure is displayed when viewing an entry.
At this point all of the updates were still running on my laptop, and with everything in place and tested the final step was to migrate everything to our test server. I don’t have direct access to the server so needed the help of someone with this access in order to complete tasks such as creating and populating the new Solr cores, and thankfully Luca agreed to help out. The process went remarkably smoothly and by the end of Thursday the update was complete, with our test server displaying the new POS fields. I also made a number of other minor updates to the display of entries via the XSLT files that had been requested and it’s now over to the DSL editors to test everything out and make sure all is working as it should.
I spent the remainder of the week dealing with emails I’d received whilst I was away. I also had a meeting with Joanna about the DOST Auld Laws project, which is nearing completion now, and had a meeting with Renu regarding the display of polygons in the map for the HiMuJe Malabar project.
After being off sick on Thursday and Friday last week I returned to work for a full five days this week, although I was still in the throes of ‘post viral fatigue’ and it wasn’t until Friday that I stopped feeling exhausted. Despite this I still managed to get a lot done for several projects this week. I had been planning on continuing with the redevelopment of Mapping Metaphor towards the end of last week, and as I was unable to do so I decided to focus on this on Monday.
As I looked through the partially updated site I spotted an issue when combining E and OE data: Sometimes the metaphorical connection between two categories has the categories the other way round in the other dataset (i.e. in E we have cat1 and cat2 whereas in OE we have cat2 and cat1). This was resulting in metaphors in the combined view sometimes being treated as different when they’re actually the same, which was affecting some counts and also sometimes making the figure for number of OE lexemes appear next to the wrong category in the card view. I managed to sort this, and I also ensured that if the category order needs to be swapped then the direction is also swapped.
I then spent most of the day implementing the combined table view, and it should be fully operational now. It was extremely tricky to implement this as the script that generates the data for the table view can handle many different data requests and (unlike the visualisation script) also processes individual metaphor connections for display, including directionality and examples of metaphor. There was a lot to update and check, and it was probably not the best of choices of things to work on during my first day back at work. However, I got there in the end and was able to share the update with Wendy and Carole by the end of the day. I still need to update the CSV download option, but that will need to wait for another day.
I also spent about half a day or so on the Playbills project this week. I fixed the few remaining plays that didn’t have links to canonical records so that every play now includes such a link. I then moved onto looking at performers. I wrote a script that outputs a spreadsheet that lists all performers arranged alphabetically by surname with the corresponding play, date of performance and venue. I’m hoping we’ll be able to do something with this to figure out which performers are actually the same person, but with almost 52,000 rows we’ll probably want to try some automated processes rather than figure it all out manually. I’m just not sure how best to proceed with this.
For example, there are 17 ‘Mrs Ashton’ performers, with performances from 1827 to 1834. 12 of these are at the Theatre Royal, Birmingham in 1827, then there’s nothing until 1832 at Bristol (4 performances) then one final performance in 1834 in Edinburgh. Would we consider these all to be the same performer? Or should we treat the Bristol and Edinburgh ones as different people? This will need some discussion with Deven.
I also began looking into generating canonical roles for plays. I wrote a script that for a given canonical play ID returns all plays that have the ID and for each lists its name and all associated roles. The individual play names are links through to the play page, from which it’s possible to access the playbill page and check the original image. At the very bottom of the output is a list of all unique roles with a count of the number of times they appear. This is only an initial version but it has highlighted many issues due to variant spellings (either in the original playbill or due to the AI text extraction). I can further tweak this, for example making all lower case, removing punctuation, removing text in brackets, replacing “M’” with ‘Mac’ or ‘Mc’ to see if these match. We could also use Levenshtein distance to try and find forms that differ by one character, which would match ‘Baillie Nicol Jarvie’ and ‘Balie Nicol Jarvie’ with ‘Balie Nicol Jarvie’ and ‘Captain Thorton’ with ‘Captain Thornton’, for example. We’d have to watch out for false positives, though (e.g. ‘Jane’, ‘Jean’ and ‘Janet’), and also make sure the form that has the most hits is taken as the canonical form, rather than it just being the first form found. But it’s also possible that asking AI to do this might be a better approach.
As I was working on this I noticed the interestingly named role ‘MacSycophant’ in the list, which is definitely an AI hallucination! Looking at the original playbill image it should be ‘MacStewart’. I also spotted that many of the actors are incorrectly assigned, with ‘Mr Harrold’ and ‘Mr Felton’ being swapped and Rob Roy listed as being performed by Miss A Murray instead of Mr Pritchard). Checking the original YAML shows that the issue was with the AI extraction and thankfully it was not any subsequent processing scripts that have introduced the errors.
Following on from this, Deven has suggested that she and some other researchers may manually proofread all of the actors and roles across the entire dataset. This would really help to ensure that the data is more accurate, but it is also a huge amount of work, as there are 1902 playbills containing tens of thousands of performers and roles.
I spent some time proposing a means of editing the data so that the correct updates are made in the database. We would need to generate lists of performers from the database so we don’t lose the work we’ve already done to split the names and assign genders, and presumably the task won’t just involve swapping roles between performers, but will involve role names being edited too (e.g. to fix things like ‘MacSycophant’). I think it’s likely that some issues with performer names that have been extracted incorrectly too will also be spotted and need fixing too. We’ll need to ensure any updates to performers’ names and role names can be tracked to the relevant record in the database using the corresponding ID fields.
To achieve all this I generated a spreadsheet that includes information about the playbill, play, date and venue, with a link to the page for each playbill for checking the original image. The Role data and Performer data then follow on each row. Using this a proofreader could then make changes to the ‘Role Name’ field where required, but also make changes to the performer fields too if anything needs corrected there, and also add in new rows if required.
I then returned to the DOST Auld Laws project, for which I created an initial version of a user interface for the site. This is still just a work in progress and may change depending on feedback given, but below is a screenshot:
I spent most of the rest of the week working on created a system to manage travel routes for the interactive map for the HiMuJe Malabar project. My first task was to create a test route based on data provided by project Co-I Ines. I added this to a spreadsheet template I’d created, and which we’ll hopefully be able to use for future routes.
With this data in place I then wrote a script that converts the spreadsheet into JSON data for use on the map, so if further travel routes are created (one per spreadsheet) with the same structure I’ll be able to add them to the map.
Once I’d completed my updates to the map interface, when you press on the ‘Travel Routes’ section of the map menu each travel route is now listed (there is currently only one). Each route appears with its title and type (we can maybe split and/or limit the list by type in future) and the description given in the spreadsheet. Pressing on the checkbox or the route name adds the route to the map. This currently highlights the relevant locations with a red border and adds a dotted red line connecting each location in the itinerary. We may want to use different colours when multiple routes are added at the same time to help differentiate them.
As the route can get lost in amongst all of the other data on the map I’ve added an option to show or hide unrelated places. If you deselect the ‘Show unrelated places’ checkbox any locations that are not part of the travel route are removed from the map, making it much easier to see the route. The following screenshot shows this:
The legend is removed from the map when the ‘Travel Routes’ map menu is active. This is because we need the space to display the information about a specific stage in the travel route itinerary, as described below. Also, if the legend was visible users may end up removing a map layer (e.g. ‘Temple’) that contains locations that appear in a selected route, which would cause confusion. When viewing travel routes, all of the categorisation layers are set to on for this reason. If you navigate from the ‘Travel Routes’ menu to another menu any selected travel routes are removed from the map, the legend is reinstated and all locations are added to the map.
If you press the ‘Explore’ button for a route this adds the route to the map (if you’ve not already added it by selecting the checkbox) and the map will reposition to display the first stage in the itinerary. This displays an info box in the top right of the map (in the place the legend would otherwise be). This box displays the route title and the information about the first stage in the itinerary. This can include any information about the stage in the itinerary (including duration, if of relevance). I’ve also included information about the number of stages in the route and the number of the current stage (e.g. ‘Stage 2 of 10’), as I figured this would be useful for people. There is also a ‘close’ icon in the top right that closes the info box and ‘Next’ and ‘Previous’ buttons at the bottom that can be used to traverse the travel route. Pressing on one of these buttons repositions the map to the next or previous stage in the itinerary and loads the information about this stage into the info box. Using these options you can step through the travel itinerary and view all of the information relating to it. The following screenshot shows the map with ‘Stage 2’ of the travel route loaded. Note that you can also still open the popup for any location on the route or manually scroll the map between locations.
This is only a first version of the feature and there’s still a lot to do. Displaying multiple routes at the same time is going to require further work, and I also want to add the selected route options to the page URL to enable citation / sharing / bookmarking of specific route selections and also possibly individual stage selections. We may also want to add information about relevant routes to the location popup (in a new tab) so users can tell at a glance which routes a location appears in. I guess adding links from this to the relevant stage in each route would also be useful.
Also, the current system is only set up to work with a purely linear route (e.g. a->b->c). If we are to include routes that branch off (e.g. a->b then b->c and also b->d then c->e and also d->e) then we’ll need to consider how to handle this as the traversal via ‘Next’ and ‘Previous’ buttons would not work.
Also this week I made a few further tweaks to the Burns Supper Map, including updating the site title to include ‘Worldwide’, fixing some typos in the data and adding in some new videos. I’ll be continuing with this next week.
I began feeling unwell on Monday this week, but managed to struggle through until Thursday morning, by which point I just couldn’t sit at my desk any more. I was then off work sick on Thursday and Friday.
Despite coming down with something I still managed to get quite a bit done on Monday to Wednesday this week. On Monday I mainly focussed on the Playbills project. I manually sorted all but three of the remaining plays that didn’t have canonical records and asked Deven to further investigate the remaining three. I then added the canonical play data to the output produced by the API and wrote a script that would output all of the playbill data for every playbill as JSON files. This will be used for offline analysis by Deven.
On Tuesday I continued to work on the map for the HiMuJe Malabar project, mostly tidying up some loose ends with the map. I implemented the URL shortener and set up the ‘share / cite’ options so that now if you open a place record and press on the ‘Share / Cite’ tab this loads content, with the citation text being dynamically generated based on the map options that are currently selected. It’s also now possible to press the ‘Share’ button at the bottom of the map menu to share or cite a specific map view as opposed to a specific record.
I also implemented the ‘Reset map’ option on the ‘Home’ menu and pressing on this resets the map to the default view and categorisation type and I finally sorted out the polygons on the map so they should always now appear behind the markers, even when turning layers on and off in the legend.
The last thing I implemented was the ‘Table view’. If you press on this button at the bottom of the map menu it opens a popup containing all of the data in tabular form. I haven’t included the references in this as this is too much data for such a table, but I have included a count of the number of references for each place. Similarly, I’ve only included the latitude and longitude for each place and not the full GeoJSON shapes for the polygons. You can press on a column heading to order the table by that column (press a second time to reverse the order). I think it will be quite a useful view, for example if you know the name of a place but are not sure exactly where it is located you can order the table by placename and find it. Pressing on a placename in the table closes the popup, centres the map on the location and (after an intentional brief delay so you can appreciate where on the map you’re looking at) the relevant record popup opens. I might see about highlighting the place’s marker to make it clearer which is the relevant one in areas where there are many.
I still need to implement categorisation by certainty, but as of yet we don’t have any certainty data in the JSON file so I’m leaving this for now. I’m intending to start on the travel routes soon. We don’t have any routes available yet, but I’m going to create a test route and create the interface for plotting the connections along the route so we can see how this might work.
On Wednesday I continued to work on the DOST Auld Laws project. I managed to fix the issue with multiple results in one line causing the in-page navigation to stop working. This required quite a bit of reworking as the results were identified by line and instead I needed to add in a new way of identifying that a result may be on the same line but is actually a different result. I did this by counting the number of results per line and using this figure in addition to the line ID to track the results, adding this counter to the URL that is used to reach the document page from the search results, with the document page then taking this figure and using it to ascertain which result is the current one and allowing the results traversal on the document page to function.
I then moved onto looking at results page ordering. Previously the search results were ordered by document name, document page and line, but I wanted to add further sort options via a drop-down list to the right of the ‘you searched for…’ box. In addition to the default ordering, this allows you to order by highlighted term (useful for searches like ‘*other’ where the start of terms may differ) and a concordance-like word position. The latter allows you to sort the results by up to 5 words to the left or right of the term, with the selected position highlighted in cyan. The screenshot below shows the search for ‘*other’ ordered by the second word to the left of the term:
This was a particularly large and complex update to implement as many parts of the code had to be rewritten, but I think it’s worth it. The final update I implemented was to ensure that the ‘return to search results’ button in the search bar of the document page now takes you back to the specific results page and retains the ordering you’ve selected rather than taking you to the first page of results and the default ordering.
Also this week I set up a bare-bones WordPress site for Craig Lamont’s new project, and I’ll meet with him over the summer to get this fully set up. I also tweaked the Burns Supper Map to ensure long subtitles for the charts don’t end up overlapping with the ‘hamburger’ menus for each chart and read through the findings of the user survey for the Dictionaries of the Scots Language that I’d been sent, and which was on the whole very positive.
The project I spent the most time working on this week was the DOST Auld Laws project, which I hadn’t worked on for a couple of weeks. The last time I worked on the project I’d managed to get an initial version of the quick search working, but there were some issues with it. A phrase search wasn’t working, wildcards at the start of a search term were not working, the full term was not getting highlighted in the results where an <expan> is present in the term, I hadn’t implemented the pagination of search results and I still needed to update the page view to enable results traversal directly from the page when accessed via the search results.
I was struggling somewhat with phrase searching and full term highlighting and ended up going round in circles with ChatGPT for several hours and getting nowhere. I eventually asked my colleague Luca, who has considerably more experience and knowledge of eXist-db and XML document querying, for some advice. Thankfully he was able to come up with a query script that did exactly what I needed it to do, which was a massive help as I was really struggling. This is definitely an example of where a conversing with actual person is much more effective than AI and I really must give Luca credit for the help he gave.
By the end of the week my updates meant that it was possible to use wildcards at the start of a search term, as you can see from the following screenshot that shows the results for ‘*other’:
With Luca’s help, phrase searches now work, and the following screenshot shows the results of a search for “the landis of”:
Again with Luca’s help, the search term in each result is now fully highlighted even when an <expan> is present in the word, so for example a search for ‘pebillis’ previously found two results but only highlighted ‘pebill’ in the results as the term was recorded as ‘pebill<expan>is</expan>’. But now the full term is highlighted.
I also implemented results pagination, with is currently set to display a maximum of 20 results per page. The following screenshot shows a search for ‘r?cht’ with the pagination in place:
And now when you reach a document page that is in the search results (e.g. by selecting it from the search results) a search results navigation bar appears above the document navigation bar, allowing you to navigate to the next or previous search result or return to the full search results, as the following screenshot demonstrates:
I’ll probably add in a ‘clear search results’ button here too, and I still need to fix an issue with the results navigation when there are multiple results on one line. At the moment there is no way for the code to differentiate these so the ‘next’ and ‘previous’ links get stuck. I also need to look into speed issues with the search too. But some good progress has been made this week and I feel much more confident using eXist-db now.
Also this week I spent a bit more time on the Burns Supper Map, adding in videos and images for the recently imported supper records. I also had an email conversation with the Dictionaries of the Scots Language people about parts of speech and new data imports, which will probably be taking place in the next few weeks. I also attended a ‘coffee and catch-up’ with the other developers in the College, which was really valuable as always.
The rest of my week was divided between the Playbills project and the HiMuJe Malabar project. For Playbills I processed a spreadsheet containing around 400 performers that Deven had manually processed last week. I ended up doing a bit more manual tweaking after investigating the appearance of some of the performers in the playbill images, and when I ran the spreadsheet through my import script we then had 52,178 performers in the system, and of these some 52,048 of these have a gender assigned, which I pretty amazing.
I then moved onto processing the plays based on the spreadsheet Deven had worked on before Easter that notes which performances actually feature the same play, even if it is not referred to in exactly the same way. Using this I created canonical records for plays, I set some marked plays as ‘special attractions’ and I deleted certain plays that had been marked for deletion. Before I did this I updated the database so that all tables include an ‘isactive’ field, and updated the API so that it only includes data where ‘isactive’ is set to ‘Y’. Then when it came to deleting plays I didn’t actually delete them but set them (and all associated data such as performers and roles) to ‘isactive = N’. This means if we realise something has been ‘deleted’ that needs to be reinstated I’ll just need to update the relevant fields back to ‘isactive = Y’ rather than having to find and re-insert all associated data.
For the most part the script was successful and result in 261 plays being deleted and 1098 converted to special attractions. It then created 2039 canonical records and 4134 other plays were then assigned to these. There were a few issues with some plays referencing canonical records for other plays that were set to be deleted (so canonical records weren’t created for them), and I passed these on to Deven so she could look into them.
I then created an API endpoint and front-end pages for listing the canonical plays and the details for a selected canonical play. Below is a screenshot of part of the list of canonical plays, ordered by number of plays:
It’s just an initial version that lists the titles, the record type (either play or ‘special attraction’) and a count of the number of associated plays. You can press on column headings to order the table by the column and if you press on a canonical play name you can access a list of plays that are associated with it, as you can see below:
This lists each associated play’s name, date, playbill, venue, location and genres, and you can click on each linked item to reach the relevant page (e.g. the details for a play or the associated playbill page). This is just a work in progress and we’ll probably want to update it, for example to include a genre filter on the canonical plays list, or including access to lists of associated roles and performers. I also updated the playbill page to add links through from play titles to the relevant play page, and I’ve added a ‘Type’ field for each play that either displays ‘Play’ or ‘Special Attraction’.
For the Malabar project I sorted out the legend for source texts, alphabetising the list and ensuring the pane has a maximum width. I also added in the two other categorisation options that I’d included in my specification document: Number of references and placename languages. Number of references categorises the markers and polygons based on the number of times each place is referenced in the source texts. I set up the categories to match the available data (0, 1-5, 6-10, 11-20, 21+) but these can very easily be altered as the data and number of references grow. I think it will prove quite useful to be able to quickly identify the places that are referenced the most in the texts. Below is a screenshot showing the currently available data categorised by number of references:
The other new categorisation option was ‘Placename languages’ and this categorises the places based on the languages of the placename variants included in each record. This will likely need some further work, such as adding in full language names rather than the codes, and possibly filtering out ‘en’ as I’m guessing these would not have been found in the original sources, but it’s still interesting to use the categorisation – for example finding all placenames that have a Hebrew form. Here is a screenshot showing placename languages:
The data itself is still being compiled and I still need to work on the marker colours (and icons) and also to ensure the polygons always appear behind the markers and don’t make the markers unclickable, as is sometimes the case at the moment. Also this week I added the selection of base map, menu section and categorisation type to the URL, meaning it’s now possible to bookmark / share / cite specific views of the map. You can also link directly to a specific record as well.
This was another week of many projects, the first being the Burns Supper project. This week I finished implementing all of the data visualisations on the ‘facts and figures’ pop-up, with a mixture of bar, column and pie charts depending on how many data types there are. All charts are ordered by number of suppers, other than the pie charts. I also added in a statement above each visualisation about the number of suppers that supplied the data and a note when a supper may have more than one type of data. The screenshot below shows part of the popup with three visualisation types (mostly) visible:
I haven’t updated the narrative text to update the percentages to be based around the number of suppers that supplied the data rather than the total number of suppers and to include additional explanatory text, but I’ll do this next week.
For the HiMuJe Malabar project I have now incorporated the updated JSON file that contains cats and subcats into the map. I have also incorporated the GeoJSON polygons that project RA Renu has created so far. Where a place has a polygon this is used instead of a marker, and polygons feature a tooltip on hover-over in the same way as markers. I added the type and subtype to the tooltip as well. I also made a start on the categorisation of places based on cat and subcat, although this still needs some work. I added in a ‘legend’ box on the right of the map and have split the places into layers based on their cat and subcat. Each of these appears in the legend, and you can turn each layer on or off. You can also select or deselect all layers, which I find quite useful (e.g. deselect all then only add in the layers you’re interested in). There is also a count of the number of places in each layer as part of the layer label. Note that the cats and subcats only appear if there is at least one place that has latitude and longitude assigned to the subcat, which is why (for example) ‘Mosque’ is not currently listed. Below is a screenshot showing the categorisation options that are currently in place:
There’s still quite a lot to do and some issues that need to be addressed. At the moment only places that have a subcat appear. I still need to implement the top-level cats, so for example ‘Sri Lanka’ that has cat ‘Region’ and no subcat is not currently appearing on the map. I’m hoping to update the legend to give it a two-level hierarchy (as mentioned in my specification document) but I didn’t have time to implement this. Also, marker and layer colours are currently arbitrarily assigned. I think we will eventually have icons as well as colours, but in addition to this I will update the colours, probably to have different shades for each subcat in a cat. I also need to ensure that the polygons always appear behind the markers as this is not currently the case. E.g. if you turn the ‘Region: Kingdom’ layer off and then on again the layer is then added to the front, meaning it’s no longer possible to press on any markers located within the polygons. Also, there will be several other categorisation options in addition to cat and subcat (as discussed in the specification document) and these still need to be implemented.
Also this week, I returned to working on the Playbills project for the first time since before Easter. This included writing an executing a script to merge genre classifications, which reduced the number of genres from 188 to 69 and reassigned any plays that were assigned to a deleted genre. I also executed my scripts that process performers to extract individual names and titles from the free-text ‘full name’ column and assigned gender based on the title. My scripts process individual performers and also split multiple performers into individual records, resulting in the number of performers going up from 49498 to 52146. In these cases the newly extracted performers have also been assigned to the same role as the existing row. Of the 52146 performers we now have a Male or Female gender assigned to 51724, which I think is pretty good. I also updated the playbill and play pages so that the extracted names and genders now appear, and multiple performers for a role all now appear. I still need to deal with the ‘unprocessed’ performer spreadsheet, which is probably going to need some manual work as this contains the ‘edge cases’ that my processing scripts were unable to tackle. I hope be able to continue with this next week.
My fourth project of the week was the DOST Auld Laws project, for which I continued to work on the search facilities. This has involved setting up an Exist-db XML database and learning how to run full-text queries on the documents contained in it and how to generate KWIC (keyword in context) snippets for the results. There’s still a lot to do, but an initial version of the quick search is now operational, as the screenshot below demonstrates:
You can also use an asterisk wildcard to represent any number of characters, for example ‘wyn*g’ finds all terms beginning ‘wyn’ and ending in ‘g’. A question mark wildcard can also be used to represent a single character, e.g. ‘r?cht’ matches ‘richt’ and ‘rycht’. I haven’t added in pagination of results yet, so they currently all appear on one page. Each result features the document name and the page number where the result is found, plus the snippet with the term highlighted. Pressing on the snippet loads the page with the line where the term is found highlighted in both the image and the text.
It’s definitely something of a milestone to get the quick search working, but there is still a lot to be done. The highlighting of the term in the snippet is not currently working properly when the term includes characters in <expan>. The search finds the term, but the highlighting is only getting applied to the part of the term before the <expan> tag. Also, Exist-db uses the Lucene full-text engine and this does not support wildcards at the beginning of terms, so while a search for ‘grant*’ finds all terms beginning with ‘grant’, it’s not possible to find all words ending in ‘*ting’, for example. I’m not sure how big an issue this is, but if such functionality is required I’ll have to investigate an alternative. Also, phrase searching is not currently operational. I have a query that should be able to handle this, but I haven’t had time to add this in yet. It’s been quite tricky to implement as phase searches need to cross line and page boundaries whilst still returning the line and page IDs where the start of the phrase is found.
In addition, Boolean searches are not yet working (e.g. term1 AND term2, or term1 OR term2). However, we might want to consider how useful these would be and how they would work. Currently the search looks for occurrences of the term in entire documents and returns snippets showing the context of the term. If a Boolean search is using an entire document as its source then how useful would it be if (for example) term1 is found on page one and term2 is found on page 29? I’m just not sure how helpful this would be. I guess an OR search would be useful in finding variant spellings?
I also still need to add in results traversal to the document page, so you can navigate directly through results when looking at one specific result, and I need to add in a link back to the results page from the document page. There’s also still a lot of work to do on the user interface, which is still not finalised and I’ll work on this (e.g. colours, fonts, layout) once I’ve finished with the search facilities.
My fifth and final project of the week was the redevelopment of Mapping Metaphor. I didn’t have much time left to spend on this, but I did manage to tidy up the loose ends from last week. Where a combined metaphor has both OE data and E data and the E data does not begin in the OE period the timeline in the visualisation card popup now includes OE selected and highlights the period when the E data begins. This second highlight currently uses the green used for the visualisation background but can be changed. We should also maybe include some explanatory text so users understand why two periods are selected. The following screenshot shows the timeline with two periods highlighted:
I also updated the card popups so that both E and OE IDs appear in the header of the card (where applicable) now, as you can also see in the above screenshot. Also, where a metaphor connection exists in E but not OE (or vice-versa) counts of lexemes in each joining category in the other period are now displayed.
Also this week I had an email conversation with Craig Lamont about an online resource for his new research project and arranged for a subdomain to be set up for it. I also updated the DSL survey so that it only appears on mobile devices and responded to a request from Kirsteen McCue regarding stats for song downloads on the Editing Burns and Burns Choral websites.
This was a pretty busy week that had me dividing my time between five main projects. I spent most of Monday working on the Burns Suppers project, beginning development of the ‘facts and figures’ popup. I added a button in the ‘Home’ menu labelled ‘Facts & figures’, that features a pie chart icon. Pressing on this opens the facts and figures popup part of which you can see in the screenshot below:
The popup features the narrative summary section, with the figures in this section being dynamically generated. It took quite some time to write the code to generate them, but at least there will be no further work to do when we import more data. For now all percentages are the percentage of the total number of suppers rather than a percentage of the number of suppers that supplied the data, so for example, the ‘annual supper’ figure is 39% rather than displaying 83% (based on there being 420 suppers that actually have frequency data). This is because I realised that without a lot of additional explanatory text users will likely think our figures are wrong. If people only see the total number of suppers (886) and the number that are annual (347) then displaying 83% will be misleading. We’d have to include lots of additional data such as “of the 420 suppers that included frequency data, 347 (83%) were annual events”. Depending on feedback I may implement this later.
I’ve only implemented one graph in the pop-up so far, which is the graph of countries. This is a long list, but I think it works ok. I also included an option to switch from a graph ordered by number of suppers to an alphabetical version, and this is all operational. I’ll continue to add further graphs next week.
I spent most of Tuesday working on the interactive map for the HiMuJe Malaber project. I’ve created an initial version of the map now, which uses my map menu interface that I’ve used on several other resources, and below is a screenshot:
By default the map uses the ‘Watercolour’ base map, as we used for a static map on the main site (https://himuje-malabar.glasgow.ac.uk/about/summary/). You can also switch to a satellite map using the ‘Change the base map’ buttons. Currently markers as just displayed as red dots, and if you hover over them the ‘Preferred Name’ from the JSON file is displayed as a tooltip (although I’ve noticed that some places don’t seem to have a ‘Preferred Name’). There are no popups or filters or anything like that yet. Also note that any places in the JSON file that don’t currently have latitude and longitude values are not displayed as there is nowhere to ‘pin’ them.
You can zoom and pan the map as with Google maps, and the icon in the bottom right makes the map full screen. The map also works on mobile devices (hiding the left-hand menu using the ‘<’ button above the menu helps when using a mobile device). Other than the base map selection options, nothing works in the left-hand menu yet. ‘Places’ will eventually include the place filters. ‘Travel Routes’ will list travel routes involving people and organisations once this data is available. There’s still a lot to do, and I’ll hopefully begin work on the place popups and the categorisation options next week.
I spent most of Wednesday working on the Place-names of Armagh project. I’ve updated my ‘Irish form / translation’ script to remove ‘More details: Unverified’ and to also attempt to split the translation up based on apostrophes. This has been a bit of a nightmare, firstly because apostrophes are not only used to denote the translation but appear within the text, and secondly because what looks like an apostrophe can actually be many different characters, including straight, curly opening and closing apostrophes and several HTML codes that are rendered as apostrophes but are stored as codes. This has all made splitting the text up rather challenging. However, I’ve got something that mostly works.
The script now deals with things like Ir. <em>Coill Uí Fhloinn ‘O’Flynn’s </em>wood’ and outputs “O’Flynn’s wood” as the translation. Where there are multiple sections some unnecessary apostrophes appear, for example: “Ir. <em>Tír Garbh </em>’rough land or district’ or perhaps Ir. <em>Baile Uí Aodha </em>'<em>O’Hugh’s</em> homestead or townland’” results in: “rough land or district’ or perhaps Ir. Baile Uí Aodha ‘O’Hugh’s homestead or townland” This will probably need some manual fixing. The script also now processes rows that don’t have ‘Ir.’ and italics, for example “Críonchoill ‘withered decayed wood’” now has Irish form “<em>Críonchoill </em>” and translation “withered decayed wood”.
I sent an Excel version of the script output to the project team as this will likely require some (but hopefully not too much) manual intervention to fully sort out. For example, the text “Eng./Sc. ‘hill of the military camp’ or perhaps Ir. <em>Mullach na Críne </em>’hilltop of decay’ or Ir. <em>Mullach na Craoibhe </em>’hilltop of branch, tree’” will need some work as the translation omits the ‘Eng./Sc.’ Text as it’s before the first apostrophe.
The other thing I’ve managed to do today is to set up an initial version of the public map interface. This currently takes quite a long time to load as it’s processing a lot of data. As with the other sites, I’ll create a cached version of the data once we’re ready to launch, which will be much faster to load. The reason it’s not in place now is that a new version of the cache will need to be generated any time you want subsequent changes made to the CMS to appear on the map, and we’re still very much working on the data.
For now, three base maps are available (satellite, satellite with labels and relief). We can add more in later. There are also no parish boundaries or townland / barony boundaries as I don’t have this data yet. I also still need to work on the colours of the markers as there are no colours for a lot of the classifications so they’re defaulting to purple. There’s also nothing in the elements glossary as we don’t have this data yet. But the search, browse and categorisation options all work. For example, here are the placenames beginning with ‘A’ categorised by altitude:
There are some issues with the data. For some reason there are three place-names miles away from Armagh, around Ballybofey, and there are a couple of place-names appearing in the Irish Sea. We also have an issue of the same coordinates being used for multiple places, thus resulting in markers appearing on top of markers. There’s also an issue with the accuracy of markers too. For example, the marker for ‘Lowry’s Lough’ is found about 500m south of the actual body of water. But the good thing about having this map available (despite the speed issues for now) is that it will help when working on the data.
I spent all of Thursday and most of Friday morning learning how to use the Exist-DB XML database that I’m hoping to use for the DOST Auld Laws project and developing the query that will eventually power the quick search, and form the basis for the advanced search. I installed Exist on my laptop and managed to set up a collection for the project’s XML files, which I then uploaded into the system. I followed the documentation available on the Exist website and was able to create a full-text index for the collection, and I followed a useful tutorial here: https://dh.obdurodon.org/php-xquery.xhtml about how to query Exist using the REST interface. Setting up and querying the texts in Exist was all new to me and there was a lot to try and take in, with many configuration options that were not all that easy to follow in the Exist documentation. I ended up using ChatGPT quite a lot to help me figure out how everything should work and why some of my initial tests were not producing any results. This proved to be extremely useful and really helped increase my understanding of Xquery. I was able to get a search working that queries the full text and returns contextual snippets for each result, together with the IDs of the line, page and document, which is everything I’ll need for the quick search. On Friday I asked Luca to help set up the necessary collection on the server and hopefully I’ll be able to get an initial version of the quick search working on the website next week.
This left me with a few hours on Friday afternoon to devote to the redevelopment of the Mapping Metaphor resource, for which I’m creating a unified view of the data, joining both the English and Old English datasets together. I used this time to implement the visualisation card view when viewing connections between specific categories. The combined card view shows a bidirectional arrow if the OE and E directions differ. It also defaults to the E strength. The counts of lexemes in each category include both full and OE counts and the examples of metaphor feature both OE and E examples, with OE coming first. If the metaphor exists in the OE data then the ‘start era’ now defaults to OE (but as of yet I’ve not added in a further highlight in the timeline to show when the first non-OE occurrence was documented). Below is an example of the combined card view:
In addition to the above I also looked into an issue raised by Ann Fergusson for the Dictionaries of the Scots Language regarding accented characters, search results and entry slugs. This took some time to investigate but I think my response proved useful. I also had email discussions with Deven Parker about her Playbills project, which she now has time to look into again. I’ll probably be working on this again next week.
This was a four-day week as Monday was a bank holiday. I spent most of Tuesday and Wednesday continuing to work on the DOST Auld Laws project, working with the XML files. The XML files generated by Transkribus and exported by the tool as TEI XML contained many elements that were not valid TEI elements, such as the custom <Aitken> element that had been applied to notes added by A J Aitken. My first task of the week was to write and apply transformations to the XML files to convert them into fully valid TEI. I achieved this using XSLT, which is a language I find very unintuitive and frustrating to work with, no doubt exacerbated by the fact that I don’t work with it very often. I struggled to get any transformations to run initially, and ended up turning to AI to figure out why my scripts were not working. In this instance AI proved to be extremely useful as it identified what the problems were (e.g. I hadn’t declared the correct namespace or used it when writing the rules) and really helped me to understand how everything fitted together. I still wrote the bulk of the code myself, but AI was very helpful in identifying errors or issues. By the end of Tuesday I had generated (and checked) a collection of TEI files that successfully validated in Oxygen, which was a good milestone to reach.
On Wednesday I then worked on the front-end for the project, figuring out how to transform the valid TEI XML into HTML for display in the ‘text and image’ and ‘text only’ views of document pages. As this was once more using XSLT I enlisted the help of AI to figure out specific issues that I was unfamiliar with. The biggest of these was how to pick out and process one single page from a document’s XML file based on the <pb/> element. I had no idea how to achieve this, and despite this being a fairly fundamental issue when processing TEI documents I didn’t manage to find any useful information online. However, AI came up with a solution (and equally importantly an explanation) in a few seconds and I was then able to incorporate this into my code, transforming the contents of one page, whose ID was passed to the XSLT script as a parameter, to HTML with a variety of styles applied to the various elements.
I also updated the ‘click on a line in the image’ feature, and it’s now possible to deselect the line if it’s already selected – previously once you’d highlighted a line you couldn’t get rid of the highlighting, only move it to a different line but now if you press on the highlight it’s removed. The lines are also now connected to the text view – pressing on a line in the image also highlights the corresponding line in the text. You can also press on a line in the text to highlight it and the corresponding line in the image.
I also updated the height of the text pane so that it matches the height of the image pane and if the text is longer the pane scrolls. This ensures that if the text is very long it’s still possible to see the image, rather than having the entire page scrolling, which may result in the image not being visible when you’re at the end of the text. It is how we did things in Books and Borrowing, but having a scrollbar in a section of the page in addition to the browser’s scrollbar can be annoying for some people so I might revert to the previous layout depending on feedback. Here’s a screenshot showing the transformed text, a highlighted line and some of the formatting that’s been added:
Also this week I continued to work on the Place-names of Armagh project. I wrote and executed a script that generated Irish grid references and ITM values for all places based on their latitude and longitude, a task that I completed using the ‘GridRefUtils’ scripts as detailed here: https://www.howtocreate.co.uk/php/gridrefapi.php. I’ve used these scripts before on previous place-name projects and they’ve been hugely useful. I then ran a further script to generated altitude for the place-names by connecting to the Google Maps API, so we now have complete geospatial data for all of the places (other than the 386 that didn’t include Easting and Northing data in the original spreadsheet). I also added in a new parish and barony in a different county that one place-name requires, and had discussions with the team about the splitting of analysis data into Irish forms and translations. Removing the Irish forms from the translation field is going to be rather tricky to automate as the Irish forms often form part of the translation. It’s looking like I’ll need to automate the transformation of some of these, with the rest then requiring manual intervention.
I also continued working on the redevelopment of the Mapping Metaphor resource to create a combined English and Old English map, something I began last week. This week I managed to get the combined view of the drilldown of the visualisation working. The counts represented by the yellow circles also use the combined data, and these are also displayed in the pop-up card, for example, if you select 1K02 Creation and press on the yellow line or circle for 1B Life. In the combined map there are connections to 6 categories in Life whereas there are 5 in the E map and 2 in the OE map (one of the OE ones is also present in the E map, which is why the combined total is 6). The combined pop-up card also now lists the number of OE lexemes in addition to the E lexemes, for example “1K02 Creation 1088 lexemes / 127 OE lexemes”.
I still need to implement the combined visualisation view of the search results, and also the card pop-up between individual categories, which is going to take quite some reworking (possible direction changes, combined example lexemes, timeline updates etc). I’ll hopefully find some time to continue with this next week.
Also this week I processed and added some more images to a few Burns Suppers, had a chat with Garrick Allen about a research project he’s wanting me to be involved with later in the year, and contacted Lindsay Balfour about a proposal she’s writing that will have some technical requirements. I also made a couple of tweaks to the Thesaurus of Old English website after Fraser go in touch with some suggestions, made some tweaks to the survey popup on the Dictionaries of the Scots Language website, gave some mapping advice to Renu of the HiMuJe Malabar project, and created an alternative version of one of the Anglo-Norman Dictionary’s textbase documents that strips out all Latin text.