I had my PDR this week, which meant spending some time preparing and then attending the session, which was all very positive.  Other than this I spent quite a lot of time this week working for the Anglo-Norman Dictionary, mostly working on scripts to rationalise the parts of speech used across the dictionary’s entries.  The largest aspect of this was writing a script that would go through all of the live entries in the database and find all occurrences of a particular POS (e.g. ‘s._pl.’) and replace these with a standardised form (e.g. ‘s.pl.’).  This involved going through the entry XML files to make changes not only to the entry-level <pos> tag, but also to look at the <senseInfo> tags for every sense and subsense, both main and locutions, to replace the ’pos’ attributes found there.  The script also then needed to update the ‘pos’ table in the database (which hold a list of all parts of speech associated with each entry) to delete the old forms and add in the new forms.  It was clearly important to fully test this script, and make backups of the data prior to running it, as it would need to run through every live dictionary entry, making updates to the XML where necessary.  I tested everything out on an instance of the database running on my local PC, and once I was absolutely sure the script did what was required (and nothing that wasn’t required) I ran it on the live database, making changes to numerous parts of speech and updating at least a couple of thousand of entries.

I also had to update the list of parts of speech in the dicionary’s DTD, adding in a few new options that had been supplied to me.  Unfortunately some of these had spaces in them and these are invalid characters in list options in a DTD.  I hadn’t spotted this so making the update unfortunately stopped the dictionary entries displaying on the website, but thankfully once we did notice this it was a very quick issue to fix.  I also ran a few more iterations of the POS updating script to target other parts of speech that had spaces in them that had managed to find their way into entries over the years.

I also had a lengthy email conversation with the AND editor Geert about how the new POS search facilities would function, as these steps to rationalise the parts of speech are leading up to the implementation of new search options.  A POS selector will be added to the headword search, but we’ve still not fully decided which other searches should also incorporate such a selector.  I had thought possibly ‘citations’ and ‘English translations’ might benefit from a POS option, but Geert didn’t seem to think so.  He did suggest the ‘label’ and ‘language’ searches should possibly be updated, though.

These searches will not be able to just use the ‘entry’ level POS list that we already have – instead we’d only want to target specific citations where the <senseInfo> POS matches, otherwise entries that have multiple POS elements may bring back results that don’t actually match.  For example

This will require updating the database to store POS for each citation and translation, regenerating all of the citation and translation data for every live entry to populate the new fields and updating the DMS and batch ‘upload entries’ scripts to ensure this additional data is extracted and stored.  For example, a label search for ‘alchem.’ brings back https://anglo-norman.net/entry/arder where the label appears in senses that have ‘p.pr. as a.’ and ‘p.p. as a.’ as their POS.  If we add a POS filter that only works at entry level then someone selecting ‘v.trans.’ and ‘alchem.’ Would still find this entry, even though the label isn’t found in the ‘v.trans.’ section.  This is going to need some further work.    For now I’m waiting for Geert to send me a list of the parts of speech he would like users to be able to select through the search before I take this further.

Also this week I responded to a few requests from the DSL and also Eleanor Lawson regarding a new proposal she’s putting together.  I also worked with Andrew McHugh of RCaaS to migrate the old Emblems sites to a new server, although we’re encountering what looks to be a DDoS attack on this site.  I can’t really go into too much detail about it here, but I spent quite some time this week with both Andrew and Luca investigating the situation and looking for solutions.  As of yet the attack is still ongoing and we’re still trying to find a solution.

In addition to the above I also spent a bit of time processing some data for a new edition of Burns music that will be added to the Robert Burns website.  I haven’t finished working with this data yet, as I had some questions for the researchers, but hopefully the information will be available in the next week.

Finally this week I spent a few more hours processing some of the messy data for the Place-names of Fife project.  This mostly involved fixing errors in the historical forms relating to sources, and I wrote a few scripts to fix several patterns of error that I encountered.  The Fife data was originally in a Word file and several years ago I managed to convert this into structured data, but some edge cases in the almost 30,000 historical forms were not successfully processed and it’s these I’m currently trying to tidy up.  I think I’m just about there now, and on Friday I migrated the historical forms to a new table that will eventually be used for a new interface.