Week Beginning 10th May 2021

I continued to work on updates to the Anglo Norman Dictionary for most of this week, looking at fixing the bad citation dates in entries that were causing the display of ‘earliest date’ to be incorrect.  A number of the citation dates have a proper date in text form (e.g. s.xii/xiii) but have incorrect ‘post’ and ‘pre’ attributes (e.g. ‘00’ and ‘99’).  The system uses these ‘post’ and ‘pre’ attributes for date searching and for deciding which is the earliest date for an entry, and if one of these bad dates was encountered it was considering it to be the earliest date.  Initially I thought there were only a few entries that had ended up with an incorrect earliest date, because I was searching the database for all earliest dates that were less than 1000.  However, I then realised that the bulk of the entries with incorrect earliest dates had the earliest date field set to ‘null’ and in database queries ‘null’ is not considered less than 1000 but a separate thing entirely and so such entries were not being found.  I managed to identify several hundred entries that needed their dates fixed and wrote a script to do so.

It was slightly more complicated than a simple ‘find and replace’ as the metadata about the entry needed to be regenerated too – e.g. the dates extracted from the citations that are used in the advanced search and the earliest date display for entries.  I managed to batch correct several hundred entries using the script and also adapted it to look for other bad dates that needed fixing too.

I also created a new feature for the Dictionary Management System: an entry proofreader.  It allows an editor to attach a ZIP file containing XML entries and it then displays all of these in a similar manner to the live site, only with all entries on one long page.  The editor can then select all of the text, copy it and then paste it into Word and the major formatting elements will be retained (bold, italic, superscript etc.).  I tested the feature by zipping up 3,178 XML entries and although it took a few minutes to process, the page displayed properly and I was able to copy the text to Word (resulting in a 1,029 page Word file).  After finishing the initial version of the script I had to tweak it a bit, as I wrote the HTML and JavaScript with the expectation that there would be one dictionary item on the page and some aspects were not working when there were multiple items and needed updating.  I also ensured that links to sources in entries work.  In the actual dictionary they open a pop-up, which clearly isn’t going to work in Word so instead I made the link go to the relevant item in the bibliography page (e.g. https://anglo-norman.net/bibliography/B#bib-Best).  Links to other dictionaries, labels and other AND entries also all now work from Word.

In addition, cogrefs appear before variants and deviants, commentaries appear (as full text, not cut off), Xrefs at the bottom now have the ‘see also’ text above them as in the live site, editor initials now appear where they exist and numerals only appear where there is more than once sense in a POS.

Also this week I did some further work for the Dictionary of the Scots Language based on feedback after my upload of data from the DSL’s new editing system.  There was a query about the ‘slug’ used for referencing an entry in a URL.  When the new data is processed by the import script the ‘slug’ is generated from the first <url> entry in the XML.  If this <url> begins ‘dost’ or ‘snd’ it means a headword is not present in the <url> and therefore the new system ID is taken as the new ‘slug’ instead.  All <url> forms are also stored as alternative ‘slugs’ that can still be used to access the entry.  I checked the new database and there are 3258 entries that have a ‘slug’ beginning with ‘dost’ or ‘snd’, i.e. they have the new ID as their ‘slug’ because they had an old ID as their first <url> in the XML.  I checked a couple of these and they don’t seem to have the headword as a <url>, e.g. ‘beit’ (dost00052776) only has the old ID (twice) as URLs: <url>dost2543</url><url>dost2543</url>, ‘well-fired’ (snd00090568) only has the old ID (twice) as URLs: <url>sndns4098</url><url>sndns4098</url>.  I’ve asked the editors what should be done about this.

Also this week I wrote a script to generate a flat CSV from the Historical Thesaurus’s relational database structure, joining the lexeme and category tables together and appending entries from the new ‘date’ table as additional columns as required.  It took a little while to write the script and then a bit longer to run it, resulting in a 241MB CSV file.

I also gave some advice to Craig Lamont in Scottish Literature about a potential bid he’s putting together, and spoke to Luca about a project he’s been asked to write a DMP for.  I also looked through some journals that Gerry Carruthers is hoping to host at Glasgow and gave him an estimate of the amount of time it would take to create a website based on the PDF contents of the old journal items.