I had a couple of meetings this week, the first of which was with the Books and Borrowing team to discuss a potential new project. I had to do a fair bit of preparation for this and had several discussions following the meeting too. It’s all looks really great, but I can’t go into any more details at this stage. My second meeting was with Deven Parker to discuss her Playbills project. This was also a very productive meeting and plans are beginning to come together. I’ve been invited to participate in a meeting Deven is having with some AI people (erm, that’s people studying AI rather than ‘AI people’… at least I hope so) in Computing Science next month, which I’m pretty excited about.
I spent quite a lot of time this week continuing with my mockups of a new interface for the Dictionaries of the Scots Language website. I can’t share any screenshots of my work yet, but this week I added in the ‘Add yogh’ button to the ‘Older Scots’ search bar (both index page and entry page). This appears as a button to the left of the bar with ‘What is yogh’ added as a link underneath. Pressing on the ‘Add’ button adds the yogh to the input. Pressing on the ‘What…’ link opens a modal overlay featuring the explanatory text. I also created a mockup of the ‘advanced search’ form. As with the live site, the advanced search features a tab for entries and another for bibliography. Each has a search form section and a help section and the layout is pretty similar to the live site, but has been modernised and tidied up. As with the live site, pressing on the ‘In’ buttons in the entry search changes which search options are visible and the layout works a lot better on mobile screens than the live site does.
Finally I completed a mockup of the ‘About’ page. The page features boxes for the ‘top level’ information, each of which appears as a link (not currently linking to anywhere). Where an information type has subpages these appear in a list within the box. Initially I wasn’t going to add a quick search box to the ‘About’ pages, but decided it would be better to do so. However, as these pages are not within a selected dictionary section the search bar needs to include a dictionary selector. I’ve added this to the left of the search and hopefully it should be intuitive to use. At the moment the yogh information only appears when ‘Older Scots’ is selected and is hidden when ‘Modern Scots’ is selected. When Older Scots is selected the search bar text input does get a bit small on mobile screens, but it’s still perfectly usable. I’d envisage such a bar appearing on all of the ancillary pages, but we would retain the specific dictionary searches when in a dictionary.
I also continued with updates to the Anglo-Norman Dictionary this week, using my newly created workflow to add a further six texts to the site’s Textbase. I also rearranged the ‘browse’ page so that the texts are now arranged by genre, and there are buttons to jump straight to a genre you’re interested in. You can view the updated feature here: https://anglo-norman.net/textbase-browse/ and below is a screenshot:
Also this week I added further content to the Speechstar website (more videos and an additional ‘Phonemic target’ metadata field for all records listed here: https://www.seeingspeech.ac.uk/speechstar/disordered-child-speech-sentences-database/. I also went through the other site (https://speechstar.ac.uk/) to rename to project from ‘Speech Star’ to ‘SpeechSTAR’ wherever this text appears. I also had to spend rather a lot of time creating timesheets for every month I’ve worked on the project since July 2021 due to my time having been costed incorrectly. As you can imagine, this was quite a long and tedious task, but thankfully it was made easier by having this blog to consult.
I’ll be on holiday next week so there will be no more from me until the start of July.
My time this week was devoted mostly to dictionaries, with about half my time devoted to the Anglo-Norman Dictionary and the other half working for the Dictionaries of the Scots Language. For the AND I continued with the creation of a workflow to add new texts to the Textbase (https://anglo-norman.net/textbase-browse/). Last week the editor Geert had created a complete XML version of a new text to be added using the Oxygen XML editor and a limited set of tags based on the existing Textbase texts. As I discovered last week, while these texts use some TEI elements they are not actually valid TEI as they don’t link to a DTD and include undeclared non-TEI elements, but we’re going to have to stick with this structure for the new text to ensure compatibility with the older ones.
I spent some time this week working on the scripts that would then import this text, based on the scripts I’d initially created to batch import the older Textbase texts. It was a bit of a lengthy process as it’s not just a case of importing the text but also generating the search terms including KWIC for the concordance and proximity searches, but I managed to get everything working with the new text, testing things on a version of the site running on my laptop rather than the live site, as adding a text directly to the live site would immediately make it live.
One issue with the text is that it is tagged as one continuous page, with folio breaks used throughout rather than page breaks. This means that all of the footnotes are added to the very end of the text. I suggested to Geert that the text could be split using the <pb> tag for folios instead, which would help break the text up and make the footnotes easier to use. It turned out that Geert had been using the folio break tag for both paragraph numbers and folio breaks, which wasn’t quite right. Swapping to using pagebreak tags purely for actual folio breaks would work much better as there are only 20 or so folios while there are many more paragraphs.
I decided to work on the XML myself to restructure it before sending it back to Geert for final tweaking. I replaced the proper folio breaks (as opposed to paragraph IDs) with page breaks and have given them an ID beginning with ‘fol-‘, e.g. ‘fol-1ra’. Geert wanted a different format including spaces, but we can’t include them as the values end up being used as JavaScript IDs and spaces are invalid in IDs. I think the resulting page markers look ok, though. I then changed the way the paragraph markers were recorded. These are now just an ‘n’ attribute of the <p> element, e.g. ‘<p n=”1.1″>’ which I reckon makes a lot more sense than having them as part of the actual text. I updated the XSLT so that when a paragraph has an ‘n’ attribute this gets displayed in bold square brackets before the paragraph text. The only concern here is that the XSLT is applied to all texts in the textbase so if any existing texts have paragraph IDs these are now also going to end up displayed like this.
I also ensured the import scripts extracted all of the text for the Textbase concordance and proximity search. This involved many stages, including splitting the text into pages, then splitting the pages into individual words, logging counts of the number of words on each page and generating the ‘Keyword in Context’ for every word in the text. It took a bit of trial and error to get the scripts working properly with the new text, but I got there in the end.
With the scripts in place I was then able to run the new text through the import process on the server, resulting in the text being available through the ‘Browse’ facility and fully incorporated into the search facilities. You can view the new text here: https://anglo-norman.net/textbase-browse/sjer. Soon after completing work on the import of this text the editor Delphine sent me a further text that she had been working on. This was very handy as I was able to test the import scripts on a new text whilst everything was still fresh in my mind. Thankfully the scripts worked perfectly and this new text was also successfully incorporated and can be found here: https://anglo-norman.net/textbase-browse/otinel. I then suggested to Geert that we should maybe update the ‘Browse’ feature to break up the list by genre as currently the texts are all just listed alphabetically by title, which is possibly not all that helpful. I’m going to investigate this next week.
I spent most of the remainder of the week working for the DSL. I’d previously discussed with the team how the existing website design was getting a bit long in the tooth, as it will be 10 years this September since we launched the site. Users of the site increasingly access it on mobile devices and while the interface was designed to work on much devices it doesn’t make best use of the space available on such screens. I therefore wanted to experiment with a new interface for the website, which is what I did this week. I’m not able to share any screenshots of my experiments at this stage as I haven’t even heard back from the DSL people yet, but I’ll give an overview of what I’d been working on so far.
The interface I created uses the Bootstrap front-end toolkit, which is an ideal starting point for creating mobile-friendly sites. I carried over some design aspects from the live site, and have also borrowed some elements from my work on the Anglo-Norman Dictionary. I created a working mockup of the interface for the homepage and dictionary entry pages and shared this with the DSL team.
The biggest structural change to my test interface is something that has been discussed before: Splitting up SND and DOST. The mockup site only has four top-level menu items: ‘Modern Scots’, ‘Older Scots’, ‘Advanced Search’ and ‘About’. I’ve gone with ‘Modern Scots’ and ‘Older Scots’ as these are more understandable than ‘SND’ and ‘DOST’ and there is no space for the full titles or dates. As it currently stands all four tabs fit on one line on a mobile device (tested on my Android phone and an iPhone 15).
The default landing page for the site is ‘Modern Scots’. It features a large ‘Search Modern Scots’ input and some brief text about the SND, which I’ve taken from the current homepage and the ‘about SND’ page. This is followed by three info boxes as found on the right-hand side of the current homepage. On mobile devices these are stacked. Note that I haven’t added the ‘Add Yogh’ button to any quick search form yet. There is also a separate ‘Older Scots’ homepage, which is very similar but with information about DOST.
I then created a mockup of an SND entry page for ‘Dreich’. The entry page is now two-column. A ‘Search Modern Scots’ input appears at the top of the main column. The entry itself has a summary section at the top (influenced partially by the new OED site). This includes the headword and POS (as already found in the ‘sticky’ header that appears when you scroll down the entry page on the live site), information about the dictionary, the ‘about’ text and options to show/hide things etc. Dates of attestation and the sparkline are also included here. I’ve gone with a rounded-end for the sparkline to make it look a bit more swish. The main entry itself is exactly as it appears on the live site. Note that a ‘Top’ button appears in the bottom right as you scroll and pressing on this scrolls the page back to the top. For now I haven’t included a ‘sticky’ header as we have in the live site but I can add this in. The right-hand column is also not ‘sticky’ as I’m not convinced it’s really necessary and wouldn’t work on mobile devices anyway as the column will appear beneath the entry.
The right-hand column is adapted from the Anglo-Norman site. It defaults to the ‘Browse’ but has further tabs for ‘Results’ and ‘Log’. Neither of these currently work, but the former would feature the search results as currently found in the live site’s left-hand column and the latter would contain a list of all entries you’ve looked at during the current session, which is quite a handy feature I added to the AND site. The ‘Share’ box is also present in the right-hand column. On mobile screens the right-hand column appears below the entry column.
I also made a similar mockup for DOST, using the DOST entry for ‘Scunner’. It’s currently identical in structure to the SND entry, but gives you another example of the entry page, with a more interesting sparkline. We could differentiate SND and DOST by using a different colour scheme for the DOST entry page (e.g. a different shade of blue for the entry header) but for now everything is the same.
I haven’t added in the search results page yet, but it will be fairly similar to the live site. I’ve also not added in a mockup of the Advanced Search form yet, but this will be the only place to search both dictionaries. It will be pretty similar to the live site. The ‘Results’ tab on the entry page will work in a similar way to the left-hand column on the entry page on the live site. Where an advanced search involving both dictionaries is performed the results will be split like the live site. We could make the ‘About’ tab a drop-down menu but there are so many ‘About’ pages that I think it might work better if it just led to a page that listed the various sections (‘Our Publications’, ‘About Scots’ etc) and gave links to everything. Similar to the OED’s ‘Information’ page.
I did consider having both a top section of each page under the selected tab, and the selected tab itself in blue like the entry header, with this then containing the search and the entry header and then the rest of the page below this being white. This would sort of give a similar approach to the OED’s entry page. However, I went with a white tab and all white page to make it clearer that all of the content ‘belongs’ to the tab. In terms of the site font I’ve used an updated version of the one used on the live site, but we can change this, or use multiple fonts for different sections. I added a subtle gradient to the background of the header and footer (it gets lighter to the right), but this can be changed.
I’ll just have to see what the DSL people make of my experiments and we’ll see if any aspects of them find their way into the live site.
I returned to working on the Iona place-names interactive map this week, which I’d not worked on for a while. The team had demonstrated the resource at an event last week and had a few suggestions after this. They’d noticed that the map was rather cluttered on mobile devices with both the side panel and the map legend opened and I therefore added in a check for screen width when the page initially loads. If the width is less than 500 pixels the legend is hidden by default, which stops multiple panels appearing and overlapping. I decided against hiding the left panel as this might be more confusing as users will generally need to use this panel and if it’s off by default they might not realise it’s there.
I also made a couple of other tweaks. Firstly, I’d noticed the ‘Select all’ option in the legend was remaining visible when the legend was hidden and I sorted this. It was actually rather tricker than I’d expected to get this working but it’s sorted now. I also replaced the ‘Show / Hide’ text that was the only text in the legend button to just display ‘Legend’ instead (along with the up/down icon). I figured this would make it clearer what the section was when it’s hidden by default. Secondly, the tabs in the record popup were overlapping on narrow screens when the tabs were split over two lines so I added a vertical margin to the tabs to fix this. I don’t think there are any other issues with the layout on mobile devices, but I’ll just need to see if anything else gets reported. Below is a screenshot of the map on a mobile device, showing the legend closed by default and the left-hand panel open:
I was also alerted to the fact that I hadn’t implemented the display of images associated with place-names yet so I implemented this. I had to update the API to ensure image data was outputted and for places that have images the record popup now features an ‘Images’ tab. Pressing on this displays the image and its caption, for example:
For the moment if there are multiple images for a place they will each appear in the tab, one after the other. We might want to consider some sort of slideshow instead. I also ensured that sounds associated with place-names are now also set to be returned by the API, but as of yet I haven’t added in a tab for them, as we don’t currently have any sounds in the system.
The team also wanted cross references between place-names to appear in the mini-popup in addition to the main one, so I added these in. However, whilst working on this I realised there was a bit of a problem with how cross references were working: The links would only work if the cross referenced place-name was also part of the currently visible map data. For example, a quick search for ‘buaile’ results in only 6 matching place-names on the map. The cross reference for ‘Buaile Staonaig’ is ‘Loch Staonaig’, which is not part of these results and therefore pressing on the cross reference wasn’t doing anything. This was a bit of a headache to sort out, resulting in some pretty major changes to the code.
What happens now is if the cross referenced place is also on the map then clicking on the link will move you to it as you’d expect. But if the place is not on the map then when you press on the link the map resets itself, loading in all place-name data. Once the data is loaded in then the system identifies the cross referenced place and moves the map to it. It’s maybe a bit confusing to have all of the data loading in, but I just couldn’t think of another way to do it without messing up the view of the map with place-names that don’t match the criteria.
Also this week I spent some time making final alterations to the websites for the three new Speak For Yersel regions. This included adding in logos, replacing some of the images used in the surveys, adding in new questions and making tweaks to existing questions, adding in the ‘About’ pages and removing all of the test users and their data from the systems ahead of piloting. The resources are ready to go now, although it’s likely that some further tweaks will be required after the piloting phase.
I also spent about a day further researching a new potential project on 18th and 19th century playbills for researcher Deven Parker. I can’t really go into too much about this here, but there was a lot to consider and I sent a lengthy email to Deven containing many discussion points.
My other main task of the week was to investigate adding new texts to the ‘Textbase’ of the Anglo-Norman Dictionary (see https://anglo-norman.net/textbase-browse/). Geert had sent me a new text for me to try and add to the resource and I spent some time working with it and the various import scripts that I’d created for the existing Textbase texts.
Unfortunately the structure of the new file is quite different to the existing XML files. I was hoping I’d be able to just run the XML through the same scripts but this isn’t going to work without a lot of changes to the scripts. There are no <pb> tags in the new file. These are used to denote page breaks and are important for the search facilities, as words extracted for search purposes are associated with a page record in the system. If having pages really doesn’t matter I can easily just add a <pb> around all of the text, which should sort the issue. There may also be an issue with the ‘jump to page’ feature, which would of course only then contain one page. Related to this, as there are no page breaks all notes will end up appearing at the very end of the text as technically there will be no ‘pages’. The XML (which was generated using a tool called the Classical Text Editor) contains a lot of data relating to the style of elements (within <tagsDecl>) and a lot of the text is contained within <hi> tags, although it’s not clear why. E.g. we have <hi rend=”font-size:12pt;”>Nous lisoms en anciens estories les uns avoir environee les provinces, et avoir alee as novels poeples, avoir passee les meers, q’eux verroient devaunt</hi> but in the PDF view of the text there is no discernible reason why ‘devaunt’ is the last word in this tag as the first word in the following tag (ycestes) follows directly on from it. Also, the numbers at the start of paragraphs are not specifically marked up but are instead just part of the text, (e.g. <hi rend=”font-size:12pt;”>[1.1] Frere Ambros, a moi portaunt…). This means we wouldn’t be able to separate out the numbers from the text or apply a different style to them. There was also a lack of metadata about the text, and what was there was not in the same structure as in the existing textbase texts.
I suggested to the editor Geert that as the Classical Text Editor seems to add an awful lot of unnecessary (from our point of view) tags that result in a very messy XML file whether the texts could instead just be produced directly in Oxygen. I spent a bit of time creating a sample XML document in Oxygen, including just the first few paragraphs of the text.
It seemed pretty feasible to me to create the texts in this fashion, but I did spot an issue. The existing Textbase texts are not actually valid TEI XML. The texts don’t link to a DTD or the TEI namespace and when I did try doing so with my sample file in Oxygen it produced a lot of errors. E.g. the ‘fb’ element used for folio breaks in the Textbase texts is not a TEI element. Unfortunately we’d need to conform to the old Textbase XML as that is what my processing scripts are set up to work with. However, we’d have to end up with such non-valid (from a TEI point of view) XML in any case if we want to incorporate new texts so perhaps this isn’t an issue, but just something we will have to bear in mind. I’ll continue with this next week.
I also updated the entry page to make language tags searchable. You can now click on language tags (for example in this page https://anglo-norman.net/entry/alabrot) and perform a search for the corresponding language.
Monday was the late May bank holiday this week, so it was a four-day week for me. On Tuesday I had an online meeting with Tony Harris, a developer at Cambridge who will be working on a project about Middle English Lexicon. The project involves Louise Sylvester, who is in charge of the Bilingual Thesaurus of Everyday Live in Medieval England (https://thesaurus.ac.uk/bth/) which I developed, and this new project is going to expand upon the data held in this resource. We had a good chat about the Bilingual Thesaurus and the technologies I’d used to put it together, and discussed some ways in which the new project might function from a technical perspective. It’s likely that we’ll meet again in the coming months to expand upon our ideas and I’ll probably be involved with the project in some small capacity.
Also on Tuesday fellow Arts developer Stevie Barrett and I met with the ‘Technical Champion’ for the College of Arts and Humanities, Aris Palyvos who is a technician in Archaeology. We had a good chat about the role of technicians in the College and how we can improve our visibility. We now have a Teams group for technicians and hopefully we’ll be able to meet up with some of the others in the College in the coming months.
Last week I’d started work on a content management system for Burns correspondence and I spent a bit of time this week finishing things off. I’ve given Craig and Pauline in Scottish Literature access to the CMS now and they have someone starting next week who will be using the system so I’ll just need to see how they get on and if they request any changes.
Also this week I made a few further tweaks to the new Speak For Yersel survey regions, fixed a couple of typos on the Speech Star website and helped to resolve a fairly serious issue with the Books and Borrowing website. The IIIF server that the website uses had gone offline, meaning none of the images of register pages were loading. Our usual IT guy at Stirling was out of office, but thankfully someone else there was able to get things up and running again.
I also completed my work on the migration of the British Association for Romantic Studies’ journal the BARS Review (https://www.bars.ac.uk/review). This has taken quite some time over the past few weeks to get sorted but it’s now up and running. To get it working I needed to switch the PHP version the site was using from a rather ancient version to the current version, and thankfully the other parts of the site that don’t use the OJS system were not adversely affected by this change. I also took the opportunity to add Google reCAPTCHA to the site for registration and login, which should hopefully stop the spam registrations. Registration also now requires the user to verify their registration via an email. I also made a few additional security updates that I’d better not discuss here.
I spent the rest of the week working for the Dictionaries of the Scots Language, making a few tweaks to the advanced search on our test server, investigating some issues and replying to emails. I also made a fairly major change to the sparkline data for entries so that dates of attestation beyond the period of each dictionary are handled in a different manner. Previously all such dates were bundled together as the start or end date and then this date was used to generate blocks for the sparkline visualisation. For example, ‘Abeich’ in SND has a first date of attestation of 1568, a long time before the official start date of the dictionary, which is 1700. Previously this start date was being converted to 1700. Our ‘cut off’ point for generating blocks of continuous attestation is now set to 50 years, meaning that if there are two or more attestations 50 years or less from each other this results in a block of continuous usage in the sparkline visualisation. As the next date of attestation for the entry was 1721 the resulting sparkline therefore gave a continuous block from 1700 to 1721, which did not affect the underlying data, plus the sparkline text then included ‘1700-1721’ which was not at all accurate. See the following screenshot to see what I mean:
I updated the code that generates the data for the sparklines so that any dates prior to 1700 result in the text ‘<1700’ appearing and the code no longer uses such dates as a starting point for a ‘block’ in the visualisation. After the update we’re now presented with the following sparkline, which has a line at the start of the visualisation representing ‘<1700’ and then a gap from this point until 1721, which is the first attestation in the dictionary’s official period:
In order to get this working I needed to regenerate the data for Solr and then update the Solr core with the new data. For now this is only running on my laptop and I have put in a request with our IT people to update the online Solr cores, as I don’t have access to do this myself. Once the change has been made our online test site will be updated and hopefully it won’t be too much longer before we can actually update the live site and make this new feature available to everyone.