Seeing speech
The resource provides ultrasound tongue imaging (UTI) and lip video of speech, magnetic resonance imaging (MRI) video of speech, and 2D midsagittal head animations based on MRI and UTI data. Find out more
Month: November 2024
Week Beginning 18th November 2024
I spent a fair amount of time this week working on a paper about Speak For Yersel and its new regions for the DH2025 conference. I worked on this with the PI Jennifer Smith and after several revisions and a meeting on Tuesday we had something that was ready to submit. I also spent some time finalising the code for the tool that I created to generate the new survey areas. We’ve decided to make this publicly available for others to download and reuse via GitHub but before I could add the code I needed to get it ready for publication – doing things like ensuring libraries like Bootstrap and jQuery are referenced externally rather than including them directly, ensuring no passwords and keys are included, updating the setup instructions document, preparing the example data and running through a few test installations to make sure the thing actually works. For sample data we’re making the Wales data available. This consists of not only the questions and answer options for the survey but all of the geolocational data that is needed in order to ensure submitted answers can be correctly pinned onto the map. You can now access all of the code for SFY here: https://github.com/baitken/sfy.
Also this week I spent some time going through the website mockups and sample data that I’d been sent for the VARICS project. They are creating a database of speech files with filters, similar in many respects to the facilities I created for the STAR project and I’ll be developing a resource for the project over the coming months. We have a project meeting scheduled in a couple of weeks, and I have a lot of questions to ask when we meet.
I heard some good news this week regarding project funding. The Anglo-Norman Dictionary has had its final phase approved by the AHRC, after a few weeks of uncertainty, which is fantastic. Also Joanna Kopaczyk’s DOST project has received funding and I’ll be working with her on this in the new year, which is great.
Other tasks this week included updating the embedded Scots School Dictionary in Speak For Yersel to replace this with the new Concise Scots Dictionary that is now available from the DSL, plus giving some advice to the DSL editor William Ashford about which versions of the ancillary pages of the DSL site and its test equivalent to update. I also updated the links from the Thesaurus of Old English to the Dictionary of Old English as their URLs have changed, and discussed the creation of a new place-names site for Nairn with Thomas Clancy, submitting the request to have a subdomain created for this.
Week Beginning 11th November 2024
Last week I was sent the latest dataset from the Dictionaries of the Scots Language’s editing system that included fixes to the way citation dates were generated and the reinstatement of the bibliographical data structure that my scripts were written to work with. On Monday I ran this updated dataset through my import scripts running on an instance of the DSL site hosted on my laptop and all went very smoothly.
The new bibliographical data contains 7045 entries (4794 SND, 2251 DOST), up from 6108 in the live data (4099 SND, 2009 DOST). The new data features 5348 authors, down from 5434, and 9791 titles, down from 9908.
When I processed the previously exported dataset a couple of weeks ago I spotted that there had been an issue with the generation of the citation date elements during the export process, which meant that almost 29,000 entry citations featured no date field. Thankfully this issue has been successfully resolved in the latest export and the number of citations featuring no dates had dropped dramatically to just 56.
With both the export from the DSL’s editing system and the import of the data into the website system now working smoothly we are in a position to roll out the update on the live DSL site. The DSL team still have other things to do before this happens, however, and we’re likely to not go live with the update until the new year.
I spent a large part of the week continuing to work on the interactive map of the correspondence of Robert Burns, and I have now fully completed a first version of the resource. Last week I worked on facilities to enable specific views of the map to be bookmarked, shared and cited by adding information about the view to the hash in the page URL. This week I completed work on this by enabling a specific timeline entry to be shared. I was worried that this would be rather tricky to implement as the timeline data is loaded in sequentially each time the user chooses to proceed to the next item and initially I had been intending on loading all correspondence for prior timeline items when a specific item was loaded from a URL. Instead I decided to not feature the earlier data on the map until the user navigates back through the items, much in the same way as later timeline items are added. This was simpler to implement than populating the map with every earlier item, and the timeline is primarily meant to be accessed using the ‘previous’ and ‘next’ buttons rather than by manually moving about the map so I didn’t think this was a huge issue.
Whilst using the timeline feature I realised that it was somewhat difficult to return to the full map data once the timeline tab has been pressed on. When a user enters the timeline the map is cleared of data and each correspondence is then added to the map in date order as the user navigates through the timeline items. But pressing on another menu tab leaves the contents of the map at the point at which the user left the timeline, meaning not all data is necessarily visible. The only way to return the full dataset to the map is to reset it or re-apply a filter, which is not very intuitive. I therefore added an ‘Exit timeline’ button to the timeline view that when pressed on reinstates the full dataset that matches the user’s chosen filters and returns the user to the ‘Filter’ tab. You can see this new button in the following screenshot:
Also visible in the above screenshot is the new ‘Cite’ button, which I added to each menu tab and also to the bibliography popup. When pressed on this opens up a popup with various citation styles and the text of the citation varies depending on what you’re viewing, as the screenshot below demonstrates (with the URL of the resource redacted as it’s not ready to launch yet). The screenshot shows that the view of the map if filtered to only show ‘historians’, the map uses the ‘historical’ base map and the ‘filter’ tab is visible.
Citations of the bibliographies are handled slightly differently as the biography is already in a popup. For this reason when you press on the ‘cite’ link the existing popup content is replaced with the ‘cite’ options, with a link back to the biography rather than opening a further popup. I also implemented a URL shortener that I developed for another project, which essentially stores a short code in the database to represent the full URL. The short code is then used in the citation, and can be shared. When the short code is passed the full URL is retrieved and the relevant view and data are loaded.
I then updated the filters tab to include includes options to view the data in a table and download the data as a CSV, as the following screenshot demonstrates:
Again I was thankfully able to adapt existing code from previous projects to implement these options, although I did still have to spend some time customising things to ensure that the data was properly formatted in the CSV and the in-page table view. I think that is now everything in place and it’s now time for others to test out the resource and give me some feedback. I’m sure there will be at least some things that will need tweaked, or some bugs that crop up.
Also this week I had an email conversation about the hosting arrangements for the OHOS project and also set up a new user for the project’s website. Eleanor Lawson sent me some further updates to the STAR website, consisting of some improved animation videos and I ensured the older versions were successfully replaced with the new ones. I also had a chat with Jennifer Smith about the DH2025 conference and whether we should write a paper about the new Speak For Yersel regions for it. I’ll need to give this some further thought next week.
Week Beginning 4th November 2024
I continued to work on the interactive map of the correspondence of Robert Burns this week, spending quite some time working on the facilities that would allow people to bookmark, share or cite specific views of the map. Last week I’d added the specific location, zoom level and base map selection to the hash in the page URL, ensuring that when the URL was shared these specific settings were loaded in. This week I moved onto adding in some of the more tricky selections. Following my work it’s now possible to bookmark or share a specific location that a user has selected, meaning that when the URL is shared the chosen location marker and its connecting lines are given the selected style (red), the ‘correspondence’ tab in the map menu is visible and any correspondence involving the location are listed.
I then added in a further option to ensure that when a person’s name is pressed and the biography pop-up loads this is registered in the URL as well. This enables specific biographies to be linked to, with the pop-up automatically opening when a person’s ID is included in the URL. After that I then tackled the data filter options. This was rather complicated to implement as not only did all of the various filter options needed to be added to the URL, but when included in the URL when a page loads the system needed to process these and automatically perform a search using them. As you can maybe imagine, it took quite some time to get this working properly.
Finally, I added in the possibility of sharing the timeline view, although I didn’t manage to fully complete this. I’ve added in the method for logging the specific timeline entry that is visible in the URL and I hope to be able to allow the exact entry to be linked to but I’ll need to leave this for next week. As mentioned in last week’s post, this is going to be tricky to implement as the timeline data is loaded into the map sequentially and if we allow people to jump to a specific point in it I’ll need to ensure that all previous entries are also added.
I’ve had to do a lot of testing and debugging as I’ve worked on all of this as a lot of complexities have been added to the resource, some of which have introduced unintended consequences. It’s possible there are still some bugs in there that will need to be sorted. Hopefully next week I’ll be able to complete a first version.
Also this week Garrick Allen sent me an updated set of documentation for his ERC proposal and I spent some time reading through all of this and giving some feedback. The editor of the Anglo-Norman Dictionary also sent some further information about their new AHRC proposal and I looked over this too. I responded to some queries from Amy Wilcockson and Mridhula Murali about the projects they are involved with (Robert Fergusson and VARICS). I’m also involved in these projects and will have some work to do for them over the coming months.
The remainder of my week was spent working for the Dictionaries of the Scots Language. I set up a Google Analytics property for a new website that the DSL is developing and gave some advice on how this could be used. I also received a further export of the DSL data from their editing system that will have addressed some of the issues with dates that I identified in the previous output. There were still some issues with the bibliographical data being in a different format to previous outputs and I had some discussions with the DSL people about this. This led to a further output being sent to me in the format that my scripts are set up to process. I haven’t processed this new dataset yet but will do so next week.
Week Beginning 28th October 2024
On Thursday this week the three new Speak For Yersel regions were successfully launched. These can now be accessed here: Northern Ireland: https://speakforyersel.ac.uk/ni/, Republic of Ireland: https://speakforyersel.ac.uk/roi/, Wales: https://speakforyersel.ac.uk/wales/.
I spent most of my week working on dictionary projects. Last week I was given access to a new export of the Dictionaries of the Scots Language data that the team wanted me to import into a new test version of the website. The aim was to check how the work on the dates was progressing and if there were any issues. Setting up a new test instance of the website and importing all of the new data took a day or so and involved setting up new Solr caches, running many different scripts to process the data and generate the data for Solr and test everything out on a temporary version of the website running on my local PC.
As I prepared to process the bibliographical data I noticed that the XML now appears to have a different structure and I made the decision to hold off importing it until I hear more from the team. Running the XML data through my existing scripts would not extract anything as the expected structure is not found. Previously the source dictionary was specified in an attribute of the ‘record’ element, e.g.
<record id=”bib014623″ oldid=”sb1″ sourceDict=”snd”><bibl><author>Abel, George</author>. <title>Wylins fae my Wallet</title>. 1916 (Abd.)</bibl></record>
But in the new data the ‘bibl’ tag is wrapped in either a ‘snd’ or ‘dost’ tag, e.g.
<record oldid=”sb1″ id=”bib014623″><snd><bibl oldid=”sb1_1″ id=”bib014623_001″><author>Abel, George</author>. <title>Wylins fae my Wallet</title>. 1916 (Abd.)</bibl></snd></record>
I can update my extraction script to deal with this change, but it looks like there are other changes too (e.g. the ‘bibl’ element now having its own ID) so I’ll need to talk through all of the changes with the team before I implement any updates.
When it came to processing the dictionary entry data, the import all went very smoothly. There were the 11 ‘PROBLEM’ dates (citation dates that give errors when passed through the date normalisation scripts that are part of the process that exports the data from the DSL’s editing software). I extracted the citations that feature these and their corresponding entry IDs and sent them to the team; it looks like these will be pretty easy to fix.
There are, however, some 28,730 citations for which no date has been extracted. This is down from 37,010 in the previous version of the data. I also extracted these and sent them to the team for consideration. Looking at a few of these it would appear that the issue is that the citations don’t feature ‘date’ tags but only feature a ‘bibDate’ attribute for the ‘cref’ element, for example:
<cit> <cref refid=”bib012649″ bibDate=”1375″><title>Ib.</title> <sc>iii</sc>. 429; etc.</cref><q>In a nycht and in a day</q></cit>
I recall that we’d discussed these previously and I was told to ignore these as they’re still being worked on. As they stand I can’t use them without further processing as they often include affixes, for example:
<cit> <cref refid=”bib013662″ bibDate=”a1400″><title>Ib.</title> xli. 294; etc.</cref><q>Of a mylk and of a clath</q></cit> <cit>
There are also citations that feature no actual quote text. There are 26,672 such citations in the new data, up from 25,929 in the previous dataset. I’ve informed the team about these.
In the new dataset DOST has 48,349 entries containing 200,026 forms and 577,528 citations. This compares to 50,212 entries containing 200,202 forms and 577,265 citations in the previous dataset. SND has 29,981 entries in the new dataset, containing 67,080 forms and 171,388 citations, compared to 30,107 entries containing 67,414 forms and 170,605 citations in the previous dataset.
With all of the data extraction (other than the bibliographies) complete I contacted the team and will await to hear back from them before I proceed further.
I then moved onto the Anglo-Norman Dictionary, for which I updated the CSV output functionality of the API. When I originally developed the API I’d left these unfinished as it didn’t look like the options would end up being used anyway. In fact, the CSV output of single entries was entirely broken. However, CSV output is now going to be used by the team in order to share the project’s data with third parties so I needed to get the functionality working.
It was slightly complicated to deal with the part of speech, label and form fields in the search results output as each entry can have any number of such fields, each of which can comprise of multiple types of data (e.g. each label has five fields). For the data to be useable in a spreadsheet each bit of data needs to occupy its own cell, but across multiple search results the columns of data need to be in alignment. For example, one entry may have one label and another may have ten. Forms are listed as columns after labels so in order for the form columns to align for all entries the code needs to ascertain which entry features the most labels, create headings for all of these and then pad out the entries that feature less labels so that their form columns still appear in the correct place.
And some entries have a LOT of parts of speech, labels and forms. For test purposes I was using a headword search for ‘*ment’, which gives 1924 results. At least one entry in these results features 7 different parts of speech, 17 different labels and 442 forms (not necessarily the same entries in each case).
The CSV output now handles all of these correctly, rather than just displaying ‘array’ in the pos, label and forms columns. It now means you can order the resulting spreadsheet by the columns to figure out which entries have the most pos, labels or forms, for example. Plus if you’re really not interested in all of the separate pos, labels and forms you can just ignore or delete the columns.
After dealing with the pos, label and forms columns I thought I’d sorted the search results CSV output, but it turned out that there was rather more still to do. We actually offer several different search types, such as citations and languages and these bring back different types of data. Some of these include different array-based fields such as translation and I also needed to sort these out in the CSV output. Thankfully I could apply the same methodology I’d used for fields such as pos so it didn’t take too long to sort this out.
For the rest of the week I returned to working on the interactive map of the correspondence of Robert Burns. I haven’t worked on this since before my holiday and it took a bit of time to familiarise myself with the resource again. Whilst doing so I spotted that when viewing the map in full-screen mode the autocomplete and tooltips in the filter popup were not working. I’d used a similar approach to the new ‘map first’ place-names interface so checked these resources too and discovered they were similarly affected. It took an awfully long time to get to the bottom of this. Initially I had thought it was a simple ‘z-index’ issue but this was not the case. When the map is in full-screen mode it’s treated as a separate view to the standard page, and the autocomplete and tooltips were still associated with the ‘in-page’ view and therefore not appearing.
The autocomplete and tooltips are initialised when the user opens the filter form modal by pressing on a button. When in full-screen view the autocomplete and tooltips were still being attached to the ‘in-page’ view, which wasn’t visible, rather than the ‘full-screen’ view. For the jQuery-UI auto-complete widget there is an ‘appendTo’ option that allows you to specify which element the autocomplete is attached to. By specifying this I was able to get the autocomplete to work.
Unfortunately the jQuery-UI tooltips didn’t appear to have have a similar option and I went round in circles for hours trying to figure out how to get the tooltips to appear. I thankfully got there in the end, though. It turned out that it was possible to append the tooltips to an element, but this was only required when the map was in full-screen mode and would break the tooltips if the map wasn’t in full-screen. The code that sorted it is:
$(".tip").tooltip({
open: function (e, ui) {
if(document.fullscreen)
ui.tooltip.appendTo("#map");
}
});
It was such a relief to finally get this working, and with this in place I then began working on the major update to allow specific views of the map to be bookmarked / shared / cited. I’m still in the middle of this as it requires a significant number of changes to be implemented. I added in the Leaflet hash plugin that stores the location and zoom level in the hash of the page URL. I also ensured that the currently active base map (e.g. satellite view) is also similarly remembered, as it the currently active tab in the map menu. It’s taken most of the day to get this far, and there’s still more I need to do, namely ensuring any selected filters are remembered and any selected location is highlighted and its correspondence loaded into the correspondence tab.
I’ll also need to see what I can do about remembering the position in the timeline. This is going to be tricky as currently the data is loaded in sequentially each time the user presses the ‘Next’ button. If we want to remember that someone is (for example) viewing item 13 when they bookmark the map then I’ll need to also ensure that the previous 12 items are loaded. I’ll see what I can do, but it might end up that the timeline always reverts to the first item. I’ll hopefully continue with this next week.