Seeing speech
The resource provides ultrasound tongue imaging (UTI) and lip video of speech, magnetic resonance imaging (MRI) video of speech, and 2D midsagittal head animations based on MRI and UTI data. Find out more
Category: STELLA Apps
Week Beginning 17th February 2025
For the Anglo-Norman Dictionary this week I made a couple of major updates to the data. Firstly, a researcher had gone through the whole of ‘D’ to compile links from each entry to the DEAF dictionary. This took the form of a spreadsheet and I adapted the script I’d previously written to deal with DEAF links in order to work with this new data. The script updated the XML of each entry so I made sure I backed up the data first, and ran everything on a test version prior to updating the live site, but thankfully all went smoothly and entries such as https://anglo-norman.net/entry/dubbeour now have links out to the DEAF.
The second major update was a new batch of entries and major revisions of existing entries that the team have been working on over the past few months. This covers all of ‘W’ plus may other entries across the entire dictionary. As with previous updates, I ran and tested everything locally before updating the live site, as the first step in such an update is to delete the entirety of the letter that’s being updated. After running everything on my laptop and testing things out I spotted a couple of issues with the entries and in discussion with the editor Geert we managed to sort them out. In total the update added or edited 1918 entries, of which 1161 were ‘main’ (rather than cross-reference) entries. These included 1245 main senses, 34 subsenses, 77 locutions, 41 locution senses, 1 locution subsense and 2460 citations.
Also during this process we spotted that cross references were not being listed properly in the dictionary’s content management system. The code that regenerates the cross references after changes are made to the data wasn’t specifically identifying and targeting the ‘active’ versions of entries and sometimes the links that were stored were to inactive versions, which then weren’t displaying, as the code for displaying them was specifying active records only. However, I updated the code and regenerated the cross references so they all now appear.
Also this week the editor Karen spotted that links to the FEW dictionary from commentaries weren’t working and I updated the code to fix this. I also spotted that the dictionary browse feature on the entry page wasn’t working when entries featured upper case letters. I tried a few approaches to sorting this, but the ‘slug’ field in the database needs to be case sensitive as this is the only way to make the field accent sensitive too, which is needed as we have slugs for different entries that are identical other than for an accent. I did try swapping the field the browse targets from the slug to the lemma, but this introduced further issues because the lemma can include brackets, plus the ‘next’ and ‘previous’ links in the browse rely on the slug field. I’m going to have to return to this next week.
Also this week I created a mini-site for Alan Riach’s media. This involved extracting audio and video files from CDs and DVDs, which was slightly time consuming. The DVDs were rather tricky to extract as they were in multiple parts that then needed joined together. Also, one of the DVDs was corrupted and I was only able to extract half of it. I also had to migrate transcriptions of four hour-long audio files from Word to HTML and of course create the site structure and design. The site is password protected and is intended for students and educational use only but here is a screenshot:
Also this week I addressed some issues that had been uncovered whilst testing the advanced search facilities of the test version of the Dictionaries of the Scots Language that is set up with the latest version of the data. The first issue was that in bibliography searches some entries appear twice in the drop-down list. This was happening because entries can have both long and short titles and authors and the drop-down list was accommodating both. I’ve updated this so that just the author and title that are picked as the ‘display’ versions during import are now displayed, so there are no more duplicates. However, this does mean it’s not always clear why an entry appears in the list. For example, if you select ‘author’ and enter ‘ham’ you’ll see ‘Belhaven, J. H.’ and it’s not until you click through that you can see that ‘Ham’ comes from ‘Hamilton’.
The second issue, also relating to the bibliographies was that some items were not being found in the search. This was because the import script was only picking out titles and authors for bibs that had an ‘oldid’. Bibs that didn’t have these were still being processed and display titles and authors were being stored, but as no additional titles and authors were being extracted the searches therefore weren’t working. This has now been sorted – the display titles and authors are also now being searched in addition to the other titles and authors.
Whilst testing this out I also discovered that selecting items from the drop-down list that featured apostrophes and colons broke the search and led to an empty search results page. For example a title search for ‘cottager’ and then selecting the “cottager’s daughter” item. This is still broken on the live site, but is now fixed on the test site.
Also this week I liaised with our IT Services and technical people at The National Archives to arrange for the OHOS domain to be transferred to the TNA. I also responded to a query about the Bilingual Thesaurus of Everyday Life in Medieval England, had an email conversation with Matthew Creasy about his Mallarmé project and spent some time making updates to the Iona map interface following our team meeting last week.
I think I’ve covered everything that was raised at the meeting. I created a space for each categorisation at the top of the legend with some placeholder text for now (e.g. ‘Info about language classification’). For now when you hide the legend the description remains visible, as I figured it might be useful to know what the markers represent, but I can make this hide along with everything else. I’ve also updated the altitude classification to 20m increments and added in language tooltips wherever languages appear (e.g. in the small popup and the full record). Parts of speech in the full record also now have tooltips, although I’m afraid there are a few that I can’t put a name to and there are others I may have got wrong, so these may need further work. I updated the satellite views so that they now feature markers with a white border. And I’ve fixed issue with the top legend tick box ending up unticked in Chrome. In addition, all language combinations in the legend for language now feature tooltips (if they are abbreviated). I’ve applied most of these updates to the other place-names resources too. I still have to add in the tooltips (points 3 and 4) to the other sites, but I will do so next week.
Week Beginning 19th August 2024
I spent quite a bit of time this week completing the redevelopment of the ‘Readings in Early English’ resource. This was the first of the old STELLA desktop applications that I converted into an app and web resource more than a decade ago and the interface was in need of some modernisation. Last week I’d put together a first attempt at a new interface and this week I refined it and migrated all of the content across. This took quote some time as I needed to reformat much of the content for the 28 texts. Here are a couple of ‘before’ and ‘after’ shots. Firstly the homepage:
And one of the texts:
The ‘notes’ for each text were previously hidden in a pop-up but I’ve mode these permanently visible as the notes can include a lot of useful information that could otherwise be missed. I haven’t gone live with the new interface yet, but will do so this week. I also still need to decide whether to pursue the publication of app versions or whether we’ll just stick with web-based resources now. I’m in two minds about this. It takes a lot of time and effort to publish apps on the app stores and I’d rather not have to jump through all of these hoops on an annual basis in order to ensure the apps remain visible. But having the resources in app form would make them more visible to a broader audience, so it may be worth the effort. I’ll need to think about this some more.
Also this week I spent some time working for the Dictionaries of the Scots Language. I’d been sent some video recordings of presentations made at a recent team meeting, discussing new projects that are underway. I watched these and gave some feedback on how these projects may require the front-end to be updated. I also participated in a Teams call with the DSL people on Thursday, at which we discussed further upcoming projects and ongoing tasks such as the new date searches. I’d also been sent a few further updates to make to the new date searches so implemented these. This included adding in the total extent of attestation as text to the left of the sparklines in the search results and updating the way the ‘filter by date’ works when single years are entered, bringing this into line with how the ‘search by date’ boxes work. I also updated the algorithm that generates the sparkline text to tweak the way entries with no end date are displayed. This took a bit of time to update and test, and after I was satisfied with the output I then needed to regenerate the Solr index data as the sparkline text is stored in these indexes. As I can’t update the online Solr instance myself I submitted a Helpdesk job to get things sorted, but as of yet this has not been acted upon. I’m sure it will all get sorted next week.
Week Beginning 12th August 2024
I devoted some time this week to preparing for my Performance and Development Review, as it’s that time of year again. This involved going through all of my blog posts for the past year and picking out my main accomplishments then writing the necessary sections of the PDR documentation.
Also this week I continued to work on the old STELLA apps. I’d contacted the app people in IT Services on Monday and someone got back to me later in the week, although I’m still not certain whether we will be able to relaunch the resources as iOS and Android apps. We would have to commit to releasing updated versions of the apps at least once a year, and as these resources are reference works for subjects that don’t exactly change much (Old English, English grammar) releasing regular updates isn’t going to be particularly suitable. I’ll need to have a think about this, as it might be better if we just release updated web-based resources instead.
I focussed for some of this week on creating a new interface for the oldest of the STELLA apps: Readings in Early English. I created this more than a decade ago and an update is long overdue (currently the old version can be accessed here: https://stella.glasgow.ac.uk/readings/). I created a potential new interface for the resource this week, although I still have to add in all of the content and try out some alternatives so there’s still lots to be done.
Other than these things I had an email conversation with Thomas Clancy about another place-name project that wanted some advice on developing online resources. I also fixed a bug in the new map interface for place-names. When viewing a restricted set of place-names (e.g. as the result of a search or browse) and then pressing on a cross-reference link in a place-name pop-up, if the place-name labels are set to ‘always on’ and the cross-referenced place-name is not in the restricted set of names currently visible then the full dataset will be loaded into the map. When this happens the ‘always on’ labels should be reset of ‘off’ as having a map with thousands of ‘always on’ labels slows the interface to a crawl and is impossible to navigate. However, although the labels option in the menu is reset to ‘hover only’ the labels were continuing to be displayed. This isn’t a major bug as clearly it only triggers in very specific circumstances, but I wanted to sort it and thankfully I was able to, updating the code on both the Iona and Ayr websites (the maps are not yet publicly available on either of these resources). I also updated the Ayr resource to add in a ‘Loading’ spinner when the data first loads in. The initial load can take several seconds, during which it looks like the resource is just an empty map, so having a ‘Loading’ spinner on a greyed out map makes it clearer that something is actually happening. The initial load takes so long because of the parish boundary GeoJSON data rather than the place-name markers themselves. The parish boundary data is pretty large so takes some time to pull in.
Also this week I gave some advice to the researcher in charge of the Robert Fergusson website, had a chat with Geert of the AND about future AND work, watched some videos from a recent team meeting that Rhona of the DSL sent to me and resent the staff credentials for the new Speak For Yersel resources to Mary.
Also this week the old Italian Emblems website (https://www.italianemblems.arts.gla.ac.uk/) was finally reinstated after some recent reworking of it.
Week Beginning 5th August 2024
Back at the start of the year I spent a bit of time creating a new version of ‘An Anthology of 16th and Early 17th Century Scots Poetry’, a website that was originally created 25 years ago featuring 155 poems. The site was originally a plain HTML 4 site, with the poem text and glosses laid out in table rows and columns. The site is still actively used for teaching, and in redeveloping the site I decided to migrate the poems to TEI XML, separating out the data from the presentation and ensuring the data could then be reused more easily in future. It took rather a long time to migrate the poems and create the site, but I’d completed the work by March. However, the person in charge of the site, Theo Van Heijnsbergen, wanted to make some updates to the content before I replaced the old site so I had to wait until he got back to me. After a bit of prompting Theo emailed me with the updates this week, and I was finally able to launch the new site, which can be accessed here: https://stella.glasgow.ac.uk/anthol/. The site gives access to the XML source files for all of the poems and I’ve ensured that all pages of the old site redirect to the corresponding new page. It’s great to finally have the new site publicly available at last.
Also this week the researcher I’d given some advice to last week about displaying map data got back to me with an update and a couple of further questions, and I spent a bit of time thinking of possible solutions and replying to him. The editor of the Anglo-Norman Dictionary also got in touch this week, asking me to generate a list of all main entries (as opposed to cross-references) beginning with letters A to K. I ran the necessary query and exported a spreadsheet for him featuring the lemma, homonym and ‘slug’ fields. There are 14,736 entries beginning with these letters.
I spent the rest of the week returning to app development. Many years ago I’d created a series of iOS and Android apps primarily based on old STELLA teaching resources, but I hadn’t been able to update these since 2019 as the Mac I was using became too old to receive updates, and you can’t publish iOS apps without a Mac. Also, up until this point I was managing the University’s iOS and Android accounts but control of these was then taken over by IT Services and I no longer had access to them. As the apps hadn’t been updated in so long they ended up being taken down from the app stores. Last year I took delivery of a new Mac Mini which could be used for app development, but since then I haven’t had the time to get started. This week I had a bit of time so took the plunge.
To develop the apps I use a tool called Apache Cordova (https://cordova.apache.org/) which allows you to develop apps using HTML, CSS and JavaScript, which Cordova then ‘wraps’ in the native code for whichever platforms you’re targeting. It’s a very handy tool, but setting up a computer for app development using Apache Cordova is not straightforward. It would be great if you could just press a button and automatically have a working platform, but there are an awful lot of dependencies, software installations, system configurations and updates that all need to be handled manually. To make matters worse the documentation is often not up to date and doesn’t necessarily reflect what actually needs to happen. I’m not going to go into any detail, but suffice to say that it took several days of installing, tweaking, configurating, upgrading and in some cases downgrading, processing and building before the platform was working and I had the default test app working in the iOS and Android emulators, my iPad and my Android phone.
At this point I was able to try to get one of the old apps working, and I decided to begin with the last app I’d created, as I figured that it wouldn’t need much updating. This app is ARIES – Assisted Revision in English Style, which is available as a standard website here: https://stella.glasgow.ac.uk/aries/. I made a few tweaks to the code, but on the whole I think it holds up pretty well. I managed to wrap the app and deploy it to my iOS and Android test devices, and all is looking pretty good. The next step will be to contact IT Services to see how I should go about publishing the updated version on the app stores, assuming they’ll let me do so. I’ll see about this next week.
Week Beginning 23rd October 2023
After a delightful holiday last week I was back at work again this week. This involved spending quite a bit of time catching up with emails and dealing with the ongoing issue of migrating sites from old servers to either our new external supplier or a newer server hosted internally. I was involved with the migration of the SCOTS Corpus to a new server, with my work including fixing a few PHP errors that were cropping up on the more up to date server. There were also some issues relating to database connections as the original code (which I didn’t write) uses rather a lot of connections – more than the new server was set to allow. We had thought we’d fixed the issue but it looks like further investigation will be required.
We also migrated the thesaurus.ac.uk site and the Bilingual Thesaurus of Everyday Life in Medieval England (https://thesaurus.ac.uk/bth/) to a new server, which also required tweaking some of the code. The new server was caching scripts that generated different output each time they were run (e.g. to generate the random category on the homepage), meaning the category wasn’t random but was constantly stuck on ‘Lard a roast’, which wasn’t very helpful. Thankfully we managed to unstick the cache.
Also this week I investigated an issue with the advanced search of the Dictionaries of the Scots Language as the full-text search had stopped working. It turned out that the Solr index that powers this search had entirely disappeared from the server, which is more than a little concerning. It wasn’t a huge issue to rectify as I had the configuration scripts and the data on my PC, but we’re in the dark as to how the index could have been removed. It had also been brought to my attention that some of the video files I’d uploaded for the Speech Star project before I went on holiday had also disappeared and I’ve reuploaded them too. Our IT people are investigating what might have caused these issues and if they are linked, but it is concerning.
I also spent a bit of time looking through the old arts.gla.ac.uk server to try and figure out what needed to be retained from it. It’s mostly old subject area sites that were long ago superseded by T4, plus old conference sites that are no longer needed. A few of the other sites I’ve already previously moved to T4 myself (e.g. https://www.gla.ac.uk/schools/critical/aboutus/resources/stella/projects/starn/ and https://www.gla.ac.uk/schools/critical/aboutus/resources/stella/projects/bibliography-of-scottish-literature/). The only site that I think need to be retained are the STELLA apps that I developed from old teaching resources in around 2015. I therefore requested a new subdomain be set up to host them and migrated them over. I’ve also requested we set up external hosting for arts.gla.ac.uk, purely to host redirects from old URLs so we don’t end up with broken links. The new sites are now available (see https://stella.glasgow.ac.uk/aries/, https://stella.glasgow.ac.uk/grammar/, https://stella.glasgow.ac.uk/eoe/, https://stella.glasgow.ac.uk/metre/ and https://stella.glasgow.ac.uk/readings/) but the redirects from the old URLs are not yet in place. I’d really like to spend some time redeveloping all of these old apps (apart from ARIES, which has already been redeveloped). Maybe next year I’ll find some time.
I also set up a new project website for Rhona Brown in Scottish Literature. I’ve created a bare-bones website at the moment and I’m awaiting further instruction from her on things like themes, colour schemes, site structure and logos. I also tweaked the project website I’d set up a couple of weeks ago for Petra Poncarova in Scottish Literature to improve the URLs for the Gaelic version of the homepage and helped a project team member get access to a site I’d set up for Matthew Creasy in English Literature.
On Wednesday morning this week I participated in a networking event for the new Research Professional Staff Network. The event went well and it was very interesting to find out more about other people involved in research support across the University.
For the remainder of the week I began work on the development of the new ‘map first’ interface for the place-names projects, which I’m developing initially for the Iona project. Below is a screenshot of how things look so far:
At the moment the interface consists of a narrow bar at the top of the browser window with the site’s icon, title and subtitle using the blue colour of the site’s banner as a background. You can press on the logo or site title to navigate to the main site. The rest of the browser is taken up with the map. On the left is the side menu. As discussed in the requirements document I previously wrote, it consists of four collapsible sections, with ‘Home’ open by default. I haven’t had the time to implement the search and browse options yet, but the ‘Display options’ section is operational, as you can see above. Pressing on the section’s title will open the section and you can access the various options. You can show or hide the side menu by pressing on the button above it.
For the moment the map displays all data that has been marked as ‘on web’ in the CMS (362 records, I think). By default these are colour-coded by classification code. The legend is displayed in the top right, allowing you to turn specific features on or off. You can also show or hide the legend to free up space. In the bottom right are zoom options plus a ‘full screen’ button that does what you’d expect. You can press on a map marker to open up the pop-up. As of yet there is no link through to the full record and some Gaelic fields may be visible. These will be removed at some point.
Using the ‘Display options’ in the side menu you can change how the map markers are classified. We may need to be a little more fine-grained with start date and especially altitude. Also colours for classification codes are currently arbitrarily assigned but we might want to change this – having blue for ‘field’ seems a bit daft, for example. You can also change the base map and these options are currently the same as for the other place-name sites. We still need to figure out if / how we can integrate another map of Iona that we discussed at a meeting before I went on holiday. There is also an option to turn labels on or off.
That’s as far as I’ve got this week. There’s still a lot to do but I’ve made pretty good progress. I’ll hopefully find some time to continue with this next week. I also discovered that the Leaflet mapping library has a method to set the map view so as to show all markers at the closest zoom possible so I’ll ensure I use this when I develop the search and the browse. I’m currently already using it when the map is first opened to ensure that all of Iona, Soa in the south-west and Eilean Annraidh in the north-east are always visible, no matter what dimensions your screen / browse window are.
Week Beginning 29th November 2021
I participated in the UCU strike action on Wednesday to Friday this week, so it was a two-day working week for me. During this time I gave some help to the students who are migrating the International Journal of Scottish Theatre and Screen and talked to Gerry Carruthers about another project he’s hoping to put together. I also passed on information about the DNS update to the DSL’s IT people, added a link to the DSL’s new YouTube site to the footer of the DSL site and dealt with a query regarding accessing the DSL’s Google Analytics data. I also spoke with Luca about arranging a meeting with him and his line manager to discuss digital humanities across the college and updated the listings for several Android apps that I created a few years ago that had been taken down due to their information being out of date. As central IT services now manages the University Android account I hadn’t received notifications that this was going to take place. Hopefully the updates have done the trick now.
Other than this I made some further updates to the Anglo-Norman Dictionary’s locution search that I created last week. This included changing the ordering to list results by the word that was search for rather than by headword, changing the way the search works so that a wildcard search such as ‘te*’ now matches the start of any word in the locution phrase rather than just the first work and fixing a number of bugs that had been spotted.
I spent the rest of my available time starting to work on an interactive version of the radar diagram for the Historical Thesaurus. I’d made a static version of this a couple of months ago which looks at a the words in an HT category by part of speech and visualises how the numbers of words in each POS change over time. What I needed to do was find a way to allow users to select their own categories to visualise. We had decided to use the broader Thematic Categories for the feature rather than regular HT categories so my first task was to create a Thematic Category browser from ‘AA The World’ to ‘BK Leisure’. It took a bit of time to rework the existing HT category browser to work with thematic categories, and also to then enable the selection of multiple categories by pressing on the category name. Selected categories appear to the right of the browser, and I added in an option to remove a selected category if required. With this in place I began work on the code to actually grab and process the data for the selected categories. This finds all lexemes and their associated dates for each lexeme in each HT category in each of the selected thematic categories. For now the data is just returned and I’m still in the middle of processing the dates to work out which period each word needs to appear in. I’ll hopefully find some time to continue with this next week. Here’s a screenshot of the browser:
Week Beginning 11th November 2019
It was another mostly SCOSYA week this week, ahead of the launch of the project that was planned for the end of next week. However, on Friday this week I bumped into Jennifer who said that the launch will now be pushed back into December. This is because our intended launch date was the last working day before the UCU strike action begins, and is a bad time to launch the project, for reasons of publicity, engaging with other scholars and risks associated with technical issues that might crop up which might not be able to be sorted until after the strike. As there’s a general election soon after the strike is due to end, it looks like the launch is going to be pushed back until closer to Christmas. But as none of this transpired until Friday I still spent most of the week until then making what I thought were last-minute tweaks to the website and fixing bugs that had cropped up during user testing.
This included going through all of the points raised by Gary following the testing session he had arranged with his students in New York the week before, and meeting with Jennifer, E and Frankie to discuss how we intended to act on the feedback, which was all very positive but did raise a few issues relating to the user interface, the data and the explanatory text.
After the meeting I make such tweaks as removing most of the intro text from above the public atlas, but adding in a sentence about how to make the atlas full screen, as this is a feature that I think most users overlook. I also overhauled the footer, adding in the new AHRC logo, logos for Edinburgh and QMUL and rearranging everything. I also updated the privacy policy based on feedback from the University’s data protection people. I also updated the styling of the atlas’s menu headers to make them bolder on Macs, adding in links to the project’s API from the Linguists’ atlas and extended the height of the example selection area in the Linguists’ atlas too. I also slightly tweaked the menu header text (e.g. ‘Search Examples’ is now ‘Search the examples’ to make it clearer that the tab isn’t just a few example searches) and updated the rating selection option to make unselected ratings appear in a very faded grey colour, to hopefully make it more obvious what is selected and what isn’t. I also updated the legend so that the square grey boxes that previously said ‘no data’ now say ‘Example not tested’ instead. I also updated the pop-ups accordingly.
I also sent the URL for the public and linguists’ atlases to the other developers in the College of Arts for feedback. Luca Guariento found a way to break the map, which was good as after some investigation I figured out what was causing the issue and fixed it. Basically, if you press the ‘top’ button in the footer it jumps to the div with ID ‘masthead’ using HTML’s plain ‘if hash passed show this on screen’ option. But then if you do a full reload of the page the JavaScript grabs ‘masthead’ from the URL and tries to convert it to a float to pass it to Leaflet and things break. By ensuring that the ‘jump to masthead’ link is handled in JavaScript rather than HTML I stopped this situation arising. Stevie Barrett also noted that in Internet Explorer the HTML5 audio player in the pop-ups was too large for the pop-up area, and thankfully by adding in a bit of CSS to set the width of the audio player this issue was resolved.
Also this week I had a further chat with Luca about the API he’s building, and a DMP request that came his way, and arranged for the App and Play store account administration to be moved over the Central IT Services. I also helped Jane Roberts with an issue with the Thesaurus of Old English and had a chat with Thomas Clancy and Gilbert Markus about the Place-names of Kirkcudbrightshire project, which I set the systems up for last year and is now nearing completion and requiring some further work to develop the front-end.
I also completed an initial version of a WordPress site for Corey Gibson’s bibliography project and spoke to Eleanor Capaldi about how to get some images for her website that I recently set up. I also spent a bit of time upgrading all of the WordPress sites I manage to the latest version. Also this week I had a chat with Heather Pagan about the Anglo-Norman Dictionary data. She now has access to the data that powers the current website and gave me access to this. It’s great to finally know that the data has been retrieved and to get a copy of it to work with. I spent a bit of time looking through the XML files, but we need to get some sort of agreement about how Glasgow will be involved in the project before I do much more with it.
I had a bit of an email chat with the DSL people about adding a new ‘history’ field to their entries, something that will happen through the new editing interface that has been set up for them by another company, but will have implications for the website once we reach the point of adding the newly edited data from their new system to the online dictionary. I also arranged for the web space for Rachel Smith and Ewa Wanat’s project to be set up and spent a bit of time playing around with a new interface and design for the Digital Humanities Network website (https://digital-humanities.glasgow.ac.uk/) which is in desperate need of a makeover.
Week Beginning 28th October 2019
I split most of my time this week between the SCOSYA project and the Historical Thesaurus. The launch of the SCOSYA atlases is scheduled to take place in November and I had suggested to Jennifer that it might be good to provide access to the project’s data via tables rather than through the atlas interfaces. This is because although the atlases look great and are a nice interactive way of accessing and visualising the data, some people prefer looking at tables of data instead, and other people may struggle to use the interactive atlases due to accessibility issues, but may still want to be able to view the project’s data. We will of course provide free access to the project’s API, through which all of the data can be accessed as CSV or JSON files, or can even be incorporated into a completely new interface, but I thought it might be useful if we provided text-based access to the data through the project’s front-end as well. Jennifer agreed that this would be useful, so I spent some time writing a specification document for the new features, sending it to the team for feedback and developing the new features.
I created four new features. First was a table of dialect samples, which lists all of the locations that have dialect sample recordings and provides access to these recordings and the text that accompanies them, replicating the data as found on the ‘home’ map of the public atlas. The second feature provides a tabular list of all of the locations that have community voice recordings. Clicking on a location then displays the recordings and the transcriptions of each, as the following screenshot shows:
The third new feature lists all of the examples that can be browsed for through the public atlas. You can then click on one of these examples to listen to the example sound clips of the example and to view a table of results for all of the questionnaire locations. Users can also click through to view this example on the atlas itself too, as I figured that some people might want to view the results as a table but then see how these look on the atlas too. The following screenshot shows the ‘explore’ feature for a particular example:
The fourth new feature replicates the full list of examples as found in the linguists’ atlas. There are many examples nested within parent and sub-parent categories and it can be a bit difficult to get a clear picture of what is available through the nested menus in the atlas, so this new feature provides access to a complete list of the examples that is fully expanded and more easy to view, as the following screenshot demonstrates:
It’s then possible to click on an example to view the results of this example for every location in a table, again with a link through to the result on the atlas, which then enables the user to customise the display of results further, for example focussing on older or younger speakers or limiting the display to particular rating levels.
I reckon these new features are going to complement the atlases very well and will hopefully prove very useful to researchers. Also this week I received some feedback on the atlases from the project team and I spent some time going through this, adding in some features that had been requested (e.g. adding in buttons to scroll the user’s page down so that the full atlas is on screen) and investigating some bugs and other issues that had been reported, including some issues with the full-screen view of the atlas when using Safari in MacOS that Gary reported that I have so far been unable to replicate. I also implemented a new way of handling links to other parts of the atlas from the stories, as new project RA Frankie had alerted me to an issue with the old way. Handling internal links is rather tricky as we’re not really loading a new page, it’s just the JavaScript in the user’s browser processing and displaying some different data. As a new page is never requested pressing the ‘back’ button doesn’t load what you might expect to be the previous page, but instead displays the last full web page that was loaded. The pages also don’t open properly in a new tab because the reference in the link is not to an actual page, but is instead intended to be picked up by JavaScript in the page when the link is clicked on and then processed to change the map contents. When you open the link in a new tab the JavaScript doesn’t get to run and the browser tries to load the reference in the link, which ends up as a broken link.
It’s not a great situation and the alternative should work a bit better. Rather than handling the link in JavaScript the links are now full page requests that get sent to the server. A script on the server then picks up the link and processes a full page reload of the relevant section of the atlas. Unfortunately if the user is on the last slide of a story, then clicks a link on the last slide then presses the back button they’ll end up back at the first slide in the story not the last, as there isn’t a way to reference a specific slide in a story, but setting the links to open in a new tab by default gets round this problem.
Finally for the project this week I met with Jennifer and E to discuss the ancillary pages and text that need to be put in place before the launch, and we discussed the launch itself and what this would involve.
For the HT I generated some datasets that an external researcher had requested from the Thesaurus of Old English data, and I generated some further datasets from the main HT database for another request. I also started to implement a system to generate the new dates table. I created the necessary table and wrote a function that takes a lexeme and goes through all of the 19 date fields to generate the rows that would need to be created for the lexeme. As of yet I haven’t set this running on the whole dataset, but instead I’ve created a test script that allows you to pass a catid and view all of the date rows that would be created for each lexeme in the category so I (and Marc and Fraser) can test things out. I’ve tested it out with categories that have some complicated date structures and so far I’ve not encountered any unexpected behaviour, apart from one thing: Some lexemes have a full date such as ‘1623 Dict. + 1642–1711’. The script doesn’t analyse the ‘fulldate’ field but instead looks at each of the actual date fields. There is only one ‘label’ field so it’s not possible to ascertain that in this case the label is associated with the first date. Instead the script always associates the label with the last date that a lexeme has. I’m not sure how common it is for a label to appear in the middle of a full date, but it definitely crops up fairly regularly when I load a random category on the HT homepage, always as ‘Dict.’ so far. We’ll need to see what we can do about this, if it turns out to be important, which I guess it probably will.
Also this week I performed some App Manager duties, had a conversation with Dauvit Broun, Nigel Leask and the RDM team about the ArtsLab session on data management plans next week, and spoke to Ann Ferguson of the DSL about how changes to the XML structure of entries will be reflected in the front-end of the DSL.
Week Beginning 12th August 2019
I’d taken Tuesday off this week to cover the last day of the school holidays so it was a four-day week for me. It was a pretty busy four days, though, involving many projects. I had some app related duties to attend to, including setting up a Google Play developer account for people in Sports and Recreation and meeting with Adam Majumdar from Research and Innovation about plans for commercialising apps in future. I also did some further investigation into locating the Anglo-Norman Dictionary data, created a new song story for RNSN and read over Thomas Clancy’s Iona proposal materials one last time before the documents are submitted. I also met with Fraser Dallachy to discuss his Scots Thesaurus plans and will spend a bit of time next week preparing some data for him.
Other than these tasks I split my remaining time between SCOSYA and DSL. For SCOSYA we had a team meeting on Wednesday to discuss the public atlas. There is only about a month left to complete all development work on the project and I was hoping that the public atlas that I’d been working on recently was more or less complete, which would then enable me to move on to the other tasks that still need to be completed, such as the experts interface and the facilities to manage access to the full dataset. However, the team have once again changed their minds about how they want the public atlas to function and I’m therefore going to have to devote more time to this task than I had anticipated, which is rather frustrating at this late stage. I made a start on some of the updates towards the end of the week, but there is still a lot to be done.
For DSL we finally managed to sort out the @dsl.ac.uk email addresses, meaning the DSL people can now use their email accounts again. I also investigated and fixed an issue with the ‘v3’ version of the API which Ann Ferguson had spotted. This version was not working with exact searches, which use speech marks. After some investigation I discovered that the problem was being caused by the ‘v3’ API code missing a line that was present in the ‘v2’ API code. The server automatically escapes quotes in URLs by adding a preceding slash (\). The ‘v2’ code was stripping this slash before processing the query, meaning it correctly identified exact searches. As the ‘v3’ code didn’t get rid of the slashes it wasn’t finding the quotation mark and was not treating it as an exact search.
I also investigated why some DSL entries were missing from the output of my script that prepared data for Solr. I’d previously run the script on my laptop, but running it on my desktop instead seemed to output the full dataset including the rows I’d identified as being missing from the previous execution of the script. Once I’d outputted the new dataset I sent it on to Raymond for import into Solr and then I set about integrating full-text searching into both ‘v2’ and ‘v3’ versions of the API. This involved learning how Solr uses wildcard characters and Boolean searches, running some sample queries via the Solr interface and then updating my API scripts to connect to the Solr interface, format queries in a way that Solr could work with, submit the query and then deal with the results that Solr outputs, integrating these with fields taken from the database as required.
Other than the bibliography side of things I think that’s the work on the API more or less complete now (I still need to reorder the ‘browse’ output). What I haven’t done yet is to work on the advanced search pages of the ‘new’ and ‘sienna’ versions of the website to actually work with the new APIs, so as of yet you can’t perform any free-text searches through these interfaces but only directly through the APIs. Working to connect the front-ends fully to the APIs is my next task, which I will try to start on next week.
Week Beginning 11th September 2017
I spent more than half of this week continuing to work on the new ARIES app. Last week I finished work on an initial, plain HTML and JavaScript version of the app, and I received another couple of bits of feedback this week that I implemented. The bulk of my time, however, was spent using Apache Cordova to ‘wrap’ the HTML and JavaScript version, converting it into actual iOS and Android apps, then testing these apps on my iOS and Android devices, and then making all of the media files that an app needs, such as icon files, screenshots, app loading screens, app store graphics and things like that. This process always takes longer than I think it should. For example, I have to make more than 20 different icon files at varying resolutions, and I need to grab multiple screenshots from at least four different devices. This latter process is made trickier because my Android Nexus 7 tablet no longer connects properly to my PC – the ‘photos’ folder appears blank when I connect for photo transfer and doesn’t contain the actual updated contents when I connect for file transfer, so I have to use a third party file explorer app to move the screenshots to a different folder on the device that somehow does get updated when viewing on my PC. Regarding the icons, I came up with a few alternatives for this, based on the header image for the app, and we finally agreed on a sort of ‘marble’ effect circle on a white background. I think it looks pretty good, and is certainly better than the old ARIES logo. The app publication process was also complicated by two new issues that have emerged since I last made an app. Firstly, Apple have updated the build process to disallow any extended image metadata, I guess as a security precaution. I created my app icon PNG files in Photoshop, which added in such metadata. When I then built my iOS app in xCode I received some rather unhelpful errors. Thankfully StackOverflow had the answer (see https://stackoverflow.com/questions/39652867/code-sign-error-in-macos-sierra-xcode-8-3-3-resource-fork-finder-information) and after running a couple of command-line scripts this metadata was stripped out and the build succeeded. My second issue related to the app name on the App Store. Apple has decided to limit app names to 30 characters, meaning we could no longer call our app ‘ARIES: Assisted Revision in English Style’. And as there is already an app names ‘ARIES’ we couldn’t call it that either. This is a real pain, and seems like a completely unnecessary restriction to me. In the end we called the app “ARIES: English Academic Style”. I managed to submit the app to Apple and Google on Wednesday, and thankfully by the end of the week the new version was available on both the App and Play Stores. I also made the ‘web’ version available, replacing the old ARIES site. You can access this, and link through to the app versions from here: http://www.arts.gla.ac.uk/stella/apps/web/aries/
Other than ARIES work, I made some further changes to the Edinburgh Gazetteer keywords, replacing the old list of keywords with a much trimmed down list that Rhona supplied. I think this works much better than the previous list, and things are looking good. I also helped Alison Wiggins with some information she wanted to add to the Digital Humanities website, and I spent about half a day working with the Mapping Metaphor data, generating new versions of all of the JSON files that are required for the ‘Metaphoric’ app and testing the web version of this out. It looks like everything is working fine with the full dataset, so next week I’ll hopefully publish a new version of the app that contains this data. I also started working on the database of Burns’ paper for Ronnie Young, firstly converting his Access database into an online MySQL version and then creating a simple browse interface for it. There’s still lots more to be done for this but I need to meet with Ronnie before I can take this further.
The rest of my week was taken up with meetings. On Wednesday morning I was on an interview panel for a developer post in another part of the college. I also met with Gerry McKeever in the afternoon to discuss his new British Academy funded ‘Regional Romanticism’ project. I’ll be working with him to set up a website for this, with some sort of interactive map being added in sometime down the road. I spent Friday morning attending a network meeting for Kirsteen McCue’s Romantic National Song Network. It was interesting to hear more about the project and to participate in the discussions about how the web resource for this project will work. There were several ideas for where the focus for the online aspect of the project should lie, and thankfully by lunchtime we’d reached a consensus about this. I can’t say much more about it now, but it’s going to be using some software I’ve not used before but am keen to try out, which is great.