Seeing speech
The resource provides ultrasound tongue imaging (UTI) and lip video of speech, magnetic resonance imaging (MRI) video of speech, and 2D midsagittal head animations based on MRI and UTI data. Find out more
Month: November 2019
Week Beginning 18th November 2019
On Monday I’d arranged another Arts Developer Coffee meeting, and for the first time all four of the current developers in the College of Arts were able to attend (that’s me, Luca Guariento, Stevie Barrett and David Wilson). It was a great opportunity to catch up with them all and discuss our work and some of the issues we’re currently dealing with.
I divided my time between several projects this week. I spent some time on DSL duties. This involved making updates to the new version of the API that I developed a few months ago, that uses the most up to date version of the DSL data and a new instance of Apache Solr that I set up. Ann had gone through the search facilities and had noted some discrepancies between this new version and the live site, so I addressed these. Firstly, the new version of the quick search if the ‘autocomplete’ list was ignored and the search form was submitted was running a ‘match the characters anywhere in a headword’ search, meaning lots of results were being returned that were possibly not very relevant. I updated this to ensure an exact match would be executed in such circumstances instead. The other change I made was to the advanced search, which was previously ordering results by entry ID. This meant that for a search that returned a lot of results only DOST results would be returned before the ‘maximum results’ limit was reached. Instead I changed the Solr query to use Solr’s ‘relevance’ score instead, which uses various methods to rank results. Strangely the returned result-set was still in a different order to the ‘live’ site, which also uses Solr’s ‘relevance’ score, which I can only assume is because the methods in which this score is generated have changed between versions of Solr.
I also got stuck into the creation of a new means of ordering SND and DOST entries for browse purposes. The original browse lists supplemental entries after the main entries and doesn’t disregard non-alphanumeric characters when working out browse order, which leads to some rather odd placements of entries. After a bit of toing and froing with Ann about how the new browse should work I spent some time on Friday writing a script that would order the data-sets. The output consists of a table with one row per entry, listing the ID, Headword and POS of each, and also the ‘stripped’ version of the headword (i.e. no non-alphanumeric characters), the generated browse order and the generated POS Num, which is based on a table that Thomas Widmann had created.
In order for yogh to get positioned correctly after ‘Y’ and before ‘Z’ I’ve replaced the character with ‘yz’ in the stripped column (the stripped column will never be seen so this doesn’t really matter). There was also an ash character in DOST, which I’ve converted to ‘ae’. The ordering algorithm reorganises entries by the ‘stripped’ column initially. If this column is the same as another then the ‘POS Num’ column is used to order things. If this number is also identical then the IDX column (the numerical part of the ID column) is used. This seems to work pretty well. E.g. for ‘aiker’ in the SND output there are 3 rows for ‘n.1’, ‘,n.2’ and ‘n.3’. These are all given the ‘POS Num’ of ‘1’, so the IDX field is checked and the entries are ordered correctly because by IDX order ‘n.1’ comes before ‘n.2’ which comes before ‘n.3’.
Also this week I met with Thomas Clancy and Gilbert Markus to discuss the Place-names of Kirkcudbrightshire project. I’m going to be putting the finishing touches to the front-end for this project soon and we met to discuss what updates might need to be made. I also added some new parishes to the underlying database.
On Thursday this week there were some issues with the server that hosts the majority of our project websites, which meant I couldn’t work on the projects I had intended to work on. Instead I continued to look into overhauling the Digital Humanities Network website. I began to set up the content management system for the data and migrated some of the underlying tables across to work with. I also spent some time thinking about what new data regarding projects would need to be recorded and how this might be stored.
I spent the rest of the week continuing to make updates to the SCOSYA website. We had received feedback from a user testing session on Monday and I spent some time reading through all of this and thinking about the various issues that had cropped up. Generally the feedback was hugely positive, but there were some usability issues that kept cropping up. I met with Jennifer on Tuesday to discuss these, and implemented a number of changers after that. This included making the community voices popups narrower on smaller screens and larger with a bigger font size on bigger screens and making the atlas area narrower so it’s easy to scroll up and down the page. I also needed to change the way the sidebar opens and closes. There was an issue where pointer events were not being passed to the map in the area underneath the sidebar, meaning you couldn’t open popups in this space, and on mobile devices this space was basically the whole map.
I had to update the JavaScript to get this to work (rather than just updating the stylesheet which I did initially, but introduced new problems). Previously the left-hand menu would only get resized if the contents ended up bigger than the height of the atlas. In such cases the menu would be set to a bit less than the height of the atlas, with a scrollbar used to reach the parts that didn’t fit. If a different menu item was clicked on that resulted in a shorter menu the menu area was not resized, as the contents would automatically take up less space anyway. However, this left the menu as a transparent area taking up the full height of the atlas, an area where any map-based actions (e.g. opening markers) would not function, as the menu area was set to intercept the pointer events in order for scrolling to work.
What happens now is whenever the page loads or a section of the menu is opened or closed the JavaScript tallies up the height of all of the visible menu items (the headers plus the open body). If this total is greater than the atlas height then the menu is set to be a bit smaller than the atlas height and a scrollbar appears as before. But if the total is less than the atlas height then the menu is now resized to the height of all the visible elements combined, so that it no longer takes up the full height of the atlas as a transparent area that intercepts pointer events. It’s a much more elegant solution and works very well.
I also updated the base map for the atlas to add in place-names, something that we had made a decision to exclude previously. We had some feedback that suggested our bare, minimalist map made it difficult for people to understand where places were, so I reinstated the names. Everyone is actually very happy with how the names look, so we’re going to leave them in. I also updated a number of marker locations that didn’t have quite the right name (e.g. Boness instead of Bo’ness).
In addition I updated the atlas so that the choice of age group and markers or points is no longer remembered, but always defaults to ‘all’ and ‘markers’ when moving between examples. I added a ‘choose another story’ link that now appears at the top of every slide in a story, rather than just the first and I’ve added a button to the top of the first story slide that opens the left-hand menu and reloads the ‘how do people speak in…?’ map. I also added more explanatory text above the atlas, as feedback suggested people were not understanding how to use the interactive map. I’ve added text explaining how to show and hide the left-hand menu, how to move about the map and how to zoom in and out. I made the ‘show transcription’ links in the community voices popups buttons rather than links and fixed the links in the tabular dialect samples page, which were pointing to an older version of the atlas that no longer exists. I made the border round the atlas blue to make it clearer where the atlas area ends and the rest of the page begins, as the previous light grey border was a bit hard to see and fixed an issue with the map popups when viewing a story pane that displays data for ‘young’ or ‘old’, whereby the sentence for ‘all’ and the other age group was still appearing when it shouldn’t. I also applied a number of these fixes to the linguists’ atlas as well. Here’s a screenshot of how the public atlas now looks:
The UCU strike begins on Monday next week so I will not be back into work until Thursday the 5th of December, unless the situation changes.
Week Beginning 11th November 2019
It was another mostly SCOSYA week this week, ahead of the launch of the project that was planned for the end of next week. However, on Friday this week I bumped into Jennifer who said that the launch will now be pushed back into December. This is because our intended launch date was the last working day before the UCU strike action begins, and is a bad time to launch the project, for reasons of publicity, engaging with other scholars and risks associated with technical issues that might crop up which might not be able to be sorted until after the strike. As there’s a general election soon after the strike is due to end, it looks like the launch is going to be pushed back until closer to Christmas. But as none of this transpired until Friday I still spent most of the week until then making what I thought were last-minute tweaks to the website and fixing bugs that had cropped up during user testing.
This included going through all of the points raised by Gary following the testing session he had arranged with his students in New York the week before, and meeting with Jennifer, E and Frankie to discuss how we intended to act on the feedback, which was all very positive but did raise a few issues relating to the user interface, the data and the explanatory text.
After the meeting I make such tweaks as removing most of the intro text from above the public atlas, but adding in a sentence about how to make the atlas full screen, as this is a feature that I think most users overlook. I also overhauled the footer, adding in the new AHRC logo, logos for Edinburgh and QMUL and rearranging everything. I also updated the privacy policy based on feedback from the University’s data protection people. I also updated the styling of the atlas’s menu headers to make them bolder on Macs, adding in links to the project’s API from the Linguists’ atlas and extended the height of the example selection area in the Linguists’ atlas too. I also slightly tweaked the menu header text (e.g. ‘Search Examples’ is now ‘Search the examples’ to make it clearer that the tab isn’t just a few example searches) and updated the rating selection option to make unselected ratings appear in a very faded grey colour, to hopefully make it more obvious what is selected and what isn’t. I also updated the legend so that the square grey boxes that previously said ‘no data’ now say ‘Example not tested’ instead. I also updated the pop-ups accordingly.
I also sent the URL for the public and linguists’ atlases to the other developers in the College of Arts for feedback. Luca Guariento found a way to break the map, which was good as after some investigation I figured out what was causing the issue and fixed it. Basically, if you press the ‘top’ button in the footer it jumps to the div with ID ‘masthead’ using HTML’s plain ‘if hash passed show this on screen’ option. But then if you do a full reload of the page the JavaScript grabs ‘masthead’ from the URL and tries to convert it to a float to pass it to Leaflet and things break. By ensuring that the ‘jump to masthead’ link is handled in JavaScript rather than HTML I stopped this situation arising. Stevie Barrett also noted that in Internet Explorer the HTML5 audio player in the pop-ups was too large for the pop-up area, and thankfully by adding in a bit of CSS to set the width of the audio player this issue was resolved.
Also this week I had a further chat with Luca about the API he’s building, and a DMP request that came his way, and arranged for the App and Play store account administration to be moved over the Central IT Services. I also helped Jane Roberts with an issue with the Thesaurus of Old English and had a chat with Thomas Clancy and Gilbert Markus about the Place-names of Kirkcudbrightshire project, which I set the systems up for last year and is now nearing completion and requiring some further work to develop the front-end.
I also completed an initial version of a WordPress site for Corey Gibson’s bibliography project and spoke to Eleanor Capaldi about how to get some images for her website that I recently set up. I also spent a bit of time upgrading all of the WordPress sites I manage to the latest version. Also this week I had a chat with Heather Pagan about the Anglo-Norman Dictionary data. She now has access to the data that powers the current website and gave me access to this. It’s great to finally know that the data has been retrieved and to get a copy of it to work with. I spent a bit of time looking through the XML files, but we need to get some sort of agreement about how Glasgow will be involved in the project before I do much more with it.
I had a bit of an email chat with the DSL people about adding a new ‘history’ field to their entries, something that will happen through the new editing interface that has been set up for them by another company, but will have implications for the website once we reach the point of adding the newly edited data from their new system to the online dictionary. I also arranged for the web space for Rachel Smith and Ewa Wanat’s project to be set up and spent a bit of time playing around with a new interface and design for the Digital Humanities Network website (https://digital-humanities.glasgow.ac.uk/) which is in desperate need of a makeover.
Week Beginning 4th November 2019
I spent much of my time this week continuing with updates to the SCOSYA website ahead of the launch later this month. This included adding in introductory text in several places, creating new buttons and a new homepage, writing a privacy page for the site and adding in a cookie banner. I also created a new ‘contact us’ form using the ‘Contact Form 7’ plugin for WordPress. I used this as we wanted to ensure the form had a ‘Captcha’, a test that must be completed before the form is submitted in order to separate out spam bots from real people. The plugin uses Google’s ‘reCAPTCHA’ server, which previously presented users with an ‘I am not a robot’ button and additional tests if the user wasn’t already somehow known to Google. However, a new version of reCAPTCHA has now been released which does all of the checking in the background and isn’t in any way visible to the user. This seemed ideal, but what it means is that Google now checks the user’s interactions on every single page of the site in order to ascertain whether they are a valid user if they choose to fill out the contact form. Worse still, Google automatically adds a ‘reCAPTCHA’ logo that hovers over the site in the bottom right corner, obscuring anything else that is put there. It’s all horribly intrusive. I tried reverting back to the earlier version of reCAPTCHA with its ‘I am not a robot’ button, but this no longer works with the Contact Form 7 plugin and ends up just breaking the form submission. Instead I reluctantly returned to using the current version, and found a way to hide the badge but keep the service running (see https://stackoverflow.com/a/53986985) which apparently Google now allows you to do, so long as links to its terms of service and privacy policy are clearly displayed, which they are on the SCOSYA site. I also met with Jennifer to discuss changes to the site and the impending launch, and spoke to Gary again about some issues that he is experiencing on his computer that no-one else is able to replicate. I also made changes to the public atlas, renaming some of the atlas menus and updating all references to these, I updated the Google Analytics access to allow Jennifer to access the stats and changed the way attribute names are managed to allow HTML tags to be included in the CMS, and for the display of the names to no all be bold in the various front-ends, thus allowing certain words in the names to be made bold. On Friday Gary ran a user testing session so no doubt I’ll have some further things to change next week.
Also this week I met with Corey Gibson from Scottish Literature to discuss an online bibliographical resource he would like me to help him put together for a Carnegie funded project that he’s currently running. It was a useful meeting and we made some decisions about how the resource will function, and later in the week I put in a request to set up a subdomain for the resource. Next week I’ll create an initial version for him. Also this week I created a new project website for an RSE funded project that is involving English Literature for Eleanor Capaldi. I created an initial interface and page structure and got all of the basics in place and will update this further once Eleanor gets back to me, if further changes are required.
I had a meeting with Rachel Smith about an interactive website that she is putting together with Ewa Wanat. I’d met with Ewa about this in May but hadn’t heard anything since, but since then they have been speaking to Alistair Beith, a PhD student in Psychology, who is going to do the development work for them. Alistair was at the meeting too and we discussed the requirements of the project, the technologies that will be used and some of the implications relating to access and file formats. It was good to speak to another person with web development skills, and once the team has decided on a suitable subdomain for the project I’ll get things set up and give Alistair access to create the site.
I also met with my old friend and colleague Yunhyong Kim from Information Studies to try and get access to an ancient Apple computer that I had left in the building when I moved to Critical Studies. Thankfully I could still remember the password and we managed to get it to boot up so I could set her up with a user account. I also had some further communication with Brian McKenna from Central IT Services about the University’s App and Play Store accounts. I have been managing these for many years now, mainly because Critical Studies was prepared to pay the subscription fees when no-one else was. It looks like responsibility for this is now going to be taken over the IT Services, which makes sense. I just hope it’s not going to make the process of publishing apps even more torturous than it already is, though.
Finally, on Friday I attended an ArtsLAb session on research integrity and data management plans. This was a course that we’d tried to run several times before, but hadn’t managed to get sufficient numbers to sign up for. This time we reduced the length of the session and we got a decent number of attendees. I had previously spoken at such sessions, but as there was less time I suggested that it made sense to split the hour between Nigel Leask, who gave a very interesting talk about research integrity, and Matt from Research Data Management gave a great overview of data management and what researchers are required to do. I provided some sample data management plans for the attendees to look at in their own time and it was a very useful session.
Week Beginning 28th October 2019
I split most of my time this week between the SCOSYA project and the Historical Thesaurus. The launch of the SCOSYA atlases is scheduled to take place in November and I had suggested to Jennifer that it might be good to provide access to the project’s data via tables rather than through the atlas interfaces. This is because although the atlases look great and are a nice interactive way of accessing and visualising the data, some people prefer looking at tables of data instead, and other people may struggle to use the interactive atlases due to accessibility issues, but may still want to be able to view the project’s data. We will of course provide free access to the project’s API, through which all of the data can be accessed as CSV or JSON files, or can even be incorporated into a completely new interface, but I thought it might be useful if we provided text-based access to the data through the project’s front-end as well. Jennifer agreed that this would be useful, so I spent some time writing a specification document for the new features, sending it to the team for feedback and developing the new features.
I created four new features. First was a table of dialect samples, which lists all of the locations that have dialect sample recordings and provides access to these recordings and the text that accompanies them, replicating the data as found on the ‘home’ map of the public atlas. The second feature provides a tabular list of all of the locations that have community voice recordings. Clicking on a location then displays the recordings and the transcriptions of each, as the following screenshot shows:
The third new feature lists all of the examples that can be browsed for through the public atlas. You can then click on one of these examples to listen to the example sound clips of the example and to view a table of results for all of the questionnaire locations. Users can also click through to view this example on the atlas itself too, as I figured that some people might want to view the results as a table but then see how these look on the atlas too. The following screenshot shows the ‘explore’ feature for a particular example:
The fourth new feature replicates the full list of examples as found in the linguists’ atlas. There are many examples nested within parent and sub-parent categories and it can be a bit difficult to get a clear picture of what is available through the nested menus in the atlas, so this new feature provides access to a complete list of the examples that is fully expanded and more easy to view, as the following screenshot demonstrates:
It’s then possible to click on an example to view the results of this example for every location in a table, again with a link through to the result on the atlas, which then enables the user to customise the display of results further, for example focussing on older or younger speakers or limiting the display to particular rating levels.
I reckon these new features are going to complement the atlases very well and will hopefully prove very useful to researchers. Also this week I received some feedback on the atlases from the project team and I spent some time going through this, adding in some features that had been requested (e.g. adding in buttons to scroll the user’s page down so that the full atlas is on screen) and investigating some bugs and other issues that had been reported, including some issues with the full-screen view of the atlas when using Safari in MacOS that Gary reported that I have so far been unable to replicate. I also implemented a new way of handling links to other parts of the atlas from the stories, as new project RA Frankie had alerted me to an issue with the old way. Handling internal links is rather tricky as we’re not really loading a new page, it’s just the JavaScript in the user’s browser processing and displaying some different data. As a new page is never requested pressing the ‘back’ button doesn’t load what you might expect to be the previous page, but instead displays the last full web page that was loaded. The pages also don’t open properly in a new tab because the reference in the link is not to an actual page, but is instead intended to be picked up by JavaScript in the page when the link is clicked on and then processed to change the map contents. When you open the link in a new tab the JavaScript doesn’t get to run and the browser tries to load the reference in the link, which ends up as a broken link.
It’s not a great situation and the alternative should work a bit better. Rather than handling the link in JavaScript the links are now full page requests that get sent to the server. A script on the server then picks up the link and processes a full page reload of the relevant section of the atlas. Unfortunately if the user is on the last slide of a story, then clicks a link on the last slide then presses the back button they’ll end up back at the first slide in the story not the last, as there isn’t a way to reference a specific slide in a story, but setting the links to open in a new tab by default gets round this problem.
Finally for the project this week I met with Jennifer and E to discuss the ancillary pages and text that need to be put in place before the launch, and we discussed the launch itself and what this would involve.
For the HT I generated some datasets that an external researcher had requested from the Thesaurus of Old English data, and I generated some further datasets from the main HT database for another request. I also started to implement a system to generate the new dates table. I created the necessary table and wrote a function that takes a lexeme and goes through all of the 19 date fields to generate the rows that would need to be created for the lexeme. As of yet I haven’t set this running on the whole dataset, but instead I’ve created a test script that allows you to pass a catid and view all of the date rows that would be created for each lexeme in the category so I (and Marc and Fraser) can test things out. I’ve tested it out with categories that have some complicated date structures and so far I’ve not encountered any unexpected behaviour, apart from one thing: Some lexemes have a full date such as ‘1623 Dict. + 1642–1711’. The script doesn’t analyse the ‘fulldate’ field but instead looks at each of the actual date fields. There is only one ‘label’ field so it’s not possible to ascertain that in this case the label is associated with the first date. Instead the script always associates the label with the last date that a lexeme has. I’m not sure how common it is for a label to appear in the middle of a full date, but it definitely crops up fairly regularly when I load a random category on the HT homepage, always as ‘Dict.’ so far. We’ll need to see what we can do about this, if it turns out to be important, which I guess it probably will.
Also this week I performed some App Manager duties, had a conversation with Dauvit Broun, Nigel Leask and the RDM team about the ArtsLab session on data management plans next week, and spoke to Ann Ferguson of the DSL about how changes to the XML structure of entries will be reflected in the front-end of the DSL.