On Monday I’d arranged another Arts Developer Coffee meeting, and for the first time all four of the current developers in the College of Arts were able to attend (that’s me, Luca Guariento, Stevie Barrett and David Wilson). It was a great opportunity to catch up with them all and discuss our work and some of the issues we’re currently dealing with.
I divided my time between several projects this week. I spent some time on DSL duties. This involved making updates to the new version of the API that I developed a few months ago, that uses the most up to date version of the DSL data and a new instance of Apache Solr that I set up. Ann had gone through the search facilities and had noted some discrepancies between this new version and the live site, so I addressed these. Firstly, the new version of the quick search if the ‘autocomplete’ list was ignored and the search form was submitted was running a ‘match the characters anywhere in a headword’ search, meaning lots of results were being returned that were possibly not very relevant. I updated this to ensure an exact match would be executed in such circumstances instead. The other change I made was to the advanced search, which was previously ordering results by entry ID. This meant that for a search that returned a lot of results only DOST results would be returned before the ‘maximum results’ limit was reached. Instead I changed the Solr query to use Solr’s ‘relevance’ score instead, which uses various methods to rank results. Strangely the returned result-set was still in a different order to the ‘live’ site, which also uses Solr’s ‘relevance’ score, which I can only assume is because the methods in which this score is generated have changed between versions of Solr.
I also got stuck into the creation of a new means of ordering SND and DOST entries for browse purposes. The original browse lists supplemental entries after the main entries and doesn’t disregard non-alphanumeric characters when working out browse order, which leads to some rather odd placements of entries. After a bit of toing and froing with Ann about how the new browse should work I spent some time on Friday writing a script that would order the data-sets. The output consists of a table with one row per entry, listing the ID, Headword and POS of each, and also the ‘stripped’ version of the headword (i.e. no non-alphanumeric characters), the generated browse order and the generated POS Num, which is based on a table that Thomas Widmann had created.
In order for yogh to get positioned correctly after ‘Y’ and before ‘Z’ I’ve replaced the character with ‘yz’ in the stripped column (the stripped column will never be seen so this doesn’t really matter). There was also an ash character in DOST, which I’ve converted to ‘ae’. The ordering algorithm reorganises entries by the ‘stripped’ column initially. If this column is the same as another then the ‘POS Num’ column is used to order things. If this number is also identical then the IDX column (the numerical part of the ID column) is used. This seems to work pretty well. E.g. for ‘aiker’ in the SND output there are 3 rows for ‘n.1’, ‘,n.2’ and ‘n.3’. These are all given the ‘POS Num’ of ‘1’, so the IDX field is checked and the entries are ordered correctly because by IDX order ‘n.1’ comes before ‘n.2’ which comes before ‘n.3’.
Also this week I met with Thomas Clancy and Gilbert Markus to discuss the Place-names of Kirkcudbrightshire project. I’m going to be putting the finishing touches to the front-end for this project soon and we met to discuss what updates might need to be made. I also added some new parishes to the underlying database.
On Thursday this week there were some issues with the server that hosts the majority of our project websites, which meant I couldn’t work on the projects I had intended to work on. Instead I continued to look into overhauling the Digital Humanities Network website. I began to set up the content management system for the data and migrated some of the underlying tables across to work with. I also spent some time thinking about what new data regarding projects would need to be recorded and how this might be stored.
I spent the rest of the week continuing to make updates to the SCOSYA website. We had received feedback from a user testing session on Monday and I spent some time reading through all of this and thinking about the various issues that had cropped up. Generally the feedback was hugely positive, but there were some usability issues that kept cropping up. I met with Jennifer on Tuesday to discuss these, and implemented a number of changers after that. This included making the community voices popups narrower on smaller screens and larger with a bigger font size on bigger screens and making the atlas area narrower so it’s easy to scroll up and down the page. I also needed to change the way the sidebar opens and closes. There was an issue where pointer events were not being passed to the map in the area underneath the sidebar, meaning you couldn’t open popups in this space, and on mobile devices this space was basically the whole map.
I also updated the base map for the atlas to add in place-names, something that we had made a decision to exclude previously. We had some feedback that suggested our bare, minimalist map made it difficult for people to understand where places were, so I reinstated the names. Everyone is actually very happy with how the names look, so we’re going to leave them in. I also updated a number of marker locations that didn’t have quite the right name (e.g. Boness instead of Bo’ness).
In addition I updated the atlas so that the choice of age group and markers or points is no longer remembered, but always defaults to ‘all’ and ‘markers’ when moving between examples. I added a ‘choose another story’ link that now appears at the top of every slide in a story, rather than just the first and I’ve added a button to the top of the first story slide that opens the left-hand menu and reloads the ‘how do people speak in…?’ map. I also added more explanatory text above the atlas, as feedback suggested people were not understanding how to use the interactive map. I’ve added text explaining how to show and hide the left-hand menu, how to move about the map and how to zoom in and out. I made the ‘show transcription’ links in the community voices popups buttons rather than links and fixed the links in the tabular dialect samples page, which were pointing to an older version of the atlas that no longer exists. I made the border round the atlas blue to make it clearer where the atlas area ends and the rest of the page begins, as the previous light grey border was a bit hard to see and fixed an issue with the map popups when viewing a story pane that displays data for ‘young’ or ‘old’, whereby the sentence for ‘all’ and the other age group was still appearing when it shouldn’t. I also applied a number of these fixes to the linguists’ atlas as well. Here’s a screenshot of how the public atlas now looks:
The UCU strike begins on Monday next week so I will not be back into work until Thursday the 5th of December, unless the situation changes.
It was another mostly SCOSYA week this week, ahead of the launch of the project that was planned for the end of next week. However, on Friday this week I bumped into Jennifer who said that the launch will now be pushed back into December. This is because our intended launch date was the last working day before the UCU strike action begins, and is a bad time to launch the project, for reasons of publicity, engaging with other scholars and risks associated with technical issues that might crop up which might not be able to be sorted until after the strike. As there’s a general election soon after the strike is due to end, it looks like the launch is going to be pushed back until closer to Christmas. But as none of this transpired until Friday I still spent most of the week until then making what I thought were last-minute tweaks to the website and fixing bugs that had cropped up during user testing.
This included going through all of the points raised by Gary following the testing session he had arranged with his students in New York the week before, and meeting with Jennifer, E and Frankie to discuss how we intended to act on the feedback, which was all very positive but did raise a few issues relating to the user interface, the data and the explanatory text.
Also this week I had a further chat with Luca about the API he’s building, and a DMP request that came his way, and arranged for the App and Play store account administration to be moved over the Central IT Services. I also helped Jane Roberts with an issue with the Thesaurus of Old English and had a chat with Thomas Clancy and Gilbert Markus about the Place-names of Kirkcudbrightshire project, which I set the systems up for last year and is now nearing completion and requiring some further work to develop the front-end.
I also completed an initial version of a WordPress site for Corey Gibson’s bibliography project and spoke to Eleanor Capaldi about how to get some images for her website that I recently set up. I also spent a bit of time upgrading all of the WordPress sites I manage to the latest version. Also this week I had a chat with Heather Pagan about the Anglo-Norman Dictionary data. She now has access to the data that powers the current website and gave me access to this. It’s great to finally know that the data has been retrieved and to get a copy of it to work with. I spent a bit of time looking through the XML files, but we need to get some sort of agreement about how Glasgow will be involved in the project before I do much more with it.
I had a bit of an email chat with the DSL people about adding a new ‘history’ field to their entries, something that will happen through the new editing interface that has been set up for them by another company, but will have implications for the website once we reach the point of adding the newly edited data from their new system to the online dictionary. I also arranged for the web space for Rachel Smith and Ewa Wanat’s project to be set up and spent a bit of time playing around with a new interface and design for the Digital Humanities Network website (https://digital-humanities.glasgow.ac.uk/) which is in desperate need of a makeover.
Also this week I met with Corey Gibson from Scottish Literature to discuss an online bibliographical resource he would like me to help him put together for a Carnegie funded project that he’s currently running. It was a useful meeting and we made some decisions about how the resource will function, and later in the week I put in a request to set up a subdomain for the resource. Next week I’ll create an initial version for him. Also this week I created a new project website for an RSE funded project that is involving English Literature for Eleanor Capaldi. I created an initial interface and page structure and got all of the basics in place and will update this further once Eleanor gets back to me, if further changes are required.
I had a meeting with Rachel Smith about an interactive website that she is putting together with Ewa Wanat. I’d met with Ewa about this in May but hadn’t heard anything since, but since then they have been speaking to Alistair Beith, a PhD student in Psychology, who is going to do the development work for them. Alistair was at the meeting too and we discussed the requirements of the project, the technologies that will be used and some of the implications relating to access and file formats. It was good to speak to another person with web development skills, and once the team has decided on a suitable subdomain for the project I’ll get things set up and give Alistair access to create the site.
I also met with my old friend and colleague Yunhyong Kim from Information Studies to try and get access to an ancient Apple computer that I had left in the building when I moved to Critical Studies. Thankfully I could still remember the password and we managed to get it to boot up so I could set her up with a user account. I also had some further communication with Brian McKenna from Central IT Services about the University’s App and Play Store accounts. I have been managing these for many years now, mainly because Critical Studies was prepared to pay the subscription fees when no-one else was. It looks like responsibility for this is now going to be taken over the IT Services, which makes sense. I just hope it’s not going to make the process of publishing apps even more torturous than it already is, though.
Finally, on Friday I attended an ArtsLAb session on research integrity and data management plans. This was a course that we’d tried to run several times before, but hadn’t managed to get sufficient numbers to sign up for. This time we reduced the length of the session and we got a decent number of attendees. I had previously spoken at such sessions, but as there was less time I suggested that it made sense to split the hour between Nigel Leask, who gave a very interesting talk about research integrity, and Matt from Research Data Management gave a great overview of data management and what researchers are required to do. I provided some sample data management plans for the attendees to look at in their own time and it was a very useful session.
I split most of my time this week between the SCOSYA project and the Historical Thesaurus. The launch of the SCOSYA atlases is scheduled to take place in November and I had suggested to Jennifer that it might be good to provide access to the project’s data via tables rather than through the atlas interfaces. This is because although the atlases look great and are a nice interactive way of accessing and visualising the data, some people prefer looking at tables of data instead, and other people may struggle to use the interactive atlases due to accessibility issues, but may still want to be able to view the project’s data. We will of course provide free access to the project’s API, through which all of the data can be accessed as CSV or JSON files, or can even be incorporated into a completely new interface, but I thought it might be useful if we provided text-based access to the data through the project’s front-end as well. Jennifer agreed that this would be useful, so I spent some time writing a specification document for the new features, sending it to the team for feedback and developing the new features.
I created four new features. First was a table of dialect samples, which lists all of the locations that have dialect sample recordings and provides access to these recordings and the text that accompanies them, replicating the data as found on the ‘home’ map of the public atlas. The second feature provides a tabular list of all of the locations that have community voice recordings. Clicking on a location then displays the recordings and the transcriptions of each, as the following screenshot shows:
The third new feature lists all of the examples that can be browsed for through the public atlas. You can then click on one of these examples to listen to the example sound clips of the example and to view a table of results for all of the questionnaire locations. Users can also click through to view this example on the atlas itself too, as I figured that some people might want to view the results as a table but then see how these look on the atlas too. The following screenshot shows the ‘explore’ feature for a particular example:
The fourth new feature replicates the full list of examples as found in the linguists’ atlas. There are many examples nested within parent and sub-parent categories and it can be a bit difficult to get a clear picture of what is available through the nested menus in the atlas, so this new feature provides access to a complete list of the examples that is fully expanded and more easy to view, as the following screenshot demonstrates:
It’s then possible to click on an example to view the results of this example for every location in a table, again with a link through to the result on the atlas, which then enables the user to customise the display of results further, for example focussing on older or younger speakers or limiting the display to particular rating levels.
Finally for the project this week I met with Jennifer and E to discuss the ancillary pages and text that need to be put in place before the launch, and we discussed the launch itself and what this would involve.
For the HT I generated some datasets that an external researcher had requested from the Thesaurus of Old English data, and I generated some further datasets from the main HT database for another request. I also started to implement a system to generate the new dates table. I created the necessary table and wrote a function that takes a lexeme and goes through all of the 19 date fields to generate the rows that would need to be created for the lexeme. As of yet I haven’t set this running on the whole dataset, but instead I’ve created a test script that allows you to pass a catid and view all of the date rows that would be created for each lexeme in the category so I (and Marc and Fraser) can test things out. I’ve tested it out with categories that have some complicated date structures and so far I’ve not encountered any unexpected behaviour, apart from one thing: Some lexemes have a full date such as ‘1623 Dict. + 1642–1711’. The script doesn’t analyse the ‘fulldate’ field but instead looks at each of the actual date fields. There is only one ‘label’ field so it’s not possible to ascertain that in this case the label is associated with the first date. Instead the script always associates the label with the last date that a lexeme has. I’m not sure how common it is for a label to appear in the middle of a full date, but it definitely crops up fairly regularly when I load a random category on the HT homepage, always as ‘Dict.’ so far. We’ll need to see what we can do about this, if it turns out to be important, which I guess it probably will.
Also this week I performed some App Manager duties, had a conversation with Dauvit Broun, Nigel Leask and the RDM team about the ArtsLab session on data management plans next week, and spoke to Ann Ferguson of the DSL about how changes to the XML structure of entries will be reflected in the front-end of the DSL.