This is my 200th weekly report since starting this job, which is something of a milestone! I received emails from three members of staff who required my help this week. Maria Dick in English Literature is putting together a project proposal and wanted some advice on a project website so I had an email conversation with her about this. Carolyn Jess-Cooke in English Literature wanted my help to set up a website for a small unfunded project she is starting up, and I spent some of the week getting this set up. Finally Michael Shaw of The People’s Voice project got in touch with me because he has created a new flyer for the project and wondered if the project website design could be updated to reflect some of the elements from the flyer. I played around with some of the graphics he had sent me and came up with a new interface that I think looks a lot nicer than the current one. I sent screenshots of this new interface to Michael and he is going to show them to the rest of the team to see whether we should replace the current website design.
On Monday this week I completed work on Android versions of the ‘ARIES’ and ‘English Grammar: An Introduction’ apps. I took screenshots of the apps running on my Android tablet and completed the Play Store listing for the apps and then went through the various processes required to prepare the app package file for submission to the store. This is a slightly cumbersome process, involving various command-line tools such as zipalign and certification and signing tools. But by the afternoon I had submitted the apps to the Play Store and the following day the apps were available to download. You can download ‘ARIES’ here: https://play.google.com/store/apps/details?id=com.gla.stella.aries and ‘English Grammar: An Introduction’ here: https://play.google.com/store/apps/details?id=com.gla.stella.grammar. They are free to use so give them a try.
On Tuesday I met with Jean Anderson to discuss the ‘Basics of English Metre’ app that I’m currently redeveloping. Jean had tested out the app for me and had noted down a few areas where improvements could be made and a couple of places where there were bugs. It was a really useful meeting and we spent some time thinking about how some aspects of the app could be made more intuitive. I spent some time during the week implementing the changes Jean had suggested, namely:
Some of these changes took rather a long time to implement, especially the introduction of new colours for certain types of content as this meant updating the JSON source data file and some parts of the JavaScript code for both the ‘app’ and ‘web’ versions of the tool (these versions have some differences due to the different libraries that are used in each). I am now just waiting for Jean to supply me with some ‘about’ text and then hopefully I’ll be able to start creating iOS and Android versions of the resource. BTW, you can view the ‘web’ version of the resource here if you’re interested: http://arts.gla.ac.uk/stella/apps/web/metre/
On Wednesday I had a SCOSYA meeting with Jennifer and Gary to discuss where we’re at with the technical aspects of the project. They both seem pretty satisfied with the progress I’ve made so far and our discussions mostly focussed on what developments I should focus on next. This was good because I had pretty much implemented all of the items that I had on my to do list for the project. After our meeting I had lots more things added, namely:
E.g. Attributes A,B,C are selected, joined by ‘OR’. These are given the colours Red, Green and Blue. Location X has A and B so has a marker split in two – half red, half green. Location Y has A, B and C so has a marker split into thirds, one red, one green, one blue. Location Z only has attribute C so is coloured solid blue. The ratings are not represented in the markers – just whether the attribute is present or not. I’m going to see whether it might be possible to use D3.js for this.
E.g. There are 6 categories within Negative Concord and the limit options are ‘rated by 2 or more people with a rating of 3-5’. The atlas will count how many of these 6 categories meet the criteria at each location. This will then be split into 5 grades (so as to be able to handle any number of categories within one or more parents). If the 6 categories (100%) meet the criteria at Location X then the map maker will be dark (e.g. black). If only 3 categories (50%) match then the marker will be lighter (e.g. grey). If no categories (0%) match then the marker will be white. This will allow us to show the density of particular forms by location. Note that some categories may need to be excluded by the project team. This will be handled manually once Gary knows which attributes might need taken out (e.g. because they are just not present anywhere).
So, lots more to do for the project, and I’ll probably start on this next week.
I spent a fair amount of time this week working on the SCOSYA project again. We had a project meeting planned for Wednesday but unfortunately Gary had to postpone this to next week, but I decided to tackle some of the outstanding items on my ‘to do’ list of the project nonetheless. The first thing I implemented was a ‘history’ feature for the Atlas. This was based on the ‘history’ feature I previously created for the Mapping Metaphor project, allowing you to return to a previous version of the visualisation interface that you had been looking at. For SCOSYA, the Atlas now ‘remembers’ every search you do during a session and lists these on a ‘history’ page, allowing you to return to something you’ve previously looked at. A history entry is logged whenever you press one of the ‘show’ buttons to change the atlas display criteria. Changes in zoom level and map position don’t currently get logged as a new history entry. I can add these, but I thought it would result in too much information being logged. However, it is possible to save a snapshot of a particular position and zoom level by pressing the ‘save’ button again. The history feature can be accessed by pressing on a ‘Browse History’ button that I have added to the bottom of the ‘Atlas display options’ panel. The list of items shows what you’ve searched for, when you searched for it (with the most recent at the top) and includes buttons that allow you to load the search item. There’s also a ‘clear history’ option too.
The ‘load history’ aspect took a bit of time to get working as the page content depends purely upon the data contained in the ‘hash’ of the page (the part of the URL after the ‘#’. The things is, when anything in this part of the URL changes it doesn’t trigger a page reload, meaning pressing on the ‘load’ button didn’t actually do anything. I investigated just refreshing the page content on the client side when the ‘load’ button was pressed, but I already had a lot of code in place to load the data on initial page load based on the values passed in the hash (not just loading the data but prepopulating all of the search boxes) so I thought it would be simpler just to trigger all of this rather than write something new. It turned out to be pretty easy to trigger a full page load. First of all I updated the hash with the new value from the ‘load’ button’s ‘href’ attribute using ‘location.replace’ and then I reloaded the page using ‘location.reload’.
The last SCOSYA item on my ‘to do’ list for the atlas was to add the code descriptions to the atlas. The atlas display options slide-out for the atlas has a series of drop-down lists for selecting one or more attribute codes. We wanted to ensure descriptions for these codes were also visible to give users a bit more information about what each code is. Initially I added these as ‘tooltips’ when you hover over an attribute in the drop-down lists in ‘Atlas options’ using the ‘tooltipster’ jQuery plugin (http://iamceege.github.io/tooltipster/) that I’d previously used for the Thesaurus of Old English. The screenshot below shows how this works.
Unfortunately I came to realise that there were a couple of fairly major limitations with these tooltips in the atlas. As touchscreens don’t have a ‘hover over’ action it means the tooltips won’t work on tablets or smartphones, and I can’t make the tooltips open on click because clicking selects an option in a drop-down list. Also, the behaviour of drop-down lists is dependent on the operating system and unfortunately tooltips just won’t work in drop-down lists in MacOS, iOS or Android devices. Also, I realised that the tooltips don’t work when the ‘full screen’ mode is activated either. I realised that the only way I’d to be able to get around all (or possibly only some) of these limitations would be to get rid of the drop-down lists and replace them with something that’s purely JavaScript based and therefore works independently of the underlying operating system. I decided to try the jQuery UI Selectmenu (https://jqueryui.com/selectmenu/) first of all, as the jQuery UI library is what I normally use for such widgets.
I’m afraid this just didn’t work out. I tried replacing the standard drop-down list with the ‘selectmenu’ and while this did work in MacOS it failed to work at all when in ‘full screen’ mode – i.e. the drop-down list wouldn’t even drop down, effectively breaking the interface. I then thought about other ways we could allow an attribute to be selected, as the drop-down list is rather long and unwieldy. I thought about maybe having the selection in a new pop-up that would give more space for the attribute list and the descriptions. For popups I use the jQuery UI ‘modal dialog’, and unfortunately it looks like none of the widgets are going to work when in ‘full screen’ mode. I figured out that the reason for this is the ‘full screen’ mode hides everything other than elements contained within the ‘map’ section of the page, and as the jQuery UI widgets exist elsewhere on the page they simply don’t show up when in ‘full screen’ mode, even though all associated JavaScript triggers and runs properly. This is the reason the ‘tooltipster’ tooltips weren’t displaying in ‘full screen’ mode either.
I would consider the ‘full screen’ mode to be hugely important for the atlas as it makes the interface so much more usable. Therefore I don’t think we should attempt to use any user interface elements that don’t work in this view. For this reason I abandoned the idea of adding tooltips to the drop-down lists. Instead I’ve added in something that is a bit more ‘low tech’ but which works across all operating systems, screens and in ‘full screen’ mode: When you select an attribute from the list its description then appears below the drop-down list, as the following screenshot shows:
This does unfortunately mean that you can’t see what the description is until you have selected the attribute, but at least it is possible to view the information. I will also create a link to a full list of attributes and their descriptions that users could consult in conjunction with the atlas interface. I can always return to this once I start on the public version of the atlas and see if there are any further solutions. For example, it still might be possible to update the ‘full screen’ view so it does allow non-map elements to be used – possible by ensuring all jQuery UI elements are located somewhere within the map element.
In addition to the above I’ve also reworked the pop-ups that appear on the map so that code descriptions appear. I’ve also rearranged the individual ratings to group them by their code so the code name and other information isn’t repeated for each rating but appears in a heading above the ratings.
Other than my work for SCOSYA, I spent some time on AHRC review duties and I had a meeting with the Burns people on Wednesday where we discussed the timetable for updates t the website for the next six months or so. I then spent the rest of the week working on the STELLA Apps. I’m meeting with Jean Anderson next week to go over the ‘Metre’ app, so in the meantime I’ve decided to return to the two older apps that I never got round to making Android versions of: ‘English Grammar: An Introduction’ and ‘ARIES’. I have created Google Play listings for these apps, have generated a new version of the app and have tested this out on my phone. I have generated icons and screenshots (on my phone, I still need to do this on my Android tablet) and am just about ready to begin the process of building the release versions of the app code. I’ll hopefully be able to submit the apps to the store next week, if I can get screenshots on my tablet sorted out and the pesky process of building and signing the apps doesn’t take too long.
I had expected to return to SCOSYA duties this week as we had a team meeting scheduled for Monday morning, but the meeting was pushed back t next Wednesday so in the meantime I decided to focus on other projects instead. I did tick off one item on my SCOSYA ‘to do’ list this week, though: I managed to track down a bug which was causing the CSV view of the map data to include an extra column for some data, which was offsetting some of the rows and thus messing up the structure. The problem was caused by the ‘data that doesn’t meet your criteria’ being included. This data includes an extra field so that the system knows that its points on the map should be coloured grey. In the CSV view this extra field was being added to the middle of each spreadsheet row, which was pushing everything else along one cell. I decided to remove this data from the CSV as it’s not really necessary to include it. The CSV instead should focus on the relevant data, not the stuff that doesn’t match.
I spent some time on Monday updating the ‘Metre’ app after receiving some initial feedback from Jean Anderson. She didn’t like the colour scheme so I’ve chosen a more restrained one instead. I also created the ‘web’ version of the resource. This version has the UoG look and feel, but is otherwise mostly the same. There are some slight differences in the JavaScript as this version doesn’t use the jQueryMobile framework so some UI components needed to be replaced. You can access and try out the ‘web’ version here: http://www.arts.gla.ac.uk/STELLA/apps/web/metre/ but do be aware that this version is still undergoing testing and may break or change without warning.
I had to undertake a few more Apple developer account related tasks this week, which took up a bit of time. I also spent some time doing further AHRC review duties and responded to a request from Sarah Jones of the DCC regarding a workshop about Technical Plans. I also fixed some issues in the rather ancient ‘Learning with the online Thesaurus of Old English’ website. I spent the best part of Wednesday working on the REELS project. One of the project team would like export and import facilities to be added to the CMS to enable the data to be exported to an Excel compatible format. As the structure of the data is rather complex (13 inter-related tables), converting this to a flat spreadsheet is going to be rather tricky and will take a lot of planning. I spent several hours thinking about the implications and replying to the team member’s original email. Once I receive some clarification on a number of issues I raised I will set about implementing the required feature.
On Tuesday I met with Marc and Fraser to discuss updates to the Historical Thesaurus based on the new data supplied to us from the OED people. We discussed the scripts I had already created to analyse the data and considered our next steps, which for me would involve the creation of further scripts that Fraser would then be able to access in order to check the data. It is going to be rather tricky to ensure all of the correct changes are implemented. For example, the OED has revised dates for words and we generally want to use these instead of the dates we currently have in the HT, but in some particular cases our dates will still be the more accurate. There are also issues relating to matching up words in the HT with words in the OED data as sometimes words have changed category, or words are formatted differently (hyphens or spaces etc) and therefore don’t match up.
My first task following the meeting was to update the scripts I had created that show which categories in the two datasets don’t match up. The OED data includes lots of empty top-level categories that don’t have a part of speech, and as these would never match up with any HT category I removed them from my script. I also updated the script I created that lists words within matching HT / OED categories so that words are converted to lower case before comparison happens as there were some mismatches between the datasets (e.g. ‘Arctic Circle’ and ‘Arctic circle’ weren’t being matched).
I then thought it might be useful to be able to view all of the words within categories that didn’t match across the two datasets, as a means of possibly identifying where problems might be and which category should match which. Marc emailed me on Thursday with an urgent request for some figures relating to the updates we’re doing so I got those to him as soon as I could. I then set about creating a new script that looks at the distinct words in the HT database and then compares every ‘sense’ of the word across the two datasets. Basically counting the number of categories each word appears in within each dataset and seeing how these differ. The script outputs three tables of data. The first set shows all those words that appear more often in HT categories than OED categories (with counts of categories and details of each category included). The second set lists words that appear more often in OED categories than HT categories. The this lists those words that appear in the same number of categories, but only those ones where those listed categories are not exactly the same in each set (i.e. pinpointing where words have moved category).
In the set for ‘appears more in OED categories than HT categories’ I also included any distinct words found in the OED dataset that were not present at all in the HT dataset. Rather surprisingly there were 45804 words in the OED data that were not found in the HT data. This is surprising because there are 369888 distinct non-OE words in the HT data and only 322635 distinct words in the new OED data. It took a bit more investigation to figure out what was going on, but it looks like a lot of the remaining ‘new’ words are phrases, or have brackets, or hyphens and I suspect a lot of these are just variants of words we already have in the HT rather than being new words per se. I think we’re going to have to be careful when adding ‘new’ words.
On Friday I managed to complete work on these new scripts and save the output as static HTML tables for further analysis by Fraser. There are 57420 words that appear more often in the HT data than the OED data, 59085 words that appear more often in the OED data (including ‘new’ words), and 11959 words that appear the same number of times in each dataset but appear in different categories. That’s a lot of analysis to carry out, but hopefully we can come up with a series of rules to properly align the majority of these.
This week was rather an unusual one due to my son being ill with the winter vomiting virus. I had to take Tuesday off as annual leave at short notice to look after him and then I also ended up having to work from home on Wednesday in addition to my usual Thursday. I spent quite a bit of Monday creating a new version of the ‘Essentials of Old English’ app that fixed an issue with the Glossary. I’d started on this last Friday but ended up spending ages just installing updates and getting to the point where I could build new versions of the app. Thankfully I managed to get this sorted on Monday, although it still took an awfully long time to build, test and deploy the iOS and Android versions. However, they are now both available (version 1.2) through the App and Play stores now. I also spent a couple of hours on Monday replying to a rather detailed email for the REELs project and also spent a bit of time getting some of the Hansard sample data to Marc.
I was off work on Tuesday, which unfortunately meant rescheduling the meeting I was supposed to have with Marc and Fraser about updating the Historical Thesaurus with new data from the OED people. This is going to take place next Monday now instead. As I haven’t heard back from the SCOSYA people since the last updates I made I decided to hold off with any new developments here until after we have our next meeting (also next Monday), so this gave me some time I could spend on the ‘Basics of English Metre’ app. I spent most of Wednesday, Thursday and some of Friday afternoon on this, and I’m very pleased to say I have now completed a first draft of the app. IT took a bit of time to get back into developing the app as it had been a while since I last worked on it. There were also some rather tricky exercises that I needed to implement as part of Unit 3 of the app, which took some planning, testing and refining. It feels very satisfying to have all of the exercises fully operational now, though. The next stage will be to get people to test it, make required updates, create a ‘web’ version with the University website’s look and feel and then start the arduous wrapping and deploying of the actual app versions. It feels like the end is in sight now, though, and I’ve already started to think about what old resource to tackle next. Having said that, I still need to make Android versions of the ‘Grammar’ and ‘ARIES’ apps first.
On Friday I dealt with some issues relating to the University’s iOS developer account, created some text for Marc in relation to getting access to a collection of historical textual data and created a new feature for the Medical Humanities Network that lists the email addresses of all members in a format that can be pasted into Outlook. Next week I’ll return to SCOSYA work and will no doubt be working on the Historical Thesaurus again.