It was another unsettled week for me, as on Tuesday my uncle died. He’d been suffering from cancer for a while and had been steadily getting worse, but we hoped he would make it until after Christmas, so it was a bit of a shock. I was very close to him, and he introduced me to hillwalking, which is one of my favourite pastimes, so it’s all very sad.
However, I still managed to work just about a full week of work and get a lot done. I’d met with Jennifer Smith several weeks ago to discuss a pilot project she was putting together that would require an online questionnaire that schoolchildren would fill in. On Monday Jennifer phoned me up to say that the pilot school had been in touch and she would need the website in place for next Monday. Thankfully I had intended to spent a fair amount of time this week on the SCOSYA project (another of Jennifer’s projects) so I could simply divert my time without causing too much inconvenience to anyone.
I met with Jennifer and her RA Derek Henderson on Monday to discuss what needed to be done. When we’d met previously we’d decided that a simple Google Form would probably work well enough as the questionnaire, but after further discussions it turned out this wasn’t going to work so well. Derek had been playing about with Google Forms but it just wasn’t flexible enough to get things laid out exactly as he wanted. Plus, we needed to password protect the form and create different user accounts for different schools, and as a Google form is hosted on a Google server at a Google URL it’s not possible to do such a thing (I did some investigation and it is possible to set up a password, but only in JavaScript on the client side, which means anyone looking at the page source can see how to bypass it). So for these reasons I decided I’d just set up the questionnaire myself at a subdomain on one of our servers.
As the questionnaire is to be filled in on mobile devices as much as traditional PCs I used the jQuery Mobile framework to set up the user interface. This worked very nicely as it provides lots of great widgets for form elements. It’s also possible (and indeed recommended) to use one single page to process multiple jQuery Mobile powered pages, which allows for nice transitions between pages, and also works very nicely with forms split over multiple pages (the questionnaire was in four parts) as it means the previous stages aren’t actually submitted but are just ‘hidden’ by the framework. This allows the user to navigate back to the earlier stages and for all the content they’ve entered to still be there.
So, I set up a database structure for questions, answers, user groups and such things. I designed a simple but pleasing user interface, aided by the jQuery Mobile theme roller, I picked out a couple of nice fonts for the header and the main site text, I added in some validation of certain boxes, made some boxes hidden until certain options had been selected, and wrote a small amount of PHP to handle the submission of the form and the inserting of the data into the database. I also created a script that exports all of the data as a CSV file for Jennifer and Derek to use.
Other than this I had a further email conversation with the DSL people about future updates and make a couple of tweaks to the live website. I also participated in an email discussion about the way the Thesaurus of Old English links out to the Dictionary of Old English. Previously, we were able to link from a word on our site directly into the search facility on the DOE website, allowing users to go from an OE word on our site to entries on about the word on the DOE website. However, a while back the DOE unveiled a new website, which was much improved but unfortunately changed the way their search worked. All searches are now handled by AJAX in the client’s browser and it’s not possible for us to hook into this. However, DOE are keen on allowing us to hook in as before so I made some suggestions as to how this might be possible, and explained exactly how the old system worked. It looks like we might be able to get something working again soon.
On Wednesday I met with Pauline Mackay to discuss a few upcoming issues. Firstly, we talked about a new database she’s been putting together that I’m going to create an online version of. Pauline has created this in Access and I should be able to migrate this to an online version and create a nice search / browse interface fairly easily. Secondly, we talked about updating the ‘Editing Burns’ website to incorporate the new ‘Phase 2’. We’re going to have some sort of top level of tabs allowing users to choose two different sites that have different colour schemes. There’s a project meeting in December that I’ll be going to where this will be discussed further. Thirdly, we discussed a new project Pauline is in the process of putting together that will have a small technical component. I can’t say much more about this for now, but I’m going to help out with the Technical Plan.
I also spent a bit of time this week on AHRC review duties and I have a brief chat with Scott Spurlock about a possible way of getting his parish records project funded. And I updated Gavin Miller’s SciFiMedHums project website to disable user registrations as these had started to attract spammers and the project doesn’t require user registrations at the moment anyway. Oh, on Monday I was on an interview panel for a technical job in another part of the College. Finally, I spent a bit of time upgrading all of the WordPress instances I manage as a new version of WordPress (version 4.9) was recently released. No doubt there will be a 4.9.1 to install before too long.
I mostly split my time this week between three projects: The Dictionary of the Scots Language, SPADE and SCOSYA. For DSL I managed to complete the initial migration of the DSL website to WordPress and all pages and functionality have now been transferred over. Here’s what’s in place so far:
I emailed Thomas and Ann at SLD about the new version of the site and they are going to play around with it for a while and get back to me. I think I’ve finished work on this rather more swiftly than they were expecting so it may be a while before any further work is done on this.
For the SPADE project I met up with Rachel again and we spent a morning continuing to work on the Polyglot server. Following some helpful advice from the team in Montreal we managed to make some good progress this week with running some sample texts through the Polyglot system. Rachel had picked out six audio files and accompanying text grid files from the ‘sounds of the city’ project and we prepared these to be run through the system, and updated the system config files so it knew where to find the files, and where to place the outputs. With it all set up we ran the script for extracting sibilant data, and it began processing the files. This took some time but progress was looking good, until we received an error message stating that PRAAT had encountered errors when processing consonants. As the error message said ‘please forward the details to the developers’ we did just that. We received some further replies but as of yet we’ve not managed to get the script to work and we’ll need to return to this next week. Progress is being made, at least.
For SCOSYA I finally managed to return to updating the Atlas interface. The Atlas uses the Leaflet.js mapping library, and when I first put the Atlas together the current stable version was 0.7. It’s now up to version 1.2 and I’ve upgraded the library to this version. As there are some fairly major differences between versions it’s taken me some time to get things working, but it’s been worth doing. With the previous version the tooltip hover-overs had to be implemented by a further library, but these are now included in the default Leaflet library, and look a bit nicer. Another advantage is the sidebar will now scroll using the mouse wheel when it’s taller than the page. I’m also now using a more up to date version of the Leaflet data visualisation toolkit library, which produces the polygon markers. These things may seem rather minor but making the update will ensure the atlas continues to function for longer.
I’ve also updated the attribute search. Previously when a code had multiple attributes the code ended up appearing multiple times in the drop-down list. This gave the impression that selecting one or other of the options would give different results, but that wasn’t the case – e.g.A8 has attributes NPPDoes and NSVA so A8 appeared twice in the list. But whether you select one or the other the search is still for A8. I’ve now amalgamated any duplicates. This actually proved to be more complicated to sort out than I’d expected. Basically I had to create a new search type in the API to search based on code rather than attribute ID. I then needed to update the Atlas search to work with this new option.
I’ve also updated the ‘or’ search to get rid of star markers, as requested, and to limit the number of differently sided polygons that are used. The ‘or’ search still brings back random colours and shapes each time, but I might change this. I made all of these changes live in the CMS and also the ‘atlas guest’ URL and I’m hoping to start work on the ‘advanced attribute search’ next week.
I had a bit of an unsettled week this week as one of our two cats was hit by a car and killed on Monday night. It’s all very sad. Anyway, in terms of work, I was on an interview panel on Tuesday so spent some of Monday afternoon preparing for this, together with a fair amount of Tuesday morning participating in the interviews. On Monday I also ran a few more queries for Fraser relating to the HT and OED data matching. I also had a few more WordPress administrative tasks to take care of. I also had to spend some of this week on AHRC review duties.
In addition, I spent about a day working on the SPADE project. Rachel, the project RA’s PC had finally been set up and the Linux subsystem for Windows 10 had been activated by Arts IT support. Our task was to install and configure the Polyglot server and database software that will be used by the project, and is being developed by our Montreal partners. Thankfully there was quite a lot of documentation about this process and we could follow the many steps that were required to install the software and its dependencies. I was a little sceptical that we would be able to do all of this without needing further administrator access on Rachel’s PC (if we need administrator access we have to ask someone in Arts IT Support to come over and entire their password), but rather wonderfully, once you’ve set up the Linux subsystem it basically works as a virtual machine, with its own admin user that Rachel is in control of. I have to say it was a little strange working from a Linux command prompt in Windows, knowing that this was all running locally rather than connecting to a remote server. The polyglot server sets up a Django web server through which various endpoints can be accessed. I wondered whether it would be possible to access this ‘local’ sever from a web browser in the main Windows 10 instance and the answer was yes, it most certainly is possible. So I think this setup is going to work rather well – Rachel will just need to open up the Linux subsystem command prompt and start the server running. After that she will be able to access everything through her browser in Windows.
We did, however, run into a few difficulties with the installation process, specifically relating to the setting up of the Polyglot database to which the server connects. The documentation got a little shaky at this point and it was unclear whether by installing the server we had also automatically installed the database, or whether we still needed to manually get this set up. We contacted Montreal and were quickly told that we didn’t need to install the database separately, which was good. We’re now at the stage where we can try to start running some tests on some sample data, although once more we’re not entirely sure how to proceed. It’s a bit tricky when we don’t actually know exactly what the software does and how it does it. It would have been useful to have had a demo of the system before we tried to set up our own. We’ll press on with the test scripts next week. Also for SPADE this week I extracted some data from the SCOTS corpus that had been missed out of the dataset that Montreal had previously been given.
I had a meeting with Graeme to discuss some development issues, and I spent most of the rest of the week continuing with the reworking of the DSL website. I updated the ‘support DSL’ page of the live site as I’d realised it had a broken link and some out of date information on it. I then continued migrating the DSL website to WordPress. The big task this week was to handle the actual dictionary pages – i.e. the search results page, the advanced search page, the bibliography page and the actual entry page itself – all of the pages that actually connect to the DSL’s database and display dictionary data. For the most part this was a fairly straightforward process. I needed to strip out any references to my own layout scripts, incorporate a link to the WordPress system and then add in the WordPress calls that display the WordPress header, footer and sidebar content. This then means any changes to the installed WordPress theme are reflected on these dictionary pages, even though the pages themselves are not part of the WordPress instance. There were of course a few more things that needed done. I’m replacing all of the PNG icons with Font Awesome icons, so I needed to update all of the occurrences of these. I also noticed that the bibliography search results page wasn’t working properly if the user enters text and then presses ‘search’ rather than selecting an option from the ‘autocomplete’ facility. The results page loads with the ‘next’ and ‘previous’ links all messed up and not actually working. I spent some time fixing this in my new WordPress version of the site, but I’m not going to fix the live site as it’s a minor thing that I’m guessing no-one has actually even noticed. With these pages all working I spent some time testing things out and it would appear to me that the new DSL dictionary pages are all working in the same way as the old pages, which is great.
With the dictionary pages out of the way I continued migrating some of the ancillary pages to the new site. I’ve managed to get all pages other than the ‘history of scots’ pages complete. I’m about half-way through the latter, which is a massive amount of content. It’s taking some time to migrate, not just because of its length, but also because it incorporates lots of images that I need to upload to WordPress, and even a couple of OpenLayers powered maps that I needed to migrate too. Hopefully I’ll get this section of the site fully migrated early next week. After that I’ll need to think about incorporating a sidebar, and I also need to tweak a few more aspects of the site, such as the HTML titles that get displayed in the entry pages. After that I’ll need some further input from SLD about how we’re going to include the pages from the main SLD website.
I had been intending to continue to make updates to the SCOSYA atlas this week, but I realised this would not be a good idea because Gary was demonstrating it to a class. I had also intended to spend some of the week setting up the Polyglot server for the SPADE project, but we are unfortunately still having some issues with getting access to the hardware on which I will install the software. We’re going to set things up on the project RA Rachel MacDonald’s new PC, which has now been delivered but has yet to be set up by Arts IT Support. However, the machine is going to had the University’s ‘Standard Staff Desktop’ environment installed on it, and the new Windows 10 version of this seriously locks down administrator access. Apparently only Arts IT Support will be able to have administrator access to a machine, which is potentially going to make managing the software (which is still in development) a little tricky. By the end of the week this issue still hadn’t quite been sorted, but hopefully next week Rachel and I will be able to get stuck into the software installation.
As I was unable to do the work I had intended to do this week, I instead focused on setting up the new amalgamated version of the SLD and DSL websites. I met with Thomas last week to discuss the plans for this, which will involve creating a WordPress version of the existing DSL site while keeping the actual dictionary search, browse and view pages separate. SLD has agreed the payment for this work and as I had some time available I thought I’d just get started.
I’d never created a WordPress theme from scratch before, having previously only ever adapted existing themes. I had to spend some time researching how themes are made and how they need to be structured, for example how to ensure that the necessary JavaScript and CSS files are loaded in and get picked up by WordPress. Integrating the menu with WordPress proved to be rather tricky as I had to ensure that my styles and JavaScript worked with the menu structure as outputted by WordPress, so as to enable the site menu to be dynamic (i.e. so staff users can update the menu contents). I have also added in a new menu that gets used on narrow. This works better than the ‘live’ site, but I still need to work on the bullet point styles. I created my own little ‘breadcrumb’ function that generates the breadcrumb trail too, and I’m replacing the old icons with nicer Font Awesome ones.
I also had to investigate how to set up ‘sidebars’ (basically any area where WordPress widgets can be added) and have created such areas for the homepage to allow some custom DSL widgets to be placed there. I also then created these widgets, such as a DSL quick search widget that connects into the existing DSL AJAX scripts in order to display an ‘auto-complete’ list of options as you type, and a ‘Word of the day’ widget that connects to the DSL API in order to display the current day’s word. I added in a couple of new free-text widgets for the other homepage boxes too, such as the ‘news’ box. I had considered just tying this into the ‘SDL News’ blog posts, but decided it would be better instead to allow staff to manage exactly what appears in this homepage box, rather than just getting the content automatically from whatever new item was most recently posted.
There’s still lots to do (e.g. actually integrating the dictionary pages) but I’m pretty happy with the progress I’ve made so far. In the first instance I’m aiming to produce a version of the DSL that should be just about identical to the ‘live’ site but is actually WordPress powered. After that we can add new content, tweak the layout and such things.
I probably spent the best part of four days working on the DSL site. The rest of the week was spent doing some further work on the OED data for the Historical Thesaurus for Fraser and updating all of the WordPress sites I manage to the most recent version of WordPress.