Week Beginning 18th September 2017

On Monday this week I spent a bit of time creating a new version of the MetaphorIC app, featuring the ‘final’ dataset from Mapping Metaphor.  This new version features almost 12,000 metaphorical connections between categories and more than 30,000 examples of metaphor.  Although the creation of the iOS version went perfectly smoothly (this time), I ran into some difficulties updating the Android app as the build process started giving me some unexplained errors.  I eventually tried dropping the Android app in order to rebuild it, but that didn’t work either and unfortunately dropping the app also deleted its icon files.  After that I had to build the app in a new location, which thankfully worked.  Also thankfully I still had the source files for the icons so I could create them again.  There’s always something that doesn’t go smoothly when publishing apps.  The new version of the app was made available on the Apple App and Google Play stores by the end of the week and you can download either version by following the links here: http://mappingmetaphor.arts.gla.ac.uk/metaphoric/.  That’s Mapping Metaphor and its follow-on project MetaphorIC completely finished now, other than the occasional tweak that will no doubt be required.

I spent the bulk of the rest of the week working on the Burns Paper Database for Ronnie Young.  Last week I started looking at the Access version of the database that Ronnie had sent me, and I’m managed to make an initial version of a MySQL database to hold the data and I created an upload script that populated this table with the data via a CSV file.  This week I met with Ronnie to discuss how to take the project further.  We agreed that rather than having an online content management system through which Ronnie would continue to update the database, he would instead continue to use his Access version and I would then run this through my ‘import’ script to replace the old online version whenever updates are required.  This is a more efficient approach as I already have an upload script and Ronnie is already used to working with his Access database.

We went through the data together and worked out which fields would need to be searchable and browseable, and how the data should be presented.  This was particularly useful as there are some consistency issues with the data, for example in how uncertain dates are recorded, which may include square brackets, asterisks, question marks, the use of ‘or’ and also date ranges.

After the meeting I set to work creating an updated structure for the database and an updated ‘import’ script that would enable the extraction and storage of the data required for search purposes.  This included creating separate tables for year searches, manuscript types, watermarks and countermarks, and also images of both the documents and the watermarks.  It took quite some time to get the import script working properly, but now that it is in place I will be able to run any updated version of the data through this in order to create a new online version.  With this in place I set to work on the actual pages for searching and browsing, viewing results and viewing an individual record.  Much of this I managed to repurpose from my previous work on The People’s Voice database of poems, which helped speed things up considerably.  The biggest issue I encountered was with working with the images of the manuscript pages.  The project contains over 1200 high-resolution images that Ronnie wants users to be able to zoom into and pan around.  In order to work with these images I had to batch process the creation of thumbnails and also the renaming of the images, as they had a mixture of upper and lower case file extensions, which causes problems for case sensitive servers.  I then had to decide on a library that would provide the required zoom and pan functionality.  Previously I’ve used OpenLayers, but this requires large images to be split into tiles, and I didn’t want to have to do this.  Instead I looked at some other JavaScript libraries.  What I really wanted was a ‘google maps’ style interface that allowed multiple levels of zoom.  Unfortunately most libraries didn’t seem to offer this.  I found one called ‘jQuery Panzoom’ (http://timmywil.github.io/jquery.panzoom/demo/) that fitted the bill, and I tried working with this for a while.  Unfortunately, my images were all very large and the pane they will be viewed in is considerably smaller, and it didn’t seem very straightforward to reposition the zoomed image so that it actually appeared visible in the pane when zoomed out by default.  Instead I tried another library called magnifier.js (http://mark-rolich.github.io/Magnifier.js/) that can be set up to have a thumbnail navigation window and a larger main window.  I spent quite a bit of time working with this library and thought everything was going to work out perfectly, but then I encountered a bug:  If you manually set the dimensions of the pane in which the zoomed in image appears and these dimensions are different to the image then the zoomed in image is distorted to fit the pane.  After investigating this issue I discovered it had been raised by someone in 2014 and had not been addressed (see https://github.com/mark-rolich/Magnifier.js/issues/4).  As a distorted image was no good I had to look elsewhere once again.  My third attempt was using the ‘Elevate Zoom’ plugin (http://www.elevateweb.co.uk/image-zoom/examples).  Thankfully I managed to get this working.  It also can be set up to have a thumbnail navigation window and then a larger pane for viewing the zoomed in image.  It can also be set up to use the mouse wheel to zoom in and out, which is ideal.  The only downside is without physical zoom controls there’s no way to zoom in and out when using a touchscreen device.  But as it’s still possible to view the full image at one zoom level I think this is good enough.  By the end of the week I had pretty much completed the online database and I emailed the details to Ronnie for feedback.

Other than the above I also did a little bit of work for the SPADE project, beginning to create a proper interface for the website with Rachel MacDonald, and I had a further chat with Gerry McKeever regarding the website for his new project.

Week Beginning 11th September 2017

I spent more than half of this week continuing to work on the new ARIES app.  Last week I finished work on an initial, plain HTML and JavaScript version of the app, and I received another couple of bits of feedback this week that I implemented.  The bulk of my time, however, was spent using Apache Cordova to ‘wrap’ the HTML and JavaScript version, converting it into actual iOS and Android apps, then testing these apps on my iOS and Android devices, and then making all of the media files that an app needs, such as icon files, screenshots, app loading screens, app store graphics and things like that.  This process always takes longer than I think it should.  For example, I have to make more than 20 different icon files at varying resolutions, and I need to grab multiple screenshots from at least four different devices.  This latter process is made trickier because my Android Nexus 7 tablet no longer connects properly to my PC – the ‘photos’ folder appears blank when I connect for photo transfer and doesn’t contain the actual updated contents when I connect for file transfer, so I have to use a third party file explorer app to move the screenshots to a different folder on the device that somehow does get updated when viewing on my PC.  Regarding the icons, I came up with a few alternatives for this, based on the header image for the app, and we finally agreed on a sort of ‘marble’ effect circle on a white background.  I think it looks pretty good, and is certainly better than the old ARIES logo.  The app publication process was also complicated by two new issues that have emerged since I last made an app.  Firstly, Apple have updated the build process to disallow any extended image metadata, I guess as a security precaution.  I created my app icon PNG files in Photoshop, which added in such metadata.  When I then built my iOS app in xCode I received some rather unhelpful errors.  Thankfully StackOverflow had the answer (see https://stackoverflow.com/questions/39652867/code-sign-error-in-macos-sierra-xcode-8-3-3-resource-fork-finder-information) and after running a couple of command-line scripts this metadata was stripped out and the build succeeded.  My second issue related to the app name on the App Store.  Apple has decided to limit app names to 30 characters, meaning we could no longer call our app ‘ARIES: Assisted Revision in English Style’.  And as there is already an app names ‘ARIES’ we couldn’t call it that either.  This is a real pain, and seems like a completely unnecessary restriction to me.  In the end we called the app “ARIES: English Academic Style”.  I managed to submit the app to Apple and Google on Wednesday, and thankfully by the end of the week the new version was available on both the App and Play Stores.  I also made the ‘web’ version available, replacing the old ARIES site.  You can access this, and link through to the app versions from here: http://www.arts.gla.ac.uk/stella/apps/web/aries/

Other than ARIES work, I made some further changes to the Edinburgh Gazetteer keywords, replacing the old list of keywords with a much trimmed down list that Rhona supplied.  I think this works much better than the previous list, and things are looking good.  I also helped Alison Wiggins with some information she wanted to add to the Digital Humanities website, and I spent about half a day working with the Mapping Metaphor data, generating new versions of all of the JSON files that are required for the ‘Metaphoric’ app and testing the web version of this out.  It looks like everything is working fine with the full dataset, so next week I’ll hopefully publish a new version of the app that contains this data.  I also started working on the database of Burns’ paper for Ronnie Young, firstly converting his Access database into an online MySQL version and then creating a simple browse interface for it.  There’s still lots more to be done for this but I need to meet with Ronnie before I can take this further.

The rest of my week was taken up with meetings.  On Wednesday morning I was on an interview panel for a developer post in another part of the college.  I also met with Gerry McKeever in the afternoon to discuss his new British Academy funded ‘Regional Romanticism’ project.  I’ll be working with him to set up a website for this, with some sort of interactive map being added in sometime down the road.  I spent Friday morning attending a network meeting for Kirsteen McCue’s Romantic National Song Network.  It was interesting to hear more about the project and to participate in the discussions about how the web resource for this project will work.  There were several ideas for where the focus for the online aspect of the project should lie, and thankfully by lunchtime we’d reached a consensus about this.  I can’t say much more about it now, but it’s going to be using some software I’ve not used before but am keen to try out, which is great.

Week Beginning 4th September 2017

I spent a lot of this week continuing with the redevelopment of the ARIES app and thankfully after laying the groundwork last week (e.g. working out the styles and the structure, implementing a couple of exercise types) my rate of progress this week was considerably improved.  In fact, by the end of the week I had added in all of the content and had completed an initial version of the web version of the app.  This included adding in some new quiz types, such as one that allows the user to reorder the sentences in a paragraph by dragging and dropping them, and also a simple multiple choice style quiz.  I also received some very useful feedback from members of the project team and made a number of refinements to the content based on this.

This included updating the punctuation quiz so that if you get three incorrect answers in a quiz a ‘show answer’ button is displayed.  Clicking on this puts in all of the answers and shows the ‘well done’ box.  This was rather tricky to implement as the script needed to reset the question, including removing all previous answers, ticks, and resetting the initial letter case as if you select a full stop the following letter is automatically capitalised.  I also implemented a workaround for answers where a space is acceptable.  These no longer count towards the final tally of correct answers, so leaving a space rather than selecting a comma can now result in the ‘well done’ message being displayed.  Again, this was rather tricky to implement and it would be good if you could test out this quiz thoroughly to make sure there aren’t any occasions where the quiz breaks.

I also improved navigation throughout the app.  I added ‘next’ buttons to all of the quizzes, which either take you to the next section, or to the next part of the quiz, as applicable.  I think this works much better than just having the option to return to the page the quiz was linked from.  I also added in a ‘hamburger’ button to the footer of every page within a section.  Pressing on this takes you to the section’s contents page, and I added ‘next’ and ‘previous’ buttons to the contents pages too, so you can navigate between sections without having to go back to the homepage.

I spent a bit of time fixing the drag / drop quizzes so that the draggable boxes were constrained to each exercise’s boundaries.  This seemed to work great until I got to the references quiz, which has quite long sections of draggable text.  With the constraint in place it became impossible for the part of the draggable button that triggers the drop to reach the boxes nearest the boundaries of the question as none of the button could pass the borders.  So rather annoyingly I had to remove this feature and just allow people to drag the buttons all over the page.  But dropping a button from one question into another will always give you an incorrect answer now, so it’s not too big a problem.

With all of this in place I’ll start working on the app version of the resource next week and will hopefully be able to submit it to the app stores by the end of the week, all being well.

In addition to my work on ARIES, I completed some other tasks for a number of other projects.  For Mapping Metaphor I created a couple of scripts for Wendy that output some statistics about the metaphorical connections in the data.  For the Thesaurus of Old English I created a facility to enable staff to create new categories and subcatetories (previously it was only possible to edit existing categories or add / edit / remove words from existing categories).  I met with Nigel Leask and some of the Curious Travellers team on Friday to discuss some details for a new post associated with this project.  I had an email discussion with Ronnie Young about the Burns database he wants me to make an online version of.  I also met with Jane Stuart-Smith and Rachel MacDonald, who is the new project RA for the SPADE project, and set up a user account for Rachel to manage the project website.  I had a chat with Graeme Cannon about a potential project he’s helping put together that may need some further technical input and I updated the DSL website and responded to a query from Ann Ferguson regarding a new section of the site.

I also spent most of a day working on the Edinburgh Gazetteer project, during which I completed work on the new ‘keywords’ feature.  It was great to be able to do this as I had been intending to work on this last week but just didn’t have the time.  I took Rhona’s keywords spreadsheet, which had page ID in one column and keywords separated by a semi-colon in another and created two database tables to hold the information (one for information about keywords and a joining table to link keywords to individual pages).  I then wrote a little script that went through the spreadsheet, extracted the information and added it to my database.  I then set to work on adding the actual feature to the website.

The index page of the Gazetteer now has a section where all of the keywords are listed.  There are more than 200 keywords so it’s a lot of information.  Currently the keywords appear like ‘bricks’ in a scrollable section, but this might need to be updated as it’s maybe a bit much information.  If you click on a keyword a page loads that lists all of the pages that the keyword is associated with.  When you load a specific page, either from the keyword page or from the regular browse option, there’s now a section above the page image that lists the associated keywords.  Clicking on one of these loads the keyword’s page, allowing you to access any other pages that are associated with it.  It’s a pretty simple system but it works well enough.  The actual keywords need a bit of work, though, as some are too specific and there are some near duplications due to typos and things like that.  Rhona is going to send me an updated spreadsheet and I will hopefully upload this next week.

Oh yes, it was five years ago this week that I started in this post.  How time flies.

Week Beginning 28th August 2017

This week was rather a hectic one as I was contacted by many people who wanted my help and advice with things.  I think it’s the time of year – the lecturers are returning from their holidays but the students aren’t back yet so they start getting on with other things, meaning busy times for me.  I had my PDR session on Monday morning, so I spent a fair amount of time at this and then writing things up afterwards.  All went fine, and it’s good to know that the work I do is appreciated.  After that I had to do a few things for Wendy for Mapping Metaphor.  I’d forgotten to run my ‘remove duplicates’ script after I’d made the final update to the MM data, which meant that many of the sample lexemes were appearing twice.  Thankfully Wendy spotted this and a quick execution of my script removed 14,286 duplicates in a flash.  I also had to update some of the text on the site, update the way search terms are highlighted in the HT to avoid links through from MM highlighting multiple terms.  I also wrote a little script that displays the number of strong and weak metaphorical connections there are for each of the categories, which Wendy wanted.

My big task for the week was to start on the redevelopment of the ARIES app.  I had been expecting to receive the materials for this several weeks earlier as Marc wanted the new app to be ready to launch at the beginning of term.  As I’d heard nothing I assumed that this was no longer going to happen, but on Monday Marc gave me access to the files and said the launch must still go ahead at the start of term.  There is rather a lot to do and very little time to do it in, especially as preparing stuff for the App Store takes so much time once the app is actually developed.  Also, Marc is still revising the materials so even though I’m now creating the new version I’m still going to have to go back and make further updates later on.  It’s not exactly an ideal situation.  However, I did manage to get started on the redevelopment on Tuesday, and spent pretty much all of my time on Tuesday, Wednesday and Thursday on this task.  This involved designing a new interface based on the colours found in the logo file, creating the structure of the app, and migrating the static materials that the team had created in HTML to the JSON file I’m creating for the app contents.  This included creating new styles for the new content where required and testing things out on various devices to make sure everything works ok.  I also implemented two of the new quizzes, which also took quite a bit of time, firstly because I needed to manually migrate the quiz contents to a format that my scripts could work with and secondly because although the quizzes were similar to ones I’ve written before they were not identical in structure, so needed some reworking in order to meet the requirements.  I’m pretty happy with how things are developing, but progress is slow.  I’ve only completed the content for three subsections of the app, and there are a further nine sections remaining.  Hopefully the pace will quicken as I proceed, but I’m worried that the app is not going to be ready for the start of term, especially as the quizzes should really be tested out by the team and possibly tweaked before launch.

I spent most of Friday this week writing the Technical Plan for Thomas Clancy’s new place-name project.  Last week I’d sent off a long list of questions about the project and Thomas got back to me with some very helpful answers this week, which really helped in writing the plan.  It’s still only a first version and will need further work, but I think the bulk of the technical issues have been addressed now.

Other than these tasks, I responded to a query from Moira Rankin from the Archives about an old project I was involved with, I helped Michael Shaw deal with some more data for The People’s Voice project, I had a chat to Catriona MacDonald about backing up The People’s Voice database, I looked through a database that Ronnie Young had sent me, which I will be turning into an online resource sometime soon (hopefully), I replied to Gerry McKeever about a project he’s running that’s just starting up which I will be involved with, and I replied to John Davies in History about a website query he had sent me.  Unfortunately I didn’t get a chance to continue with the Edinburgh Gazetteer work I’d started last week, but I’ll hopefully get a chance to do some further work on this next week.