Week Beginning 18th January 2016

Two new projects that I will be involved with over the coming months and years started up this week. The first one was the People’s Voice project for Catriona MacDonald and Gerry Carruthers. I will be developing a database of poems and establishing a means of enabling the team to transcribe poems using the TEI guidelines. This is a good opportunity for me to learn more about text encoding as although I’ve been involved in some text encoding projects before I’ve never had sole responsibility for such aspects. Since starting back after Christmas I’ve been getting up to speed with TEI and the Oxygen text editing tool and this week I met with the team and we had a two hour introductory session to transcription using TEI and Oxygen. I spent quite a bit of time before the session preparing a worksheet for them and getting my head around the workings of Oxygen and the workshop went very well. It was the first time some of the people had ever written any XML and everyone did very well. It will obviously take a bit of practise for them to be able to transcribe poems rapidly, but hopefully the worksheet I prepared together with the template files I’d made will allow them to get started. There is still lots to get the project up and running and we will be meeting again in the next few weeks to get started on the database and the website, but so far things are progressing well.

The second new project I was involved with this week was Carole Hough’s REELS project (Recovering the Earliest English Language in Scotland). This project will be analysing the placenames of Berwickshire and I’ll be developing a content management system to enable the team to record all of the data. We had a project meeting this week where we went over the project timetable and discussed how and when certain tasks would start up. It was a useful meeting and a good opportunity to meet the rest of the team and we have now arranged a further, more technical meeting for next week where we will think in more detail about the requirements for the database and the CMS and things like that. I also put a request in for a subdomain for the project website and got some shared drive space set up for the project too.

As well as these two projects beginning, another project I’ve been involved with launched this week. Over the past few months I’ve been developing the technical infrastructure for a Medical Humanities Network website for Megan Coyer. This project had its official launch on Friday evening, and all went very well. The project can be accessed here: http://medical-humanities.glasgow.ac.uk/

In addition to these projects I also spent a bit of time trying to figure out what was preventing the Curious Travellers WordPress installation from connecting to external services. Using my test server I managed to fix one of the issues (the instance will now connect to the WordPress RSS feeds) but other services such as Akismet and the searching of new plugins fail to connect. The strange thing is if I copy both the database and the files for the site onto my test server external connections work, which would suggest that the problem is with the server configuration. I’ve spoken to Chris about this and he has done some investigation but as far as he can tell there is nothing at server level that is any different to other server setups. It’s all very odd and we may have to consider moving the site to a different server to see if this fixes the problem.

I also spent some time working with the Grid and preparing the Hansard data for processing on the Grid. Gareth Roy from Physics has been helping me with this and he’d sent me some instructions on how to submit a test job to the Grid before Christmas. This week I managed to successfully submit process and extract the output for my test script, which is encouraging. Gareth thought that splitting my 10Gb text file into small chunks for processing would make the most sense so I wrote a little script that split the file up, with 5000 lines per file. This resulted in about 1200 files with sizes varying from 5Mb to 16Mb, which should hopefully be relatively easy to load and process. I now have to figure out how to write a shell script that will load a file, process it and export SQL statements to another text file. I’ve never written a shell script that does anywhere near as much as this before, so it’s going to take a bit of time to get the hang of things.

My final project of the week was Metaphor in the Curriculum. We had another project meeting this week and as a result of this I added a new feature to the Mapping Metaphor website and did some further work on our prototype app. The new feature is a ‘Metaphor of the Day’ page that does what you’d expect it to: displaying a different metaphorical connection each day. You can view the feature here: http://mappingmetaphor.arts.gla.ac.uk/metaphor-of-the-day/

For the prototype I updated the structure so that the homepage of the app is now a list of links to subsections rather than just displaying the list of quizzes. This list has now been moved to a subsection. The structure is now in place to be able to add other subsections to the prototype, such as the ability to browse the metaphors and access the visualisation. As the app will need to be used without an internet connection I’m going to have to ensure that all of the data can be accessed by the app locally. For the main website the data is stored in a MySQL database and there are a bunch of AJAX calls to some PHP scripts that then generate the required HTML fragments or JSON data that the website’s Javascript then uses. For the app everything is going to have to be handled in Javascript, with the full dataset made available as JSON files. With this in mind I created some scripts that generate the JSON files I will require. In the next few weeks I will then have to figure out how to update the code to work directly with these files.

Week Beginning 11th January 2016

The Medical Humanities Network ‘soft launched’ on Friday this week so I had quite a bit of last minute tweaking and adding of features to manage before this happened. This included updating the structure to allow people and collections to be associated with each other, fixing a number of bugs, ensuring ‘deleted’ content could no longer be accessed through the site (it was previously available for test purposes), adding a new ‘contact’ section and adding a feature that ensures people agree that they have the rights to upload images. It’s all looking pretty good and as far as I’m aware it’s going to be officially launched in a week’s time.

I also received the final pieces of information I required this week to allow paid apps to be published through the Apple App Store and the Google Play Store. This is something that has been dragging on for a while and it is really good to get it out of the way. It’s actually something that’s required by people outside of the Critical Studies and getting it sorted took quite a bit of effort so it’s especially good to get it sorted.

I also took ownership of the Pennant project’s technical stuff this week. This is a temporary arrangement until a new developer is found, but in the meantime I noticed some problems with the project’s WordPress installation. There is some sort of issue that is stopping WordPress connecting to external servers. It’s being blocked somehow and as this is affecting things such as the Akismet anti-spam plugin I thought I’d better try and investigate. I had thought it was some kind of server setting, but I installed the site on my test server and it gave the same errors, even though another WordPress site I had on the server worked fine. I tried a variety of approaches, such as updating the version of WordPress, replacing the data with data from a different instance and deactivating each plugin in turn, and I eventually figured out that it’s something within the wordpress options table that’s causing the problem. If I replace this with a default version the connection works. However, this table contains a lot of information about valid WordPress plugins so I’ll have to carefully go through it to identify what has caused the problem. I’m fairly certain it’s one of the plugins that has somehow managed to block external connections. I’ll need to continue with this next week.

I met with Gary Thoms this week to discuss the technical aspects of the SCOSYA project. The .ac.uk domain name has still not come through yet so I contacted Derek Higgins, who is the person who deals with JANET, to ask him what’s going on. Happily he said JANET have now approved the domain and are awaiting payment, so we should be able to get a project website set up in the next few weeks at least. In the meantime I set Gary’s laptop up so that it could access the test version of the site I developed last year. This now means that he can use the content management system to upload and edit survey data and things like that.

I also tried to help Fraser Dallachy out with a problem he was encountering when using the command line version of the SAMUELS tagger. When running the script on his laptop he was just getting memory errors. I updated the command he was running and this at least got the script to start off, but unfortunately it got stuck loading the HT data and didn’t proceed any further. Fraser spoke to Scott at Lancaster about this and he thought it was a memory issue – apparently the script requires a minimum of 2Gb of RAM. Fraser’s laptop had 4Gb of RAM so I wasn’t convinced this was the problem, but we agreed to try running it on my new desktop PC (with 16Gb of RAM) to see what would happen. Surprisingly, the script ran successfully, so it would appear that 4Gb of RAM is insufficient. I say it ran successfully, which it did with a test file that only included ‘the cat sat on the mat’. Unfortunately, no matter what we did by way of changing the input file, the output resolutely produced the output for ‘the cat sat on the mat’! It was most infuriating and after trying everything I could think of I’m afraid I was stumped. Fraser is going to speak to Scott again to see what the problem might be.

I spent the rest of the week getting up to speed with TEI and Oxygen for the People’s Voice project. Although I have a bit of experience with TEI and XML technologies I have never been responsible for these aspects on any project I’ve been involved with before, and to become the resident expert is going to take some time. Thankfully I found some very handy online tutorials (http://tei.it.ox.ac.uk/Talks/2009-04-galway/) aimed at complete beginners, which I found to be a very useful starting point, despite being a few years old now. With a sample poem from the project in hand and Oxygen opened on my computer I managed to make some pretty good progress with transcribing and figuring out how to cope with the variety of content that needed to be marked up in some way. The thing about TEI is there are often several different ways something could be encoded and it’s difficult to know which is ‘right’, or perhaps most suitable. Thankfully I had Graeme Cannon around to offer advice, which was hugely helpful as Graeme has been using these technologies for a long time now and knows them all inside out. By the end of the week I had familiarised myself with Oxygen as a tool, had created a RelaxNG schema using the TEI P5 Roma tool, had created a stylesheet for the Author View, had transcribed the poem to a standard that Graeme was happy with and had begun work preparing the workshop I’m going to be leaving for the project team next Wednesday.

Week Beginning 4th January 2016

I returned to work on Wednesday this week, after very enjoyable but seemingly all too brief Christmas holiday. I spent a bit of time on Wednesday sorting through my emails, replying to things and checking that there was nothing lurking in my inbox from last year that I’d failed to deal with. I also spent a few hours helping Simon Taylor out with some problems a user of his Fife Placenames website had encountered. I had previously helped Simon to extract the placename data from Word files and created scripts that would split these up and generate a relational database for the resulting data, but it would appear that in a few cases the splitting script hadn’t quite worked properly and some places were being appended to the entries for the previous place. I think I managed to get this all sorted now, though.

I also spent a fair amount of my time this week on the Medical Humanities Network website and database. The project is going to be launching this month and there have been a number of further tweaks and updates that Megan and Hannah have requested, such as allowing Twitter feeds to be associated with collections, changing the way teaching materials are organised and updating the list of icons for projects. Everything is coming together quite nicely, though.

Gavin Miller also contacted me about his SciFiMedHums project. I’d previously created a WordPress plugin that allows him and his RA to create and manage bibliographical data and now he is hoping to get the public involved in submitting data too. We had an email discussion where I described a few possible ways in which this could be handled and I’ll just need to wait and see how Gavin wants to proceed with this.

I also worked on the Metaphor in the Curriculum project, greatly expanding the amount of content that is currently available in the prototype app that I’ve created. Previously we had one quiz available (metaphor and war) but before Christmas Rachael had uploaded some new quizzes to the project’s shared drive (in Word format) so I created interactive versions of these. There are now four quizzes available, each with their own nice background image. Things are shaping up quite nicely, I think.

So, a bit of a short report this week. Next week I’m going to have lots to do, such as starting work on the People’s Voice project, continuing with the SCOSYA project and working with the Grid to try and get the Hansard data processed.