Monday was a bank holiday so this was a four-day week for me. I had yet more AHRC review duties to perform this week so quite a lot of Tuesday was devoted to that. I also had an email conversation with Murray Pittock about the technical plan for a proposal he is currently putting together. We’re still trying to decide on the role of OCR in the project, but I think a bit of progress is being made. I also spent some further time helping to sort out the materials for the new section of the Burns website. This mainly consisted of sorting out a series of video clips of song performances, uploading them to YouTube and embedding them in the appropriate pages. For Mapping Metaphor, Wendy had sent me on some new teaching materials that she wanted me to add to the ‘Metaphoric’ resource (http://mappingmetaphor.arts.gla.ac.uk/metaphoric/teaching-materials.html). This involved uploading the files, adding them to the zip files and updating the browse by type and topic facilities to incorporate the resources.
My two big tasks of the week were working with the Hansard data and starting the redevelopment of another old STELLA resource. As mentioned in previous posts, Gareth Roy of Physics and Astronomy has kindly set up a database server for me where the Hansard data can reside as we’re processing it. Last week I added all of the ancillary tables to the database (e.g. Information about speakers) and I ‘fixed’ the SQL files so that MySQL could process them at the command line and I wrote a very simple Shell script that takes the path to an SQL file as an argument and then invokes the MySQL command to import that file in to the specified database. I tested this out on the first output file, running the Shell script on the test server I have in my office and it successfully inserted all 581,409 rows contained in the file into the database. It did take quite a long time for the script to execute, though. About an hour and 20 minutes, in fact. With that individual test successfully completed I wrote another Shell script that would submit a batch of jobs to the Grid. It took a little bit of trial and error to get this script to work successfully within the Grid, mainly due to needing to specify the full path to the MySQL binary from the nodes. Thankfully I got the script working in the end and set it to work on the first batch of 9 files (files 2-10 as I had already processed file 1). It took about four hours for the nodes to finish processing the files, which means we’re looking at (very roughly) 30 minutes per file and there are about 1200 files so it might take about 25 days to process them all. That is unless processing more than 9 jobs at a time is faster, but I suspect speed might be limited to some extent at the database server end. I submitted a further 20 jobs before I left the office for the weekend so we’ll just need to see how quickly these are processed.
The STELLA resource I decided to look into redeveloping is ‘The Basics of English Metre’. The old version can be found here: http://www.arts.gla.ac.uk/stella/Metre/MetreHome.html. It’s a rather out of date website that only works properly in Internet Explorer. The exercises contained in it also require the Flash plugin in order to function. Despite these shortcomings the actual content (if you can access it) is still very useful, which I think makes it a good candidate for redevelopment. As with the previous STELLA resources I’ve redeveloped, I intend to make web and app versions of the resource. I spent some time this week going through the old resource, figuring out how the exercises function and how the site is structured. After that I began to work on a new version, setting up the basic structure (which in common with the other STELLA resources I’ve redeveloped will use the jQueryMobile framework). By the end of the week I had decided on a structure for the new site (i.e. which of the old pages should be kept as separate pages and which should be merged) and had created a site index (this is already better than the old resource which only featured ‘next’ and ‘previous’ links between pages with no indication of what any of the pages contained or any way to jump to a specific page). I also made a start processing the content, but I only got as far as working on the first exercise. Some of the latter exercises are quite complicated and although I will be able to base a lot of the exercise code on the previous resources I had created it is still going to require quite a bit of customisation to get things working for the sorts of questions these exercises ask. I hope to be able to continue with this next week, although I would also like to publish Android versions of the two STELLA apps that are currently only available for iOS (English Grammar: An Introduction and ARIES) so I might focus on this first.