I continued to work on the Online Exhibition for the Robert Fergusson project this week, spending much of Monday and Tuesday on the task and completing an initial version that I sent to the team for feedback on Tuesday afternoon. As I worked with the materials and got to know them a bit better I ended up creating a more fully-featured first draft than I was originally intending to make. It is still an initial version, and any aspects can be changed as required. I took inspiration from the Books and Borrowing online exhibition to a certain extent. As with this site, the pages all have an ‘Explore’ section with image-based buttons leading to the various sections, and each page has links to the next and previous sections at the bottom.
I can’t share screenshots or the URL just yet, but the interface is designed to work on all screen dimensions from large monitors to mobile phones and as you scroll down the pages sections animate into place. There are currently two alternative animations. Pages such as ‘Publications’ have a ‘zoom’ animation whereby the section zooms and slides up into place as you scroll. The alternative can be viewed on pages such as ‘Depictions’ where sections are already visible but the content slides in from the left and right as you scroll (except the final video section of ‘Depictions’ which zooms up). Certain pages have the first section fixed (e.g. ‘Publications’) while other pages animate the first section too (e.g. ‘Scots poems’). We might not want to mix and match the approaches so much – I included alternatives so the team can see the options.
The pages currently consist of sections with three different background colours that are alternated (a greeny-grey, a dark grey and a sort of grey-teal colour). The header font is the same one as the main site and the paragraph text is fairly large. Where a section has one image this appears to the left or right (this alternates) in a box that also features the caption. Where there are multiple images (e.g. ‘poems on various subjects’ on the ‘publications’ page) a carousel is used and you can press the arrow buttons to scroll between images. In such cases there is still currently one single caption underneath the carousel.
On the ‘depictions’ page I’ve embedded the YouTube video in the final section. On the ‘Scots poems’ page I’ve included the three audio recordings in a final section with some placeholder text. Where images appear, I’ve applied some post-processing on many of them to remove the yellowing and make them brighter. I think this makes the images look nicer as part of an online exhibition, but the team may prefer to use the originals for the sake of authenticity. I’m pretty happy with how the initial version of the exhibition has turned out. During the rest of the week I had an email conversation with the RA Amy Wilcockson about adding in new content or rearranging the content that was already there, and I spent some of my time making such updates, some of which where rather time-consuming to implement but needed to be done.
On Tuesday afternoon I met with Moira and Leo from Archives and Special Collections to discuss some of the old resources I’d created for the archives around 15-20 years ago. It was great to catch up with them both and to hear their plans for the resources. It turns out that the University Story site has been completely lost – no copy of the site and its database remains on any University servers or other hardware. This was quite a shock as a huge amount of effort went into writing the content of this site – thousands of biographies as well as structured data about (for example) holders of professorships throughout history. When I stopped working for the Archives some 13 years ago a lot of work continued to be done to the site, both in terms of the content and reworking by subsequent developers, but I wondered whether I might still have a backup of the site as I left it somewhere at home.
On Wednesday I hunted about for the data. I found my old backup external hard drive in my attic, but unfortunately this did not include anything I hadn’t already discovered on my more recent external hard drive. I then managed to locate the old laptop that I used when working at the Archives back in the day, thankfully complete with power cable. This took ages to boot up, and I couldn’t remember the password I used to use for it. And the laptop was taking more than half an hour between entering a password and coming up with the ‘incorrect password’ warning. I then decided to set up a ‘boot from USB’ installation of Ubuntu to run on the laptop instead of booting the ancient version of Windows and this worked perfectly. I was able to access the laptop’s hard drive and copy all of the files relating to my Archives work onto my external hard drive. This included one single dump of the UGS database from June 2011, plus all of the code, images, and a bunch of other stuff relating to other projects. I now have this on my desktop PC as well as on the external hard drive. I loaded the SQL dump into a MySQL instance on my new laptop and it looks like the data is complete (albeit from 2011). It consists of 93 tables, incorporating the main UGS site, the international story, the ‘world changing’ site and the WW1 roll of honour. There are records for 19,228 people and of these 1,799 have biographies. There are 1,111 image records and these link to the image files contained in the ‘images’ directory in the code. There are three derivatives per image, which is why there are so many image files compared to image records.
The code contains the code for the site and the CMS, as it was in 2011. I did try getting the site to run on my laptop, but it would take a lot of work to get it operational due to changes in technology and also because it references stylesheets from the main University website from 2011. An awful lot was added to the site after June 2011, but hopefully having the database structure and the data up to this point will be a useful starting point. Stats on the homepage from the Internet Archive snapshot from April 2018 state that there are 2390 images and 3678 biographies on the site, which is a lot more than we had in 2011.
If it’s possible to request a copy of the UGS data from the Internet Archive the Archives people could then write a script to compare the people IDs in the 2011 database with IDs embedded in the URLs of pages from the Internet Archive (e.g. ‘WH10083’ from https://web.archive.org/web/20150908140047/http://www.universitystory.gla.ac.uk/biography/?id=WH10083&type=P&o=&start=0&max=20&l=a) to then make a list of all of the people that are not found in the 2011 database. The same could be done with images (e.g. ‘UGSP01400’ in the URL https://web.archive.org/web/20130704220509/http://www.universitystory.gla.ac.uk/image/?id=UGSP01400&o=&start=0&max=20&l=A&biog=WH10083&type=P&p=2). Of course there’s no guarantee that the Internet Archive captured all of the data for every person and image, but this would at least enable the archives to create a more limited list of people and images that could then be added to the 2011 data, rather than having to process all of the Internet Archive data from scratch.
The archives people could then write a script that could extract data only from the people / image pages in the list and insert this into the database. The pages are pretty well structured so it might not be too tricky to write such a script. Of course any edits made to the existing data since 2011 wouldn’t be spotted by this approach, as only new records would be targeted.
I wouldn’t recommend using the old code for the site and CMS, which is all very outdated now. The archives would be better off building a new system around the database structure, referencing the Internet Archive version of the old site to help figure out how things used to work. They might also want to rationalise the database structure as it’s possible not all tables are actually needed, or could be structured more efficiently.
Also this week I had an informal meeting with Marc Alexander to discuss my work, and I read through an edited version of the upcoming Books and Borrowing article to make sure the edits all made sense.
For the rest of the week I looked into a task that has been on my long-term ‘to do’ list for ages: Redeveloping the Fife Place-names website (https://fife-placenames.glasgow.ac.uk/). I made this as a proof of concept many years ago, having written scripts to extract and structure the data from Word documents. What I want to do is migrate the data to the system I created for the other place-names resources, which would then mean I could create a map interface for Fife that would be comparable to the Berwickshire one, for example (see https://berwickshire-placenames.glasgow.ac.uk/map/).
The first task was to attempt to generate latitude, longitude and altitude data for Fife, as currently the data only has grid references. I wrote a script that would do this, thinking I’d just be able to leave it running. Unfortunately when I ran it I realised that many of the grid references have problems. Many records have malformed grid references, such as ones with an additional number, or a zero character instead of an ‘O’, while for some others the data extraction script did not successfully grab the grid reference. In many such cases I had to do research to discover the actual grid reference using a combination of Google Maps, the historical maps on the Berwickshire website, the UK Grid Reference Finder (https://gridreferencefinder.com/) and the NLS GB1900 website (https://maps.nls.uk/projects/os1900). As you can imagine this took quite a lot of time, and I hadn’t managed to complete the georeferencing by the end of the week.
I’m on holiday for the next week and a bit, and I’ll return to this task once I’m back.