I had several meetings this week, the first being to discuss the development of the map for Ophira Gamliel’s HiMuJe Malabar project.  In the run-up to the meeting it became clear that no-one had read the specification document I’d written regarding the development of the map, which I’d sent out several weeks ago, but thankfully after I raised this it was distributed before the meeting and we had a very useful discussion about the map and how to prepare the data for the map.  It feels like some real progress has been made in our joint understanding of the data and what needs to be presented, and following the meeting I sent out a proposed structure for the JSON data that will be generated from the digital edition files and a proposal for how we would document travel routes on the map.  I also met with project RA Renu a couple of days later to discuss geoJSON data and the creation of polygons for certain locations in the data.

My second meeting was with the Place-names of Armagh people, and this was also very useful.  We discussed some additional explanatory text found in the original data that I hadn’t spotted before, the splitting up of the existing ‘analysis’ data into Irish forms and translations and the creation of the front-end.  Now all I need to do is actually find the time to work on all of this.  My third meeting was with the Burns Supper team, at which we went over the ‘facts and figures’ popup that I will be developing.  We also discussed the launch of the resource and a demo that was given on Monday and was very well received.

I spent a fair amount of time this week working on the DOST Auld Laws project, beginning with an analysis of the XML files in order to work out what non-standard elements generated by Transkribus needed to be converted into which TEI elements.  It took some time to go through everything but it was very useful to get it all documented and subsequent email discussions with Joanna and Pia were really helpful.

I then ran a script to update the XML files to change the filenames, add in IDs and replace the referenced image filenames (both in the XML files and the image files themselves) with a more rational naming structure as I’d previously discussed with the team.  For example, the file with the title ‘115 – Peebles B. Rec.’ was given the ID ‘peebles-b-rec’ and image files and references were renamed from (for example) ‘0001_115 001.jpg’ to ‘peebles-b-rec_001.jpg’.  With this in place I then arranged with Luca for the images to be uploaded to the IIIF server, after which I could begin working on the front-end on the server, as opposed to on my laptop as I’d been doing previously.

As part of this work I also decided to migrate the document data I’d extracted from the XML from JSON to a MySQL database.  The reason for this was that some of the document JSON files were rather large (a couple more than 6MB) and it seemed rather inefficient to load all of this data in just to work out counts of the number of pages and such things.  With all of the data in a relational database it’s much easier to query and return just the data that is needed.

During the week I completed work on an initial version of the image parts of the site.  The user interface is by no means complete and is fairly rudimentary – we will decide thinks like fonts, colour schemes, illustrative images and ancillary content later on.  For now there is just placeholder text on all but the ‘Documents’ pages and the search option doesn’t work yet.  However, it is possible to browse the documents and all facsimile images contained in each.  Pressing on the ‘Documents’ menu item brings up a sub-menu listing the documents, plus an ‘overview’ page which is currently empty.  If you select a document you’re presented with an overview page.  This can feature a description of the document (there’s just placeholder text for now) a link to open the document at the first page and a randomly selected image displayed to the right of the description for illustrative purposes.

 

Beneath this is a section through which you can see a count of the number of pages and access a list of thumbnails of every page.  For some of the longer documents it can take some time to load in all of these thumbnails.  We can maybe have this section hidden by default, or possibly paginated.  Pressing on the ‘open document at first page’, the random image or any of the image thumbnails opens the page in question, a screenshot of which is shown below:

I’ve borrowed much of the layout of this page from the work I did on the Books and Borrowing project.  The page features a navigation bar at the top and bottom, through which you can navigate to the next or previous pages, or use the ‘jump to’ feature to select a specific page.  There are three views of the page: an image and text view which displays both side by side and then text or image only views.  When you select a view it is remembered as you use the navigation options.

As of yet there is no content in the text view (processing the XML will be my next task) so for now it’s all about the images.  You can zoom and pan the image or open it full screen.  You can also press on a line to highlight it.  Eventually this will allow highlighting of the text from the image and vice-versa, but for now all that happens is the section of the image is highlighted.  As mentioned in an earlier post, I adapted an existing ‘simplify coordinates’ script (https://github.com/dariok/page2tei/blob/master/simplify-coordinates.xsl) to generate simple rectangular sections for each Transkribus polygon, but we may need to have more complex shapes as in some pages (particularly the handwritten ones) the boxes are not very accurate.  I’ve also spotted an issue on touchscreens (on my Android phone using Firefox, at least) whereby the boxes for the lines stop the ‘pinch to zoom’ feature working, meaning zoom will only work using the icons in the top left.  I’ll need to investigate this further.

My next task will be to work with the XML, firstly replacing the Transkribus tags with valid TEI ones and then generating the text view for the website, which will link each line to the image.  Once this is in place I’ll begin to think about the search facilities.

On Friday I spent a bit more time working on Sara Ponz-Sanz’s AHRC application and spent the rest of the day beginning work on the redevelopment of the Mapping Metaphor website.  It took a while to refamiliarize myself with the code and to work out how things function, but once I made some progress with this I decided to start work on a new ‘combined’ version of the map.  This will display the combined E and OE data but will not initially allow you to switch the display to just E or OE.  There is a LOT of work that needs to be done to get this working fully, and I only made a start on things this week.  For now only the visualisation works, and only the top level view of this works.

Working out the totals for the visualisation is not as simple as just getting the E and OE data and adding them together, as this would duplicate data for metaphors that appear in both sets.  Instead, the new version of the visualisation grabs all of the E data and then only adds OE data for connections that are not already present.  So for example, in the new visualisation pressing on ‘1K’ displays  a connection to ‘3E’, which is not present in the E map but is present in the OE map.

In the combined map with strength set to ‘Strong’ and with ‘1K’ selected, pressing on the yellow line or circle for ‘1B’ the card shows 18 connections.  If you do the same for the E map you’ll also see 18 connections, while the OE map shows 3 connections.  The reason the combined map shows 18 rather than 21 connections is because the three OE connections already exist in the E data.  Setting the strength to ‘Both’ in the combined map shows 38 connections between these categories, whereas there are 35 in the E map and 10 in the OE map, and this is because there are three additional weak connections between categories in ‘1K’ and ‘1B’ in OE that are not found in E.  There is going to be a lot of figuring out how exactly to amalgamate the data across all views and all levels, which is going to be a pretty large undertaking.  But I feel that I’ve begun to figure it out, at least.