I’d taken Monday off this week and on Tuesday I continued to work on the Speak For Yerself follow-on projects.  Last week I started working with the data and discovered that it wasn’t stored in a particularly consistent manner.  I overhauled the ‘lexis’ data and this week I performed a similar task for the ‘morphology’ and ‘phonology’ data.  I also engaged in email conversations with Jennifer and Mary about the data and how it will eventually be accessed by researchers, in addition to the general public.

I then moved on to looking at the GeoJSON data that will be used to ascertain where a user is located and which area the marker representing their answer should be randomly positioned in.  Wales was missing its area data, but thankfully Mary was able to track it down.

For Speak For Yersel we had three levels of locations:

Location:  The individual places that people can select when they register (e.g. ‘Hillhead, Glasgow’).

Area: The wider area that the location is found in.  We store GeoJSON coordinates for these areas and they are then used as the boundaries for placing a random marker to represent the answer of a person who selected a specific location in the area when they registered.  So for example we have a GeoJSON shape for ‘Glasgow Kelvin’ that Hillhead is located in.  Note that these shapes are never displayed on any maps.

Region: the broader geographical region that the area is located in.  These are the areas that appear on the maps (e.g. ‘Glasgow’ or ‘Fife’) and they are stored as GeoJSON files.

For the new areas we didn’t have the ‘region’ data.  I therefore did some experimenting with the QGIS package and I found a way of merging areas to form regions, as the following screenshot demonstrates:

I was therefore able to create the necessary region shapes myself using the following method:

1. I opened the GeoJSON file in QGIS via the file browser and added the OpenStreetMap XYZ layer in ‘XYZ Tiles’, ensuring this was the bottom layer in the layer browser
2. In the layer styling right-hand panel selected the ‘ABC’ labels icon and chose ‘County’ as the value, meaning the county names are displayed on the map
3. In the top row of icons selected the ‘Select Features by area or single click’ icon (the 23rd icon along in my version of QGIS)
4. I could then do ‘Ctrl+click’ to select multiple areas
5. Then I selected the ‘Vector’ menu item and selected ‘Geoprocessing’ then ‘Dissolve’
6. In the dialog box I had to press the green ‘reload’ icon to make the ‘Selected features only’ checkbox clickable then I clicked it
7. Then I pressed ‘Run’ which created a new, merged shape.
8. The layer then needed to be saved using the layer browser in the left panel.
9. This gave me separate GeoJSON files for each region but I was then able to merge them into one file by opening the ‘Toolbox’ by pressing on the cog icon in the top menu bar, searching for ‘merge’ then open ‘Vector general; -> Merge Vector layers, selecting the input layers and ensuring the destination CRS is WGS84, then entering a filename and running the script to merge all layers.

I was then able to edit / create / delete attributes for each region area by pressing on the ‘open attribute table’ icon in the top menu bar.  It’s been a good opportunity to learn more about QGIS and next week I’ll begin updating the code to importing the data and setting up the systems.

Also this week I created an entry for the Books and Borrowing project on this site (see https://digital-humanities.glasgow.ac.uk/project/?id=160).  On Friday afternoon I also investigated a couple of issues with the search that Matt Sangster had spotted.  He noticed that an author surname search for ‘Byron’ wasn’t finding Lord Byron, and entering ‘Lord Byron’ into the surname search was bringing back lots of results that didn’t have this text in the author surname.

It turned out that Byron hadn’t been entered into the system correctly and was in as forename ‘George’, surname ‘Gordon’ with ‘Lord Byron’ as ‘othername’.  I’ll need to regenerate the data once this error has been fixed.  But the second issue, whereby an author surname search for ‘Lord Byron’ was returning lots of records is a strange one.  This would appear to be an issue with searches for multiple words and unfortunately it’s something that will need a major reworking.  I hadn’t noticed previously, but if you search for multiple words without surrounding them by quotes Solr searches the first word against the field and the remaining words against all fields.  So “surname ‘Lord’ OR any field ‘Byron’”.  Whereas what the query should be doing is “surname ‘Lord’ AND surname ‘Byron’”.  This is something that will probably affect all free-text fields.  I’m going to have to update the search to ensure multi-word searches without quotes are processed correctly, which will take some time and I’ll try to tackle next week.  I also need to create a ‘copy’ field for place of publication as this is being tokenised in the search facet options.  So much for thinking my work on this project was at an end!

Also this week I spent many hours going through the Iona map site to compile a spreadsheet listing all of the text that appears in English in order to make the site multilingual.  There is a Gaelic column in the spreadsheet and the plan is that someone will supply the appropriate forms.  There are 157 separate bits of text, with some being individual words and others being somewhat longer.  By far the longest is the content of the copyright and attribution popup, although we might also want to change this as it references the API which might not be made public.  We might also want to change some of the other English text, such as the ‘grid ref’ tooltip that gives as an example a grid reference that isn’t relevant to Iona.  I’ll hold off on developing the multilingual interface until I’m sure the team definitely want to proceed with this.

Finally this week I continued to migrate some of the poems from the Anthology of 16th and Early 17th Century Scots Poetry to TEI XML.  It’s going to take a long time to get through all of them, but progress is being made.