I divided most of my time this week between finishing work on the Scottish Life Writing CMS and continuing to develop the new website for the Dictionaries of the Scots Language.  I’d started work on the Scottish Life Writing CMS last week and I completed a first version this week.  It took a bit longer than I’d anticipated to develop it, mainly due to the issue of versions.  Each work may have any number of associated versions (e.g. a new edition, an adaptation for stage) and each of these versions may have any number of associated people (e.g. multiple editors, or a translator and an adaptor).  The data structure for this is pretty straightforward, but creating the front-end and processing for a form that allows this to be easily managed took some time.  Versions needed to be dynamically added to the form as required, and then the option of adding multiple people to each also had to be added in, with each person box working as an auto-complete to bring back a list of people that match the characters the user types in.  However, I managed to complete an initial version by Wednesday and then populated it with some sample data for test purposes.

For the DSL there was some follow-on stuff relating to the new search and filter options I’d developed last week to contend with.  Vasilis has suggested that when a region is selected in the advanced search form or as a filter on the search results page that any areas contained with the selected region should also be searched for – so for example if you select the ‘Highlands’ dialect area then this should automatically search all regions that are part of this area, such as Argyll.

I wasn’t aware that searching for one area should also include all other areas contained within it.  This was always going to be part of the map (and it is) so you can visualise area intersects and overlaps, but I didn’t realise this was also desired for the search, and I had some misgivings about this approach.  It is likely to generate more results than we allow for many searches.  For example, ‘Eastern Scots’ contains 102 areas, each of which may also contain multiple areas.  If all are included then the search isn’t going to be usable as the results limit will be reached.  It might be better to allow users to make their own selections based on the intersects on the map.  This would also avoid confusion.  For example, ‘Bute’ in the file of coverage includes ‘Arran’, so if we include everything then someone selecting ‘Bute’ would be unable to exclude the results for ‘Arran’, and may be confused as to why ‘Arran’ quotations are being returned when it’s a completely separate island and they didn’t select it.

Another issue is that the search indexes were not set up with this ‘contains areas’ requirement in mind.  Each quotation in the index only has its specific areas listed – they do not also include the other regions that would also match the quotation.  This would make results filtering by region rather tricky and the search queries very complicated.  E.g. for ‘Eastern Scots’ the front-end would have to formulate a query that includes all 102 areas (and any areas each of these include) before passing it to the Solr index.  This would be pretty awful to have to create the code for and would likely be slow.  The other alternative would be to generate this data when the Solr index is being created (e.g. so that when a quotation has the area ‘Mid Scots’ it should also have the area ‘Eastern Scots’).  That way a simple search for ‘Eastern Scots’ on its own would find all ‘Mid Scots’ quotations.  But the index has not been set up like this so such an approach is not currently possible.  After discussing this matter we agreed to not proceed with the ‘contains areas’ approach for now, which is a relief.

I was then able to acti on the other feedback I’d received to make updates to the work I’d done last week.  I fixed the issue with the ‘back to search results’ option when loading an entry from the search results not taking selected filters into consideration.  This should now work as intended.  On request, I set the map to open in a new tab, but I personally don’t like this this.  Different tabs are not very obvious on mobile phones and we’re forcing users to open the map in a new tab, whereas with a regular link they always have the option of opening it in a new tab if they want.  Also, if you’re consulting the map to then perform searches by selecting a region in the map your search then runs in the new tab, then if you press the ‘open map’ button again you get yet another new tab and you can very easily end up with lots of DSL tabs open without meaning to.  It’s all a bit messy.  I personally prefer to let users to choose to use new tabs themselves if they want.  I also moved the ‘open map’ button in the Advanced Search form, to hopefully make it clearer that it’s not the ‘Search’ button.

I was also asked to restrict the region filter to 1700-present only, but I didn’t proceed with this as my concern is that ‘Both’ is the default and I suspect lots of people won’t bother changing the coverage, meaning they may never even notice the region filter exists.  I was also asked to make the region filter be cleared in the Advanced Search when ‘Before 1700’ is selected, but I didn’t do this either, as it means if someone has spent time selecting multiple regions and then switches to ‘Before 1700’ and then swaps to ‘1700-present’ their selections are all lost.  Instead what I did was to ensure that if ‘Before 1700’ is selected and the search form is submitted no selected regions are passed on.

I updated the results page so that when only one result is found the results page now displays, rather than opening the entry directly.  I also updated the Region filter in the results so it now has ‘Regions’ as a header, and I fixed a bug displaying a double comma in the header for quotations searches.  I also added the text ‘Entry’ to the POS filter labels and ensured that if a POS filter is applied in the Advanced Search form, only those POS that were chosen appear in the results filter, even if other parts of speech are present in some of the entries in the results.  I also updated the map so that if a search term is present when you load the map, it now continues to be present if you select a region to search through the map interface.

With these updates in place I could then focus on going through my ‘To Do’ list and checking off other items on it.  A big change I implemented was to remove ‘Older Scots’ from the site menu and to rename ‘Modern Scots’ to ‘Home’.  I also changed the navigation menu from tabs to ‘pills’ as the tab design didn’t really work any more as entry pages (the bulk of the site) no longer have an active menu tab.  I also slightly reworked the footer to add in a ‘Make a donation’ button (which links through to the ‘Support us’ page) and I fixed the ‘Download results’ option to ensure the POS displays.  I also spotted that a full description of the selected search options (including filters) needed to be updated based on the new URL structures and filter options so have updated this too.

Finally, I made a number of tweaks to the search results, entry and bibliography pages.  If an advanced search for both dictionaries only gives results for one dictionary the other dictionary now appears with zero results in the Search Results tab on the entry page, and when one dictionary is searched, the link to search the other dictionary in the ‘Results’ tab now links to the homepage or the DOST page.  I have replaced ‘Modern Scots’ and ‘Older Scots’ with the new labels throughout the site and I’ve removed the ‘Show/hide etymology’ button the entry header.  Also, the dictionary names in the entry and bibliography headers are now links to the info about the respective dictionary.

I also spent a bit of time working on the HiMuJe Malabar project.  Renu had sent me some new, more detailed GeoJSON data for five areas around the Malabar coast.  One of these proved to be a little tricky, as the map is only set up to feature one polygon per location and it contained four separate polygons.  Thankfully I was able to use the QGIS package and some advice here: https://gis.stackexchange.com/questions/386382/merge-several-polygons-features-into-one-continuous-area-not-working to merge the separate polygons into one.  There were still a few random lines left over in the boundaries between the kingdoms, but I was then able to delete these by importing the updated geojson into https://geojson.io where I could easily remove them.  This then gave me one single polygon for the whole of Malabar.  I then replaced the existing polygons on the map with the new versions and set the maximum zoom level of the map to 10, rather than 14, which was its previous setting.  It’s definitely looking a lot better, as you can see in the following screenshot:

Also this week I responded to a further query from Sara Pons-Sanz about her AHRC application, I spoke to Faidon about the migration of the Iona Place-names site and I fixed an issue with the Historical Thesaurus search facilities that someone had contacted us about.  I also had my PDR session on Tuesday and this went very well.