Week Beginning 23rd August 2021

This week I completed work on a first version of the textbase search facilities for the Anglo-Norman Dictionary.  I’ve been working on this over the past three weeks and it’s now fully operational, quick to use and does everything that was required of it.  I completed work on the KWIC ordering facilities, adding in a drop-down list that enables the user to order the results either by the term or any word to the left or right of the term.  When results are ordered by a word to the left or right of the search term that word is given a yellow highlight so you can easily get your eye on the word that each result is being ordered by.  I ran into a few difficulties with the ordering, for example accented initial characters were being sorted after ‘z’, and upper case characters were all sorted before lower case characters, but I’ve fixed these issues.  I also updated the textbase page so that when you load a text from the results a link back to the search results appears at the top of the page.  You can of course just use the ‘back’ button to return to the search results. Also, all occurrences of the search term throughout the text are highlighted in yellow.  There are possibly some further enhancements that could be made here (e.g. we could have a box that hovers on the screen like the ‘Top’ button that contains a summary of your search and a link back to the results, or options to load the next or previous result) but I’ll leave things as they are for now as what’s there might be good enough.  I also fixed some bugs that were cropping up, such as an exact search term not appearing in the search box when you return to refine your results (caused by double quotes needing to be changed to the code ‘%22’).

I then began thinking about the development of a proximity search for the textbase.  As with the old site, this will allow the user to enter two search terms and specify the maximum number of words before or after the first term the second one appears.  The results will then be displayed in a KWIC form with both terms highlighted.  It took quite some time to think through the various possibilities for this feature.  The simplest option from a technical point of view would be to process the first term as with the regular search, retrieve the KWIC for each result and then search this for the second term.  However, this wouldn’t allow the user to search for an exact match for the second term, or use wildcards, as the KWIC only contains the full text as written, complete with punctuation.  Instead I decided to make the proximity search as similar to and as consistent with the regular textbase search as possible.  This means the user will be able to enter the two terms with wildcards and two lists of possible exact matches will be displayed, from which the user can select term 1 and term 2.  Then at this point the exact matches for term 1 will be returned and in each case a search will be performed to see whether term 2 is found however number of words specified before or after term 1.  This will rely on the ‘word order’ column that I already added to the database, but will involve some complications when term 1 is near the very start or end of a page (as the search will then need to look at the preceding or following page).  I ran a few tests of this process directly via the database and it seemed to work ok, but I’ll just need to see whether there are any speed issues when running such queries on potentially thousands of results.

With this possible method in place I began working on a new version of the textbase search page that will provide both the regular concordance search and the new proximity search.  As with the advanced search on the AND website, these will be presented on one page in separate tabs, and this required much reworking of the existing page and processing scripts.  I had to ensure that HTML elements that previously used IDs but would need to be replicated in each tab and would therefore no longer be unique would still be valid.  This meant some major reworking of the genre and book selection options, both in the HTML and in the JavaScript that handles the selection and deselection.  I also had to ensure that the session variables relating to the search could handle multiple types of search and that links would return the user to the correct type of search.  By the end of the week I had got a search form for the proximity search in place, with facilities to limit the search to specific texts or genres and options to enter two terms, the maximum number of words between the terms and whether term 1 should appear before or after term 2 (or either).  Next week I’ll need to update the API to provide the endpoint to actually run such a search.

Also this week I had an email from Bryony Randall about her upcoming exhibition for her New Modernist Editing project.  The exhibition will feature a live website (https://www.blueandgreenproject.com/) running on a tablet in the venue and Bryony was worried that the wifi at the venue wouldn’t be up to scratch.  She asked whether I could create a version of the site that would run locally without an internet connection, and I spent some time working on this.

Looking at the source of the website it would appear to have been constructed using the online website creation platform https://www.wix.com/.  I’d never used this before, but it will have an admin interface where you can create and manage pages and such things.  The resulting website is integrated with the online Wix platform and (after a bit of Googling) it looked like there isn’t a straightforward way to export pages created using Wix for use elsewhere.  However, the site only consisted of 20 or so static pages (i.e. no interactive elements other than links to other pages) so I thought it would be possible to just save each page as HTML, go through each of the files and update the links and then the resulting pages could potentially run directly in a browser.  However, after trying this would I realised that there were some issues.  Looking at the source there are numerous references to externally hosted scripts and files, such as JavaScript files, fonts and images that were not downloaded when the webpage was saved and these would all be inaccessible if the internet connection was lost, which would likely result in a broken website.  I also realised that the HTML generated by WIX is a pretty horrible tangled mess, and getting this to work nicely would take a lot of work.  I therefore decided to just create a replica of the site from scratch using Bootstrap.

However, it was only after this point that I was informed that the local site would need to run on a tablet rather than a full PC.  The tablet is an Android one, which seriously complicates matters as unlike a proper computer, Android imposes restrictions on what you can and can’t do, and one of the things you can’t do is run locally hosted websites in the browser.  I tried several approaches to get my test site working on my Android phone and with all of the straightforward ways I can get the HTML file to load into the browser, but not any associated files – no images, stylesheets or JavaScript.  This is obviously not acceptable.  I did manage to get it to work, but only by using an app that runs a server on the device and by using absolute file references to the IP address the server app uses in the files (relative file references just did not work).  The app I used was called Simple HTTP Server (https://play.google.com/store/apps/details?id=jp.ubi.common.http.server) and once configured it worked pretty well.

I continued to work on my replica of the site, getting all of the content transferred over.  This took longer than I anticipated, as some of the pages are quite complicated (artworks including poetry, images, text and audio) but I managed to get everything done before the end of the week.  In the end it turned out that the wifi at the venue was absolutely fine so my replica site wasn’t needed, but it was still a good opportunity to learn about hosting a site on an Android device and to hone my Bootstrap skills.

Also this week I helped Katie Halsey of the Books and Borrowing project with a query about access to images, had a look through the final version of Kirsteen McCue’s AHRC proposal and spoke to Eleanor Lawson about creating some mockups of the interface to the STAR project websites, which I will start on next week.