This week began with Chris informing me that someone had managed to upload a rogue script to the Burns server that had been sending out spam messages. Not the best of starts to the week! I helped Chris clean the server on Monday morning and we managed to strengthen the security somewhat. It was also interesting to view the rogue script itself – a small but deadly script that enabled the hacker to run any code they wanted on the server. Hopefully that’s the hole successfully plugged now though.
I managed to spend a bit of time this week working on the redevelopment of the DSL site, something I’ve not had as much time to work on as I would have hoped recently. I managed to tick off quite a few items on my ‘to do’ list including providing a ‘fixed’ entry heading so that when a user scrolls down the entry page the heading remains visible. I also added the ‘About this entry’ box to the entry page, which will contain information about when the entry was originally published. I then implemented the facilities to show / hide quotations and the etymology, using HTML5 local storage to record the user’s choice so that this is remembered whenever they go to the ‘entry’ page. The ‘Cite this entry’ option was also added – a popup the will work in a similar manner to the HT ‘cite’ options (I still need to know what citation options need to go in this box, though). I also added in a ‘share’ box, including options to email, tweet, link to facebook etc for each entry and added a ‘browse’ box that will eventually feature the dictionary browse facility. I also changed the site font to a sans-serif one purely to try this approach out as I was getting a bit fed up with the serif font, and I updated the layout of the quotations to make them appear in a smaller font with less padding to cut down on the amount of vertical space that was being used.
There was another DROG meeting this week, the first one this year. It was another good opportunity to let people know about the projects and bids I’ve been involved with and also to hear what other developments have been taking place across the School and beyond.
The majority of the rest of the week was spent preparing for and attending the Mapping Metaphor colloquium, which took place on the Friday and also the Saturday morning. I wrote my presentation and prepared the walkthrough of the website, and also made some last minute tweaks to the website, for example adding in facilities to view a category’s keywords from the search results page and also the category page itself. I attended the full day of the colloquium on the Friday, which was very interesting. The Saturday morning was when the focus was on the visualisations and the website, though. Thankfully this all went very smoothly. The presentation and walkthrough both went well, there were no issues with the server and the 20 or so people who attended for the ‘hands on’ session managed to access the website and visualisations without any major issues. The participants filled in some feedback forms (which I have not looked at yet) and the overall impression I got was that people were pretty pleased with how things were developing.
Another mostly Metaphor week this week, with a sprinkling of other projects added in too. For Metaphor I completed work on the version of the website that we will be using at the colloquium next week. This included an update of the underlying data that Flora helpfully prepared for me. This new data upped the metaphor connections from around 9,000 to over 19,000, but thankfully the visualisations and the other data views available through the website seem to be scaling up nicely with the increase in data. A lot of time was spent this week fixing some of the bugs that had crept into the visualisations and other parts of the website and I’d noticed over the weeks. These included the ‘metaphor strength’ selector failing to work with some of the visualisations and the metaphor cards sometimes not displaying the correct information. I also finally managed to get the ‘show categories with connections to both x and y’ visualisation working properly, which proved to be surprisingly complicated compared to the ‘show categories with connections to either x or y’ view. I also tweaked the interface to the website in a number of areas, such as ensuring the ‘close’ button in the metaphor card pop-up stays visible even if the user has to scroll down the card to view all of its content.
The biggest task I tackled this week was to overhaul the search facilities. Previously the search brought back a list of metaphor connections, but after discussions with Marc and Ellen it turned out what was really wanted was something closer to the way the HT search works – i.e. you perform a search and this returns a list of matching categories. You then select one of these to view the metaphor connections within it. So I set up a new ‘category’ page that listed the data for the category within three tabs (tabular, visual and card) as used to be found on the search results page. I then realised that I should probably update the browse facilities as these were showing the tabular view of a category within the browse structure. Far better (I decided) to instead link to the new category page I had developed and therefore give users the option of not only viewing the category as a table but also the visualisation and the card view. I then rewrote the search page so that it would return a list of matching categories. It isn’t that simple, however. Matching categories are indeed returned (much like the HT’s ‘recommended categories’) but the search script also now searches each category’s keywords too, listing all categories where the entered search term matches a keyword. These are now presented below the ‘matching categories’ section. Finally, I added in another search option, this time to perform a word search on the HT data, returning any categories that feature one or more words that match the entered search term. I decided to include this as a separate AJAX search that the user must choose to initiate as the search itself can be quite slow and in many cases wouldn’t really be necessary. It all seems to be working rather well across all major browsers. I enhanced the functionality of the category pages in a few other ways too, such as allowing the user to order and reverse the order of the tabular view of the data by clicking on a table heading. I also started to think about what I would say at the Saturday session of the colloquium next week and attended a meeting with Marc and Ellen to discuss the morning session.
Also this week I spent a little bit of time on the DSL, although this was mostly spent in email conversations rather than any development tasks. I also produced a big long ‘to do’ list of outstanding items for the front-end and I will begin working through this next Monday. The other projects I was involved with this week were Burns, for which I provided advice to the Choral Burns project and helped with the upload and integration of new songs for the main Burns C21 website. I also made a couple of minor tweaks to the cogtop website and attended a meeting for the SAMUELS project with Scott Piao from Lancaster. He demonstrated the current version of the tagger, which is shaping up very nicely.
A very hectic week this week, with so much outstanding work to do for Mapping Metaphor ahead of the colloquium at the end of the month. This was made slightly more difficult by my son coming down with chicken pox this week, meaning he was barred from nursery until his spots cleared up, which in turn meant I had to attempt to work from home whilst looking after him for two and a half days. It actually turned out ok, thankfully, due to a combination of odd working hours (e.g. 6:30-8:30 while the boy slept, 19:30-21:30 when he was back in bed) and the mesmerising effects of Pixar films which kept him occupied for a few hours during the days.
Anyway, as previously mentioned, a lot of this week was spent on Mapping Metaphor duties. I left for my holiday last week having dropped the bombshell that we had too much data for the visualisation options that we had decided upon. However, on Monday I worked hard at addressing this problem by implementing a solution I had come up with in the meantime, specifically creating a hybrid of our top-level and drill-down diagrams that show just one level 2 category open while aggregating the metaphorical connections to the remaining categories. It took a while to get this approach working, but having done so it actually works pretty well. You can view the MMCats in a level 2 category, see which other level 2 categories each on connects to then open one of these categories to view the MMCats within it and all of the connections to the MMcats of the first category you’re looking at. It still needs some tweaks but on the whole I think the approach is acceptable.
On Tuesday I focussed on implementing the metaphor card view of the search results. It was agreed at the meeting last week that I should create different version of the card based on the PDF mock-ups that Marc had created and display these through the metaphor card view of the search results. This would then allow people at the colloquium (and project staff) to decide which cards (or which elements of cards) they like best. It took the best part of the day to get the layout and CSS figured out for 6 different card styles, and these are now available through the search results page.
My final Mapping Metaphor task this week was to develop the browse facilities for the metaphor connections. This is an alternative to the visualisations for people who can’t necessarily get them to work on their computer (e.g. if their browser somehow doesn’t support SVGs). It took about a day to develop the browse, which works as a sort of accordion style (although for now all the data is managed server side on page load rather than using any jQuery and AJAX. I wanted to avoid anything too fancy as the browse is supposed to be a ‘low-tech’ solution. How it works is you can click on a top level category (external / mental / social world) to view the level 2 categories contained within. You can then click on one of these to expand a section containing the MMCats and from here you can click on an MMCat to view the metaphorical connections in a tabular format as with the search results. You can also close sections by clicking on the headers again. It works pretty well and doesn’t swamp the user with too much information. And there are links to the visualisations from the metaphor connection table too.
Other than Metaphor stuff I also spent a fair amount of time on DSL development, addressing the issues raised by Ann and other team members in a feedback document that was sent around last week. I also replaced the static sample XML entries for DOST and SND with the real thing, which Peter had made available through the API. So now it’s possible to view the full DSL entries through the new website, which is a real step forward. I had to tweak the CSS and the XSLT a little to work with the real data, but after doing so the entries I’ve looked at all seem to be laid out nicely. I also implemented wildcard searches and had a couple of email discussions with Ann and Peter about the development of the front end and the API. I think the new website is coming along nicely.
Also this week I helped Fraser out with some queries regarding the HT for the SAMUELS project. I also gave some advice on digitisation and OCR to Bryony Randall in English Literature and made some further modifications to the Burns Choral website for Vivien. I also provided some responses to the technical review of the Pennant bid that came back this week.
A short week this week as I took Thursday and Friday off as a holiday. Most of my three days of work was spent on the Mapping Metaphor website, working on the visualisations and the search. I attended a project meeting on Monday to talk about some questions that had built up regarding the website, and also to discuss plans for day two of the colloquium, which is fast approaching. On day two I will give a 15 minute talk about the development of the visualisations and a walkthrough of the website and then we will ask people to try out the website and hopefully give us some constructive feedback.
Returning to the development of the visualisations and the website, I finally got round to updating the algorithm that pulls metaphor connections out of the database this week. Previously it was pulling out connections based on the cat1 column only – e.g. show all category B connections would only return any connections with cat1 starting with B. Now the algorithm is looking at both cat1 and cat2 – e.g. showing all category B connections now finds connections with a B in cat2 in addition to those with a B in cat1. This has greatly increased the amount of data that gets returned and I’m afraid in many cases the visualisations were getting completely swamped. I only uncovered this towards the end of Wednesday, but my two days off gave me time to consider some possible solutions to the problem.
Also for the Metaphor project this week I made some arrangements regarding the DH2014 conference in Lausanne. It’s looking like I will be attending this event, which is really good news. There has been a lot of toing and froing with Wendy, Marc and Fraser about the arrangements but by the end of the week the big decisions had been made (e.g. which project is paying for what, travel arrangements, hotels).
Other than Mapping Metaphor stuff, this week I did a bit more tweaking of the CogTop website, some further work on the technical plan for George Pattison’s project and a little more work on the Choral Burns website.
Another week, another technical plan. I spent some time this week writing a technical plan for a bid George Pattison in Theology is putting together. The technical component of his project is rather less complicated than for Jennifer and Carole’s projects, so it didn’t take as long to put together. I did still need to spend some time reading through the project material, meeting with George, writing the plan and firing off a few emails full of questions. I also spent a little time this week following up on a few issues regarding Jennifer’s technical plan.
I met with Vivien to discuss the Burns Choral project and spent some time reworking certain aspects of the website for the project. The technical side of this project is almost complete now and the project website should be launching shortly.
The rest of the week was split between the two big projects I’m currently involved with – the redevelopment of the Dictionary of the Scots Language and Mapping Metaphor. For the DSL I added some further functionality to the website. I added in the predictive search option to the quick search boxes, meaning that as a user starts to type in a word the website connects to the API and brings back headwords that match what the user has typed in. If the user then selects one of these options the quick search is fired off immediately. It works quite nicely and is pretty speedy, even when connecting to Peter’s test server in Edinburgh. The other major this I tackled for the DSL this week was to create the XSLT files that will transform the DOST and SND entry XML into HTML. Ann had previously sent some explanatory documents about the structure of the files and how aspects should be represented on screen, and also some sample XML files too, which was very helpful. I used PHP’s XML library in order to dynamically pull in an XML file, an XSLT file, perform the transformations and then output the resulting HTML. It works very well with the test files and once the API has been updated to include the actual entry XML it should be straightforward to plug these into the PHP script.
For Mapping Metaphor I began to get the actual website in some kind of proper order, rather than there being just a bunch of test files. The website now has placeholder pages wherever a real page should go, a proper layout script that provides the layout for all pages and proper dynamically generated menus, enabling pages to be added and removed very easily. I also started working on the search facilities for the website, getting a very simple quick search facility that displays a table of metaphor connections that feature metaphors with names (or numbers) that match the supplied text. I also updated the underlying data, using the new sample data supplied by Flora. The visualisations are now using 9292 rows (7151 coded as ‘metaphor’) rather than 5119 (4031 coded as ‘metaphor’), and they seem to be handling this increase quite nicely.