Investigations of a Dog

In a previous post I mentioned experimenting with taking photos at the Public Records Office/National Archives. Getting good photos is only part of the problem. The real work starts when you get them home. How do you organise them and make sense of them? It should be no surprise that Zotero is really useful for this, but I’ve discovered a few tricks to make it even better.

One of the main challenges of the Commonwealth Exhequer Papers (SP 28), is that although the class contains a huge amount of useful information about the British Civil Wars it isn’t very well sorted or catalogued. It includes lots of different kinds of documents created by different organisations at different times for different purposes. These documents range from warrants and bills on a single sheet of paper to account books with over 100 pages. Having photos of every page is a huge advantage over having barely legible handwritten notes. It’s almost like having the original document at home on your desk, but using Zotero we can make it better than the original document without losing anything. I can create a Zotero item corresponding to the document (whether it’s a pay warrant, an account book, or anything else) and attach the page images to it. Then I can add all kinds of metadata using the item fields, notes, and tags, and rearrange the items however I want, while still having instant access to images of the original, and still being able to get them back into their original order. That’s the theory. Today I’ve been trying it to see how it might work in practice, using some test shots of pay warrants that I took last year.

First I had to work out what kind of document I was looking at, then create a new item for it, then attach the correct images. I’ve found that the best way to do this is to use another Firefox plugin called MozImage. Sadly it isn’t a picture of Morrissey, but it’s almost as good: it lets you browse image files on your local hard drive through Firefox. [Edit: I've just noticed that it can also browse directories on a web server, which could be useful for collaboration] You can easily click forwards or backwards through a folder of photos taken in the archives to see how many pages there are in a document. Viewing it full size in the browser window, with the option to zoom in and out, makes it easy to identify what kind of document it is. Once you’ve created the item you can attach a snapshot or link to each image just as you would with a web page.

It would be nice if Zotero had some similar way of letting you scroll through item attachments. Being able to click forward and back buttons to view the next/previous attached file would be easier than having to click on each one.

Creating an item for each document can get quite tedious and repetitive, especially as there might be up to 1,000 pay warrants in each box of SP 28, each one of which needs to be catalogued as a separate document. Fortunately I thought of a quick and easy hack which lets you create an item from a template without having to do any actual programming:

1. Create a new item in Zotero and fill in the bits that you need to apply to every document of that type. For example I might set the Author as “Earl of Essex”, Title as “Pay Warrant”, Repository as “TNA: PRO”, and Loc In Archive as “SP28″.
2. Select the item and export it as Zotero RDF.
3. Open the RDF file in Firefox.
4. The magic icon should appear in the address bar. If you click on it you get a new item with the fields pre-filled with the data you set up.
5. You can also take a snapshot of the RDF file in Zotero so that you can easily find it and open it whenever you need it.

It would be even better if I could export several items to the same RDF file and choose one from the drop down menu on the folder icon, but it doesn’t seem to work like that – I just get a single icon and clicking it imports every item in the file. [Edit: selecting from multiple items isn't really necessary as I've thought of a better way to do it now]

The basic item fields are quite useful for adding metadata and sorting documents – especially author, date, repository, and location – but there’s a lot of data I need which won’t fit. This is where tags come in. I can use these to sort documents by type (pay warrants, quartering bills, accounts etc), or by content. For example I could pull out all warrants relating to cavalry troops, all warrants for purchase of horses etc. The number of officers and units is finite, so it might be feasible to have a tag for each unit or its commander (although I’d have to standardize the spelling for this to work properly).

There’s some data which tags can’t deal with. For example I need to extract and add up amounts of money. Once custom fields have been implemented (and this is a planned feature of Zotero, although I don’t know when it’s coming) this should be easy. Until then I’m wondering if I could use notes with the data marked up by XML tags. Using the template trick it would be easy to create an item with a note with the XML tags already in it. Then I could probably write a Python script to pull out the data and do something with it.

I’m also thinking about how to export data to the Your Archives wiki. It should be possible to use a script, or maybe just a citation style, to generate document references and summaries in wiki format. But this is all speculation. I’ll have to do some more thinking and experimenting. For now I’m just really pleased that Zotero exists and is free.