Investigations of a Dog

As part of the research for my book (saying that still feels a bit weird, but I’m sure I’ll get used to it) I’m going through indemnity cases in class SP 24 in the UK National Archives (aka the PRO). The Indemnity Committee was set up by parliament in 1647 to protect soldiers and officials from prosecution for actions that they had carried out under the authority of parliament, such as requisitioning things for the army or arresting royalists. It also dealt with disputes over sequestered rents and debts, and helped to enforce parliament’s order that apprentices who joined the army should be allowed to count military service towards their term of apprenticeship. If someone was prosecuted in court for acts which were covered by the Indemnity Ordinance (and many were despite the Ordinance banning people from bringing cases of this kind) the defendant could send a petition to the Indemnity Committee asking for protection. In SP 24 there are 58 boxes of petitions and other papers relating to cases, such as depositions and lists of expenses. Unlike some classes these are quite well sorted: papers relating to each case are grouped together and sorted in roughly alphabetical order of the plaintiff’s name (although confusingly the plaintiff in an indemnity case is the defendant in the corresponding criminal prosecution). I’m particularly interested in cases relating to horse requisitioning. According to Ian Gentles, about 30% of the military cases involve horses, although from what I’ve seen so far military cases seem to be a minority as many cases are disputes between civilians over payment of rents and debts due to sequestered estates. It usually takes me less than an hour to skim through a box, look at the first petition in each case to see if it’s about horses, and photograph the relevant cases. Sometimes I get cases that look interesting for other reasons, but I try not to wander too far off topic too often. Since I’m photographing these papers for my research, and since the National Archives allow document images to be uploaded to Flickr, that’s just what I’m doing. I’m also putting transcripts or summaries of the documents, along with links to the images, on the Your Archives wiki. You can see what I’ve done so far, and follow my progress in future, via a Flickr collection and Your Archives category.

So far I’ve uploaded cases from the first 2 boxes. I have another 16 boxes ready to be uploaded, but I’m working on some Python scripts to automate the process. The trial run on the first two boxes proved that doing it all manually is quite labour intensive. First I copied the image files from my camera and sorted them into directories for each box. The directory structure is based on the archival reference, so there’s a directory called “SP 24” with sub-directories called “30”, “31” etc. Then I went into each of these directories and made sub-directories for each case, so it looks like this:

• SP 24
  • 30
    • 1 Abeary vs Windebanke
    • 1 Adams vs Haughton
    • 2 Alford vs King
    • etc
  • 31

And the path to a particular case would be:

SP 24/30/2 Alford vs King

Which looks quite similar to the archival reference.

The numbers at the start of the case name are the part number (each box usually contains three folders called part 1, part 2 and part 3 but I decided not to make directories for these). Up to here it has to be done manually as arranging cases into directories involves looking at the documents to see where a new case begins and to check the names. But from here a lot of it can be automated.

Each directory containing one case needs to have its own photoset on Flickr. I used Postr to upload one case at a time and then used Desktop Flickr Organizer to create a set and add photos to it (I got both of these applications from the Ubuntu repository – if you’re on Windows then… stop using Windows!). Then I used the Organizr on the Flickr website to drag each set into the “SP 24 Indemnity Cases” collection. Once the Flickr photos and sets were in place I went to the web page for each set, manually created a Zotero item for the case, and attached a link to the page. Finally I created a Your Archives page for each case and attached a link to it in Zotero. This includes a template that I made for indemnity cases which gives some basic information in a standardized form and includes a link to the relevant Flickr set. Doing all this manually for each case is quite tedious and takes a long time, so I’m working on some Python scripts to automate the process. What I want the scripts to do is:

1. Upload photos from multiple directories
2. Create a separate photoset for each directory, with a name based on the directory name and path
3. Get the ID of each set and write the IDs and names to a CSV file
4. (At this point I’ll manually edit the CSV file to add data that will be needed for Your Archives and Zotero and which can only be got by looking at the document images, eg full names of plaintiffs and defendants, date of the petition, summary of the case, categories/tags)
5. Use the data from the CSV file to construct a wiki page with the correct template and upload to Your Archives through the MediaWiki API
6. Export an XML file which can be imported into Zotero

So far I’ve written a Flickr upload script which does the first three steps and more or less works. Rather than working directly with the Flickr API I’m using the Python Flickr API library, which makes things very easy. It provides a flickr class with methods to handle API calls and authentication. Before using it you have to go to the App Garden and request an API key, but that doesn’t take long to do. App pages can be kept private, which is what I’m doing in this case as I don’t really have the time or skills to make my scripts fit for public consumption. The next step is to add error handling as the script only works as long as nothing goes wrong. In the real world, there are lots of things that could go wrong. The library throws an exception if it gets an error response from the API. Until I add some exception handling this means that the script just stops on an error. The script will need to keep track of what has and hasn’t been done (photos uploaded, sets created, photos added to sets) so that I can run it again if anything was left undone, and so that it doesn’t try to do the same thing again if it’s already been done. One annoying thing about Flickr’s public API is that it provides no way to create a collection or add sets to a collection. I assumed I’d be able to automate that part of the process but it looks like I’ll still have to do it manually.

For step 5 I’ll be using the Pywikipediabot library. I’ve already done some simple tests on a local MediaWiki installation and it seems quite easy to create a page. Once I’ve finished the script and thoroughly tested it I can ask for a bot account on Your Archives. Step 6 will involve learning a bit more about Zotero RDF. The easiest way to find out how to generate the right code is to export some similar existing items and look at the results.

So just because I’m writing a monograph it doesn’t mean I’ve abandoned digital history. I’ll still be using lots of digital tricks in the background, but they won’t necessarily be obvious in the text of the book. New technology is certainly making my research quicker and cheaper than it used to be. The stuff that I’ve written about above isn’t exactly revolutionary: it saves labour but it doesn’t offer new insights that couldn’t have been found before. But later in the project I’m planning to do some text mining which I hope will show me things that I couldn’t otherwise have found. I’ll also be revisiting phonetic algorithms for place name identification. And if I can’t think of anything else to blog about, there are likely to be some interesting stories in the indemnity cases.

This is a selection of First World War photos from my collection, mostly bought from ebay. I’ve posted some horse photos over at The horse in history and culture. The ones here have more of a gender theme. Click on the thumbnails to see bigger versions.

Four male prisoners of war, two in drag. This was taken in the theatre at Cottbus PoW camp, where my great-grandad was held from 1917 to 1918. He performed in the theatre but there’s no evidence that he dressed as a woman. One of the paradoxes of the hyper-masculine environment of the 20th century British Army was that it often forced men into stereotypically feminine roles in order to stand in for the women who were excluded.

Royal Army Medical Corps group, taken in France, 1919. It clearly shows how uniforms reinforced gender roles. The men are wearing army service dress, just like combat soldiers, although their role is to provide medical care. The women are wearing long skirts and big head-dresses. Also notice that some of the men are very short. The man on the left of the middle row, standing between the corporal and the nurse with a dog at their feet, looks shorter than some of the women. If you look very closely you can see that some of the group are holding puppies.

A man and woman called Fred and Kitty, but I don’t know their surnames. Fred is a sergeant in the Army Service Corps, and Kitty is in civilian clothes. The poses reinforce the differences in dress, suggesting male dominance and female submission.

Territorial Royal Field Artillery corporal with a small boy. Probably taken in Cardiff or Pontypridd. Like the Sergeant in the previous photo, the corporal is wearing spurs. These were standard equipment for troops classed as mounted, which included field artillery and service corps because they relied on horses for transport. I love the little boy’s pose. Although man and boy are both male, they illustrate the hierarchy of masculinity: the corporal is more of a man because of his age, independence and military service.

A group of female munitions workers. The unprecedented expansion of both the British Army and the arms industry in the First World War, along with the assumption that women couldn’t or shouldn’t fight, led to more women working in munitions factories. This temporarily gave some women increased pay and freedom, but 90 years on women as a group still earn less than men as a group. Although the uniforms make some concessions to the practicalities of working in a factory, they also signify femininity.

There has been some bad news for historians recently: the RHS Bibliography of British and Irish History has lost its direct government funding and is being privatised in a move disturbingly reminiscent of PFI (and to add insult to injury the IHR claims to be “delighted” about this!); the UK National Archives (or PRO to most of us who use it) can no longer afford to open on Mondays or offer free parking.

But it’s not all bad. There’s also some good news from the National Archives which has got much less attention than the bad news – in fact I’m not even sure exactly when it happened. They are now allowing and encouraging users to upload photos of public records held at Kew to Flickr and similar photo sharing sites. Crown Copyright had already been waived to allow republication of the text of public records but previously publishing images of documents didn’t appear to be allowed. Now it’s confirmed that uploading images to Flickr is allowed (provided that you’ve taken them yourself – this doesn’t cover documents bought from DocumentsOnline or Ancestry). This is a win situation for everyone, because these documents will be made freely available without it costing the archives anything – a major advantage when budgets and funding are being cut drastically.

The NA has its own Flickr account, and a group for visitors. Combined with the Your Archives wiki this could lead to some really exciting stuff. Some people are already using Flickr and Your Archives to publish Metropolitan Police leavers’ registers. The possibilities are endless. I’m certainly going to upload all the photos I take in the course of my research. To start with I’ve put up the service record of my ancestor Tom Wenham from the First World War (photographed from the screen of a microfilm reader).

Still to come are some indemnity cases from SP24, and sooner or later I’ll have loads of SP28 to share. It would be fantastic if other archives would do this too, although some will probably be too conservative to try it. The British Library still doesn’t allow digital cameras, which just makes me not want to bother with BL manuscripts.

I’m finally back to blogging after a few months off. This post is a quick roundup of some things that happened while I was away.

First of all, the Military History Carnival always needs more hosts. Although it’s not my responsibility any more I’d like to see it continue, so if you can help please contact TJ at tj$linzy$@$gmail$.$com (remove the dollar signs)..

I’ve deactivated my Facebook account as I was finding the whole thing too annoying. I’ve opened an account at LinkedIn but I’m not sure if I’m going to use it. It seems very much aimed at The Businessman In His Suit And Tie. Does anyone use it, and is it any use? I’d like to see a social networking site specifically designed for academics. Maybe with facilities to self-archive our publications.

The Great War Archive (which I posted about here) opened on time in November, and looks really good. They’re also continuing to collect submissions through a Flickr group. Reviews in History have published a review by Esther MacCallum-Stewart and a response by Stuart Lee. This project gives us proof that there are lots of interesting documents in private hands and that at least some people are willing to share them on the web if given the chance. Although 6,500 documents sounds like a lot I suspect it’s a tiny fraction of what’s still out there.

The 1911 census has been released early (but some counties aren’t available yet). Although access is relatively expensive you get very good quality colour photos of the original documents: much better than the 1901 census.

EThOS, the British Library’s new online thesis sevice, is now in public beta. I’ll probably post about it in more detail later in the week, but for now I’ll just say that I’m very impressed.

Via Archaeozoology, an interesting but difficult to spell blog about about the archaeology of non-human animals, I discovered another interesting archaeology blog. Middle Savagery is written by Colleen Morgan, a PhD student at UC Berkeley. She’s doing lots of innovative things with Flickr, YouTube, Facebook and Second Life (don’t let the Goreans get you!).

I think maybe historians and archaeologists don’t talk to each other enough despite supposedly having a common interest in the past. My BA was originally going to be archaeology but I was bored with it after two terms and switched to history – I don’t think I would’ve done very well if I’d stuck with it. That bad experience has affected me for far longer than it should have done, and it’s about time I got over it. I was similarly disgusted with history after finishing my PhD but it only took me 5 years to get over that. (Disgust is a vice.) Studying the non-human is one obvious place where historians and archaeologists need to get together.

The web could well offer a way of breaking down barriers between disciplines. Since getting involved in blogging I’ve come into contact with lots of different ideas which I wouldn’t ever have thought about if I’d just been doing history in the traditional way. Reading blogs has given me easy access to literary theory, philosophy, cognitive psychology, evolutionary biology, linguistics, various feminist perspectives and much more. Writing my blog allows me to try out ideas that are outside my specialist area without investing too much in them. And trying to think differently benefits my “proper” work.

I’m trying to get some “proper” English Civil War related work done this week, but at the weekend I did some more First World War stuff. In April I posted about World War I on Flickr, when I uploaded my great-grandfather’s photos from Cottbus PoW camp. Now I’ve added his letters, and another photo which I got from ebay. Although he isn’t on it, it was taken in the theatre at Cottbus and one of the men has the same “Bing Bong Boys” navy outfit:

I’ve now put each letter/postcard in its own set to make the link between the front and back of the same document more explicit. The sets are then arranged into collections. Some people on the Great War Forum were able to help me locate Cottbus Camp No. I, so now most of the photos have been placed on the map.

I also discovered that another Wenham brother might have died in the Great War. I don’t know why I hadn’t ever looked for Wenhams on CWGC before, but I found a Charles Wenham who could well be one of William’s brothers. Some of the evidence is circumstantial and I need to do more digging to be sure, but the epistemic probabilities are quite high. So far it looks like he joined 10th Lincolnshire Regt (Grimsby Chums), served overseas, was wounded and sent back to England but died of his wounds. Unlike the soldiers who died overseas, his body was brought home and buried in Cleethorpes cemetery. Again the Great War Forum has been a great help, and you can see more details on this thread.

And with regard to the other World War, I played some more of Brothers In Arms: Earned In Blood. I was still a bit curious about the post-Hill 30 storyline, but so far it’s been quite boring, and I gave up when I got into a silly tank level that’s suspiciously similar to the silly tank level in Road To Hill 30 that I complained about before. But there are more trees this time…

I nearly called this post “the Great War on Flickr” but that could easily have been misunderstood…

It’s really about using the photo hosting site Flickr for photos and other documents from the First World War. I’ve uploaded some photos of my great-grandfather here as an experiment in digital history. Flickr is much more than just an image hosting service. I’ve been trying out some of its advanced features to see how useful they can be to digital historians, and what the limitations might be.

(more…)