Investigations of a Dog

• The latest Military History Carnival is up at Wig-Wags.
• The Institute of Historical Research is carrying out a survey to find out what people think about the possibility of podcasting/vidcasting research seminars. Go and tell them what you think. Their digital seminars project also has its own blog.
• Ross Mahoney linked to a UK National Archives project which involves post blogging the Second World War on Twitter using cabinet papers: @ukwarcabinet
• Meanwhile, the National Archives wiki Your Archives is starting a project to  create a glossary of historical terms. See the current list of wanted terms, sign up and add what you know.

Knowing how to program can save you from tedious repetitive tasks, such as inserting templates into a wiki page. Recently I’ve been spending more time editing the UK National Archives wiki Your Archives. I created a category for women’s wills, and while I was adding pages to it, I found that a lot of them didn’t have the correct template. Wills that were proved in the Prerogative Court of Canterbury are held by the National Archives and can be downloaded from their DocumentsOnline service. Transcripts of these wills can be posted on Your Archives, and we have a template for them which automatically creates a link back to DocumentsOnline based on an ID code, and formats some key data (testator’s name, dates, catalogue reference) in a standard form. Most of the data which goes into the template can be found in the DocumentsOnline index. We used to copy and paste each value manually, which was not the best use of a human’s time. Faced with the prospect of doing this an awful lot, I decided to write a program to do it automatically. First I threw together a Python script, which was alright for me but no use for people who don’t have Python and BeautifulSoup (and I also wrote it in such a way that it relied on Linux with xclip installed). So then I decided to rewrite it in JavaScript, so that other people could use it in their browsers. You can find the finished version and documentation on the PCC Will Bookmarklet page. Below is a walk through of how I did it.

(more…)

Last year computer programming was out, but now it’s back in. For me anyway. Having finished my data entry job in October I’ve got more spare computer time, which means I can be more active in digital history again. Some things are different now. Zotero has groups and syncing. The Programming Historian has moved since the last time I looked at it. I can finish my digital edition of Sandall’s history of the 5th Lincolns because Major Teall’s epilogue came out of copyright in the UK at the start of this year. But the biggest change is that I’ve switched my operating system from Windows to Linux. When I built my new desktop PC (codenamed Zen) I installed Ubuntu, and I love it. My laptop (codenamed Orac) still has Windows Vista, but I don’t use it much.

Changing to a completely different operating system might sound like a big step but it was actually really easy. This is partly because most of the applications I use are cross platform. I use Firefox more than any other application (and possibly more than all other applications put together). Don’t think that I spend all my time idly browsing the web: Firefox is vital for my historical research and writing. I use Zotero to store, sort, and access all the bibliographic data plus associated notes and PDFs for my research projects. These can all be synced between my PCs via the Zotero server and my own WebDAV server. My works in progress are now drafted on a private wiki which is also necessarily accessed through my web browser. This is much more powerful and flexible than writing in Word like I used to. Every page has an edit history so I can easily compare versions and revert to an earlier one. Wikilinks make it easy to fit sections together in different orders and link to supplementary information. Thanks to Google my e-mail and RSS feed reader are also on the web. When I’m not using Firefox, I still mostly use cross-platform applications. For the last few years I’ve used oXygen for XML editing and jEdit for find and replace operations, both of which are written in Java. Python can run on Linux, Windows and Macs, and although that doesn’t necessarily make individual scripts cross-platform it doesn’t really matter when I’m writing them for myself. The only Windows specific app that I’ve relied on in the last few years is MS Access. Even that was mainly because I was getting paid good money to put data into it for someone else. For my own research I’ve got some old databases from my PhD research, but all I ever need to do with them is export data into other formats.

Given all this, changing to Linux was not likely to be much of a problem, but that would be understating things. In fact it turned out to be a big advantage. Ubuntu is actually much quicker and easier to install and set up than Windows. It just works out of the box and comes with most of the things that most people need to get started. Open Office, Firefox, and even Python are all pre-installed. Once I’d added my favourite Firefox extensions and synced my Zotero library I was ready to do most of what I need to do. The only tricky things were manually installing a proprietary graphics driver and setting up DVD playback, but even this wasn’t too hard. If you don’t have a powerful new graphics card and don’t need 3D performance out of it, the pre-installed open source driver will be adequate for desktop stuff. Even setting up a network printer was completely painless.

Adding new applications is generally much easier than on Windows. Instead of buying a CD or downloading an executable file you can just access software repositories via a menu and tick boxes to select apps you want to be downloaded and installed. Because most of these apps are free in every sense of the word (like Ubuntu itself) you won’t have to pay money or agree to a licence that sells your soul to the devil. Via the repositories I could easily install Geany (a code editor which I now use for Python programming: I actually like it more than Komodo), gFTP (FTP client), the aforementioned jEdit, and the BeautifulSoup library for Python. It only took a few simple commands at the terminal to install and set up an Apache server with PHP and MySQL for local testing. oXygen had to be downloaded and installed manually as it’s a proprietary application, but the academic licence is cheap and cross-platform: I originally bought it for Windows but my licence automatically carries over to Linux. To get it working properly I had to install the proprietary Sun version of Java, but that was easy to do via the repository. There is a thing called WINE which lets you run some Windows programs in Linux, but so far I’ve only used it for listening to music with Spotify.

With everything set up to my liking, Ubuntu has made me fitter, happier and more productive. It’s faster, more secure, more stable, and less annoying than Windows. You can start using it as soon as the desktop appears on the screen instead of waiting for it to finish starting, or dealing with a patronising storm of pop-ups about how your anti-virus might be out of date or how you’ve got unused icons on your desktop. The Blue Screen of Death is now just an unpleasant memory. Linux users generally don’t have virus scanners or software firewalls because we don’t need them. The only major problem I’ve had so far is when an upgrade to a new version didn’t agree with my proprietary graphics driver and made it impossible to boot to the desktop from the hard disk. Even that was surprisingly easy to recover from, as being able to run the operating system from the LiveCD makes it very easy to rescue any files which aren’t already backed up before doing a clean reinstall (and the reinstall process is quicker and easier than for Windows).

So those are my reasons for preferring Ubuntu to Windows. If you haven’t tried Linux before you can download Ubuntu, burn it onto a CD, and then boot from the CD, which gives you an option to try it out without actually installing it on your PC. And it won’t cost you anything. Meanwhile I’ll be getting on with my research, writing and programming. And blogging about those things…

It didn’t take long before I decided to start a Zotero group. It’s called War and Gender and is dedicated to collecting and sharing any material relating to the intersections of these two very important things. There are no limits on period or place, membership is open, and all members can add to the group library. So if you’re interested, and if you’re using Zotero 2.0, get stuck in.

You can see from my profile that the groups I’m involved with so far are all quite specific and tend towards things that relatively little has been written about yet. I think we’re all still finding our way and sticking to things that are likely to be manageable. In the future it’ll be interesting to see if more general groups appear and whether they work out. I could start a British Civil Wars group, but it would be potentially huge. I already have over 900 items in my library tagged with “english civil war”, and these are mostly biased towards my research interests in England 1642-46. I don’t have very much on Scotland, Ireland, the Second Civil War, the Commonwealth or the Protectorate. Maybe more specialist sub-fields will be the way to go, but we’ll see eventually.

Ian MacInnes has set up a Zotero group for The horse in history and culture where we can pool references to horses. This was the incentive I needed to finally sort out my collections and upgrade to Zotero 2.0. I’ve set up my profile on the Zotero website, added a CV and shared my library so anyone can browse it. Although it looks like I’ve got an awful lot of collections, I’ve simplified the hierarchy and started making better use of tags. I’m not sharing notes at the moment, but maybe I will later. (NB: if you uncheck the “share notes” box in the privacy settings it only hides notes that are attached to items, not standalone notes.) There’s also another group for early-modern animal studies where you can find stuff about other species as well as horses. Now I’m wondering what other groups would be useful, but I’m not sure if I want the responsibility of owning a group yet.

There has been some bad news for historians recently: the RHS Bibliography of British and Irish History has lost its direct government funding and is being privatised in a move disturbingly reminiscent of PFI (and to add insult to injury the IHR claims to be “delighted” about this!); the UK National Archives (or PRO to most of us who use it) can no longer afford to open on Mondays or offer free parking.

But it’s not all bad. There’s also some good news from the National Archives which has got much less attention than the bad news – in fact I’m not even sure exactly when it happened. They are now allowing and encouraging users to upload photos of public records held at Kew to Flickr and similar photo sharing sites. Crown Copyright had already been waived to allow republication of the text of public records but previously publishing images of documents didn’t appear to be allowed. Now it’s confirmed that uploading images to Flickr is allowed (provided that you’ve taken them yourself – this doesn’t cover documents bought from DocumentsOnline or Ancestry). This is a win situation for everyone, because these documents will be made freely available without it costing the archives anything – a major advantage when budgets and funding are being cut drastically.

The NA has its own Flickr account, and a group for visitors. Combined with the Your Archives wiki this could lead to some really exciting stuff. Some people are already using Flickr and Your Archives to publish Metropolitan Police leavers’ registers. The possibilities are endless. I’m certainly going to upload all the photos I take in the course of my research. To start with I’ve put up the service record of my ancestor Tom Wenham from the First World War (photographed from the screen of a microfilm reader).

Still to come are some indemnity cases from SP24, and sooner or later I’ll have loads of SP28 to share. It would be fantastic if other archives would do this too, although some will probably be too conservative to try it. The British Library still doesn’t allow digital cameras, which just makes me not want to bother with BL manuscripts.

A while ago I was wondering whether there were any good social networking sites for academics, and whether I really needed one. Now it looks like Zotero is going to fill the gap. Zotero 2.0 is now in beta, and Dan Cohen has given a rundown of the new features. Users will get a personal page on the Zotero site allowing us to network and interact with people with similar research interests, and users will be able to create and join groups to make sharing of bibliographic data easier. This all looks really exciting. I’ll probably try it once I’ve got my Zotero collections and tags in better order.

Via Ben Brumfield and Archaetech I’ve heard about the InterFace conference on humanities and technology, which will take place at Southampton University in July. Although it looks a bit more formal than THATCamp, the speed dating and lightning presentations should make it very different from traditional humanities conferences. The deadline for submissions is 1st May. I’ll probably put one in myself if I can come up with a good enough idea.

As I mentioned last week, EThOS is now open to the public. This is the British Library’s new online service for delivering copies of UK PhD theses, replacing the old British Thesis Service which used to supply copies on paper or microfilm. Although the site is officially in beta most features seem to be fully working. Right now there’s only a basic search (you can’t limit your search to specific fields) but it accepts wildcards and should be enough to find what you want if you know what you’re looking for. Searches can be limited to theses which are available for immediate download.

Theses are being digitized on demand. If someone has already requested and received a copy of a thesis then it will be available immediately for anyone else who wants it. There is no charge for downloading a thesis which has already been digitized, but you have to pay the cost of printing and binding if you prefer a hard copy. The publicity last year suggested that the first person to request a thesis would have to pay the costs of digitization, but now it looks like this will rarely happen because many university libraries have agreed to pay for digitization of their own theses as part of their commitment to open access. Once a new thesis has been requested digitization is promised in 10 working days although a notice on the site says there might be delays because of heavy demand. I ordered an undigitized thesis today (D. E. Lewis on the parliamentarian ordnance office – something I would have read during my PhD if I’d known about it), so we’ll see how long it takes.

Searching for “english civil war” limited to theses already available I got a couple of hits (there are lots more for “first world war”) and downloaded David Evans’s thesis on Edward Massey. I found it slightly annoying that I had to go through a checkout process even though the download was free, but it’s still an awful lot more convenient than paying around £50 and waiting several weeks the last time I ordered a thesis. The digital file is a PDF but it comes inside a zip file. That seemed slightly pointless as it didn’t make the download significantly smaller (12MB zipped, 14MB unzipped) and means that you can’t view the thesis straight away in your browser. It might make sense if multiple orders were combined in the same zip file, but even if you have more than one thesis in your basket you still have to download and unzip each one separately.

It looks like most UK universities are participating in the scheme, but significantly Oxford and Cambridge aren’t. Although their theses show up in the search results they can’t be ordered through the site. This might just result in fewer people bothering to read and cite theses from the big two, so it could be their loss as much as anyone else’s.

Overall I’m really impressed with this site. There are some minor things that could be improved, and it crashes occasionally, but it’s obviously going to be a very useful resource. I’m particularly pleased that in most cases users won’t have to pay for theses. I hope this will encourage people to be more adventurous about which theses they consult.

I’m finally back to blogging after a few months off. This post is a quick roundup of some things that happened while I was away.

First of all, the Military History Carnival always needs more hosts. Although it’s not my responsibility any more I’d like to see it continue, so if you can help please contact TJ at tj$linzy$@$gmail$.$com (remove the dollar signs)..

I’ve deactivated my Facebook account as I was finding the whole thing too annoying. I’ve opened an account at LinkedIn but I’m not sure if I’m going to use it. It seems very much aimed at The Businessman In His Suit And Tie. Does anyone use it, and is it any use? I’d like to see a social networking site specifically designed for academics. Maybe with facilities to self-archive our publications.

The Great War Archive (which I posted about here) opened on time in November, and looks really good. They’re also continuing to collect submissions through a Flickr group. Reviews in History have published a review by Esther MacCallum-Stewart and a response by Stuart Lee. This project gives us proof that there are lots of interesting documents in private hands and that at least some people are willing to share them on the web if given the chance. Although 6,500 documents sounds like a lot I suspect it’s a tiny fraction of what’s still out there.

The 1911 census has been released early (but some counties aren’t available yet). Although access is relatively expensive you get very good quality colour photos of the original documents: much better than the 1901 census.

EThOS, the British Library’s new online thesis sevice, is now in public beta. I’ll probably post about it in more detail later in the week, but for now I’ll just say that I’m very impressed.