I’ve just been looking into Google Base, which lets you upload structured data in XML format and make it searchable on Google (although so far Base pages don’t seem to show up in the standard web search). The data is described using item-types and attributes, and although Google provides recommended types and attributes you can also make up your own, for just about any purpose you want. This kind of semantic markup gives the potential for much more specific and accurate search results than a normal web search.
Now I’m wondering if this could be a possible solution to a big problem that I’ve been thinking about for a while: pulling together a list of all the British soldiers who served in the First World War and everything that’s known about them. This would be in the region of several million names. Many of the details are already available online in various places but they’re not linked together. The CWGC has a more or less complete database of personnel who meet their criteria of having died as a result of the war, although new names are discovered every so often (their own search engine can only search by name, not by regiment or service number). Surviving service records (only about 30 to 40% survived the Blitz!) are being put online by Ancestry, although it’s subscription only and the indexing and transcription are reputed to be really terrible. The UK National Archives has made the medal index cards available online (I’ve seen several transcription errors in the index but it’s apparently not as bad as Ancestry). This collection contains nearly 5.5 million records and should mention every soldier who qualified for a campaign medal by serving overseas (although there are unsubstantiated rumours on the Great War Forum that some cards were lost in transit). The medal cards also include men who were awarded a Silver War Badge for being discharged as unfit for service, even if they hadn’t served overseas. Officers are more problematic because if they survived the war they had to apply for their campaign medals and there are many known examples of officers who didn’t make a claim and so have no medal card. Commissions and gallantry medals are shown in the London Gazette, which is available online, but its search engine is notoriously difficult to use. Then there are various personal websites of people who are researching their families or a particular unit. And there’s the Great War Forum, which contains a huge number of posts on individual soldiers, often pulling together information from many different sources, from the most well known online databases to obscure local newspapers and family collections.
In theory something like Google Base could help to pull all this stuff together and make it easier to find information on specific people. For example, you could create an item type for soldiers and give it attributes like name, rank, regiment, battalion, service number etc. First of all a lot of thought and consultation would need to go into defining the item and attributes to make it as useful as possible to as many people as possible. This is definitely something to think about for the future.
However, there are some limitations which mean it isn’t going to happen soon. The biggest problem I can see is that you have to manually upload the records to your account, and edit them whenever they change. You can use the API to automate this but I think it would be much better if you could just embed Google Base metadata in a webpage and let Google’s spiders pull it out automatically. Another thing is that there doesn’t seem to be any scope for collaboration. Once you’ve uploaded your data no-one else can edit it. This is quite disappointing because sharing is a big part of Google Docs. In my experience many expert Great War researchers do not have advanced IT skills and so we need things to be as simple as possible, and easy ways of helping less IT literate people by being able to edit their stuff directly. The Your Archives wiki has shown that this can work really well: it doesn’t matter if people haven’t formatted their pages properly or don’t know how to insert a link. As long as you put up some relevant information, someone else can sort it out.
But these are changes that Google could make in the future, so it’s something to watch out for. There must be lots of other ways that historians could use Google Base. It’s already good enough for smaller data sets which have already been compiled by one person, so I might be able to put up some of my English Civil War data.
In my last post I posted my first attempt at writing Python code to do calculations with pre-decimal currency. With a lot of help from Ben Brumfield I’ve rewritten it so that it now does a lot more with less code. The classes and functions have been completely rearranged, everything is easier to read, and there is more scope for dealing with uncertainty. This is yet another example of the benefits of blogging. Without Ben’s input I’d still be using some pretty mediocre code, but by posting my first attempt on the blog and brainstorming with readers I’ve made a vast improvement in only a couple of days. More details below.
(more…)
And more adventures with Python programming. One of the trickiest problems in British history is dealing with pre-decimal currency. Until 1971 British currency was a bit strange to say the least. There were 12 pence in a shilling and 20 shillings in a pound (so a pound had 240 pence). This is obviously not something that most off-the-shelf software can deal with, but doing calculations on old money is something that historians need to do quite a lot. During my PhD, when I was using Access databases, I had to decimalize amounts of money before I could do anything with them. That was awkward because some values in the pence column (I seem to remember that 4 and 8 were particularly annoying) gave a recurring fraction. To make things easier I arbitrarily rounded the pence values to the nearest multiple of 3, which meant that my figures were less exact than they could have been, but in practice I could live with it.
These days I can do better. Below are some technical details of how I approached the problem in Python (I like traaaainspotting…).
(more…)
Since my last post I’ve been doing some more experiments to see how Zotero can be used for cataloguing previously uncatalogued administrative records from the English Civil War. I’ve now put some more of my ideas into practice in demo form and they seem to work. Linking images to Zotero items and adding metadata went very smoothly. The idea of adding extra data by putting XML tags in notes also works, although this is just a stopgap until they implement custom fields. Once you have data in Zotero it’s very easy to export it as XML and do whatever you want with it. More details below, but it gets a bit technical and even includes some sample code (formatting code in Wordpress is hard, and it’ll probably screw up the layout for some people). If you’re not A. Nerd and you’re not doing the shopping for your mum you might want to stop reading now.
(more…)
In a previous post I mentioned experimenting with taking photos at the Public Records Office/National Archives. Getting good photos is only part of the problem. The real work starts when you get them home. How do you organise them and make sense of them? It should be no surprise that Zotero is really useful for this, but I’ve discovered a few tricks to make it even better.
(more…)
The new early-modern edition of the Carnivalesque blog carnival is now up at Cardinal Wolsey’s Today In History.
History Nexus is a new social bookmarking site which focuses on history websites. It uses similar technology to Digg, allowing users to vote and comment on websites as well as post links to them. Although it seems to be in its early stages it has huge potential both for promoting your own history site and for discovering other sites, so I hope it attracts more users. For example, there’s a link to an innovative First World War site which I didn’t know about before:
The Christmas Truce is a collaborative project to collect reports of the Christmas 1914 truce from local newspapers. I know from my own research that local papers are a very valuable source for the Great War but that finding the information you want involves spending hours searching through microfilm, so I’m definitely in favour of any attempts to make that information more accessible. This project would be even better if it used a wiki - so far it looks like it’s based on e-mailing newspaper transcripts to the editors - but it’s good that the work is being done.
I’ve just read an anecdote about some Londoners who had never seen a cow before. It was from 1644. Ah! Pull back and reveal! My expectations were confounded…
The anecdote in question comes from Robert Harley, an officer in Sir William Waller’s army. His letter is mostly known as an eyewitness account of the battle of Cheriton, but sandwiched between a matter-of-fact relation of some minor skirmishing is the cow story (HMC, Portland, iii, 107):
The enemy faced us this day with about three thousand horse. Here you should have seen the Londoners runne to see what manner of thinges cowes were. Some of them would say they had all of them hoornes, and would do greate mischiefe with them, then comes one of the wisest of them cryeth ‘Speake softly’. To end the confusion of their opinions they pyled up a counsel of warr, and agreed it was nothing but some kind of looking glasse, and soe marched away. Wee had some light skirmishes but with little hurt on either side.
What’s going on here? Were there really 17th century Londoners who had never seen a cow? Surely Smithfield market was full of them (cows and Londoners!), and cows tended to be slaughtered at the butcher’s shop. Maybe there were some poor areas of the East End where nobody could afford beef so the butchers didn’t buy cows, but if that was the case it would imply that the residents never went very far from home. The countryside, the richer areas of London, and the livestock market at Smithfield should all have been within walking distance.
The alternative is that Harley was making it up or exaggerating for comic effect. In that case it might tell us something about stereotyping. Since he was from a gentry family (son of Sir Robert and Lady Brilliana Harley of Brampton Bryan) it wouldn’t be surprising if he had a negative view of the lower classes. But it could also be a case of rural against urban. Maybe the sterotype of ignorant townies goes back further than I thought.
Another possibility is that “cow” has some special meaning here - perhaps a different breed with bigger horns compared to the ones Londoners were used to seeing. Did the cows bred for the London meat market have their horns removed?
Ultimately I don’t really know what to make of it, but it’s definitely something to think about. Any suggestions welcome.
The 9th (not 7th!) edition of the Military History Carnival is now up at the Official Osprey Publishing Blog. Thanks to Mike for a great job. The next edition will be at Walking The Berkshires on 7th January. E-mail submissions to greensleevesenviro [at] sbcglobal [dot] net or use our carnival submission form.
Being able to download PDFs of articles from JSTOR is a lot more convenient than photocopying them out of the journals, but one of the things we’ve lost is seeing the enigmatic (and sometimes bizarre) fragments of other articles on the pages facing the start and end of the article you want. This book review, from the 1957 issue of the Journal of the Society for Army Historical Research, looks slightly disturbing now, and is the sort of thing that used to give military history a bad name:
GUNS. By S. E. ELLACOTT. (Methuen & Co., Ltd. London. 1955. 8s. od.)
This is described as “primarily for boys,” but everyone interested in weapons will enjoy reading this volume and studying the author’s 86 illustrations in clear and telling line-drawings. One constant problem of a Society like ours is the recruitment of new members, and the strategic present of a book like this is likely to develop an early intelligent interest in one aspect at least of our researches. “Brown Bess” gets a raw deal (p. 26), but since the author, besides including sections on naval and land artillery, discusses sporting as well as military weapons and has lively pages on machine guns and pistols, he may be forgiven. The story goes from the first guns in the fourteenth century to the F.N. type B automatic and “Atomic Annie,” and the whole production is well worth reading. Other books in this series of “Outlines” which can be recommended are “Castles and Fortresses” and “The Crusades,” both by R. R. Seilman.
Lively pages on machine guns: what more could you want? But remember, it’s primarily for boys. If you’re a girl, stop reading now.
