There has been some bad news for historians recently: the RHS Bibliography of British and Irish History has lost its direct government funding and is being privatised in a move disturbingly reminiscent of PFI (and to add insult to injury the IHR claims to be “delighted” about this!); the UK National Archives (or PRO to most of us who use it) can no longer afford to open on Mondays or offer free parking.

But it’s not all bad. There’s also some good news from the National Archives which has got much less attention than the bad news – in fact I’m not even sure exactly when it happened. They are now allowing and encouraging users to upload photos of public records held at Kew to Flickr and similar photo sharing sites. Crown Copyright had already been waived to allow republication of the text of public records but previously publishing images of documents didn’t appear to be allowed. Now it’s confirmed that uploading images to Flickr is allowed (provided that you’ve taken them yourself – this doesn’t cover documents bought from DocumentsOnline or Ancestry). This is a win situation for everyone, because these documents will be made freely available without it costing the archives anything – a major advantage when budgets and funding are being cut drastically.

The NA has its own Flickr account, and a group for visitors. Combined with the Your Archives wiki this could lead to some really exciting stuff. Some people are already using Flickr and Your Archives to publish Metropolitan Police leavers’ registers. The possibilities are endless. I’m certainly going to upload all the photos I take in the course of my research. To start with I’ve put up the service record of my ancestor Tom Wenham from the First World War (photographed from the screen of a microfilm reader).

Still to come are some indemnity cases from SP24, and sooner or later I’ll have loads of SP28 to share. It would be fantastic if other archives would do this too, although some will probably be too conservative to try it. The British Library still doesn’t allow digital cameras, which just makes me not want to bother with BL manuscripts.

Mercurius Politicus linked to Gilbert Mabbott, a new blog about print culture in the English Civil Wars and Interregnum. From this blog I discovered that Calendar of State Papers Domestic is starting to appear on Google Books. There’s a James I volume available with full access. I’m hoping that the rest of the series, particularly the Charles I volumes, will follow soon. There’s no reason why they shouldn’t as they’re all in the public domain. Since the original documents were under Crown Copyright and the calendars were published by HMSO in the 19th century the copyright must have expired by now.

Having decided to leave my 5th Lincolnshire First World War project for a while, I got an offer I couldn’t refuse: someone from the Great War Forum sent me a transcript of the battalion’s medal citations from the regimental archive so that I could publish them on my site and link them in to the index of people that I’d created for the book. The document contains information that can’t be found elsewhere, as although awards of the Military Medal were listed in the London Gazette, full citations were not normally published. There are also three awards not mentioned in Sandall’s list, and citations for 10 people who were recommended for awards but turned down.

I received the list as a Word file with no semantic markup on Wednesday morning, started working on it on Thursday morning, and published it on the web this afternoon. It looks very basic but it’s not bad for two days, and it’s all linked in to the index of people for Sandall’s book. First of all I copied the text into jEdit and used Find and Replace to insert some basic TEI XML markup. Then I pasted it into a new TEI document in oXygen. With the automatic validation it was easy to track down and correct errors in the markup, so by lunch time I had a completely valid TEI file. In the afternoon I spent about 3 or 4 hours on linking records by inserting key attributes into <persName> tags. In most cases I already had the keys that I used for linking names in Sandall, but sometimes I had to change them in the light of new evidence from the citations, such as full names of people who I previously only knew by their initials. This also allowed me to clear up some ambiguities . This morning I finished the linkage by creating new keys for the 13 people not mentioned by Sandall, then got started on writing some XSLT. That was easy as I could copy or adapt a lot of the code from the style sheet for Sandall. As well as generating the HTML version of the citations, this XSLT generates an extra JSON file which is imported into the Sandall index of people to allow linking the citations. Again this only required some minor adjustments to the Exhibit page. After some testing and corrections I had a live site up this afternoon.

This demonstrates the potential value of the techniques I’ve been using for marking up texts, but it also raises some problems for digital history. I decided to trust a transcript from a random person off the internet. I have no way of knowing how accurate the transcript is, or even if the source document really exists! It could be Hugh Trevor Roper and the “Hitler Diaries” all over again. Therefore I’m going to think more carefully before putting myself in this situation again. There’s also a possibility that I’ve miscalculated the copyright situation. Based on internal evidence and comparison with other documents my best guess is that the list was created by the army and is therefore under Crown Copyright (and being unpublished and available for inspection in a public record repository should come under waiver of Crown Copyright), but without seeing the original it’s hard to be sure. I might be wrong, and even if I’m right the holders of the manuscript might not agree. So technology makes some things easier, but there are other problems that it can’t solve.

Nearly a year ago I started a project to digitize T. E. Sandall’s history of the 1/5th Lincolnshire regiment in the First World War, and in summer I published an interim version. I would’ve finished it a long time ago if I didn’t have anything else to do, but original work in peer reviewed printed journals has to come first because it’ll look better on my CV. Now I’ve got time to do some more work on it, and having had a break from it I can reassess what I’m trying to do. Below is an update on what’s new.

First of all I discovered from searching the Times Online archive that Major Teall died in 1939, so I’ll be able to publish his epilogue to the book when it comes into the public domain in 2010. I’m still not sure about the photos. I’ve fought my way through the 1988 Copyright Act, which seems to say that copyright in any photos published before 1988 or taken before 1957 exists until it would have expired under the 1956 Copyright Act, which is 50 years after publication for published works, or 50 years after the death of the author for unpublished works. If I’ve understood correctly that would put the photos from this book (published in 1922) in the public domain. I’m still going to leave them out for now because it’s an extra complication, and because they didn’t scan very well.

Since I last did some work on this project, the new version of the TEI guidelines (P5) has been released. Not everyone will need to upgrade from P4 to P5, but I want to use some of the new features. There are now elements and attributes designed for linking to page images, which will be very useful. The upgrade was quite easy as most existing elements haven’t changed much. P5 is based on a schema rather than a DTD. oXygen 9.1 (which I’ve just bought a licence for – only $48 for the academic version, and well worth it) comes with TEI P5 schemas, style sheets, and templates pre-loaded. Because I need to use the modules for names and dates, and digital facsimiles, I need to generate my own schema, but this is easy using Roma, a user-friendly online tool.

As well as making the move to P5, I decided to make some other changes to the XML before moving on to the next stage. I removed the original line breaks, which were marked up with <lb> elements, and soft hyphens, which were represented with entity references. This means I’ve lost something from the original text which won’t be easy to get back, but I don’t really have any use for these line breaks and they were causing more trouble than they were worth. Rather than define a new element to represent soft hyphens just in case someone somewhere might want to see the text with original line breaks, I decided to get rid of the problem. Anyone who does want to see the original layout will be able to look at the page images anyway. Removing these elements was easy using Find and Replace with regular expressions in jEdit.

I also decided to get rid of the various elements which were used to represent double quotes. These were <term>, <distinct>, <foreign>, and <soCalled>. I already knew that I wasn’t going to be using these elements for anything. There are so many possibilities in TEI that it’s easy to get carried away and start marking up features just because you can. Now I’ve decided to only use markup which is required by the TEI guidelines, or which has a specific purpose for the resource I’m trying to create.

My first attempt at transforming the text to HTML using the TEI style sheets wasn’t very promising. I thought then that I just needed to play around with the parameters to make things better, but now, having played around with the parameters quite a lot, I don’t think it’s going to be much use. I was always going to have to write some custom XSL to deal with people, places, and dates, but it looks like the TEI XSL won’t even give me a basic HTML version split into separate files for each chapter. Although it supposedly does this, it looks like it’s based on the assumption that internal links will use query strings, whereas I just want plain old relative links to plain old HTML files at this stage. It also puts a lot of junk into the HTML whatever parameters you set. For example, if you want an id for each paragraph it doesn’t add an id attribute to the <p> tag, it inserts an empty anchor at the start of every paragraph! And every <a> tag has an XML namespace declaration, even if you’ve asked for HTML 4! So I’ll have to get better at XSL and write my own style sheet from the ground up.

The next step is to generate id and facs attributes based on page numbers for every <pb> element, and put in <ref> tags in the index to point to these page numbers. I also need to decide whether to keep the original table of contents or generate one automatically using XSLT. After that I can either write some XSLT and publish another interim version, or move straight on to marking up people, places, and dates.

It’s been a long time since I wrote anything about my First World War digitization projects, but I now have some progress to report: today I published an interim version of Sandall’s History of 5th Lincolnshire Regiment. It’s still a work in progress, and there’s a lot more to be done, but you can see it here. It’s just a plain HTML version (and not strictly valid HTML), and the whole text is on one page (at least it makes it easy to search the whole text with your browser’s Find feature!), there’s no name linkage yet, no page images online, and no mechanism for submitting corrections. However, even in this form it should be useful to people who are researching the battalion and can’t get hold of the original book. More details on what I’ve done and how I’ve done it below.

That I haven’t posted about this project since February might suggest that it’s been slow and difficult, but actually I was just busy with other things (writing articles, applying for jobs, organising the Military History Carnival). Until then, things were going surprisingly smoothly and quickly. I was hoping that I’d be able to get everything finished before my Oxygen free trial finished, but because I was doing other things it’s now expired, and although a licence doesn’t cost much, I need the money for other things (especially work on peer reviewed things which might get published in a “proper” journal and look better on my CV than a self-published digital edition). Fortunately the last thing I did on the project in February was try an XSLT transform to make a test HTML version of the text. Initially this just showed me that I needed to play around with the transform parameters to get what I wanted, but I’ve now decided that it’s good enough to form the basis of an interim edition. (In theory I could use the free version of Saxon to do more transforms, but it’s a scary command line application!)

The HTML file that came out of the transform was over 900K but I cleaned it up using Find/Replace and regular expressions in jEdit, getting it down to 385K. That might still take a while to download if you’re on dial-up but it’s not too bad for a whole book. Every paragraph element had been given a name attribute, which isn’t necessary for this version as only chapters are linked to from the contents, so I stripped them all out. There were also some xmlns attributes which didn’t appear to be serving any purpose and must have added a huge amount to the file size, and a huge amount of superfluous white space.

As well as getting the file size down, I needed to make some other adjustments. Some of these were down to the transform settings that I’d used in Oxygen for the test run, but others showed up some possible limitations of TEI/XML. The master XML document preserves the original line breaks and hyphenation. I also left these in during the test transform but in practice an HTML version without line breaks is more useful so I used jEdit to take out all the <br> elements. Here I encountered a potential problem. Following the advice of the TEI guidelines I used the ­ entity ­ to represent soft hyphens. In practice this isn’t very helpful. If the HTML version is to keep the original line breaks then the soft hyphens need to be converted to hard hyphens so that they display properly, but if the original line breaks aren’t kept then the hyphens need to be removed along with the breaks. I’m not sure if XSLT can actually do this. My understanding is that it only deals with XML tags, but I could be wrong there. If it can’t do anything with entity references then there would be a need for some extra finding and replacing, but there might be anyway depending on how good I can get the output of the XSLT. Maybe it would be better to have an XML element which represents a soft hyphen, or add an attribute to the <lb> elements to indicate that they’re preceeded by a soft hyphen. If the line breaks and hypens could be dealt with properly by XSLT then it would make sense to not have a space before any <lb> elements in the source XML.

I found some erroneous paragraph breaks in the middle of sentences. These turned out to be mistakes in the master XML file. I’m not sure how they came about, but they’re easy to track down and fix with a regexp (just find any <p> element immediately preceeded by a comma or a letter). The XSLT had used <em> tags to highlightwords and phrases marked up with <term> or <distinct> but I changed these to double quotes to match the text. In future (if I ever digitize another book) I’m intending to leave in the original quotes rather than marking up their meaning.

It was quite ironic to discover that the XSLT had automatically inserted a copyright statement at the end of the text! (Copyright is more than just a law: it’s an insidious ideology) I corrected it manually to make it clear that the work is in the public domain. Other manual adjustments included moving the contents to before the preface (more convenient for readers) and adding some CSS to limit the width of the text divs.

So now there’s a basic version of the text online in a form which should be of some use to at least some people. It’s certainly an improvement on taking requests on the Great War Forum and posting the excerpts that people ask for. There’s still a lot to be done but it’s good to be making some tangible progress again.

The latest early-modern edition of Carnivalesque is up at Blogging The Renaissance.

I’ve turned off the comment timeout plugin, so comments on most old posts are open again, and should stay open as long as they don’t attract huge amounts of spam. I’ll be manually closing comments on posts which are getting spammed too much but I hope most of them will stay open.

Good news: Calendar of State Papers Domestic, one of the most important printed sources for British history, will be available online later this year. Bad News: it’s a paid subscription service. It remains to be seen how much it costs, but it’s particularly annoying because the project is funded by a charity, and the material is probably in the public domain, having been published by HMSO more than 50 years ago. More details at the IHR website.

Battle Through Time was a computer game released for the Commodore 64 in 1984. It featured a time travelling car and levels based on World War 1, World War 2, Korea, and Vietnam. Just another example to bring up when lazy journalists say there aren’t any WW1/Korea games, or that WW2/Vietnam games didn’t start to be made until this century. And to emphasise the links between cinema and gaming, the background music included “Suicide Is Painless” for the Korea level and “Ride of the Valkyries” for Vietnam.

And I’m still looking for Military History Carnival Hosts for September and afterwards. If you’re interested, e-mail me or leave a comment.

This is a progress report on the First World War digitization projects I outlined previously in my post on planning.

Copyright:

I now have definite proof that the work of T. E. Sandall is in the public domain, so I can proceed with digitizing the main text of the book without fear of ending up in court. His death certificate confirms that Thomas Edward Sandall MD, Deputy Commissioner of Medical Services at the Ministry of Pensions, died at 2 Montalt Road, Woodford Green, Essex, on 31st May 1930. A notice of death in The Times carried the same information also noting that he was former CO of 1/5th Lincolnshire Regiment, tying everything together nicely (I already knew that he was a doctor before the war, as well as being a Territorial officer). Therefore the main text of the battalion history has been out of copyright in the UK since 1st January 2001.

I still have no idea when Major George Harris Teall died, and since he was born in 1880 he could potentially have lived into the 1980s. If I can’t find out when he died, or if his work turns out to be still in copyright, it can easily be omitted. The postscript which he wrote deals only with the reconstitution of the Territorials in 1920, and while that’s probably an under-researched topic, it’s also likely to be less interesting to most Great War enthusiasts. The two works are treated separately on the title page and each is the sole work of its respective author, so I don’t believe that anyone could plausibly claim that the book as a whole is a joint work.

I was going to omit the plates, but there is now a slim possibility that I won’t have to. Steve Bramley and Chris Bailey, who are writing a new history of the battalion, would also like to use some of the photos in their book, so we’ve been discussing copyright issues. They might be contacting the publishers of Sandall’s book (Blackwell, so at least they’re easy to track down) to see if there is any record of the copyright holders of the illustrations. There might not be any record, or the copyright holders might require a fee for reproduction, but we can still hope that copyright has expired.

My contact with Chris and Steve has also shed new light on the family photos of my great-grandad. We need to look into copyright law in more detail, but it has been suggested that copyright in unpublished photographs created before 1957 expires 70 years after creation, regardless of the life or death of the author. If this turns out to be true, it automatically removes concerns about these photos, which were all taken in or before 1918. If it turns out to be false, it won’t be a complete disaster. The photos taken at Cottbus are credited to Paul Tharan, the camp photographer. He must have taken thousands of photos of PoWs, some of which turn up on ebay from time to time, but details of his life are surprisingly obscure. So far a request on the Great War forum, usually a mine of esoteric information, hasn’t produced anything. If I can’t determine copyright status I’m still prepared to publish the photos but take them down if a copyright holder appears.

Two of the wartime photos are almost certainly not by Paul Tharan. One has been identified (thanks to Chris Bailey) as the 5th battalion football team taken on the day of their match against the Grimsby Chums at Blundell Park on 22nd October 1914 (view photo). The other is of my great-grandad in khaki uniform (view photo). It was sent home after he was captured, but I can’t be sure when or where it was taken. In both cases there are no photographers’ names on the postcards. Although this makes it impossible to prove that they are definitely out of copyright (unless the 1957 rule is true), it also makes it impossible for anyone to prove that they are the copyright holder, so there probably isn’t much to worry about there.

Scanning:

All images have now been captured. Sandall’s book was scanned in monochrome at 300dpi, two pages at a time. This resulted in 132 images, taking about two to three hours. They are saved as JPEGs which take up about 1.5 to 2.5MB, depending on the amount of text on a page. I’m probably going to split them into separate files for each page, which won’t take long using batch processing as the position of the centre of the book is always more or less the same in every scan.

I decided to scan the family letters and photographs in full colour at 600dpi, keeping archive copies as uncompressed TIFFs. Photos can benefit from being scanned at the highest resolution possible, because enlarging them on screen can bring out detail which is invisible to the naked eye on the original prints. At the Great War forum there’s a whole sub-culture devoted to identifying units from internal evidence in photos.

Here are a couple of examples taken from the photos I linked to above. In this one I’ve enlarged the portrait of my great-grandad in uniform to show that he’s wearing a pocket watch in his left pocket, with what looks like a small cross hanging from the lanyard:

This is from the photo of the battalion football team at Blundell Park. You can see a uniformed soldier in the crowd, with what could be a Lincolnshire Regiment cap badge:

The only potential drawback with 600dpi TIFFs is file size: a scan of one side of a postcard is over 20MB! In this case the small size of the collection means there are no problems, but scaling the same technique up to a larger collection might be problematic. I’m not sure if it was strictly necessary to scan the letters at the same resolution as the photos, as all the writing would be just as legible at 300dpi and blowing it up beyond a certain point doesn’t really add anything. On the other hand, I found it convenient to keep the same scanner settings and scan the whole collection as one batch. This took about two and a half hours, resulting in 62 images.

Next Steps:

I want to do more background reading before I go too much further. I’ll also be playing around with various software, finding out what it can do and how to use it. And tomorrow I’ll be spending a whole day away from my computer as I’m going down to London for a seminar at the IHR on Cromwellian studs (it’s obvious what I meant).

In my New Year post, I mentioned that I’m thinking about carrying out a couple of digital history projects in connection with my First World War research. These projects are very small and should be relatively easy to carry out on my own, but there will almost certainly be challenges. Overcoming these will give me more experience of carrying out a digital history project (this is starting to sound like a job application again!), and produce useful resources. After that, I can move on to consider some more advanced issues, such as collaborating with other people, and dealing with seventeenth-century manuscripts. To make the experience even more useful, I’m trying to blog it as I go. This post is an outline of my plans so far. Now that I’ve published my plans I’ll have to carry them out!

Background:

Before starting any digital history project, you should read Digital History by Dan Cohen and Roy Rosenzweig which is available online free of charge. It’s an easy to understand introduction which will tell you about different approaches to digital history, costs, benefits, potential problems, and generally help you to think clearly about what you want to achieve and how you can achieve it. The authors ask aspiring digital historians to get the right balance between caution and risk-taking. While a certain amount of planning is always necessary, there is also a risk that academic historians will spend too much time thinking rather than doing.

Having read the book and thought about the potential problems to be overcome, I think I’m in a good position to put my IT experience to use. I’ve been running my band’s website since April 2000, so I’m a veteran HTML coder. There have been many changes in web technology in that time, and I’ve always kept up with new developments while being careful not to jump on the latest bandwagon too soon. I have a good knowledge of (X)HTML, CSS, PHP, MySQL, image editing, Apache server management, and web accessibility issues.

I’ve studied the Old Bailey Proceedings, a large collection of digitized source material marked up with XML, got acquainted with the Text Encoding Initiative standards, and have experimented with XML myself. Recently I volunteered my services to Distributed Proofreaders, and have already proofread over 100 pages of text. I’ll be continuing with this, and once I’ve done 300 pages I can get experience of the next stages of the process. I’ll also be reading A Companion to Digital Humanities, another free online book.

I’m breaking my own rule that you should never rely on just one person when creating digital texts, but this is an experiment to see how much I can do on my own. I also want to avoid the added complications of managing a collaborative project. If things go well, I can start to look at ways of working as a team on unofficial and unfunded projects.

Project Outlines:

Project Sandall: This project will produce a digital edition of T. E. Sandall, A History of 5th Battalion Lincolnshire Regiment (Oxford, Blackwell, 1922). The text will be marked up with TEI compliant XML and published on the web. It will include a hyperlinked index of people, places, and organizations.

Project Wenham: This project will digitize and publish the correspondence of my great-grandfather, William A. Wenham, relating to his experiences as a prisoner of war during the First World War. The collection consists of a number of letters and postcards sent to his family from prison camps in Germany, with some with photographs. The text will be transcribed and marked up with TEI compliant XML, and published on the web, along with background information written by me. There will be an index of people, and possibly places. Another optional extra will be selections of relevant documents from other sources, such as battalion war diaries.

Theory:

One of the great things about digitization projects is that you can avoid some of the more difficult theoretical controversies. The most relevant theory is information theory, and Shannon pointed out that meaning is irrelevant to information. The aim of text digitization and basic markup is to represent the information in the original document as accurately as possible, minimising the noise which can be introduced as part of the process (see my post on Historical Information and Noise). Once the text is published, individual users can decide what it means to them and how far, if at all, it relates to the reality of the past.

However, record linkage brings problems of meaning and epistemology into play. This is likely to be worse in early modern manuscripts which are more noisy to begin with. For example, the same name might be spelt several different ways. Dealing with 20th century sources is easier because of standardized spelling, but linkage still involves taking an epistemological position. One advantage of TEI XML is that it has a built in mechanism for representing uncertainty and quantifying epistemic probabilities using the <certainty> element. Furthermore, XML preserves the original text. Editorial decisions made during the tagging process have little or no impact on the integrity of the text itself. If the XML code is made freely available to users under GPL, they can download the source files and edit them according to their own needs. Therefore, users who don’t agree with an editorial decision can easily change it in their own local copy of the text.

Copyright:

I believe that T. E. Sandall’s work is out of copyright in most of the world, since the book was published in 1922 and the author died in 1931, but I will be looking for more definite proof before I proceed with the project. The copyright status of the epilogue written by G. H. Teall is unknown, but it can be omitted if I can’t determine that it’s definitely out of copyright. The plates are more problematic and will probably be omitted.

William Wenham’s letters are more straightforward. There is no possibility of anyone outside the family having a claim on the copyright and no possibility of anyone being able to make money out of their publication. The status of the photographs taken at Cottbus is unknown. They might be out of copyright or they might be orphan works which are still under copyright. In this case I’m prepared to risk publishing them, but will remove them if a genuine copyright holder objects to their publication.

Image Capture:

I have a cheap A4 size flatbed scanner which gives adequate quality. The main limitation is that it’s slow, but the quantity of images to be captured is relatively small. A digital camera can capture images faster, but the quality of text is lower than scanning and would most likely lead to more OCR errors. Sandall will be scanned as 300dpi monochrome JPEGs. There is no need to create a high quality archival copy for preservation purposes. The main aim of the project is to make the text more accessible and add value in the form of XML markup and indexing. The family papers will be scanned as full colour 300dpi TIFFs which will be kept as archival copies. These masters can be used to create 100dpi JPEGs for web publication. It is intended to make images of the documents freely available in addition to the digital text.

Digitizing Text:

The scanned images of Sandall will be converted to digital text using OCR software. I have two free OCR packages: IRIS (which came free with a scanner) and Microsoft Office Document Imaging, but both have limited features and would be inadequate for this project. Abbyy Finereader Pro has more advanced features and is recommended by Distributed Proofreaders. For this project I can use the free trial version, and if I want to do more projects in the future I can buy a licence for under £100. PC Pro magazine considers Omnipage to be superior, but it costs around £400! Since no OCR is perfect, it will be necessary to proofread and correct the entire text. Finereader asks for user input when it encounters a doubtful character, and its interface should allow manual proofreading alongside that.

There is no possibility of using OCR for handwritten text, so the letters will have to be transcribed. Although we have the originals to work from, there might be some need to work from digital images. Apart from saving wear and tear on the documents, digital images are more flexible. Difficult text can be enlarged on screen, and contrast can be adjusted to bring out faded text. However, there is the added problem of viewing an image and typing at the same time, which might require specialised software. I’ve found that Zotero notes can be very useful for transcribing text from images or PDFs and might be adequate to start with. I could also use HTML/PHP/MySQL to cobble together something like the Distributed Proofreaders interface for my own use. The front end is a simple web based interface, and although I don’t know how their back end works, a local version just for my own use could be much simpler.

XML Markup:

XML can be daunting precisely because it’s so powerful and flexible: where do you start and how far do you go? Using TEI will give me a basic framework which covers the needs of the project and guarantees a certain amount of cross-compatibility. However, there are many different ways to implement TEI compliant XML and many subjective decisions to be made. At its most basic TEI markup need not be much more complex than HTML, but at the other extreme there are provisions for extremely detailed and complex markup, such as stylistic analysis of the structure of sentences. I’ve decided to split the markup into three phases:

First: mark up the basic structure of the text, such as paragraphs, headings, and page numbers. It might be possible to do some of this automatically, but manual checking and correction will be necessary.

Second: tag names of people, places, and organizations. For the printed text of Sandall it might be possible to automate some of this using regular expressions and Find and Replace functions. Proper nouns can be expected to have capital letters, and ranks and common referents (he, officer, man etc) should be easy to detect. Again manual checking and correction will be necessary. At this stage names will only be tagged to indicate that they are names. No assumptions about identity will be made until the third phase.

Third: link/index records by assigning ID numbers and/or regularized forms to the name tag attributes. At this point some subjective decisions might be necessary, and certainty will have to be evaluated and assigned.

XML tagging can be done in any text editor, but a dedicated XML editor might be useful. I have previously used a free version of Altova XML Spy for experimenting with XML, but some more advanced features might be necessary so I’ve downloaded a trial version of oXygen. If it proves useful I can buy an academic licence for $48.

Web Publishing:

The documents and supporting material will be published in HTML or XHTML. All authoring will be done in XML, and XSLT will be used to transform the source code into other formats. For this reason there is no immediate need to decide whether to use HTML or XHTML, and in practice there is very little difference between XHTML and good HTML (anyone who has problems with the transition to XHTML has written bad HTML to start with!). The XML source files will also be made available so that users can download and manipulate them. Ideally the site would be published in XML only, with XSLT style sheets transforming the code to (X)HTML on the client side, but this would exclude older browsers which can’t handle XML.

There will be little need for a database back-end as the sites will be small and the content will not be changed or added to frequently. A search engine which recognises XML fields (as at the OBP) would be useful, but as names will be linked to an index page, Google might be adequate for full text searches.

Publicity:

This is not much of an issue as the content will only appeal to a niche audience and the projects do not have to justify themselves to funding bodies. The main aim is to develop and demonstrate my digital history skills, so if I get it right it will be a useful demonstration of how to do it. The main avenues for promotion will be my blog, other people’s blogs, and the Great War forum.

Bibliography

1. Daniel J. Cohen and Roy Rosenzweig, Digital History: A Guide to Gathering, Preserving, And Presenting the Past on the Web (University of Pennsylvania Press, November 2005).
2. Thomas Edward Sandall, A History of the 5th Batt. the Lincolnshire Regiment. by Colonel T. E. Sandall, Etc (pp. vi. 221. Basil Blackwell: Oxford, 1922).
3. Ray Siemens, John Unsworth, and Susan Schreibman, Companion to Digital Humanities (Blackwell Companions to Literature and Culture) (Blackwell Publishing Professional: Oxford, December 2004).