Investigations of a Dog

Exciting news via Read Write Web: Yahoo announced last week that their search engine will start searching semantic markup. Initially they’re concentrating on Microformats but eventually they will be supporting lots of other metadata standards using RDF. This opens up a lot of possibilities for finding information and making information easier to find. Although Yahoo Pipes previously offered powerful tools for doing things with structured content, one of the drawbacks seemed to be that you had to already know where to find the content you wanted to do stuff with. Once Yahoo search results take metadata into account it will save an awful lot of work.

For historians, learning new search techniques to take advantage of this development will be vital, but that’s the easy (or at least less difficult) part. There is even more work to be done on the side of creating metadata: putting adequate metadata into the digital resources we create ourselves, encouraging organisations to provide metadata in their digital collections (that means you, CWGC!), and above all helping to define standards for historical information so that the metadata is as useful as possible. This is exactly what Tom Scheinfeldt was getting at in his recent post at Found History. Although I took issue with some of the things he said about ideology, his central points were spot on: history hasn’t always been done the same way and won’t always be done the same way; digital technology is changing things right now; organizing, cataloguing, and collaboration will become more important than lone researchers working on monographs.

Having made good progress with my project to digitize Sandall’s History of 5th Lincolnshire Regiment in the last month I’m going to leave it for a while. This month I haven’t read any books or articles, haven’t written anything other than blog posts and computer code, and have only occasionally thought about historiography and theory. I kind of like it like that but I have other things to get on with now.

I’ve made some small changes since the last post. Dates now have tool tips, so if you hover over them you can see the full date. The place name index is a bit more user-friendly. I’ve replaced the hash values with query strings in the incoming links so that the Exhibit page filters the list down to the place passed in the query instead of displaying a box with the details. This means that you just have to click on “Map” to go straight to map view with only that place displayed. Once you’re there you can easily take the filter off again to see all the other places. The map view is also zoomed out further by default so that you can see Britain and Egypt. That means that you have to zoom in a long way to get to France and Flanders but I think it’s less confusing than not being able to see Grimsby or Alexandria unless you zoom out.

So the site is now in a satisfactory condition with lots of cool features, and now that I’ve worked out how to do everything I could probably get another book to the same stage within a few weeks. But there are still lots of features that could, and probably should, be added. See below for more details. (more…)

I’ve just finished clearing out my bookmarks folder and putting my links onto del.icio.us. I don’t know why I haven’t done this before as it’s much better than keeping over 300 links in my Firefox bookmarks. From now on bookmarks are only for the few sites that I access most regularly. Zotero is for pages that need to be dealt with in more detail, with snapshots, notes, and annotations, or which need to be kept together with bibliographies for projects that I’m working on. And everything else goes on del.icio.us. While I was rearranging everything I took the opportunity to add some of the best sites to History Nexus.

Some other cool things:

Operator is a Firefox plugin which detects and displays Microformats. Microformats are a simple way of embedding metadata in web pages using only HTML.

Firebug is another Firefox plugin, a bit like the Web Developer toolbar but much more powerful. It lets you inspect the code of a webpage with expanding and collapsing tags, highlights the current element on the page, displays all CSS styles which apply to an element, debugs Javascript, and even lets you rewrite the code on the fly! I’ve found it very useful for developing Exhibit pages, and it would also make it a lot easier to design or modify WordPress themes.

Yahoo Pipes is a set of tools for data mining and mashups. It’s kind of what I was wishing Google would do in a previous post. It looks complicated, but still easier than programming from scratch, and very powerful. I’ll be trying it out whenever I get time.

I’ve been playing with Google Custom Search and although it’s good it would be much better if it could recognize metadata in microformats, RDF, or any other formats that metadata might be found in. And if it could also scrape data off web pages using regular expressions (sort of like Feed43 but better). And if you could create custom search fields and define how they map to Google Base fields, Freebase fields, metadata tags, and scraped data.

And I want the moon on a stick.