Beyond crowd sourcing

How do you coordinate contributions from a hundred editors and ensure the quality of the resulting archive?  That's a challenge we face thanks to the success of the past several years of summer seminars at CHS.

The solution we've designed enables scattered teams using virtual machines to work in a collaborative work flow and document their progress in publicly visible github repositories.   The nuts and bolts of the process are increasingly thoroughly documented  (special thanks to project manager Stephanie Lindeborg and the summer 2015 team at Holy Cross for their invaluable contributions).  While this challenge applies to any collaborative digital project,  the HMT approach seems to stand apart from other digital projects, so I've posted a long overview of the technical design of our validation and verification system on the HMT github site.

The important conclusions: while a single book of the Iliad can easily surpass 10,000 words of text in a manuscript like the Venetus A, the HMT project's validation system  ensures that every word can be tracked to a region of interest on an image, and that both text and image are connected to a specific page of the manuscript by a syntactically valid URN that cites an object that really exists in the HMT archive. Every word of every text is tested against rigorous criteria that are specific to the type of the word.  Automated validation and computer-assisted human verification put the HMT archive on a solid foundation.

Verifying an inventory of scholia

One of the most important tasks the Homer Multitext project has been addressing is to compile a complete inventory of the scholia in the manuscripts we are studying.  Remarkably, this has never been done, even for the much-discussed Venetus A manuscript.  (The most thorough and accurate edition to date, Dindorf's admirable two-volume Scholia Graeca in Homeri Iliadem, excludes interior and exterior scholia from consideration.  As to Erbse's Scholia Vetera in Iliadem, often misunderstood by classicists as systematic, or as an attempt to create a comprehensive inventory, a detailed comparison of our edition of Iliad 3 and 4 with Erbse's text showed that he publishes only about 80% of the scholiastic text in the Venetus A.)

Venetus A, folio 12 recto, with the first 25 lines of the Iliad; overlays show the location of scholia, color-coded for their class of placement on the folio.

Since the summer of 2010, we have used a system of machine-assisted visual proofing to help verify that we have indeed included all scholia on a given folio in our inventory.  HMT editors create structured inventory notebooks that include a citation of visual evidence for each scholion they identify.  These notebooks can be transformed into web pages with images illustrating each folio, with partially opaque overlays showing the position of each scholion.  Editors can see at a glance whether any areas of a folio page have text outside an overlay.

This summer, as part of our effort to improve the automated management of our archival data, we have added an automated check that verifies the syntax of every image citation.  Yesterday, I completed a review of inventories for Iliad 1-5 in the Venetus A.  3503 out of 3505 entries (99.9%) were syntactically valid:   clearly, visual proofing is a pretty effective method of checking syntax as well as finding missed areas on a folio page.  Next, we plan to package the verification tool in an expanded tool kit for HMT editors, so that they can validate the syntax of their citations before ever submitting a notebook for review.  (That or course means:  they will be required to validate that 100% of their citations are syntactically valid before submitting a notebook for review!)

Of course we cannot be sure that this method will find scholia when they are nearly invisible on our photographs.  Participants in the 2011 summer seminar at the Center for Hellenic Studies made the alarming discovery that they had missed a number of exterior scholia visible in the 1901 facsimile edition by Comparetti.  It is clear that in the century since Comparetti, the small scholia on the edges of the manuscript pages have suffered the most, and we have since established a practice of routinely checking each inventoried folio against Comparetti's facsimile.  In some cases, our ultraviolet photography preserves legible text.

What does an inventory of 3500 scholia look like?  I've posted one visualization:  a pdf that offers a kind of "flip chart", of folios 12 recto through 80 recto of the Venetus A (that is, Iliad 1-5), with a very small version of our overlay image for each folio side.  You can see that visualization, as well as some work in progress from the summer of 2012, here.