Each newspaper issue consists of a set of images, one per page, and a set of text files for the OCR'd text. An item file links the set of pages into a single newspaper document. PagedImagePlugin is used to process the item files.
There are two styles of item files, and this collection demonstrates both. The first uses a text based format, and consists of a list of metadata for the document, and a list of pages. Some examples are: Te Waka o Te Iwi, Vol. 1, No. 1 (in import/09/09_1_1.item) and Te Whetu o Te Tau, Vol. 1, No. 3 (in import/10/10_1_3.item. This format allows specification of document level metadata, and a single list of pages.
The second style is an extended format, and uses XML. It allows a hierarchy of pages, and metadata specification at the page level as well as at the document level. An example is Matariki 1881, No. 2 in import/xml/23/23__2.item. This newspaper also has an abstract associated with it. The contents have been grouped into two sections: Supplementary Material, which contains the Abstract, and Newspaper Pages, which contains the page images.
Paged documents can be presented with a hierarchical table of contents (e.g. 23__1.2.1), or with a single-depth structure (e.g. 10_1_2). This is specified by the -documenttype (hierarchy|paged) option to PagedImagePlugin. Ordinarily, a Greenstone collection would have one plugin per document type, and all documents of that type get the same processing. In this case, we want to treat the XML-based item files differently from the text-based item files. We can achieve this by adding two PagedImagePlugin plugins to the collection, and configuring them differently.
plugin PagedImagePlugin -documenttype hierarchy -process_exp xml.*.item$ ...
plugin PagedImagePlugin -documenttype paged ...
XML based newpapers have been grouped into a folder called xml. This enables us to process these files differently, by utilising the process_exp option which all plugins support. The first PagedImagePlugin in the list looks for item files underneath the xml folder. These documents will be processed as hierarchical documents. Item files that don't match the process expression (i.e. aren't underneath the xml folder) will be passed onto the second PagedImagePlugin, and these are treated as paged documents.
Formatting
Unlike in Greenstone 2, where the document formatting was modified to customize the display, in Greenstone 3 we rely for the rest on Greenstone's default behaviour.