It’s been about 4 weeks since I wrote an entry. In the meantime we’ve been tidying up the last of the To Do list items for the upcoming GS2 release and several of the To Do list items for the GS3 release. Sam is now hard working on the GS3 interface alongside his other work on the Document Maker. It now looks like GS3 may be released separately, after GS2.

Some of the more involved things that required doing were:

• testing OAI (dc.Resource Identifier issues) and downloading over OAI
• The extracted embedded metadata, ex.*.metadata (e.g. ex.dc.* prefixes), needed to be handled different from ex.metadata. This required some changes in various files and a lot of testing.
• Conflicts between EmbeddedMetadataPlugin and some of the existing Plugins in the pipeline (OAI, DSpace, PDF plugins). Fortunately, Dr Bainbridge came up with fixes. After some testing, the known problems with these plugins no longer exist. With the tutorials we will continue to investigate how well other plugins interact with the EmbeddedMetaPlugin.
• The OAI validator at openarchives now had a test where GS2’s OAI server failed and a different one where the GS3 OAI server failed. These have been fixed up.
• The GS3 installer needed to have an admin page, like the GS2 installer does, where the user can enable admin pages and provide a password.
• wvware.pl is a new intermediary script to launch wvware in its own particular environment. This script is necessary in order for wvware’s required environment not to be set globally (thereby tampering with Linux’ windowing/GUI libraries)
• At the moment, after John Rose’s request, we’re in the process of merging the two server configuration files (glisite.cfg and llssite.cfg), so we can have just one, with some properties qualified by a “gli” prefix. The Server.jar code, the GS2 C++ code, the startup scripts and config files have been sufficiently modified to work with the work-in-progress on the GLI code, while still working with the stable GLI. Changing the GLI code was tricky two years ago, and made the code’s behaviour rather  complex. Now that I’m in the process of testing the latest overhaul to it, the changes I’ve just made to what was stable are still very buggy and reproducing the bugs takes some time. Fortunately, without the changes to the GLI code, everything else committed is able to work as accurately as before, which is fortunate since if I break anything, it will be just the LocalLibraryServer.java GLI code that once committed needs to be reverted.
• The above task has now been completely resolved, and changes committed after being tested thoroughly on both Windows and Linux.

Minor issues also kept popping up over the last month.

• There was a Z3950 “issue”that sidetracked me and which turned out not to be an issue after all: The Library of Congress’ Z3950 address seems to return SRU data. The fix is simply for the user to use the right module of the download pane.
• A bug in starting and stopping GS3 via GLI on windows
• One Greenstone member encountered a unicode issue that I wasn’t able to reproduce after initial investigations.
• Minor but frustrating bugs with the GLI for GS3 have been resolved (an extra nested <format/> tag appearing when all format statements have been removed, and the preview button activating itself when editing format statements in an unbuilt GS3 collection)
• Fixed GS3’s way of handling the port in the GSI application, so that it is no longer arbitrarily modified. The Do Not Modify port is still available.
• Some requests on the mailing list like porting indexed databases from one GS2 version to the next, since changes had been made to the name of an ex.metadata

Since the last time I wrote I have mostly been continuing work on the Document Maker. The back-end (the part that does all the hard work) is around 80% complete and is ready enough for me to start working more on the front-end (the part that makes the back-end easier to use).

I have also been tidying up some of the Format Manager work that the other Sam did before he left. He had modified the JQuery UI source code so that it allowed multiple nested lists of items (basically lists inside lists, which the original code did not allow for). The only problem with this approach (directly modifying the source code) is that it does not allow us to easily upgrade our version of JQuery UI in the future. To remedy this problem we downloaded the original JQuery UI source code and worked out what parts of the modified code we needed to keep. We then took these parts and put them into a different Javascript file and used the prototype functionality of Javascript to make sure that the modified code would overwrite the original code.

We originally tried contacting the JQuery UI developers to see if we could get Sam’s changes included in the official source code but they responded saying that this new list-inside-list functionality was outside the scope of what the original lists were intended for.

The Format Manager is still very much a work in progress and, although it will be included in the next release of Greenstone 3, we will be still recommending that people make their format statement modifications through editing the collectionConfig.xml file at this point.

Since my last post I have been working hard on the new Document Maker functionality that is planned for a future version of Greenstone. So far I have implemented the ability to create new documents, create new document sections, delete documents, delete document sections, copy documents, turn a document into a section of another document, turn a section of a document into a document and the ability to copy a section from one document into another document. Also planned is the ability to move documents and sections (basically the same as the copying operations except the original document or section is deleted afterwards); the ability to merge sections together or to split them apart; various document manipulations such as the ability to get and set metadata and the ability to get and set the document content.

The plan is that this Document Maker functionality will be presented to the users via a web interface, allowing users to modify their documents on the fly. We imagine that this functionality will be very useful to people who want to be able to create organised collections out of large, unorganised sets of text and images. One such example of this is the Pei Jones collection which is made up of many individual letters, photos and articles that have been OCRed.

Last week started off with requiring fixes to a bug introduced during recent GS3 code changes: suddenly metadata and titles were no longer being retrieved for normal search and browse operations. Then Sam’s recent improvement to GS3’s GLI by starting the tomcat server upon GLI startup was expanded to also stop the tomcat server on GLI’s exit.

Then it was time to move back to GS3 XSLT files once more. Recently, changes were made to GS3’s old standard skin (gs3library) XSLT files, so that the features exhibited in the DSpace Tutorial would work for GS3 as well. These changes needed to still be ported over to the new standard skin for GS3, currently called “oran” (its servlet is called “dev”). However, in trying to make sense of how to do this, it was discovered that the default dev servlet was not set to use Sam’s excellent default GS3 interface for dev. Because GS3’s format features need to be customisable, having any format statements in a collection’s configuration file would bypass Sam’s interface to show up a default one. However, this default one was not working at this stage. This was therefore fixed up to get back some rudimentary behaviour not unlike what GS2’s interface offers for hierarchical browsing and search results. To use Sam’s interface, all users would need to do is use GLI to delete any format statements in a collection’s config file.

In looking into this matter, a further minor bug was discovered in classifier.xsl that was also fixed.

Porting the GS3 changes made for DSpace tutorial into the new default skin later had to be continued later, since there was some incomplete work awaiting finishing: the week ended with continuing work to do with working with embedded metadata (such as of the form ex.dc.*).