It’s been a very long period since we blogged our progress. Rest assured we’ve been working hard to improve Greenstone 2 and 3, and it’s only the blogging about it that fell by the wayside. Here’s some of the things that I’ve been working on in the last few months:

• Securing Greenstone 2 pages: this involved significant changes to both the code and the macros files
• Updating the GTI Greenstone installation. Owing to the security changes for Greenstone 2, the Greenstone Translation Interface on nzdl needed to have the latest Greenstone2 to work again
• Fixing up a Remote Greenstone 3 authentication issue
• The FormatConversion wizard dialog. This completes the process of opening a Greenstone 2 collection in Greenstone 3. The format conversion dialog automatically generates Greenstone 3 equivalents for Greenstone 2 format statements behind-the-scenes, before presenting these to you. You can then interact with the wizard to improve any of the Greenstone 3 format statements that have been generated.

This week we’ve been hard at work testing Greenstone 3. We’ve managed to discover a few bugs and have fixed several of them already. While we’re at it we’ve also added horizontal classifiers into Greenstone 3 (frequent Greenstone 2 users may find this a very welcome addition). A horizontal classifier has the top level of the classifier (e.g. ‘A’ ‘B-C’ ‘D-F’ etc.) displayed in a row along the top of the browsing area instead of vertically down the page.

There’s not much more to report about my week as it has mostly involved testing and bug fixing, which is not particularly interesting to write about.

This week I have been testing and have fixed several minor bugs in Greenstone 3. I have also started working on a new feature (to be included in 3.06 most likely), which will theoretically allow much more advanced collection editing than what we’ve ever had before. At the moment a single page in a collection can be made up of many different files and if you want to edit something then you have to know which file it came from. For example, the basic file used for a document view is document.xsl (stored in the interfaces directory), but if someone wants to modify part of it then they could add their own template to their collectionConfig.xml file or they could create their own interface file and put it into the transform directory of their collection. This new feature will keep track of what file each change has come from and (probably) allow you to edit the file via your web browser.

While I am working on this feature Anu will be taking over testing for the release.

Sorry that I haven’t posted an update in a while, I keep forgetting to do it. Good news though, we are now in the final testing stages of Greenstone 3 and should hopefully have a 3.05 release candidate ready before the end of the week. We’ve also got some exciting things planned for 3.06 as well so we’re wanting to get 3.05 out as soon as possible.

There is not a lot to report other than that. Testing seems to be going well and assuming we don’t find any major bugs then keep an eye out for a release candidate in the near future.

Over the week, have been working on the activate.pl script (and things that it needs). The details are at http://trac.greenstone.org/ticket/825

For the latest changes made today, need to retest these changes against GS3 on Windows.

Still need to test the entire process on Linux.

• At the start of last week, finished off the task of the GS3 “debuginfo” button that now appears next to the login button.
• The Greenstone tutorial xml files can now include a MajorVersion element with number attribute to specify if the instructions are for GS3 or GS2 and will get processed by the XSLT to display or hide such elements depending on the active version.
• Joshua Scarsbrook discovered two bugs compiling GS3 on a Mac and has helped us fix these (but one of the fixes still needs to be tested on his machine). Unfortunately there were some issues with setting the Java preferences on my account on the Mac here. At present, GS3 can’t be compiled there because it requires Java 1.6.
• After Dr Bainbridge fixed error handling and display of the PDFBox Extension, it became easier to debug a PDFBox Extension bug discovered by a member on the mailing list. She helped us to track it down and it turned out that the PDFBox extension did not try to first look for and use any JRE included in a GS2 binary when running the java -version test.
• While trying to work out why searching 3 digit numbers crashed the server (when Diego wanted to try the ifl=1 parameter to the GS2 URL), I first found and tracked down a very troublesome bug that I had accidentally introduced into GS2. The documents in browse or search results would not display and their URLs looked strange (with the word handle in their path). It turned out that in January, I’d committed the -DDOCHANDLE option to CXXFLAGS in a win32.mak file that was meant for the experimental work Dr Bainbridge and Diego had been doing with REST URLs. I meant to commit only the RSS support code they had written. Dr Bainbridge then fixed the bug Diego had originally noticed to do with the ifl parameter.
• Some translation work and looked at a few mailing list questions.
• Currently started work on activate.pl which should perform in perl the task that GLI currently does of stopping the GS2 or GS3 server while moving the building to index and restarting the server again.

• <gslib:langfrag> was to replace a whole lot of XSLT statements instantiating a javascript array in header.xsl. It took a bit of messing about, but util.xsl now calls a new Java method to generate the Javascript array declaration and initiation, and this called from document.xsl and documentbasket.xsl (instead of header.xsl) using the new <gslib:langfrag> element.
• The gs2-library servlet was not working. After a lot of debugging, there were several fixes required. One was for the main servlet page, one for a collection page, one for viewing a browsing classifier. The major bug was in the way stylesheets were merged in GSXSLT.java. This has now been improved.
• During the above, the lack of clear error messages on where things were going wrong pointed to another problem: insufficient debugging information when XSLT transformations go wrong. XMLTransformer’s TransformErrorListener has now been improved (and TransformingReceptionist updated) to do better error reporting when transformations of in-memory DOM objects fail.
• Revisited recent unsatisfactory changes to the ant force-stop-tomcat target, since an ant restart did not always properly stop and start tomcat. Now the process has been improved by making the stop-tomcat target do a socket test in a wait loop to ensure that tomcat is properly stopped before proceeding. Subsequent calls to ant start therefore can only happen after this, even if tomcat commands are chained such as happens during “ant restart” or “ant stop start”.
• Running GS3-server.sh on Ubuntu wasn’t launching the browser. This was the old problem already seen in GS2 of the gs3-setup.sh script setting up env vars for wvware which then conflicted with the native libraries used by Linux for running graphics applications. The problem was already solved for GS2 by the introduction of the wvware.pl script which would set up the environment for wvware, and all that was required to get things to work for GS3 was to stop setting the env for wvware in gs3-setup.sh (and giving executable permissions to the wvware binary included in the Greenstone3 binary).
• A new target ant clean-logs has been added, which deletes catalina.out and greenstone.log and can help speed up the debugging cycle.
• Currently, am looking at a “debuginfo” button that will appear next to the login button and which display a page with the various options for o= that can be appended to the URL when debugging the XSL transformation process, as well as other URL suffixes.
• Once that’s done, I will return to testing the Greenstone 3 binary and start with attempting to perform the Greenstone 2 tutorials with GS3.

The very start of last week still required more work on the scripts that would handle translations made in Google’s Translator Toolkit. A couple of additional scripts were written.

Thereafter and until yesterday, the work has been mostly centred on GS3’s usersDB:

• getting the output of txt2usersDB to work as input to usersDB2txt and viceversa as well as allowing txt2usersDB to run in append mode,
• creating a new program to modify a user’s details in the DB which then gets called by the new targets config-user and config-admin (for setting the admin password) in build.xml
• getting the releasekit to update the admin password where the user provides this

There were a few questions in the mailing list that required some investigating, and today I finally got round to looking at getting Java to write out a bit of javascript that was previously done in XSLT where it looked unsightly and verbose. Unfortunately, I couldn’t test it when I tried out the Document Structure Editor. I got a blank section and could not type into this the new gslib:langfrag element that I was meant to test.

Instead, I decided to write some handy instructions into the Wiki’s Greenstone 2 FAQ which will help explain how to do some common tasks. The questions added to the FAQ are on how to manually build collections, how to get better error reporting in GLI, how to run GLI in debug mode, how to launch, use and copy from the DOS prompt, how to launch Windows Explorer and where to find the Windows key.  From experience, these instructions will be particularly helpful when answering Mailing List questions, as Greenstone users can be referred to these new FAQ items, armed with which they will then be better equipped to help us in the debugging process.

I’m writing this a day early this week because I will not be working tomorrow due to Easter, I still have a bit to report however.

This week I decided to re-think one of the previous decisions we made in regards to viewing documents in Greenstone 3. In Greenstone 2 you can normally only view one section at a time and therefore it is difficult to get a grasp of where you are in the document and also how big the document is. For this version of Greenstone 3 we decided that we would take the opposite approach and that, by default, viewing the document would always show the whole document instead of just a section. This approach works find for small- to medium-sized documents, but for larger documents the page begins to get too long. To resolve this problem I have decided to implement a mixture of the two approaches. Now, by default, when you view a page it will only show the section you asked for, but it will also give you a full table of contents and links on the page that allow you to view other parts of the document without needing to go to a new page. New sections are dynamically loaded when they are needed and there is also an option to expand all the sections if you do want to see the whole document at once.

Just a short entry today to say that this week has mostly consisted of working on minor features and fixes. All of the major new features are mostly done now and we are nearing the final testing stage before we can release Greenstone 3.