Fixed a server crashing bug, answered some questions on the mailing list, looked into a GS3 thumbnail issue which ultimately Dr Bainbridge ended up solving, and started on fixing the validation issues with the OAI server for Greenstone 2.

Mainly bug fixing.

There were a couple of outstanding unicode bugs that needed fixing (the rest were fixed the week before) such as in MetadataXMLPlugin. There were a few changes that had been made to GS3 which caused it to cease compiling properly which also required fixing. Finally, there was also the ticket where GS3’s List Users didn’t display all the details for a new user. That’s now been fixed too.

This week has been a very productive one for us, our list of tickets is down to only a few and soon we will be able to begin testing in preparation for release.

The Internet Explorer issue I mentioned last week turned out to be much more confusing that I had anticipated. Despite the fact that the servlet seemed to be getting the message to add a berry to the Berry Basket it would then report that there were no berries. After some debugging we eventually tracked the problem down to caching. The first time Berry Baskets is enabled it performs a call to the servlet asking how many berries there are (and of course there are none). Internet Explorer then cached this response so that the next time Berry Baskets asks what berries are in the basket it returns the same response without even asking the servlet, resulting in no berries showing up in the basket.

To fix this caching problem we decided to introduce a new parameter that can be used in a Greenstone URL to specify custom response headers. This parameter is specified in JSON and can take a list of header values. So to disable caching of the response we would use hhf=[{“name”:”Cache-Control”, “value”:”no-cache”}] in the URL of the page we request.

Another pair of tickets we closed were to do with changing various parts of the interface when a collection uses the Lucene indexer. For example, Lucene does not search for terms if they are too common (e.g. “the”, “a”, “of” etc.) so we need a message to show up on the search page informing the user that one or more of their search terms were too common and therefore were ignored.

Another minor ticket that was closed was allowing <store_metadata_coverage value=”true”/> to be placed in the collectionConfig.xml file to enable metadata coverage information to be stored at build-time. This feature already exists in Greenstone 2 and was not possible in Greenstone 3 until now.

Next week I will be working on the remaining outstanding tickets. If we manage to close them all we will move onto testing to make sure that there isn’t anything that we’ve missed.

For the first time in a month or so (maybe longer) I’ve actually managed to write this on a Friday instead of the following week!

Anyway, this week has consisted of more “oran” skin/interface development. A large part of this has revolved around repairing various aspects of the Berry Baskets functionality. Although the original Berry Baskets problem (that prevented it from functioning at all) was fixed a long time ago, new problems have been “created” with the recent changes to various parts of the interface and testing has revealed some problems we were not aware of until recently.

One of the largest problems we discovered was that the drag and drop Javascript code (the code that allows users to move berries to the Berry Basket) we use did not work at all in Internet Explorer 9, despite working fine in earlier versions of Internet Explorer as well as Firefox and Chrome. After doing some research on the subject we discovered that it was a problem with a Javascript library we use. We use the Yahoo User Interface (YUI) library to provide us with drag and drop functionality and we discovered that versions of the library before version 2.9.0 (the latest version at the time of writing this) all have this problem as well. As we were using YUI version 0.9.0 it was assumed that upgrading to this latest version would break other parts of the program. Fortunately this wasn’t the case and the upgrade went smoothly and the problem is now fixed.

With the recent changes to the way the classifiers (Subject/Organisation etc.) are displayed (see my previous blog entry) it was not surprising to find that Berry Baskets did not work properly with the documents displayed in that view. Fortunately, thanks to the previously mentioned problem we now have a much better understanding of the Berry Basket code and as a result it was fairly straight forward to fix this problem. There were also several more minor issues that have also been fixed.

Next week I will continue fixing up the interface and Berry Baskets as there is still more to fix (for example, in Internet Explorer the berry system does not properly maintain its state from one page to the next). Once these issues are resolved I will be able to move on to some of the other things we want to get done before we release 3.05.

As there wasn’t much to report two weeks ago, I will combine the previous 2 weeks together.

Work continues to go well on Greenstone 3, our list of tickets (basically our TODO list) is getting smaller at a steady rate. Only a few things remain to be done on the new skin and most of the other tickets are fairly minor in comparison to getting the new skin ready. So things continue to look good for a 3.05 release in the near future.

We found a good solution to the problem I discussed a few weeks ago, we now have it so that only sections can be collapsed/expanded in the table of contents and not the individual chapters. These can still be manipulated in the document by using the + and – icons next to the title of each chapter. The chapter links in the table of contents now simply takes you to that chapter by jumping you down the page, rather that the “focusing” idea we were considering, where clicking on a link would collapse all sections of the document except the one you clicked on. We also added a “back to top” link so that it was easy to get back up to the table of contents again.

We have used this same collapse/expand idea in the classifier pages (e.g. Subject/How to/Organisation  etc.) so that, rather than loading a new page when a user clicks a subject (for example) we now remain on the same page and “expand” the subject. This results in a much more intuitive way to browse subjects than the previous implementation which would only “expand” one subject at a time.

The query result page has been tidied up and now displays messages like “Displaying 1 to 20 of 100 documents” and has next and previous buttons go to the next and previous pages of results (this obviously existed in the old Greenstone3 skin but it’s new for this skin). If you enter only one or two search terms then it will display information such as “cat occurs 5 times in 3 documents”, however for three or more search terms it shows a compressed view such as “the (25739) farming (228) cat (5)”.

This week will mostly be focused on cleaning up the skin so that it will be easy for Greenstone users to modify it however they wish.

The last two weeks we were working on getting the OAI server for Greenstone 3 to validate against the on-line official OAI validation tool. There seemed to have been some code and configuration changes since the previous time the validation had been successful. Most of the changes were straightforward, though it took a while to work out what the OAI XML element in the build- and collectionConfig.xml files ought to be to set up an old demonstration collection for OAI (newly built collections have the element included by default). The final problem preventing validation had to do with how the earliestDatestamp for the repository was worked out and what the Datestamps for each document in a collection were. After fixing all the errors we had when trying to validate the GS3 OAI server, we finally succeeded in getting it to pass all the tests.

Work continues to go well on Greenstone 3. The new skin is looking really good and  – apart from a few small design decisions that have yet to be made – it is nearing completion. Currently what we have done is linked together the table of contents and the document so that when you close up a section in the table of contents it is also closed in the document (and visa versa), allowing you to more easily find the information you want, as well as providing a better overview as to what the document contains and how large it is.

One design decision we are currently contemplating is what to do with sections that contain other sections AND text of their own. For example, imagine a book has a chapter called Dogs, which has a section called Jack Russell. Lets say that the Dogs chapter has some text at the start of it describing dogs in general before moving on to specific types. If someone wants to collapse the Dog chapter they can simply click the – next to the Dog chapter in the table of contents, or if they want to collapse the Jack Russell chapter they can simply click the – next to the Jack Russell section. Doing it this way however does not allow the user to collapse just the text at the start of the Dog chapter, so this text will be there regardless of which section in the Dogs chapter they want to focus on, which is not ideal, especially if it is a long description.

The example above and other such design decisions are what we are currently working on solving, so that the new skin can be as easy and intuitive as possible.

Welcome to another late entry of my blog about Greenstone development. Early last week, the team had a discussion about modernising the Greenstone 3 interface. The improvements that I have made so far were well received and we discussed various ways to improve it further. For example, one of the things we are currently trialling is showing the whole document (rather than just the first section) when you visit a document in a collection. Each section can then be contracted or expanded in either the table of contents or the document itself (using little + and – symbols) to make navigating around the document more intuitive. When a search result returns a particular document section, that section is automatically expanded and the others contracted to provide a consistent interface.

We have also fixed up Realistic Books so that it works properly in the new skin, and we are also discussing ways for Realistic Books to be available by default in any collection (i.e. not only those that specify the correction option at build time).

The week started with small odds and ends: answering some questions on the mailing list, fixing Greenstone3 compilation so that a system-dependent file would get generated (instead of using the wrong ready-made binary version from SVN), checking that the port number could still be successfully changed on Greenstone 3 (we run “ant configure” to do so now). The latter half of the week was taken up by working on a couple of the tickets for Greenstone 3: While it turned out that export.pl functions well in GS3 from the command-line and through GLI, the GS3 OAIServer validation appears to be more involving. At present, though the initial operations have validated, more complex ones are still not working as they should. I’ll be continuing to work on this, this week.

Hi all, welcome to another Greenstone development blog entry. Last week we finished up our work helping the FLAX team create a binary installer for FLAX that would work for any version of the Mac operating system. We also mapped out what needed to be done before we can release Greenstone 3.05 and we have started working on that. So far we’ve made decent progress and things are looking good for a release in the near future.

One of the big things that I am working on at the moment is adding extra features to the “oran” skin, which is likely to become the new default skin for Greenstone 3. In particular I have been working on various features that are present in Greenstone 2 but not available in this development skin yet, such as search term highlighting and search term frequency.

So that is what we will be continuing to work on this week.