Hello Greenstone users, welcome to week 4 of my weekly blog about Greenstone development. This week I have spent most of my time working with the developers of FLAX to create an installer that will work on the most recent versions of the MacOS operating system. Those of you that use FLAX may know that it is built upon Greenstone 3 and, up until this week, the developers of the FLAX have used a modified version of the system that is used to create our Greenstone binary installers (which we internally call the release kits). I have been working with them to merge their modifications into the main release kit code so that they no longer need to keep a modified version and can simply have the release-kits enable FLAX compilation with a single switch. As part of this, several extra features were added to the Greenstone 3 compilation script, such as the ability to disable GLI and GEMS, and disabling JNI now correctly excludes the files that use it from the compilation process.

We now know enough about the Mac issues I discussed in my previous blog entry to create a version of FLAX that will (hopefully) work correctly on both the older versions and the new versions of the MacOS operating system. This is still an unsolved issue for Greenstone 3, but this is a low priority for us at the moment as the majority of our focus is on releasing Greenstone 2.84, which is currently in its final stage of testing.

See you all next week!

I don’t really have a lot to report from last week as I was away sick for some of it. We have been investigating problems with Greenstone 3 on the most recent versions of Mac OS. As some of you may know, Greenstone 3 is programmed heavily in Java which is ideal for cross platform development. Unfortunately for us, Apple recently decided to deprecate Java on their operating system and – to cut a long story short – it makes things much more messy for us. There is also an issue on some Macs (presumably ones with later versions of the compiler we use), where they attempt to compile Greenstone as if it is 64-bit. Although we are currently working on a pure 64-bit version of Greenstone it is currently not at the stage where it is ready to be included into the main part of the code. Unfortunately we only have limited access to computers with the latest Mac OS, so finding a solution to this problem may have to wait till we can get more permanent access to a machine like this.

Anyway, see you next week!

Welcome to week two of my new weekly blog about developing Greenstone.

This week has been quite successful as I have managed to achieve all of the things I set out to do in the previous blog and more. The ‘Oran’ skin is nearing completion now, we just need to get some real users in so we can evaluate both its functionality and its appearance.

As well as there being the small Javascript error in the Berry Basket code that I mentioned last time, there was also a bigger issue that was preventing us from displaying the quick search box on one of the Berry Basket pages. As some of you may know, Greenstone 3 uses technologies such as XML and XSLT to generate the web pages of the Greenstone Reader’s Interface. Greenstone 3 itself is made up of various Actions (that produce the XML for various types of pages) and Services (used by the Actions to do the hard work like retrieving document content or perform a query on a collection), one of which being the Berry Basket service. The quick search box requires information about TextQuery service in the collection that the user is currently within in order to generate a search box that will search that specific collection. The way one of the Berry Basket pages is created however does not make it easy for this collection information to be accessed. It uses a General action who’s ‘c’ parameter stands for Cluster instead of the more-standard Collection, so simply giving the action the current collection in the ‘c’ parameter was not sufficient as it would be for most other pages. We eventually decided that the tidiest fix was to save the collection in the ‘p.c’ parameter and to modify the Receptionist (the part of the code that produces the necessary extra information about that collection) to first look for the current collection in the ‘c’ parameter and if that did not exist then use the ‘p.c’ parameter.

If you’re still here then well done! My average day as a programmer consists of solving lots of these little problems. It may not sound very interesting to the average person but there is definitely a good sense of satisfaction when you find a way to solve these complex problems.

This week I also added a handy little feature that will prompt a user if they make a change in the preferences page and try to leave the page without hitting the “Set preferences” button. It asks them if they want to save their preferences, and if do it will go ahead and save it for them and take them to their new page. Doing this in a way that worked happily on all of the major browsers (Internet Explorer, Mozilla Firefox, Google Chrome and Apple Safari) was actually quite tricky. One way that I implemented worked happily on IE and Firefox but refused to work on Chrome/Safari. This ended up requiring a whole different approach, (involving a synchronous AJAX call in the “beforeunload” Javascript event for those who have some Javascript experience).

Anyway, that’s enough for this week. See you next week!

Ok, so one of the things us Greenstone developers are going to try to do from now on is blog each week about what we’ve achieved that week. Ideally we’ll try to do it on Fridays, before we leave for the weekend (so this entry is a few days late).

This week I have been working hard on the development of the new default skin for Greenstone 3 – codenamed the ‘Oran’ skin. For those of you who don’t know, Oran worked here before me and I effectively took his place when he moved on to a new job. One of the things that he had been working on was this new and fresh-looking skin for Greenstone 3, and this week I have been working hard to get it ready for the Greenstone 3.05 release which will happen later in the year.

One new feature that is worth mentioning is the addition of a quick search area that is permanently in the header of the page (assuming the collection has basic query capabilities). So regardless if you’re browsing or reading a document, a basic search field is always available.

I have also been working on fixing the Berry Basket feature that was added into Greenstone 3 a while ago but had stopped working due to other changes that had been made. Having fixed the main problem (a small javascript issue) I then discovered that it did not work correctly in Internet Explorer. So this week I will be focusing on that and other Greenstone 3 user interface related issues.

We are pleased to announce that an overview of results from the 2009 Greenstone User and Developer survey is available online at: http://greenstonesurvey.wordpress.com/greenstone-user-and-developer-survey-results/.

Through the survey, we were able to gather information from a geographically dispersed group of organizations and individuals who use Greenstone Digital Library Software. A primary component of the survey focused on how support resources are used and how current resources meet user needs.

The overview includes results from all major sections of the survey, which were the following:

• Background information (modes of use and basic use demographics);
• User Satisfaction;
• Support Mechanisms and Resources;
• Organizational Contexts;
• Technical Environments (Version Use and OS);
• Collections and Audiences; and
• Developers.

We’d like to thank respondents for participating in the survey and sharing information about their Greenstone experience!

Please take the Greenstone User Survey!  10-15 minutes of your time can help us learn more about Greenstone software users and developers. With your help, we will gain a better understanding of Greenstone software and support resource use and satisfaction; in what ways Greenstone users interact with and use the software; and the organizational and technical environments in which Greenstone software is used.

If you work with Greenstone to:
•    Develop digital library collections;
•    Teach about digital libraries;
•    Learn about digital libraries;
•    Develop language interfaces for Greenstone;
•    Develop other functionalities or code for Greenstone;
•    Disseminate or otherwise support use of Greenstone,
You are eligible to participate in this study.

How to participate:
Read information about the survey and your rights as a participant at:
http://greenstonesurvey.wordpress.com/infosheet/.
Then click the link at the bottom of the survey information page to continue to the survey.

In addition to the online survey, a pdf version is also available.
Contact Laura Sheble at [email protected] for a copy of the pdf survey.

Want to do more?
Help us distribute the survey:  Inform other Greenstone software users
and developers about this survey by posting a link to this blog entry
or by sending an email with the information above. Thank you for your help!

Would you like to have a Greenstone3 server in your pocket? Now you can with our port of the run-time system to Android. Fire-up Greenstone3 on your mobile phone and then access it just like any other Greenstone server, searching and browsing multimedia collections. You can connect to it over a wi-fi network, an ad-hoc wireless network (device-to-device) or via a USB cable.

You may have read about our earlier success in porting Greenstone2 to Apple devices. We have Greenstone2 running on early (3rd to 5th generation) iPods (see details of our demo) and the iPod Touch (see our paper here). Now Greenstone3, our next-generation digital library software, runs on a mobile handset.

Specifically, it runs on an HTC G1 Android-powered mobile phone. Android is a project of the Open Handset Alliance, and is an open platform for mobile devices.

How does it work?

Collections are built on a desktop computer in the same way as with standard Greenstone3. It is the runtime code that we have ported.

Greenstone3 conventionally runs as web-application of a Tomcat web server. However, it’s not tied to Tomcat, and can be used with an alternative web server such as Jetty . Of course, these and other desktop web servers aren’t going to run on a mobile device. Fortunately the people at Webtide have created i-jetty , a port of Jetty to Android, which solved our mobile web server requirements.

Greenstone3 is written in Java, as are Android applications. Normally though, Greenstone3 uses mg++ for indexing and GDBM as its database. Unfortunately both are written in C/C++ which isn’t much help when a totally Java runtime is needed. However, Greenstone3 supports the use of Lucene (indexing) and JDBM (database), both of which are Java. Using these we can build collections such that only Java is required for a fully functional Greenstone3 server.

However Android Java isn’t exactly the same as desktop Java, so some modification of the Greenstone3 runtime source code was required. This mainly relates to as yet unimplemented aspects of Android Java and its limited supported for XML processing. Some workarounds were required because of the limited memory (192Mb RAM) and processor power (528MHz) available on an actual handset.

The runtime is compiled into a JAR file. This and other necessary Java libraries, along with the standard Greenstone3 ‘web’ directory (which includes the collections) is organised into an i-jetty web application directory structure. i-jetty provides a utility to combine this into a WAR file, with Java classes converted into the byte code required by the virtual machine running on the Android device. This is then transferred to the SD card on the phone.

i-jetty is then launched on the phone with Greenstone3 available as a web application and accessible from a web browser by specifying the phone’s IP address and the webapp context as the URL.

At the moment the code is in pre-alpha release state. It works but needs some further debugging and optimization. When it’s ready we’ll make it available separately from the standard Greenstone3 distribution but eventually we’ll integrate it to the Greenstone3 package.

Any enquiries, technical or otherwise, should be sent to [email protected].

Acknowledgement: purchase of the G1 handset was supported by the ICT Science Kudos Award 2008.

The ideas encapsulated in the Greenstone Librarian Interface (GLI) are now available for people working with the Fedora Digital Repository system. Currently the software is checked into the SVN repository — we will produce binaries for this as part of the next general Greenstone release. For now, if you’re interested to try it out, follow the instructions for checking out Greenstone (works with either Greenstone 2 or 3). You will also want to install a version of Fedora.

When you check out the Greenstone Librarian Interface part of the SVN install, you also get all the necessary files for the Fedora Librarian Interface . Once you have the code compiled, where you would usually run ‘gli.sh’ (for Mac or Linux) run ‘fli.sh’. For Windows, it is fli.bat

The OAI Visualisation and metadata analysis tool has been upgraded with:

• lists of potential duplicate values for each element (using approximate string matching, i.e. edit distances)
• lists of records that are missing particular elements
• better linking to source item records
• greatly improved stability

This new alpha 2 version is now running at the same URL:

http://www.nzdl.org/greenstone3/mat

Starting now, nightly “snapshot” releases of Greenstone3 will be constructed and made available on our snapshots page.

Every night, the most recent revision of the Greenstone3 source code will be automatically checked out, compiled, and packaged up as an easy-to-use installer and put up on our website. These snapshots will be created for Linux and Windows, and soon, Mac. They will be made available in two formats: as an executable Jar, or as a native binary for your operating system. And to make things even easier, there is also a native binary which comes bundled with Java.

As of today, you no longer have to wait for a release to come out before you can take advantage of a new feature of Greenstone3. All you have to do is install the latest snapshot release, and you will have all the features added up to the previous day.

Nightly snapshot releases were made possible by our move to the open source installer Ant Installer. We have been able to construct a release “kit” for each operating system, which automatically creates releases of Greenstone3 using a number of Ant scripts, the Ant Installer software, and a simple executable wrapper program. Since creating a release is now as easy as running a single command, we decided to let the computer create them nightly.

Not surprisingly, the snapshot releases are likely to have a few flaws. These flaws will have two sources: bugs in the release kits, and bugs in the Greenstone code. As far as the release kits are concerned, in time we aim to eliminate all the bugs, so that they produce robust and usable installers. But as for the Greenstone code, we recognise that the most recent revision of the Greenstone code will always have some “bleeding edges”, so we do not expect to ever eliminate all the bugs there. (If you want thoroughly tested releases, go for one of our regular releases, not a snapshot.)

You can help us to perfect the Greenstone3 release kits by downloading and installing a snapshot release, and telling us if you hit any problems. Go on, be brave!

(For those interested, instructions for getting and using the release kits themselves are available at http://wiki.greenstone.org/wiki/index.php/ReleaseKits.)

Also, keep a look out for Greenstone2 snapshot releases and release kits, which are coming soon.