New version of JZkit Proxy Server

An updated version of the JZKit Proxy server (With an example configuration exposing a number of SOLR backends as a z3950 target) is available for download Here

Since the initial test at the Bath CRIG barcamp we've improved the error handling and diagnostics, improved the mapping of Z3950 database names, made the attribute mapping more configurable (The out out the box sample config is pretty much Bath profile level 1 OK), improved the structure==year mappings for date search and done some pretty hefty load testing which revealed a thread leak in certain situations (It's fixed now).

Please download, play and get in touch with any questions / comments.

Ian.

JISC CRIG DRY / IEDemonstrator BarCamp - Testing the Z3950/SOLR Bridge

The JZKit to SOLR Z3950 bridge has had it's first real foray outside of internal k-int projects by providing a z39.50 server to cross search Intute and the Oxford University Research Archive in a single z39.50 search. Been itching to try this for absolutely ages, and the CRIG/DRYbarcamp was the perfect opportunity to talk to the projects and quickly hack out the config needed to set up the server.

Here's the proof of the pudding, a z39.50 search for title=science translated into solr searches against the two repositories and integrated into a single interleaved result set. Easily integrated into any meta-search engine implementing z39.50. Cool!

Z> open tcp:@:2100
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 174
Name   : JZkit generic server / JZKit Meta Search Service
Version: 3.0.0-SNAPSHOT
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets negotiationModel
Elapsed: 0.006905
Z> base oxford intute
Z> format xml
Z> elements solr
Z> find @attr 1=4 science
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 803, setno 1
records returned: 0
Elapsed: 0.003859
Z> show 1+10
Sent presentRequest (1+10).
Records: 10
[SOLR]Record type: XML
No Third Party copyrightGordon L. ClarkGordon L. Clarkora:general>falsegeneral1982John Wiley & Sons, Ltd.D. T. HerbertD. T. HerbertR. J. JohnstonR. J. JohnstonSt Peter's College1982Social Sciences Division - Environment,Centre for the - Geography,School ofUniversity of OxfordGordon L. ClarkD. T. HerbertR. J. JohnstonJohn Wiley & Sons, Ltd.Transformations: Economy, Society and PlacePublisher has copyrightPeer reviewedGeographySocial Sciences Division - Environment,Centre for the - Geography,School ofClarkHerbertJohnstonGordon L. ClarkGordon L. ClarkD. T. HerbertD. T. HerbertR. J. JohnstonR. J. JohnstonJohn Wiley & Sons, Ltd.John Wiley & Sons, Ltd.Book (monograph): Section of book or chapter of bookGordon L.D. T.R. J.http://ora.ouls.ox.ac.uk:8081/10030/2034Geography and the urban environment : progress in research and applicationsuuid:89494f00-a92b-4914-8f9a-bcede5a4d9beUniversity of Oxford04711022531982enSt Peter's College41-61trueGordon L. ClarkD. T. HerbertR. J. Johnstonuuid:89494f00-a92b-4914-8f9a-bcede5a4d9beJohn Wiley & Sons, Ltd.Transformations: Economy, Society and PlacePeer reviewedPublisher has copyrightPublishedGeography2008-06-02T07:36:29.419ZPolicy science and instrumental reasontextuuid:89494f00-a92b-4914-8f9a-bcede5a4d9be5http://www.geog.ox.ac.uk/staff/glclark.phphttp://eu.wiley.com/WileyCDA/Section/index.html
[SOLR]Record type: XML
ScienceScienceurn:ISSN:0036-8075oai:epubs.cclrc.ac.uk:serial/1252oai:epubs.cclrc.ac.uk:serial/125257STFC ePublication Archive57-oai:epubs.cclrc.ac.uk:serial/1252ScienceScience

Exposing SOLR service(s) as a Z3950 server

JZKit is a pretty large toolkit for developers of search services to embed in their own systems, and it's not always easy to get to grips with ;). Partly thats because if you're using JZKit you're probably already dealing with the Z39.50 specifications along with a host of other concerns.

What developers need are simple starter apps that they can use to hit the ground running. In JZKit3 we've decided to try and address this by putting up some sample configurations of the tool that do useful stuff out of the box. First up.. Making a SOLR server visible as a Z39.50 Server using an easy to change XML config file. Why do that? Well lots of reasons, the most common one at the moment is that SOLR is being used to provide search interfaces into lots of interesting new content, not least of which are the whole new breed of digital object repository projects like Fedora and DSpace. What seems to keep coming back around is institutional librarians saying "But I want that content to be available along-side everything else".

Kewl, a problem we can do something about. For now, the gateway distro lives in our maven repository Here. If you fancy setting up a Z39.50 server to proxy for a local SOLR server, download the compressed tar file from the above URL and unpack it.

Configuration.

After unpacking, look in etc/JZKitConfig.xml you'll see a number of elements which define the SOLR services we want to search. (Actually, you can define all kinds of searchable resources here, not just SOLR, but other Z39.50 targets, SRU/SRW, OpenSearch, etc but thats for another day). You should be able to see the top points at a public SOLR server, adjust the URL to your local repository. The other elements inside the Repository element control the mapping of Z39.50 use attributes onto the SOLR search language and what record elements are requested as a part of search results. The code attribute specified what Z39.50 Database name this resource will be made available under. Having adjusted the config to point at a local server, cd ../bin and run "jzkit_service.sh start" this will give you a running Z3950 server, by default running on port 2100.

From here, it should be plain sailing, here's the output of a yaz client session:

(N.B.) XML Markup in the result record is being filtered out by blogger. Actual result record is XML (Actually, in this case it's just the SOLR element)

yaz-client tcp:@:2100
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID : 174
Name : JZkit generic server / JZKit Meta Search Service
Version: 3.0.0-SNAPSHOT
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets negotiationModel
Elapsed: 0.006948
Z> base Test
Z> find @attr 1=4 Dell
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 1
records returned: 0
Elapsed: 0.002231
Z> format xml
Z> show 1
Sent presentRequest (1+1).
Records: 1
[coll]Record type: XML
3007WFPDell Widescreen UltraSharp 3007WFP6
nextResultSetPosition = 2
Elapsed: 0.008594
Z> quit

Other components of the config file can be used to control the XML records returned, to convert SOLR records into MARC or to manage meta searching through the gateway, but thats a subject for next time.

Happy meta-searching.