2. How does it work?
Before attempting to use Proai for your own OAI-PMH needs, you should have a basic understanding of how it works.
Overview
Proai is installed as a service that sits between a source repository and OAI harvesters. On the front end, it responds to OAI requests by consulting its local XML Record Cache. On the back end, it keeps this cache up-to-date by periodically communicating with the source repository through a repository-specific interface.
Front End
When a harvester sends an OAI request, Proai interprets the request and consults the record cache, which consists of database tables and XML record files on disk.
If a response exceeds a configured number of records, Proai
takes care of splitting it into multiple parts and generating a
resumptionToken for each part. In OAI-PMH,
this is known as
Flow Control.
The front end depends solely on the record cache. It does not directly access the source repository and therefore can do its job even when the source repository is unavailable.
Back End
The job of Proai's back-end is to keep the record cache up to date. This is done through a process known as the update cycle.
During an update cycle, the cache updater communicates with the source repository via the OAIDriver interface. An update cycle progresses as follows:
- First, the updater queries for the current list of all sets to see if any have changed since the previous cycle. If so, the database is updated accordingly.
- Next, it queries for the list of new/modified records and enqueues them for processing.
- Finally, the updater processes the queue, committing each change to the record cache. For each record, this involves getting the XML from the remote location, validating it, and removing it from the queue.