Community:HarvestManager/documentation/usagedocs
From NSDLWiki
Contents[hide] |
Harvest Manager User Guide
The Harvest Manager is a Web application used by collections staff to manage and coordinate the ingest of metadata collections into the NSDL Data Repository. It displays information about each OAI collection associated with the repository and triggers the OAI ingest process to perform harvests at regular intervals or manually using the OAI settings obtained from the NSDL Collections System. The user interface provides tools to explore each OAI data provider, view details and histories of past OAI harvests, and, for authorized staff, to trigger harvests manually for testing or for library ingest. When a harvest is completed, the Harvest Manager notifies staff via e-mail and provides detailed information about the status of the harvest.
It is assumed that users have a prior knowledge of the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). For certain tasks it is also necessary to be familiar with the NSDL Collections System. Systems administrators will also want to be familiar with the OAI ingest process. As noted below, certain administrative tasks are available to staff only, requiring a login and authorization to perform.
Actions and Features of Each Page
Information about the actions and features of each page in the Harvest Manager user interface is provided below.
Explorer page
The Explorer pages allows collection builders and NSDL administrators to explore and validate the records and responses returned by a OAI data provider.
Actions and features:
- Open an OAI data provider to explore: Start by typing or pasting the desired OAI BaseURL into the Base URL text box and click 'set base URL'. To open a different Base URL, edit the value and clieck 'set base URL' again.
- Validate the records returned by a data provider: Select a Set (optional) and Format and then click 'validate response'. Records will be fetehced and validated. Once this is complete a report will appear with the validation results. If a large list is being returned from the provider, a link will appear to allow you to validate the next batch of record. Repeat this until the link no longer appears to validate all records in the list.
- View the available sets from the data provider: Below 'ListSets', click 'view response' or 'validate response'. This will display the list of available sets from the data provider.
- View the available formats from the data provider: Below 'ListMetadataFormats', click 'view response' or 'validate response'. This will display the list of available formats from the data provider.
Collections page
The Collections page lists each OAI collection that is associated with the NSDL Data Repository and displays the data of the previous and next harvests and the status of the most recent harvest. The OAI collections are managed separately using the NSDL Collections System.
Actions and features:
- Search for a collection: Start typing in the Find box. With each keystroke, the Harvest Manager will show only those collections that match the given characters in their title (not case sensitive). For example, when you type the character n, only collections that have the letter n anywhere in their title will be displayed. When you type nasa, collections that contain the string 'nasa' in their title will be displayed.
- View collection harvest details: Click on the title of the collection.
- View details of the most recent harvest: Click on the status link listed in the Status column for the for a given collection.
- Add, delete, or edit a collection (requires log-in): Click on the 'NCS' link at the bottom of the page to access the NSDL Collection System. From there, add, delete or edit collections in the ncs_collect format. After editing, click the 'Update' button to view changes.
Collection Harvest Overview page
The Collection Harvest Overview page displays information about a given collection's OAI harvests.
Actions and features:
- View a summary of harvest settings for the collection: Click on the 'Harvest Information' tab.
- Edit the collections harvest settings (requires log-in): Click on the 'Edit' button to edit the collection harvest information using the NSDL Collection System. After editing, click the 'Refresh' button to view changes.
- View the history of harvests for the collection: Click on the 'Harvest History' tab (default view). From there you can click on individual harvests to view harvest details.
- Make OAI requests: Click on the 'Make OAI Requests' tab. Use this tab to issue OAI requests and explore the data provider for this collection. Actions performed here are for testing and exploring purposes only and do not impact the ingest process.
- Trigger a harvest manually (requires log-in): Click on the 'Trigger a Harvest' tab. Choose one of three options: Full production harvest, incremental production harvest or test harvest, which performs integrity checks on the data provider without ingesting records. Then click 'Trigger harvest now.' After the test or production mode harvest is complete an e-mail notification will be sent to collections staff.
Harvest Details page
The Harvest Details page displays information about a single harvest for a given collection.
Actions and features:
- View a summary of the harvest: Click on the 'Details' tab (default view).
- View the harvest logs: Click on the 'Harvest Logs' tab. This tab displays the individual log messages that are generated by the ingest process. After a harvest is triggered, log messages are available for viewing immediately once they are written, even as the harvest is in progress.
- View harvested records: Click on the 'Harvested Records' tab. This tab provides access to each of the files generated by ingest process, including gzip archives of the metadata records obtained from the harvest. These files are available only once the ingest process has completed.
Harvest Manager Settings and Configuration page
Meant for system administrators, the settings and configuration page displays the current software settings. The page allows viewing but not editing or changing the settings.
Actions and features:
- View the Harvest Manager settings (requires log-in): The page displays the current software settings and describes how to make changes to those settings.
