TNS Internal:Applications/FeedEater

From NSDLWiki

Contents [hide] 1 Notes 2 Pre-Administration: 3 Basic Process 4 How FeedEater determines user authorization (LiveCUFConnection) 5 How FeedEater determines collections (CollectionInfoServlet)

Notes

• Uses the title of a collection (e.g. an NCS collection) but doesn't modify the collection metadata.
• Reads setSpec from the collection's metadata provider.
• FeedEater users are agent objects (and LDAP entries).
• FeedEater users are included in the wfi:agent property of a collection that they are allowed to modify.
• A FeedEater feed is any metadata provider that includes a WFIngestParms datastream.
• A FeedEater feed (metadata provider, above) may include a WFIngestHistory datastream.
• FeedEater creates a resource and metadata object for each feed item and relates them to each other.
• FeedEater sets the metadataProvidedBy property of the metadata for each resource to the metadata provider for the feed.

Pre-Administration:

• Use NCS to create the NCS_Collect record, which creates the Collections Aggregator and Metadata Provider Objects in NDR. Collections must already exist prior to use through WFI. (See NCS section for more info.)

Basic Process

For a new feed...

• reads collections info to display a list of collections for the user to choose into which collection the feed resources will go (via NDR Java Toolkit OR via NCS Rest API OR via DDS Rest API?)
• adds new Collection Metadata Provider for existing Collection Aggregator via NDR Java Toolkit
• application data about details of the ingest process for the new feed are stored in a datastream on the Metadata Provider object via NDR Java Toolkit

During ingest...

• create, update, and delete metadata and resource objects based on resources listed in the feed via Java Toolkit

How FeedEater determines user authorization (LiveCUFConnection)

1. A user is authorized by adding an wfi:authorizedForCollection relationship to the NDR Agent object for that user with a value of the aggregation handle of the collection for this this user is authorized. (LiveCUFConnection.addAuthorization)

How FeedEater determines collections (CollectionInfoServlet)

1. Find all metadata providers in the NDR with an oai:visibility property set to "public" and add them to a Set object. (CollectionInfoServlet.findMetadataProviders)
2. Run through this Set object and generate a Map object of metadata provider handles and CollectionInfo objects. For each metadata provider handle in the Set object... (CollectionInfoServlet.findTopLevelCollectionObjects)
   1. Look at all the aggregatedBy relationships for this metadata provider handle and use the first one as the aggregator for this aggregation. (CollectionInfoServlet.findAggregatorHandle)
   2. Retrieve a Set of metadata provider handles which have an aggregatedBy relationship for this aggregator and pick the first one that was created by NCS (tests the properties for NCS_STAMP_OF_APPROVAL?) to use as the NCS-managed metadata provider handle. (CollectionInfoServlet.findNcsManagedMetadataProvider)
   3. Find the setSpec property for this metadata provider handle. (CollectionInfoServlet.findSetSpec)
   4. Create a CollectionInfo object with the above aggregator and metadata provider handles and the setSpec property and add it to the Map. This effectively gets you a Map of all top-level NCS collection objects (the aggregator and its NCS metadata provider).
3. Strip out any aggregators that don't have a memberOf relationship with the "Collection of Collections" aggregator (CollectionInfoServlet.findMembersOfCollectionOfCollections).
4. Run through all the remaining aggregations (public NCS-managed collections) and fill in their information (CollectionInfoServlet.populateCollectionInfo)...
   1. Retrieve the aggregator handle for ths collection (CollectionInfo.getAggregatorHandle)
   2. Find the first metadata object with a metadataFor relationship with this aggregator handle and use that as the metadata for this aggregator.
   3. Read the nsdl_dc datastream from this metadata object. (CollectionInfoServlet.getDublinCoreFromMetadata)
   4. Extract the collection information from the nsdl_dc data. Use the aggregator handle to find the correct element and pull out the title, description and identifier fields and update the Map's CollectionInfo objects. (CollectionInfoServlet.getCollectionData)
5. Convert the Map to a HashMap and make it immutable by setting it to a private static volatile property, CollectionInfoServlet.Collections.