Post Fast (Aspire 2)

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here


Post Fast (Aspire 2)
Factory Name  com.searchtechnologies.aspire:aspire-post-fast
subType  default
Inputs  An AspireObject representation of the document as processed so far.
Outputs  Information posted to a Fast indexing system using the Content API.
Feature only available with Aspire Enterprise

The Post Fast Stage stage allows documents to by posted to a Fast indexing server via the content API.

The name (or IP address) of the Fast server is entered in to the component configuration, as is a default Fast collection name to submit the document to. However, the collection to submit to can be supplied in the AspireObject, allowing routing to be performed in previous stages.

The component supports job batching (see Branch Handler) and will maintain the contents of any batch passed to it, only submitting the batch to Fast once the batch is closed by the client. However, if the component receives non batched documents, it will add these to an internal batch, submitting to Fast when a certain number of documents have been received, or after a certain time.

This component also supports requests for call back information from external components. Components that submit batches of documents to pipeline that end up at the Post to Fast component and request the status of the batch, and the Post to Fast component will return the status as reported by the Fast servers. In this manner, the component originally submitting the document to Aspire can discover when the document has been successfully added to the Fast index.

Configuration

Element Type Default Description
fastServer String localhost:16100 The server and port for the Fast indexing system.
fastCollection String The (default) name of the Fast collection the document should be indexed in to. If collectionPath is not given, or the given collectionPath element is not found, this default is used.
idPath String /doc/fetchUrl The location of the document id in the AspireObject (required for the Fast API).
actionPath String /doc/@action The location of the action to be performed in the AspireObject. The content of this element (or attribute) should be one of unknown, noChange, insert, update or delete.
collectionPath String /doc/collection The location of the collection to submit the document to in the AspireObject. If this element (or attribute) is found in an AspireObject, that document will be submitted to the collection given in the AspireObject.
contentTag String The child under the root of the document from which content should be read. If given, every child of the configured tag will be submitted to Fast. If omitted, every child of the document root will be submitted to Fast.
fastTimeout int 600000
10 minutes
Fast API timeout in ms.
fastWaitForIdx boolean false Wait for an indexed callback from Fast before allowing the job to complete.
maxBatchSize long 1000 The maximum number of documents (not already in a batch) to add to a batch before sending to Fast. If documents are already in a batch, that batch is not affected and will be sent when the batch is closed.
maxBatchTime long 900000
15 minutes
The maximum time to wait before sending documents (not already in a batch) in a batch to Fast if the document count has not reached the maxBatchSize. If documents are already in a batch, that batch is not affected and will be sent when the batch is closed.

Example configuration

  <component name="PostToFast" subType="default" factoryName="aspire-post-fast">
    <debug>${debug}</debug>
    <fastServer>myFastServer:15100</fastServer>
    <fastCollection>myDefaultCollection</fastCollection>
    <idPath>/doc/@fastId</idPath>
    <actionPath>/doc/@action</actionPath>
    <collectionPath>/doc/@collection</collectionPath>
    <maxBatchSize>500</maxBatchSize>
    <maxBatchTime>300</maxBatchTime>
    <fastWaitForIdx>false</fastWaitForIdx>
  </component>