JMS Message Feeder 0.4

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

Aspire / Aspire Components / Simple Feeder / JMS Feeder

JMS Feeder
Description: Monitors a JMS queue, listening for xml messages. On receiving a message, publishes it as an Aspire document.
Inputs: JMS Queue
Outputs: An AspireDocument object containing data from the JMS message, published to the configured pipeline manager.
Factory: aspire-jmsfeeder
Sub Type: default
Object Type: Produces AspireDocument objects.

Other Notes


Feeder Operation

The JMS feeder monitors a JMS queue. By default the queue is configured for ActiveMQ, but may be configured using JNDI for any JMS queue provider. The queue may be a queue or a topic and may be durable.

When started, the feeder listens to the queue. When an xml message is received, the id and the action of the message are extracted from the message. The location of these items is specified as an xpath in the configuration. The message is then optionally transformed using XSLT, before being published on an event based on the action.

NOTE: There is no difference in operation between full and incremental feeding for this feeder, once started the feeder listens for messages until stopped.


Configuration

This feeder takes all parameters from the Simple Feeder plus the following:

Element Type Default Description
feederLabel string JMSFeed The feeder label submitted in the <feederLabel> of the published document and when querying the CCD.
queue None The JMS Queue to listen on. See below.
message/@idPath String /doc/fetchUrl The xpath to the document id in the received message.
message/@actionPath String /doc/@action The xpath to the document action in the received message. The action should be on of insert, update or delete.
message/@transform String None The name of the file holding the XSLT to apply to the incomming message before forwarding on to the pipeline.
events String See below The events to publish documents against. See below.


Queue Configuration

The JMS feeder listens on a configurable JMS queue.

Element Type Default Description
queue/@name String None The name of the JMS queue to listen on
queue/@clientId String None The id of this feeder given to the JMS broker
queue/@topic Boolean false True if using a JMS Topic rather than a queue
queue/@durable Boolean false True if this JMS Queue or Topic is durable
queue/@transacted Boolean false True if this JMS Queue or Topic is transacted
queue/@jndi Boolean false False if the connection to this JMS Queue or Topic is to be made using the built-in ActiveMQ libraies. True if you wish to use JNDI to specify the connection method. In this case, you should use the jndi/prop elements of the configuration to specify the required configuration. Multiple jndi/prop elements are permitted. At a mimimum you should set:
 java.naming.factory.initial
 java.naming.provider.url
queue/jndi/prop/@name String None The name of a JNDI property to set. The property will be set before the connection to the queue is connected to.
queue/jndi/prop/@value String None The value of the named JNDI property to set. The property will be set before the connection to the queue is connected to.
queue/jndi/classPath String None The class path of jar files needed for the classes used in the JNDI connection. This is a list of jar files (relative to Aspire home) separated by the path separator character (: on Unix and ; on Windows)


Event Configuration

The RDB feeder publishes differing actions against differing events meaning different pipelines can be used to handle different actions.

Element Type Default Description
events/insert/@event String onPublish The event to publish against for an insert.
events/update/@event String onPublish The event to publish against for an update.
events/delete/@event String onDelete The event to publish against for a delete.


Branch Configuration

The RDB feeder publishes documents using the branch manager. It publishes using the events configured above. You must therefore include <branches> for these events in the configuration to publish to a pipeline within a pipeline manager. See Branch Handler for more details.

Element Type Description
branches/branch/@event String The event to configure.
branches/branch/@pipelineManager string The URL of the pipeline manager to publish to. Can be relative.
branches/branch/@pipeline string The name of the pipeline to publish to.

Example Configurations

Simple Configuration

 <component name="EM3UpdateFeeder" subType="default" factoryName="aspire-jmsfeeder">
   <config>
     <feederLabel>EM3QueueFeeder</feederLabel>
     <autoStart>${autoFeedEM3Update}</autoStart>
     <queue name="EM3UpdateQueue" durable="true" topic="true" clientId="AspireQueueFeeder" />
     <message idPath="/update/@entityId" actionPath="/update/@entityAction" transform="config/jmsUpdate2Aspire.xsl"/>
     <branches>
       <branch event="onPublish" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Publish-pipeline" />
       <branch event="onUpdate" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Update-pipeline" />
       <branch event="onDelete" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Delete-pipeline" />
     </branches>
   </config>
 </component>


Complex Configuration

 <component name="EM3UpdateFeeder" subType="default" factoryName="aspire-jmsfeeder">
   <config>
     <feedWait>5000</feedWait>"
     <loopWait>3600000</loopWait>"
     <feederLabel>CrawlJMS</feederLabel>"
     <maxErrorsRetained>20</maxErrorsRetained>"
     <statsPeriod>20</statsPeriod>"
     <feederLabel>EM3QueueFeeder</feederLabel>
     <autoStart>${autoFeedEM3Update}</autoStart>
     <queue name="EM3UpdateQueue" durable="true" topic="true" clientId="AspireQueueFeeder1" jndi="true">
       <jndi>
         <prop name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
         <prop name="java.naming.provider.url" value="tcp://localhost:61616"/>
         <jar file="lib/activemq-core-5.0.0.jar"/>
       </jndi>
     </queue>
     <message idPath="/update/@entityId" actionPath="/update/@entityAction" transform="config/jmsUpdate2Aspire.xsl"/>
     <events>
       <insert event="onPublish"/>
       <update event="onUpdate"/>
       <delete event="onDelete"/>
     </events>
     <branches>
       <branch event="onPublish" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Publish-pipeline" />
       <branch event="onUpdate" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Update-pipeline" />
       <branch event="onDelete" pipelineManager="EM3Update-pipe-manager" pipeline="EM3Delete-pipeline" />
     </branches>
   </config>
 </component>

Websphere JNDI Queue Configuration

The following JNDI configuration works against a JMS queue under WebSphere 7

 <queue name="jms/topic/searchevents" connectionFactoryName="jms/cf/searchevents" durable="true" topic="true" clientId="EM3UpdateFeeder" jndi="true">
   <jndi>
     <prop name="java.naming.factory.initial" value="com.ibm.websphere.naming.WsnInitialContextFactory"/>
     <prop name="java.naming.provider.url" value="iiop://ibmdev2003:2809"/>
     <prop name="org.omg.CORBA.ORBClass" value="com.ibm.CORBA.iiop.ORB"/>
     <classPath>lib/com.ibm.ws.admin.client_7.0.0.jar;lib/com.ibm.ws.orb_7.0.0.jar;lib/com.ibm.ws.runtime.jar;lib/com.ibm.ws.sib.client.thin.jms_7.0.0.jar</classPath>
   </jndi>
 </queue>