RDB Sub Job Feeder 0.4

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

Aspire / Aspire Components / RDB Sub Job Feeder

RDB Sub Job Feeder
Description: Feeds Aspire documents from an RDB. The feed is driven from an incoming Aspire Document
Inputs: AspireDocument and RDB
Outputs: An AspireDocument object containing data from the RDB, published to the configured pipeline manager.
Factory: aspire-rdbfeeder
Sub Type: rdbSubJobFeeder
Object Type: Produces AspireDocument objects.


Operation

This stage takes an Aspire document and executes some SQL against an RDB. The returned rows are then formed in to Aspire documents and submitted to a pipeline.

The SQL to execute is held in the configuration file and differing SQL can be run based on the document action attribute. The configuration allows for parameters to the SQL to be taken from the document. The location of the parameter is given as an XPath.

Configuration

Element Type Default Description
rdbLocation String rdb The location with in the system of the rdb service. Used for database connections.
id/@column String None The name of the column that holds the primary key (id) for the row.
extract String None The various pieces of SQL to run. See below.
events String See below The events to publish documents against. See below.
branches None The configuration of the pipeline to publish to. See below.


Extract SQL Configuration

The RDB Row Extractor uses configurable SQL to specify what to add to the document. Different SQL may be configured for the differing actions that may occur: insert, update and delete.

Accordingly, <action> below may be replace by insert, update or delete.

Element Type Default Description
extract/<action>/sql String None The SQL to be run to extract data. The SQL may include ? as place holders for parameters. These will then be subsituted in in order.
extract/<action>/param/@index Integer None The parameter number of this parameter, with parameter one replacing the first ? in the SQL, parameter two replacing the second ? in the SQL etc.
extract/<action>/param/@type String Integer The type of this parameter - either String or Integer
extract/<action>/param/@docPath String None The xpath in the Aspire document to the element that holds the value for this parameter.


Event Configuration

The RDB feeder publishes differing actions against differing events meaning different pipelines can be used to handle different actions.

Element Type Default Description
events/insert/@event String onPublish The event to publish against for an insert.
events/update/@event String onPublish The event to publish against for an update.
events/delete/@event String onDelete The event to publish against for a delete.


Branch Configuration

The RDB feeder publishes documents using the branch manager. It publishes using the events configured above. You must therefore include <branches> for these events in the configuration to publish to a pipeline within a pipeline manager. See Branch Handler for more details.

Element Type Description
branches/branch/@event String The event to configure.
branches/branch/@pipelineManager string The URL of the pipeline manager to publish to. Can be relative.
branches/branch/@pipeline string The name of the pipeline to publish to.

Example Configuration

 <component name="EM3UpdateSubJobFeeder" subType="rdbSubJobFeeder" factoryName="aspire-rdbfeeder">
   <config>
     <rdbLocation>/common/EM3rdb</rdbLocation>
     <branches>
       <branch event="onPublish" pipelineManager="../EM3-pipe-manager" pipeline="EM3-add-doc-pipeline" />
       <branch event="onDelete" pipelineManager="../EM3-pipe-manager" pipeline="EM3-del-doc-pipeline"/>
     </branches>
     <id column="ID"/>
     <extract>
       <insert>
         <param index="1" type="int" docPath="/doc/id"/>
         <param index="2" type="string" docPath="/doc/entityType"/>
         <sql><![CDATA[select id, sub1, sub2, sub3 from main where ?  = (
                               case ?
                                 when 'ma' then mediaAssetId
                                 when 'pv' then productVersionId
                                 when 'p' then productId
                                 when 'pg' then productGroupId
                                 else productId
                               end)]]></sql>
       </insert>
       <delete>
         <param index="1" type="int" docPath="/doc/id"/>
         <param index="2" type="string" docPath="/doc/entityType"/>
         <sql><![CDATA[select id from main where ?  = (
                               case ?
                                 when 'ma' then mediaAssetId
                                 when 'pv' then productVersionId
                                 when 'p' then productId
                                 when 'pg' then productGroupId
                                 else productId
                               end)]]></sql>
       </delete>
     </extract>
   </config>
 </component>