Creating a New Pipeline Stage

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

An Aspire/Maven Archetype is available for quickly and easily creating new Aspire Pipeline Stages.

What is a Maven Archetype? - It is a special Maven Project which can create new Maven Projects. Basically, it sets up all of the directories and project files for your new pipeline stage automatically, so that all you have to do is edit your Java code.

Using the Archetype to Create a New Pipeline Stage

Execute the following maven command in some directory: (Note: We recommend that this directory be an Eclipse workspace directory - see Editing New Stages Inside Eclipse).

Also, choose the version of the stage archetype which corresponds to the version of Aspire which you are using.

For Version 2.2.2: (should be used for new projects)

mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-stage-archetype -DarchetypeVersion=2.2.2 -DrepositoryId=stPublic

For Version 2.2.1: (the previous version}}

mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-stage-archetype -DarchetypeVersion=2.2.1 -DrepositoryId=stPublic

Note: If the above doesn't work, try adding the following argument to the command:

 -DarchetypeRepository=http://repository.searchtechnologies.com/artifactory/simple/community-public/ 

It will prompt you for several input parameters:

Parameter Description
aspireStageClassName Enter the name for the Java Class which will be the implementation for your pipeline stage. For example: MyDoSomethingUseful
aspireStageDescription This is a user-friendly description of the stage. It will be included in the ComponentFactory.xml configuration file (stored in the component Jar) as well as in the POM file for the Maven project. It is also shown on the Admin page for the project. Usually it is just a sentence or two.
aspirePOMName This is a user-friendly name for the stage. It is stored in the POM file. It will be displayed as Maven builds your project, and it will also be shown in the list of Bundles in the OSGi/Apache Felix web console interface for your component's bundle.
groupId The Maven group-id for your new pipeline stage. Typically something like "com.searchtechnologies". Should be unique for your organization across all organizations in the world.
artifactId The Maven artifact-id for this project. This is the main Maven identifier for your pipeline stage Jar, within the GroupID. This will be used to locate the artifact on the Search Technologies repository and it will also be used to name the Jar File itself. Typically something like "aspire-rdb".
version The version number for your new stage. If it ends in "-SNAPSHOT" (for example, "0.5-SNAPSHOT"), then any update you make to the stage will be immediately available to other users of your stage. In other words, the stage is in "active development" and so everyone will receive the latest when they build their projects.

If your stage has been "released", then you should use a version such as "1.0" (i.e. without "-SNAPSHOT"). People who download your stage will only receive a single version of it. Updates to a released version of a stage will not be automatically downloaded by people using it. It was officially "released" after all, if you want to create a new version, then you should update the version number.

Note that SNAPSHOT versions use up more disk space on the repository, and so should only be used for releases that are actually in the midst of active development.

Once you have entered all of the parameters, you will be prompted to verify your selections. Enter 'Y', and then Maven will create your new stage for you.

Editing Your Stage in Eclipse

The instructions for adding new stages to an Eclipse project can be found here:

Editing New Stages Inside Eclipse

Building Your Stage Using Maven

The aspire component is now a complete and functioning component. Note that it doesn't yet do anything useful, but it can be completely built and installed into Aspire without further modification. This is what we are going to do now.

To build the stage, do the following:

cd {your stage directory}
mvn clean package

Note that {your stage directory} will be the same as the "artifactId" which you specified above when creating the stage.

Check the Maven output to see that unit tests and the build all complete successfully.

Changing The Source Code

Source Code: Will be in the "src/main/java" directory.

  • A ready-to-compile java source file for your new pipeline stage will be located in the standard Java package directory structure. The source file will contain many embedded comments and hints for writing your new stage.

Unit Test Code: Will be in "src/test/java".

  • A read-to-compile and run test program will already be in the appropriate package directory.

Other Resources: Will be in src/main/resources. Default resources are automatically provided for you by the stage archetype. These resources include:

  • The ComponentFactory.xml file - Identifies the types of components that this factory can produce. Automatically set up to produce a single "default" component.
    • Note that you can put the Java code for multiple components in the same bundle/jar file. Just add more entries to the ComponentFactory.xml file as needed.
    • The ComponentFactory.xml file also contains default configuration information for each of the default types.
  • Admin User Interface resources - Will be located in the src/main/resources/resources/{classname} directory. These include:
    • An image (image.jpeg) which will be displayed on your Admin UI web page
    • The status.xsl which transforms the XML status report from your pipeline stage into XHTML for display on the web page.
    • The result.xsl which transform the result XML for any web UI servlet commands that are sent to your pipeline stage into XHTML for presentation on a web browser.

Using the New Stage In an Aspire Installation

Using Maven Repositories

NOTE: These instructions are appropriate when using the <repository type="maven"> setting in your Aspire settings.xml file.

To Incorporate A Brand-New Component:

First, make your new component available to Aspire by doing the following:

 mvn clean install

Once this is done, you can then add your new stage to your Aspire System XML file:

  1. Make sure you add a new <component> to the list of components for your stage
    • The factory name for the component will be your component's Maven artifact ID (e.g. "aspire-uppercase").
    • If the groupId of your component is not "com.searchtechnologies", you will also need to specify the group id in your artifact name (e.g. "com.mycustomer:aspire-uppercase").
  2. Then add the new stage to the appropriate <pipeline>

Then reload your Aspire System XML file by clicking the "reload" button next to the configuration file on the http://localhost:50505/aspire Aspire application page.

If you update your component code:

These updates can be dynamically incorporated into Aspire. Again, first make your new component available to Aspire with "mvn clean install". Once this is done, click on "Check For Updates" on the http://localhost:50505/aspire page. Your new component should now be flagged as "Update Available". Then click on the reload button for your component to load the new binaries into Aspire.

When Using Distribution Repositories

NOTE: The following instructions are appropriate if you use the <repository type="distribution"> in your Aspire settings.xml file exclusively (i.e. for some reason you are not using the maven repository option in Aspire)

Once you have written and unit-tested your pipeline stage, you can use the following steps to use your stage in a new Aspire installation.

  1. Copy the component JAR file into your Aspire distribution.
    • COPY FROM: Your component's "target" directory
    • COPY TO: The Installation's "target/{distribution}/bundles/aspire" sub-directory
  2. Modify Your config/system-example.xml file to include your new component in your pipeline.
    1. Inside the <component name="standard-pipe-manager">, add your component:
      • <component name="uppercaseTitle" subType="default" factoryName="aspire-uppercase" />
    2. Inside the <pipeline><stages> add your new pipeline stage>
      • <stage component="uppercaseTitle" />
  3. Restart Aspire and test
    • Alternatively, you can click on "Check for Updates" on the Aspire home page
    1. Is your new pipeline stage a sub-component of "standard-pipe-manager" ?
    2. Is your new pipeline stage in the pipeline?
    3. What happens when you view your new stage in a web browser?
    4. What happens when you feed a new URL down the pipeline? (look at the end of the return of feedOne)

Note that your changes above may be wiped out if you use Maven to rebuild your distribution (which is only a problem if you used the Maven Distribution Archetype to build your distribution in the first place).

For more information about changing your Maven Distribution Project so it includes your new component everytime, see Using The Maven Distribution Archetype.

Exercise

Instructions: Modify the source code to convert the <title> element to all caps.

Explore the "src" directory structure of the component you just created. Locate the java class which is the pipeline stage and modify it appropriately. Also locate the unit test and modify it as well. Then re-build the component and re-deploy it in your aspire application.