Connector Framework

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

Introduction

In Aspire  (1.3 Release)  & (2.0 Release)  , we've tried to simplify the connector creation process. Previously you needed to consider scanner component, group expansion component and application bundle separately. In the new releases, we've built an archetype that creates a scanner component and application bundle in a project heirarchy that allows a single project to be imported in to Eclipse. We've also enhanced the scanner component to better handle security and to cache groups for group expansion. This cache can be queried via a generic group expansion client, which we've already configured in the application bundle for you.

You'll still need to do some work though. You'll need to:

  • Understand the repository you want to connect to
    • What are the objects, what can you extract etc
    • How does security work?
    • Check out the repository API and generate same programs to:
      • Login
      • Extract a list of all objects in the repository
      • Extract an object & metadata
      • Extract object ACLs
      • Extract a list of all users
      • Extract a list of all groups
      • Extract a list of all groups to which a user or group belongs

Generating the Archetype

Generating the connector archetype is the same as any other archeype and assumes that you have the Maven command line installed:

  1. Open up a DOS command window
  2. Use "cd" to change the directory to someplace where you want to store the new Aspire Connector
Typically this will be in an Eclipse workspace
  1. Execute the command shown below
Note the commands are slightly different in  (1.3 Release)   and  (2.0 Release)   due to a change in group id

For  (1.3 Release)   run

 mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies -DarchetypeArtifactId=aspire-connector-archetype -DarchetypeVersion=1.3-SNAPSHOT -DrepositoryId=stPublic

For  (2.0 Release)   run

 mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-connector-archetype -DarchetypeVersion=2.2.2 -DrepositoryId=stPublic

Ensure you use the correct version - SNAPSHOTs may have been released by the time you run this command.

Note that, sometimes, the "mvn" command will fail to locate the Search Technologies repository (it should be specified in your new settings.xml file). If this is a problem, add the following argument to the "mvn" command from above:

-DarchetypeRepository=http://repository.searchtechnologies.com/artifactory/simple/community-public/

After executing the command, Maven will download a bunch of different JAR files. Most of these are various Maven plug-ins which Maven itself requires to execute. These plug-ins are downloaded automatically from the "master maven repository in the sky", namely http://repo.maven.apache.org.

Finally, once Maven itself is ready to go, you will be prompted for the following information:

  • artifactId - This is the name for your connector. Typically we use the name of the repository you are trying to extract information from. All sub project names and classes will be base on this, and it is time consuming to change later, so you may wish to pause for a moment to think about it
  • version - This will be the version number of Aspire that you wish this connector to compile against. All Aspire dependencies will be at this version and the connector and app-bundle will be created with this version.
    • The current released version of Aspire is 2.2.2
  • mavenPassword - This will be the Maven password you got when you registered.
  • mavenUsername - This will be the Maven username which you got when you registered.
Both user and password are only used if you run the app-bundle as a distribution for testing, so you can get away with defaults here.

After you enter the final information and press the Enter key, you are prompted to confirm the entered information. If it's correct, just press enter. If it's not, type N and then press enter. You can then change the information you entered.

If you re-enter the information, you can also change:

  • groupId - This is the name of the organizational unit to which your connector belongs. For  (1.3 Release)   it defaults to com.searchtechnologies and for  (2.0 Release)   it defaults to com.searchtechnologies.aspire
  • aspireAdminPort - This is the port Aspire will run on if the app-bundle is run as a distribution

Here is an example output:

C:\Users\myUser\myWorkspace\myRepository>mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies -DarchetypeArtifactId=aspire-connector-archetype -DarchetypeVersion=1.3-SNAPSHOT -DrepositoryId=stPublic
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom <<<
[INFO]
[INFO] --- maven-archetype-plugin:2.2:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[WARNING] Archetype not found in any catalog. Falling back to central repository (http://repo1.maven.org/maven2).
[WARNING] Use -DarchetypeRepository=<your repository> if archetype's repository is elsewhere.
Downloading: http://repository.searchtechnologies.com/artifactory/simple/community-public/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml
Downloading: http://repository.searchtechnologies.com/artifactory/simple/community-public/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml
Downloaded: http://repository.searchtechnologies.com/artifactory/simple/community-public/com/searchtechnologies/aspire-connector-archetype/1.3-SNAPSHOT/maven-metadata.xml (792 B at 0.0 KB/sec)
[INFO] Using property: groupId = com.searchtechnologies
Define value for property 'artifactId': : myRepository
Define value for property 'version':  1.0-SNAPSHOT: : 1.3-SNAPSHOT
[INFO] Using property: package = pom
[INFO] Using property: aspireAdminPort = 50505
Define value for property 'mavenPassword': : myPassword
Define value for property 'mavenUser': : myUser@myCompany.com
Confirm properties configuration:
groupId: com.searchtechnologies
artifactId: myRepository
version: 1.3-SNAPSHOT
package: pom
aspireAdminPort: 50505
mavenPassword: myPassword
mavenUser: myUser@myCompany.com
 Y: :press return
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: aspire-connector-archetype:1.3-SNAPSHOT
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.searchtechnologies
[INFO] Parameter: artifactId, Value: myRepository
[INFO] Parameter: version, Value: 1.3-SNAPSHOT
[INFO] Parameter: package, Value: pom
[INFO] Parameter: packageInPathFormat, Value: pom
[INFO] Parameter: version, Value: 1.3-SNAPSHOT
[INFO] Parameter: package, Value: pom
[INFO] Parameter: mavenUser, Value: myUser@myCompany.com
[INFO] Parameter: aspireAdminPort, Value: 50505
[INFO] Parameter: groupId, Value: com.searchtechnologies
[INFO] Parameter: mavenPassword, Value: myPassword
[INFO] Parameter: artifactId, Value: myRepository
[WARNING] Don't override file C:\Users\myUser\myWorkspace\myRepository\aspire-myRepository-scanner\pom.xml
[WARNING] CP Don't override file C:\Users\myUser\myWorkspace\myRepository\app-myRepository-connector\distribution-files\web\components\MyRepositoryConnector
[WARNING] CP Don't override file C:\Users\myUser\myWorkspace\myRepository\app-myRepository-connector\distribution-files\web\components\MyRepositoryConnector\Main
[WARNING] CP Don't override file C:\Users\myUser\myWorkspace\myRepository\app-myRepository-connector\distribution-files\web\components\MyRepositoryConnector\Main\dxf.xml
[INFO] project created from Archetype in dir: C:\Users\myUser\myWorkspace\myRepository
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:53.118s
[INFO] Finished at: Mon Oct 07 18:26:51 BST 2013
[INFO] Final Memory: 10M/62M
[INFO] ------------------------------------------------------------------------
C:\Users\myUser\myWorkspace\myRepository>

The Folder & Project Hierarchy

When you generate the archetype, you end up with three projects that you can import in to Eclipse. At the top level is the super project for you connector. This is a very simple project that references two further projects, found in the directories below. Building at this level simply build the two child projects

The directory structure looks like this:

CFDirTop.png

In the directories below, there are projects for the connector component itself and the application bundle that will load the connector.

The directory structure of the scanner looks like this:

CFDirScanner.png

If you build at the scanner level, you will only build the scanner component project.

The directory structure of the application bundle looks like this:

CFDirBundle.png

If you build at the application bundle level, you will only build the application bundle project. By default this will build as an application bundle. If you want to build it as a distribution, add -P build-distribution to your Maven command

Creating the Connector