Unit Testing with Embedded Solr

From wiki.searchtechnologies.com
Revision as of 17:05, 3 March 2017 by Sdenny (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

This Page is Locked - Content no longer maintained - See QPL for the latest information.
Enterprise Add-On Feature

If you are programming a Solr plug-in, it is useful to do unit testing with an Embedded Solr server.

STEP 1: Add the following dependencies to your POM file

These instructions assume that you are creating a Maven project, which you probably should anyway (since it handles all of the dependencies for you).

You will need the following dependencies in your project.

Version 3.X

<dependency>
  <groupId>org.apache.solr</groupId>
  <artifactId>solr-core</artifactId>
  <version>3.6.0</version>
  <type>jar</type>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>org.apache.lucene</groupId>
  <artifactId>lucene-queries</artifactId>
  <version>3.6.0</version>
  <type>jar</type>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>org.apache.lucene</groupId>
  <artifactId>lucene-core</artifactId>
  <version>3.6.0</version>
  <type>jar</type>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>3.8.2</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-simple</artifactId>
  <version>1.6.1</version>
  <type>jar</type>
  <scope>test</scope>
</dependency>

The Lucene dependencies might not be strictly required, as they may get pulled in automatically by Solr.

Version 4.X

 <dependencies>
 	<dependency>
 		<groupId>org.apache.solr</groupId>
 		<artifactId>solr-core</artifactId>
 		<version>4.4.0</version>
 		<scope>provided</scope>
 	</dependency>
 	<dependency>
 		<groupId>javax.servlet</groupId>
 		<artifactId>servlet-api</artifactId>
 		<version>2.5</version>
 		<scope>test</scope>
 	</dependency>
 	<dependency>
 		<groupId>commons-logging</groupId>
 		<artifactId>commons-logging</artifactId>
 		<version>1.1.3</version>
 		<scope>test</scope>
 	</dependency>
 	<dependency>
 		<groupId>com.searchtechnologies.aspire</groupId>
 		<artifactId>aspire-framework</artifactId>
 		<version>2.0-SNAPSHOT</version>
 	</dependency>
 	<dependency>
 		<groupId>com.searchtechnologies.aspire</groupId>
 		<artifactId>aspire-services</artifactId>
 		<version>2.0-SNAPSHOT</version>
 	</dependency>
 </dependencies>

The Aspire dependencies are not required (they are included to help with deleting the index directory and all of the sub-directories).

STEP 2: Create a Solr Directory in your Test Data

I like to put all of my testing data in a separate directory called "testdata". Inside testadata, create a sub-directory called "solr" and copy in a "config" directory. Then create a sub-directory called "data" and copy in your sample indexes.

NOTE: Your sample indexes should be SMALL. Please do not copy in large indexes into Subversion!! Even better, set the "data" directory to "svn:ignore" (if you can create the indexes on the fly as part of the unit test)

Version 3.X

When you're done, the directory structure should look like this:

<Eclipse Project Directory>

  • src
  • target
  • testdata
    • solr
      • conf (note: not "config")
      • data
        • index

NOTE: You can start with no indexes at all, if you will be first indexing some documents with your embedded Solr instance. If this is the case, I recommend for you to delete the "index" directory in your unit test, before launching the Solr servers.

Version 4.X

In version 4.X, I've found it's a bit easier to actually have the parent directory with a core underneath. The structure which worked for me is this:

  • testdata
    • solr
      • solr.xml
      • collection1
        • conf (note: not "config")
        • data (will be created automatically if missing)

STEP 3: Create the Unit Test

Version 3.X and Earlier

The following is a sample unit test which initializes an embedded Solr server and then sends a query to it.

package com.searchtechnologies.qpl.solr;

import java.io.File;

import org.apache.lucene.queryParser.ParseException;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.core.CoreContainer;

import junit.framework.TestCase;

public class TestWithSolr extends TestCase {
 EmbeddedSolrServer server;

 protected void setUp() throws Exception {
   File home = new File( "testdata/solr" );
   File configFile = new File( home, "solr.xml" );

   System.setProperty("com.searchtechnologies.qpl.solr.builderFile", "src/main/resources/LuceneBuilder.groovy");

   CoreContainer container = new CoreContainer("testdata/solr", configFile);

   server = new EmbeddedSolrServer( container, "collection1" );

   super.setUp();
 }

 protected void tearDown() {
   server.shutdown();
 }

 public void testQParserPluginWithSolr() throws ParseException, SolrServerException {

   ModifiableSolrParams params = new ModifiableSolrParams();

   // ** Test the basics
   params.set("q", "search");
   QueryResponse qResp = server.query(params);

   SolrDocumentList docList = qResp.getResults();
   SolrDocument doc = docList.get(0);
   assertTrue(doc.getFirstValue("title").toString().toLowerCase().contains("search"));

   // System.out.println(qResp.toString());

 }
}

Version 4.X

The code is a little different in version 4.x.

import java.io.IOException;

import org.apache.lucene.search.spans.SpanTermQuery;
import org.apache.lucene.search.spans.Spans;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.core.CoreContainer;
import org.apache.solr.search.SolrIndexSearcher;

import junit.framework.TestCase;

public class TestTokenizerInSolr extends TestCase {
 EmbeddedSolrServer server;
 CoreContainer container;

 protected void setUp() throws Exception {
   // If indexing a brand-new index, you might want to delete the data directory first
   // FileUtilities.deleteDirectory("testdata/solr/collection1/data");

   container = new CoreContainer("testdata/solr");
   container.load();

   server = new EmbeddedSolrServer(container, "collection1" );
   super.setUp();
 }

 protected void tearDown() throws Exception {
   server.shutdown();
   super.tearDown();
 }

 public void testTokenizerInSolr() throws SolrServerException, IOException {
   ModifiableSolrParams params = new ModifiableSolrParams();

   // ** Let's index a document into our embedded server

   SolrInputDocument newDoc = new SolrInputDocument();
   newDoc.addField("title", "Test Document 1");
   newDoc.addField("id", "doc-1");
   newDoc.addField("text", "Hello world!");
   server.add(newDoc);
   server.commit();
   
   // ** And now let's query for it

   params.set("q", "title:test");
   QueryResponse qResp = server.query(params);

   SolrDocumentList docList = qResp.getResults();
   System.out.println("Num docs: " + docList.getNumFound());
   SolrDocument doc = docList.get(0);
   System.out.println("Title: " + doc.getFirstValue("title").toString());
 }
}