Imports and Export Packages in Maven

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

The Question

Below is an email sent out to technical staff in Search Technologies from an employee. We'll call him David (mainly because that's his name!). David posted:

 From: David
 Sent: Friday, April 15, 2016 4:45 PM
 To: ST Technical
 Subject: Error building a Aspire Staging Component?
 
 Hi all,
 
 I am getting the following error when running:  mvn clean package  (per our instructions)  to build an essentially do-nothing Aspire component in Java.
 
 mvn clean – works fine.
 
 mvn package produces the same error.
 
 ExportPackages.png 
 
 The declarations for the org.apache.felix:maven-bundle-plugin:2.3.7 in the Aspire POM file are somewhat complex (but untouched and standard).
 
 Anyone seen this error before and know what to do?
 
 The only think non-standard about this – is that I am just trying to create the component (haven’t written any Java code yet)
 but have followed our instructions to the letter (as far as I can tell)??

There were a number of replies, all of which solved the original issue, but our Chief Architect, Paul Nelson, wrote this very informative response:

Paul's Response

Hey everyone at ST Technical:

The exchange that you see between David & the core engineering team below is one which happens frequently with Aspire.

These sorts of errors happen because Aspire uses the OSGi framework (www.osgi.org, realized by Apache Felix - http://felix.apache.org/ ) for managing classes and components. This means that, for Aspire Components, OSGi manages all class loading. This means that OSGi must understand exactly where all of the classes in every java package comes from.

Where can classes they come from?

There three options for the origin of the classes

Standard classes from the JVM

The classes, obviously, can be standard classes which come from the JVM. This is the case for all standard JAVA packages (java.*, javax.*, etc.), although, on occasion, OSGi will not recognize a class, and you need to tell it where to find it.

Classes from other Aspire Components

You will be using some classes that come from other Aspire Components. This includes:

  • com.searchtechnologies.aspire.services.*
  • Speciality components, such as RDB Connection pool, Aspire Math, etc.
    • These components are specially built to export their packages so that other components can use them

Aspire has special code to load the components in the right order:

  • Components which provide classes to other components are loaded first (components which have dependents)
  • Components which use classes from other components are loaded later (components which are dependent on other components)
  • Aspire determines what components to load in what order by examining the pom.xml file.

Classes which come from jars in the component itself

Often, the classes can be found in jars stored in the component jar itself. These are jar files which are actually embedded inside the component jar file itself as part of the build process. In other words, the component is self-contained. It has everything (inside itself) to provide classes to its own code. OSGi carefully controls all class loading:

  • Therefore, two components can have the same classes (org.apache.commons.compress.* for example) and OSGi will make sure that each components classes are only available inside the component and not available to external components.
  • When programming this is a huge pain
  • But when running components this is a massive benefit because:
    • Two components can have two versions of the same classes and it works just fine!
      • Both versions are loaded and only available to the component which needs it.
    • Therefore, we virtually eliminate “dependency hell” where different parts of the code depend on different versions of the same packages.

Okay, now let’s get to the issues that David had (and some issues he may have in the future).

When creating a component, there are two points where these java class / package dependencies I’ve described above are checked.

When building your component

This check is performed by the mvn-bundle package when you do a Maven build. Maven creates the JAR file (and the OSGi Manifest file that it contains) which provides the instructions to OSGi about where all of the packages can come from:

  • Import-Packages - The classes in these packages come from other OSGi (Aspire) components, this is for:
    • Packages which contain standard Aspire Framework classes (com.searchtechnologies.aspire.services.*, com.searchtechnologies.aspire.framework.*)
    • If you are using specialized Aspire component such as the Aspire RDB Connection Pool or Aspire Math
    • If there is a standard Java language package (such as javax.something.*) which is not understood by Apache Felix (for whatever reason)
  • Export-Packages - This is only for packages which your component wants to provide to the outside world
    • In other words, if you want to create a database connection pool component and provide database connections to other components.
    • Generally, only the core engineering team does this.
  • Private-Packages - These are packages which are provided from within the component
    • In other words, the actual code for the classes in these packages is in the Jar file:
  • Embedded-Dependencies & Embed-Transitive - This identifies what jar files will be loaded into your component
    • These jar files are specified by their maven artifact ID (e.g. they specify what “Maven Dependencies” should be included in your component)
    • These jar files will be actually included inside your jar file. If you open up your component jar file (with WinRAR, for example), you will see embedded jar files at the top level [if you have any].

And there is one last trick. Sometimes, you will have code which requires classes from some java package, but the code will never, ever be executed. This happens all the time. For example, debugging code, testing code, etc. – especially deep inside of someone else’s jar file. The Maven Bundle will complain about these java packages (as it is doing below, in David’s example). It will say: “Hey you! Stupid programmer! You are using these java class files, but I can’t find them anywhere! You need to either add these jar files to your component, or add the code, or something.”. The Maven Bundle program is very picky and very literal – if it can’t find some java packages, it refuses to let you build your component. Imagine that the Maven Bundle program is like some overworked government bureaucrat who points out that you failed to fill out all of the fields on your application form and refuses to make an exception for you.

And so, there is a work-around, and that is to specify a “!package” (for example, “!org.apache.jcp.xml.dsig.internal.dom”) inside the “Import-Packages” directive. When you do this, you are telling the Maven Bundle: “Hey Bundle: I know you can’t find this package, but it is OK. This code will never be executed. I promise. So please, don’t complain about it and just build my component, please?”

When running Aspire

If you’ve gotten to this point, you will probably be OK. The Maven Bundle plugin is very picky, and if you get it to build, then probably everything will work okay when you actually run your component in Aspire.

There are three types of errors you can get when actually running Aspire:

  • Sometimes you get a run-time execution error, along the lines of “Unable to resolve java packages”.
    • This happens very quickly (as OSGi is loading your component).
    • This means you are missing jar files which should be inside your component.
    • Figure out what’s missing and make sure those jars are in the Maven Dependencies and then copied into your component jar file at the top level
  • “Class Not Found” – for simple classes used inside some source code somewhere.
    • When this happens for a simple class declaration, it means that you’re missing some jar files, AND you incorrectly specified those java packages as “!package” in the Import-Packages declaration (see above)
    • In other words, you told the Maven Bundle: “Hey, I promise this code will never be executed”, but, in fact, it was.
      • So, you need to find out which jar contains that code, add it to your maven dependencies, and then get that jar file loaded into your component
  • "Class Not Found” – for dynamically loaded classes
    • Sometimes, third-party jars will have their own dynamic class loading architectures.
    • This is the bane of existence for core engineers, and is the most difficult problem to solve.
    • Typically, the solution for fixing it is to make sure that the parent class loader is the OSGi class loader (and not the java JVM top-level system class loaded), but in practice this can be difficult unless the original programmer of the third-party jar has ever thought about running their library inside of OSGi.

More information can be found here

I hope this was helpful.

Cheers, Paul