Aspire for Hadoop Installation Pre 2.2 (Aspire 2)

From wiki.searchtechnologies.com
Jump to: navigation, search

For Information on Aspire 3.1 Click Here

Supported versions of Hadoop

The current version of Aspire for Hadoop runs on a CDH4 (Cloudera’s Distribution Including Apache Hadoop) cluster.

Prerequisites

To install a CDH4 cluster, follow Cloudera's CDH4 Installation Guide.

Download the aspire-for-hadoop-2.2.2.zip distribution. These files will be used during the installation.

Install Aspire for Hadoop

  1. Extract the content of aspire-for-hadoop-2.0.zip file. The top level folder has the following internal structure:
    aspire-for-hadoop-2.0
    bundles
    aspire
    boot
    aspire-bootloader-2.0.jar
    system
    org.apache.felix.configadmin.jar
    org.apache.felix.http.jetty.jar
    org.apache.felix.shell.jar
    cache
    config
    settings.xml
    felix.properties
    log
    felix.properties file is a modified version from the original Aspire Distributions and can be found here: felix.properties.
  2. Copy this folder to every task tracker node of the Hadoop cluster.
  3. Set read permissions over the whole aspire-for-hadoop-2.0 folder for the user running the Hadoop task trackers.
    sudo chgrp -R hadoop-group /path/to/aspire-for-hadoop-2.0
    sudo chmod -R +r /path/to/aspire-for-hadoop-2.0
  4. Aspire for Hadoop is ready to run. See Aspire Components for Hadoop and Developing Aspire Solutions with Hadoop for more information on how to run Map/Reduce jobs based on Aspire configurations.