Running a Storm Cluster on Ubuntu

Step One:

Install Ubuntu Zookeeper Package.

sudo apt-get install zookeeper

Zooker is located in /usr/share/zookeper

Revise /etc/zookeeper/conf/zoo.cfg for any config changes

Step two:

Download and unpack storm, for instance in /opt/ and create a storm symlink pointing to the distribution directory.

Edit /opt/storm/conf/storm.yaml

you should configure zookeeper:

storm.zookeeper.servers:
   - 10.101.20.218

Also create /var/lib/storm for data, or modify the relevant entry in storm.yaml

Optional

Create a user for storm and chown the distribution directory.

Step Three

Start nimbus:

sudo su storm -c ‘./bin/storm nimbus &’

The output will be visible in logs/nimbus.log

After Starting nimbus, start the ui:

sudo su storm -c ‘./bin/storm ui &’

Step Four

On the slave nodes, install the storm distribution by repeating the steps above.

Edit Storm.yaml and add the entries for supervisor:

nimbus.host: < master address >

supervisor.slots.ports: 
  - 6700
  - 6701
  - 6702

You can specify more and/or other ports, this reflects a three worker configuration.

Now start supervisor.

sudo su storm -c ‘./bin/storm supervisor &’

Running stuff

To register a topology with the cluster, execute:

./bin/storm jar < path to your jar > < topology main class > < topology name >

To kill a topology, run:

./bin/storm kill < topology name >

Advertisements
This entry was posted in Distributed Computing and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s