Installing and Using Spark easily with Python or R on Ubuntu #python #rstats

  1. Download spark from https://spark.apache.org/downloads.html (say to home/ajay/Desktop/BDA )
  2. Change to the directory from terminal cd /home/ajay/Desktop/BDA
  3. Unzip the file
    tar -xvf spark-1.4.1-bin-hadoop2.6.tgz
  4. Change to the directory created ( say you unizpped spark file above and renamed it spark) ajay@Inspiron-3542:~/Desktop/BDA$ cd spark
  5. Run the command ./bin/pyspark Screenshot from 2015-08-18 11:58:34
  6. To look at local jobs see http://192.168.1.18:4040/jobs/ (or based on what you get from your terminal after running command in step 4)

Screenshot from 2015-08-18 11:57:56

 

 

 

 

 

 

 

to do this with R just use .bin/sparkR

Screenshot from 2015-08-18 12:02:19

 

Sources- http://stackoverflow.com/questions/30483409/installing-apache-spark-on-ubuntu-14-04

 

Author: Ajay Ohri

http://about.me/ajayohri

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s