Skip to content

DECISION STATS

Better Decisions === Faster Stats

  • Interviews
  • About DecisionStats
    • Decisionstats.org
      • Ajay @ arts
        • Hackers dont die
  • Interviews
  • About DecisionStats
DECISION STATS

Tag: opendata

Running R on Amazon EC2

On my second day of bludering about high technology, I came across  http://rgrossman.com/2009/05/17/running-r-on-amazons-ec2/ which describes how to run R on Amazon EC2.

I tried it out and have subsequently added some screenshots to this tutorial so as to help you run R. My intention of course was to run a R GUI preferable Revolution Enterprise on the Amazon EC 2- and crunch uhm a lot of data.

Now go through the steps as follows-

0) Logging onto Amazon Console

http://aws.amazon.com/ec2/

Note you need your Amazon Id (even the same id which you use for buying books).

Note we are into Amazon EC2 as shown by the upper tab. Click upper tab to get into the Amazon EC2

2) Choosing the right AMI-

On the left margin, you can click AMI -Images.

Now you can search for the image-

I chose Ubuntu images (linux images are cheaper) and opendata in the search as belows- I get two images.

You can choose whether you want 32 bit or 64 bit image. Thumb rule-  64 bit images are preferable for data intensive tasks.

Click on launch instance in the upper tab ( near the search feature)

2) A pop up comes up, which shows the 5 step process to launch your computing.

Choose the right compute instance- As the screenshot shows- there are various compute instances and they all are at different multiples of prices or compute units.

After choosing the compute  instance of your choice (extra large is highlighted)- click on continue-

3) Instance Details-

I did not choose cloudburst monitoring as it has a extra charge- and I am just trying out things.So I simply clicked continue.

4) Add Tag Details- If you are running a lot of instances you need to create your own tags to help you manage them. Advisable if you are running many instances.

Since I am going to run just one- I clicked continue with adding just two things OS and Stats Package.

5) Create a key pair- A key pair is an added layer of encryption. Click on create new pair and name it (note the name will be handy in coming steps)

6)

After clicking and downloading the key pair- you come into security groups. Security groups is just a set of instructions to help keep your data transfer secure. So I created a new security group.

And I added some ways in security group to connect (like SSH using Port 22)

7)  Last step- Review Details and Click Launch

8) On the Left margin click on instances ( you were in Images.>AMI earlier)

It will take some 3-5 minutes to launch an instance. You can see status as pending till then.

9) Pending instance as shown by yellow light-

10) Once the instance is running -it is shown by a green light.

Click on the check box, and on upper tab go to instance actions. Click on connect-

you see a popup with instructions like these-

  • Open the SSH client of your choice (e.g., PuTTY, terminal).
  • Locate your private key file, decisionstats2.pem
  • Use chmod to make sure your key file isn’t publicly viewable, ssh won’t work otherwise:
    chmod 400 decisionstats.pem
  • Connect to your instance using instance’s public DNS [ec2-75-101-182-203.compute-1.amazonaws.com].
  • Example

    Enter the following command line:
    ssh -i decisionstats2.pem root@ec2-75-101-182-203.compute-1.amazonaws.com

    IMPORTANT-

    If you are choosing Ubuntu Terminal to connect- you need to change the word root from above to Ubuntu above.


    12) To launch R, just type R at the terminal

    If all goes well you should be able to see this-

    choose to install any custom packages (like

    install.packages(‘doSNOW’)

    work on R using command line

    13) IMPORTANT- After doing your R work, please CLOSE your instance (

    Go to LEFT Margin-Instances-Check the check box of instance you are running- on upper tab- Instance Actions- Click Terminate.




    Note there are other Amazon Machine Images as well which have R- I found this as well-

    Amazon EC2 Ubuntu 8.10 intrepid AMI built by Eric Hammond; Eduardo Leoni added R, many R packages, JAGS, mysql-client and subversion.

    Submitted By: Eduardo L Leoni
    US East AMI ID: ami-1b9b7c72
    AMI Manifest: PolMethImages/imageR64.manifest.xml
    License: Public
    Operating System: Linux/Uni


    2) You can install Revolution R on 32 bit Ubuntu using sudo apt install revolution-r
    Various versions of Revolution R are supported on different versions of Linux
    see http://www.revolutionanalytics.com/why-revolution-r/which-r-is-right-for-me.php
    I may be willing to try Red Hat Enterprise Linux on the ec2 with 64 bit AND Revolution Enterprise to see  the maximum juice I can get. or you can try it using an image below- It would be interesting if there could be an Amazon Machine Image for that (paid-public and private-academic)

    AMI ID: ami-8ba347e2
    Name:–
    Description:–
    Source:redhat-cloud/RHEL-5-Server/5.2/x86_64/Beta-2.6.18-92.1.1/RHEL5.2-Server-x86_64-Beta-2.6.18-92.1.1.manifest.xml
    Owner:432018295444 Visibility:Public Product Code:54DBF944
    State:available Kernel ID:aki-89a347e0 RAM Disk ID:ari-88a347e1
    Image Type:machine Architecture:x86_64 Platform:Red Hat
    Root Device Type:instance-store Root Device:– Image Size:0 bytes
    Block Devices:N/A – Instance Store
    Virtualization:paravirtual
    3) My ultimate goal is to run a parallel session using all cores on an EC2 instance and a R GUI (like R Commander or Rattle)
    4) For sake of running a test- I re did the parallel test I did on my 2 core laptop
    but using 68.4 gb Memory and 8 cores (26 Compute Units brrr)
    > check <-function(n) {
    + for(i in 1:1000)
    + {
    + sme <- matrix(rnorm(100), 10,10)
    + solve(sme)
    + }
    + }
    >
    >  times <- 100
    > system.time(for(j in 1:times ) x <- check(j))
    user  system elapsed
    20.51    0.00   20.66
    Note I still got a faster response than using parallel processors (on a 2 core 3 gb memory)  BUT the gain was not as much had I tried to use foreach for running each of the 8 cores parallely. Or running multicore package as in http://www.cerebralmastication.com/2010/02/using-the-r-multicore-package-in-linux-with-wild-and-passionate-abandon/
    5) For some reason on Ubuntu 64 bit Amazon image -I cant get Revolution R (even after using sudo apt) , and I am still learning how I can try and get Enterprise Edition fired up on a 64 bit Red Hat Enterprise Linux Amazon AMI (and maybe create an all new Machine Image 😉
    6) I was also unable to use the -X command desite having x 11 as in http://rgrossman.com/2009/05/17/running-r-on-amazons-ec2/ so was not able to see graphics,
    Also it prevents me from loogin onto as root, and asks me to login as ubuntu@amazon…. I also wanted to try logging into using my Windows session but kept shuttling between my VM Player Ubuntu session and Windows

    7) Hope this was useful. I am thankful to tips from Revolution Blog, R Grossman’s Blog, Creators of the Open Data AMI, Tal’s R Statistics Blog  and Cerebral Mastication blog on this
    35.965000 -83.920000

    Please share:

    • LinkedIn
    • Facebook
    • Twitter
    Author Ajay OhriPosted on September 25, 2010September 26, 2010Categories AnalyticsTags Amazon, amazon ec2, ami, blog, change, data, downloading, EC2, enterprise, foreach, green, grossman, groups, GUI, how to, id, images, launch, learning, linux, Login, Machine, multicore, open, opendata, os, parallel, R, r commander, R GUI, red hat, Red Hat Enterprise Linux, review, revolution, revolution analytics, revolution r, rstats, search, security, statistics, stats, tutorial, ubuntu, use, vm player, windows, Word, work8 Comments on Running R on Amazon EC2

    Blog Stats

    • 1,194,819 hits

    BitCoin Address

    BitCoin Wallet

    12FxL4og4MqtZt6pRZ1LkcMQq6ddgJK4Yg

    Latest Book

    SAS for R Users

    Books by Ajay Ohri

    Python for R USers

    R for Business Analytics


    R for Cloud Computing

    On FB

    On FB

    Archives

    • January 2021 (1)
    • December 2020 (1)
    • September 2020 (1)
    • July 2020 (1)
    • June 2020 (1)
    • May 2020 (1)
    • March 2020 (1)
    • February 2020 (2)
    • January 2020 (8)
    • December 2019 (1)
    • November 2019 (2)
    • October 2019 (1)
    • May 2019 (2)
    • March 2019 (3)
    • February 2019 (1)
    • January 2019 (1)
    • December 2018 (3)
    • October 2018 (1)
    • September 2018 (7)
    • August 2018 (3)
    • July 2018 (6)
    • June 2018 (2)
    • May 2018 (1)
    • April 2018 (2)
    • March 2018 (7)
    • February 2018 (16)
    • January 2018 (4)
    • December 2017 (10)
    • November 2017 (8)
    • October 2017 (6)
    • September 2017 (4)
    • August 2017 (6)
    • July 2017 (6)
    • June 2017 (3)
    • May 2017 (4)
    • March 2017 (1)
    • February 2017 (2)
    • January 2017 (2)
    • December 2016 (3)
    • November 2016 (7)
    • September 2016 (7)
    • August 2016 (12)
    • July 2016 (12)
    • June 2016 (8)
    • May 2016 (11)
    • April 2016 (5)
    • March 2016 (22)
    • February 2016 (6)
    • January 2016 (4)
    • December 2015 (26)
    • November 2015 (19)
    • October 2015 (44)
    • September 2015 (18)
    • August 2015 (28)
    • July 2015 (12)
    • June 2015 (6)
    • May 2015 (9)
    • April 2015 (9)
    • March 2015 (11)
    • February 2015 (5)
    • January 2015 (8)
    • December 2014 (9)
    • November 2014 (10)
    • October 2014 (3)
    • September 2014 (5)
    • August 2014 (5)
    • July 2014 (14)
    • June 2014 (21)
    • May 2014 (10)
    • April 2014 (14)
    • March 2014 (15)
    • February 2014 (11)
    • January 2014 (4)
    • December 2013 (16)
    • November 2013 (19)
    • October 2013 (23)
    • September 2013 (12)
    • August 2013 (14)
    • July 2013 (9)
    • June 2013 (8)
    • May 2013 (15)
    • April 2013 (16)
    • March 2013 (8)
    • February 2013 (16)
    • January 2013 (18)
    • December 2012 (7)
    • November 2012 (8)
    • October 2012 (10)
    • September 2012 (1)
    • August 2012 (26)
    • July 2012 (31)
    • June 2012 (12)
    • May 2012 (10)
    • April 2012 (12)
    • March 2012 (27)
    • February 2012 (31)
    • January 2012 (26)
    • December 2011 (27)
    • November 2011 (40)
    • October 2011 (32)
    • September 2011 (29)
    • August 2011 (30)
    • July 2011 (30)
    • June 2011 (33)
    • May 2011 (50)
    • April 2011 (51)
    • March 2011 (31)
    • February 2011 (38)
    • January 2011 (36)
    • December 2010 (35)
    • November 2010 (77)
    • October 2010 (73)
    • September 2010 (65)
    • August 2010 (19)
    • July 2010 (14)
    • June 2010 (11)
    • May 2010 (20)
    • April 2010 (18)
    • March 2010 (8)
    • February 2010 (10)
    • January 2010 (42)
    • December 2009 (38)
    • November 2009 (60)
    • October 2009 (41)
    • September 2009 (45)
    • August 2009 (37)
    • July 2009 (21)
    • June 2009 (28)
    • May 2009 (49)
    • April 2009 (29)
    • March 2009 (39)
    • February 2009 (34)
    • January 2009 (42)
    • December 2008 (18)
    • November 2008 (37)
    • October 2008 (20)
    • September 2008 (52)
    • August 2008 (62)
    • July 2008 (42)
    • June 2008 (16)
    • May 2008 (10)
    • April 2008 (20)
    • March 2008 (15)
    • February 2008 (30)
    • January 2008 (20)
    • December 2007 (13)

    This slideshow requires JavaScript.

    • Interviews
    • About DecisionStats
      • Decisionstats.org
        • Ajay @ arts
          • Hackers dont die
    • Interviews
    • About DecisionStats
    DECISION STATS Blog at WordPress.com.
    Cancel