Home » cloud

Category Archives: cloud

Is the biggest threat to the cloud US Government over reach

A presentation I made today on a talk

2013 Thank You Note

I would like to write a thank you note to  some of the people who helped make Decisionstats.com possible . We had a total of 150,644 views this year.For that, I have to thank you dear readers for putting up with me- it is now our seventh year.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total
13,940 12,153 12,948 13,371 12,778  12,085  12,894  11,934  9,914  14,764  12,907  10,956  150,644

I would like to thank Chris  (of Mashape) for helping me with some of the interviews I wrote here .I did 26 interviews this year for Programmable Web and a total of 30+ articles including the interviews in 2013.

Of course- we have now reached 116 excellent interviews on Decisionstats.com alone ( see http://goo.gl/V6UsCG )I would like to thank each one of the interviewees who took precious time to fill out the questions.

Sponsors- I would like to thank Dr Eric Siegel ( individually as an author and as founder chair of www.pawcon.com ) , Nadja and Ingo (for Rapid-Miner) , Dr Jonathan ( for Datamind) , Chris M (for Statace.com ) , Gergely ( Author) and many more during all these six years who have kept us afloat and the servers warm in these days of cold reflection, including Gregory (of KDNuggets.com) and erstwhile AsterData founders.

Training Partners- I would like to thank Lovleen Bhatia ( of Edureka  for giving me the opportunity to make http://www.edureka.in/r-for-analytics which now has 1721 learners as per http://www.edureka.in/)

I would also specially say Thank you to Jigsaw Academy for giving me the opportunity to create
the first affordable and quality R course in Asia http://analyticstraining.com/2013/jigsaw-completes-training-of-300-students-on-r/

These training courses including those by Datamind and Coursera remain a formidable and affordable alternative to many others catching up in the analytics education game in India ( an issue I wrote here)

Each and Everyone of my students (past and present) and Everyone in the #rstats  and SAS-L community, including people who may have been left out.

Thank you sir, for helping me and Decisionstats.com !

Wish each one of you a very happy and Joyous Happy New Year and a great and prosperous 2014!

Even more variety in Cloud Computing Instances from AWS

If you ever complain that R is slow because it stores it in RAM , well here is a whole lot of RAM for you.

 

From-

http://aws.typepad.com/aws/2013/12/amazon-ec2-new-i2-instance-type-available-now.html

The Specs
Here are the instance sizes and the associated specs:

Instance Name vCPU Count RAM
Instance Storage (SSD) Price/Hour
i2.xlarge 4 30.5 GiB 1 x 800 GB $0.85
i2.2xlarge 8 61 GiB 2 x 800 GB $1.71
i2.4xlarge 16 122 GiB 4 x 800 GB $3.41
i2.8xlarge 32 244 GiB 8 x 800 GB $6.82

 

This leaves these guys way behind

https://cloud.google.com/products/compute-engine/

High Memory

Machines for tasks that require more memory relative to virtual cores

Instance type Virtual Cores Memory Price (US$)/Hour
(US hosted)
Price (US$)/Hour
(Europe hosted)
n1-highmem-2 2 13GB $0.244 $0.275
n1-highmem-4 4 26GB $0.488 $0.549
n1-highmem-8 8 52GB $0.975 $1.098
n1-highmem-16 16 104GB $1.951 $2.196

Google Compute Engine pricing much more honest than AWS and Azure Pricing

Google compute is priced like a Taxi- 10 minutes and then in blocks of 1 minute .

The web page is clear and simple and does not confuse

Google Compute Engine Pricing is more honest in its GA

Microsoft dazzles but hides pricing between layers of pages, while Amazon does it with dropdowns and contact us baits. IBM is impossible  to get an upfront honest price from and I am still trying to figure out Oracle Cloud

Hopefully GCE uptime will be better than Gmail Uptime!!!

——————————————————————————————————————————-

From

https://cloud.google.com/products/compute-engine/

Pricing

All machine types are charged a minimum of 10 minutes. For example, if you run your instance for 2 minutes, you will be billed for 10 minutes of usage. After 10 minutes, instances are charged in 1 minute increments, rounded up to the nearest minute. For example, an instance that lives for 11.25 minutes will be charged for 12 minutes of usage.

If you would like to discuss pricing for long-term commitments, please contact sales

Machine Type Pricing

Standard

Instance type Virtual Cores Memory Price (US$)/Hour
(US hosted)
Price (US$)/Hour
(Europe hosted)
n1-standard-1 1 3.75GB * $0.104 $0.114
n1-standard-2 2 7.5GB $0.207 $0.228
n1-standard-4 4 15GB * $0.415 $0.456
n1-standard-8 8 30GB $0.829 $0.912
n1-standard-16 16 60GB $1.659 $1.825

High Memory

Machines for tasks that require more memory relative to virtual cores

Instance type Virtual Cores Memory Price (US$)/Hour
(US hosted)
Price (US$)/Hour
(Europe hosted)
n1-highmem-2 2 13GB $0.244 $0.275
n1-highmem-4 4 26GB $0.488 $0.549
n1-highmem-8 8 52GB $0.975 $1.098
n1-highmem-16 16 104GB $1.951 $2.196

High CPU

Machines for tasks that require more virtual cores relative to memory

Instance type Virtual Cores Memory Price (US$)/Hour
(US hosted)
Price (US$)/Hour
(Europe hosted)
n1-highcpu-2 2 1.80GB $0.131 $0.146
n1-highcpu-4 4 3.60GB $0.261 $0.292
n1-highcpu-8 8 7.20GB $0.522 $0.584
n1-highcpu-16 16 14.40GB $1.044 $1.167

Shared Core

Machines for tasks that don’t require a lot of resources but do have to remain online for long periods of time.

Instance type Virtual Cores Memory Price (US$)/Hour
(US hosted)
Price (US$)/Hour
(Europe hosted)
f1-micro 1 0.60GB $0.019 $0.021
g1-small 1 1.70GB $0.054 $0.059

Network Pricing

Ingress Free
Egress to the same Zone. Free
Egress to a different Cloud service within the same Region. Free
Egress to a different Zone in the same Region (per GB) $0.01
Egress to a different Region within the US $0.01 ***
Inter-continental Egress At Internet Egress Rate
Internet Egress (Americas/EMEA destination) per GB
0-1 TB in a month $0.12
1-10 TB $0.11
10+ TB $0.08
Internet Egress (APAC destination) per GB
0-1 TB in a month $0.21
1-10 TB $0.18
10+ TB $0.15

Persistent Disk Pricing

Provisioned space $0.04 GB / month
Snapshot storage $0.125 GB / month
IO operations No additional charge

Image Storage

Image storage $0.085 GB / month

IP Address Pricing

Static IP address (assigned but unused) $0.01 / hour
Static IP address (assigned and in use) Free
Ephemeral IP address (attached to instance) Free

VERSUS

http://aws.amazon.com/ec2/pricing/

Pricing is per instance-hour consumed for each instance, from the time an instance is launched until it is terminated or stopped. Each partial instance-hour consumed will be billed as a full hour.

On-Demand Instance Prices

Region:US East (N. Virginia)US West (Oregon)US West (Northern California)EU (Ireland)Asia Pacific (Singapore)Asia Pacific (Tokyo)Asia Pacific (Sydney)South America (Sao Paulo)
$  US Dollar
Linux/UNIX Usage
General Purpose – Current Generation
m3.xlarge $0.450 per Hour
m3.2xlarge $0.900 per Hour
General Purpose – Previous Generation
m1.small $0.060 per Hour
m1.medium $0.120 per Hour
m1.large $0.240 per Hour
m1.xlarge $0.480 per Hour
Compute Optimized – Current Generation
c3.large $0.150 per Hour
c3.xlarge $0.300 per Hour
c3.2xlarge $0.600 per Hour
c3.4xlarge $1.200 per Hour
c3.8xlarge $2.400 per Hour
Compute Optimized – Previous Generation
c1.medium $0.145 per Hour
c1.xlarge $0.580 per Hour
cc2.8xlarge $2.400 per Hour
GPU Instances – Current Generation
g2.2xlarge $0.650 per Hour
GPU Instances – Previous Generation
cg1.4xlarge $2.100 per Hour
Memory Optimized – Current Generation
m2.xlarge $0.410 per Hour
m2.2xlarge $0.820 per Hour
m2.4xlarge $1.640 per Hour
cr1.8xlarge $3.500 per Hour
Storage Optimized – Current Generation
hi1.4xlarge $3.100 per Hour
hs1.8xlarge $4.600 per Hour
Micro Instances
t1.micro $0.020 per Hour

Pricing is per instance-hour consumed for each instance, from the time an instance is launched until it is terminated or stopped. Each partial instance-hour consumed will be billed as a full hour.

Reserved Instances

Reserved Instances give you the option to make a low, one-time payment for each instance you want to reserve and in turn receive a significant discount on the hourly charge for that instance. There are three Reserved Instance types (Light, Medium, and Heavy Utilization Reserved Instances) that enable you to balance the amount you pay upfront with your effective hourly price.

The following tables display the Reserved Instance Prices available directly from AWS. In addition to Reserved Instances for Linux/UNIX and Windows operating systems specified below, we also offer Reserved Instances forAmazon EC2 running SUSE Linux Enterprise ServerAmazon EC2 running Red Hat Enterprise Linux, and Amazon EC2 running Microsoft SQL Server. Dedicated Reserved Instances are also available.

Light Utilization Reserved Instances

Region:US East (N. Virginia)US West (Northern California)US West (Oregon)EU (Ireland)Asia Pacific (Singapore)Asia Pacific (Tokyo)Asia Pacific (Sydney)South America (Sao Paulo)
$  US Dollar
1 yr Term 3 yr Term
Upfront Hourly Upfront Hourly
General Purpose – Current Generation
m3.xlarge $439 $0.254 per Hour $686 $0.201 per Hour
m3.2xlarge $879 $0.508 per Hour $1372 $0.401 per Hour
General Purpose – Previous Generation
m1.small $61 $0.034 per Hour $96 $0.027 per Hour
m1.medium $122 $0.068 per Hour $192 $0.054 per Hour
m1.large $243 $0.136 per Hour $384 $0.108 per Hour
m1.xlarge $486 $0.271 per Hour $768 $0.215 per Hour
Compute Optimized – Current Generation
c3.large $167 $0.093 per Hour $252 $0.082 per Hour
c3.xlarge $333 $0.186 per Hour $503 $0.164 per Hour
c3.2xlarge $667 $0.373 per Hour $1006 $0.327 per Hour
c3.4xlarge $1333 $0.745 per Hour $2012 $0.654 per Hour
c3.8xlarge $2666 $1.49 per Hour $4024 $1.308 per Hour
Compute Optimized – Previous Generation
c1.medium $161 $0.09 per Hour $243 $0.079 per Hour
c1.xlarge $644 $0.36 per Hour $972 $0.316 per Hour
cc2.8xlarge $1762 $0.904 per Hour $2710 $0.904 per Hour
GPU Instances – Current Generation
g2.2xlarge $772 $0.499 per Hour $1143 $0.392 per Hour
Memory Optimized – Current Generation
m2.xlarge $272 $0.169 per Hour $398 $0.136 per Hour
m2.2xlarge $544 $0.338 per Hour $796 $0.272 per Hour
m2.4xlarge $1088 $0.676 per Hour $1592 $0.544 per Hour
cr1.8xlarge $2474 $1.54 per Hour $3846 $1.225 per Hour
Storage Optimized – Current Generation
hs1.8xlarge $3968 $2.24 per Hour $5997 $1.81 per Hour
hi1.4xlarge $2576 $1.477 per Hour $3884 $1.15 per Hour
Micro Instances
t1.micro $23 $0.012 per Hour $35 $0.012 per Hour

Medium Utilization Reserved Instances

Region:US East (N. Virginia)US West (Northern California)US West (Oregon)EU (Ireland)Asia Pacific (Singapore)Asia Pacific (Tokyo)Asia Pacific (Sydney)South America (Sao Paulo)
$  US Dollar
1 yr Term 3 yr Term
Upfront Hourly Upfront Hourly
General Purpose – Current Generation
m3.xlarge $1034 $0.156 per Hour $1631 $0.123 per Hour
m3.2xlarge $2069 $0.313 per Hour $3262 $0.247 per Hour
General Purpose – Previous Generation
m1.small $139 $0.021 per Hour $215 $0.017 per Hour
m1.medium $277 $0.042 per Hour $430 $0.033 per Hour
m1.large $554 $0.084 per Hour $860 $0.067 per Hour
m1.xlarge $1108 $0.168 per Hour $1720 $0.133 per Hour
Compute Optimized – Current Generation
c3.large $383 $0.056 per Hour $591 $0.049 per Hour
c3.xlarge $766 $0.112 per Hour $1182 $0.097 per Hour
c3.2xlarge $1532 $0.224 per Hour $2364 $0.195 per Hour
c3.4xlarge $3064 $0.447 per Hour $4728 $0.389 per Hour
c3.8xlarge $6127 $0.894 per Hour $9456 $0.778 per Hour
Compute Optimized – Previous Generation
c1.medium $370 $0.054 per Hour $571 $0.047 per Hour
c1.xlarge $1480 $0.216 per Hour $2284 $0.188 per Hour
cc2.8xlarge $4146 $0.54 per Hour $6378 $0.54 per Hour
GPU Instances – Current Generation
g2.2xlarge $1987 $0.34 per Hour $3311 $0.294 per Hour
Memory Optimized – Current Generation
m2.xlarge $651 $0.103 per Hour $992 $0.08 per Hour
m2.2xlarge $1302 $0.206 per Hour $1984 $0.16 per Hour
m2.4xlarge $2604 $0.412 per Hour $3968 $0.32 per Hour
cr1.8xlarge $5958 $0.93 per Hour $9006 $0.735 per Hour
Storage Optimized – Current Generation
hs1.8xlarge $9200 $1.38 per Hour $14103 $1.11 per Hour
hi1.4xlarge $5973 $0.909 per Hour $9133 $0.705 per Hour
Micro Instances
t1.micro $54 $0.007 per Hour $82 $0.007 per Hour

Heavy Utilization Reserved Instances

Loading pricing data…

Reserved Instances can be purchased directly from AWS for 1 or 3 year terms. Using the Reserved Instance Marketplace, you have the flexibility to purchase Reserved Instances from AWS Reserved Instance Marketplace Sellers for terms ranging between 1 month to 36 months (depending on available selection). In either case, the one-time fee per instance is non-refundable. If your needs change, you can also request to move your Reserved Instance to another Availability Zone within the same region, change its Network Platform, or, for Linux/UNIX and Windows RIs, modify the instance type of your reservation to another type in the same instance family at no additional cost.

Light and Medium Utilization Reserved Instances also are billed by the instance-hour for the time that instances are in a running state; if you do not run the instance in an hour, there is zero usage charge. Partial instance-hours consumed are billed as full hours. Heavy Utilization Reserved Instances are billed for every hour during the entire Reserved Instance term (which means you’re charged the hourly fee regardless of whether any usage has occurred during an hour).

If Microsoft or Red Hat chooses to increase the license fees that it charges for Windows or Red Hat Enterprise Linux, we may correspondingly increase the per-hour usage rate for previously purchased Reserved Instances with Windows or Red Hat Enterprise Linux. The initial one-time payment for a Reserved Instance will be unaffected in this situation. Any such changes for Windows would be made between Dec 1 – Jan 31, and with at least 30 days’ notice. Any such changes for Red Hat Enterprise Linux would be made at least 30 days’ notice. If the per-hour usage rate does increase, you may continue to use your Reserved Instance with Windows or Red Hat Enterprise Linux with the new per-hour usage rate, convert your Reserved Instance with Windows or Red Hat Enterprise Linux to a Reserved Instance with Linux/UNIX, or request a pro rata refund of the upfront fee you paid for the Reserved Instance with Windows or Red Hat Enterprise Linux.

Reserved Instances are available for Linux/UNIX, Windows, Red Hat Enterprise Linux, and SUSE Linux Enterprise operating systems. You can also optionally reserve instances in Amazon VPC at the same prices as shown above.Click here to learn more about Reserved Instances.

Reserved Instance Volume Discounts

When you have purchased a sufficient number of Reserved Instances in an AWS Region, you will automatically receive discounts on your upfront fees and usage fees for future purchases of Reserved Instances in that AWS Region. Reserved Instance Tiers are determined based on the total list price (non-discounted price) of upfront fees for the active Reserved Instances you have per AWS Region. It is important to note that Reserved Instance Tiers do not apply to Reserved Instances purchased from the Reserved Instance Marketplace. A complete list of the Reserved Instance Tiers is shown below:

Reserved Instance Volume Discounts

Total Reserved Instances

Upfront Discount

Hourly Discount
Less than $250,000
0%
0%
$250,000 to $2,000,000
10%
10%
$2,000,000 to $5,000,000
20%
20%
More than $5,000,000

http://www.windowsazure.com/EN-US/pricing/overview/

Up to 29.5% savings vs.
Pay as You Go plan
Starting at $500/month

PURCHASE

SUPPORT OPTIONSCustomizable support options to provide the best available expertise for your needs.PURCHASE

Purchase plans

* Comparisons based on the pay-as-you-go plan.

Monthly
Committed Spend

  • $500 TO $14,999
  • $15,000 TO $39,999
  • $40,000 AND ABOVE

6-Month
Monthly Pay

  • discount *
  • 20%
  • 23%
  • 27%

VIEW DETAILS | BUY

12-Month
Monthly Pay

  • discount *
  • 22.5%
  • 25.5%
  • 29.5%

VIEW DETAILS | BUY

6-Month
Pre-Pay

  • discount *
  • 22.5%
  • 25.5%
  • 29.5%

VIEW DETAILS | BUY

12-Month
Pre-Pay

  • discount *
  • 25%
  • 28%
  • 32%

VIEW DETAILS | BUY

http://www.windowsazure.com/en-us/offers/commitment-plans

Usage Quotas

The following monthly usage quotas are applied. If you need more than these limits, please contact customer service at any time so that we can understand your needs and adjust these limits appropriately.

Cloud Services and Virtual Machines

The standard quota is 20 concurrent Standard Small (A1) compute instances or an equivalent number of other types or sizes of compute instances as determined by using the compute instance quota conversion table below.

1 COMPUTE INSTANCE IN THE FOLLOWING NUMBER OF EQUIVALENT STANDARD SMALL (A1) INSTANCES
Extra Small (A0) 1
Small (A1) 1
Medium (A2) 2
Large (A3) or A6 4
Extra Large (A4)  or A7 8

Storage

  • 5 concurrent storage accounts

Active Directory

  • 150,000 objects

Also see

http://www.ibm.com/cloud-computing/social/us/en/planspricing/

https://cloud.oracle.com/mycloud/f?p=service:java_pricing:0:::::

 

Java S1

$249 / MonthBuy Now- S1

  • 1

    Oracle WebLogic Server 1

  • 1.5 GB

    RAM for Java Heap 2

  • 5 GB

    File Storage 3

  • 50 GB

    Data Transfer 4

Java S2

$499 / MonthBuy Now- S2

  • 2

    Oracle WebLogic Servers 1

  • 3 GB

    RAM for Java Heap 2

  • 10 GB

    File Storage 3

  • 250 GB

    Data Transfer 4

Java S4

$1,499 / MonthBuy Now- S4

  • 4

    Oracle WebLogic Servers 1

  • 6 GB

    RAM for Java Heap 2

  • 25 GB

    File Storage 3

  • 500 GB

    Data Transfer 4

Understanding the Google Cloud

Google has a lot of services, so I really like this simple explanation of them. Though I may want a clickable , one more level of detail to make it interactive (esp Google cloud SQL vs Google Big Query- love in a tech documentation??)

google cloud

Source-

https://cloud.google.com/resources/articles/storage-overview

I wish technical documentation had more examples of lucid , infographic like explanations.

 

R in Oracle Java Cloud and Existing R – Java Integration #rstats

So I finally got my test plan accepted for a 1 month trial to the Oracle Public Cloud at https://cloud.oracle.com/ .

oc1 I am testing this for my next book R for Cloud Computing ( I have already covered Windows Azure, Amazon AWS, and in the middle of testing Google Compute).

Some initial thoughts- this Java cloud seemed more suitable for web apps, than for data science ( but I have to spend much more time on this).

I really liked the help and documentation and tutorials, Oracle has invested a lot in it to make it friendly to enterprise users.

Hopefully the Oracle R Enterprise  ORE guys can talk to the Oracle Cloud department and get some common use case projects going.

oc3.7

In the meantime, I did a roundup on all R -Java projects.

They include- (more…)

Amazon AWS Data Pipeline

Ok I missed this one as it came on Dec 20.  I think the AWS data pipeline is a really important step forward for cloud enabled analytics.

dp-how-dp-works-v2

http://aws.amazon.com/about-aws/whats-new/2012/12/20/announcing-aws-data-pipeline/

What is AWS Data Pipeline?

AWS Data Pipeline is a web service that you can use to automate the movement and transformation of data. (more…)

Follow

Get every new post delivered to your Inbox.

Join 744 other followers