I really liked the no hassles way Biocnoductor has put a cloud AMI loaded with RStudio to help people learn R, and even try using R from within a browser in the cloud.
Not only is the tutorial very easy to use- they also give away 2 hours for free computing!!!
Check it out-
Step 1
Step 2
Step 3
and wow! I am using Google Chrome to run R ..and its awesome!
Interesting- check out two hours for free — all you need is a browser and internet connection
There are still some graphs that cannot be yet made in R using a straightforward function or package.
One is sunburst (which is radial kind of treemap-that can be made in R). See diagrams below to see the difference. Note sunburst is visually similar to coxcomb (Nightangle) graphs. Coxcombs can also be manipulated and made- but I am yet to find a straight package to make coxcomb using a single function _histdata package in R comes close in terms on historical datasets.
The Treemap uses a rectangular, space-filling slice-and-dice technique to visualize objects in the different levels of a hierarchy. The area and color of each item corresponds to an attribute of the item as well.
The Sunburst technique is an alternative, space-filling visualization that uses a radial rather than a rectangular layout. An example Sunburst display is shown below. citation- http://www.cc.gatech.edu/gvu/ii/sunburst/
Coxcomb Below-
Other is cartogram -whose packages are MIA -RCartogram is very basic package http://www.omegahat.org/Rcartogram/ – It is better to use Toad Scraper software than R for this kind of map.
Cartograms are used to produce spatial plots where the boundaries of regions can be transformed to be proportional to density/counts/populations. This is illustrated in plots such as
Mark Newman’s plot of People living with HIV/AIDS
Citation: Friendly, Michael (2001), Gallery of Data Visualization, Electronic document, http://www.datavis.ca/gallery/,Accessed: 03/23/2012 18:23:33
A common example in business analytics data is to take a random sample of a very large dataset, to test your analytics code. Note most business analytics datasets are data.frame ( records as rows and variables as columns) in structure or database bound.This is partly due to a legacy of traditional analytics software.
Here is how we do it in R-
• Refering to parts of data.frame rather than whole dataset.
Using square brackets to reference variable columns and rows
The notation dataset[i,k] refers to element in the ith row and jth column.
The notation dataset[i,] refers to all elements in the ith row .or a record for a data.frame
The notation dataset[,j] refers to all elements in the jth column- or a variable for a data.frame.
For a data.frame dataset
> nrow(dataset) #This gives number of rows
> ncol(dataset) #This gives number of columns
An example for corelation between only a few variables in a data.frame.
> cor(dataset1[,4:6])
Splitting a dataset into test and control.
ts.test=dataset2[1:200] #First 200 rows
ts.control=dataset2[201:275] #Next 75 rows
• Sampling
Random sampling enables us to work on a smaller size of the whole dataset.
use sample to create a random permutation of the vector x.
Suppose we want to take a 5% sample of a data frame with no replacement.
Let us create a dataset ajay of random numbers
#This is the kind of code line that frightens most MBAs!!
Note we use the round function to round off values.
ajay=as.data.frame(ajay) nrow(ajay)
[1] 20
> ncol(ajay)
[1] 10
This is a typical business data scenario when we want to select only a few records to do our analysis (or test our code), but have all the columns for those records. Let us assume we want to sample only 5% of the whole data so we can run our code on it
Then the number of rows in the new object will be 0.05*nrow(ajay).That will be the size of the sample.
The new object can be referenced to choose only a sample of all rows in original object using the size parameter.
We also use the replace=FALSE or F , to not the same row again and again. The new_rows is thus a 5% sample of the existing rows.
Then using the square backets and ajay[new_rows,] to get-
You can change the percentage from 5 % to whatever you want accordingly.
I modified the query I wrote earlier at http://www.decisionstats.com/using-google-analytics-with-r/to get multiple dimensions and metrics from the Google Analytics API, like hour of day,day of week to get cyclical parameters.We are adding the dimensions, and metrics to bring more depth in our analysis.Basically we are trying to do a time series analysis for forecasting web analytics data( which is basically time -stamped and rich in details ).
Basically I am modifying the dimensions and metrics parameters of the query code using the list at
http://code.google.com/apis/analytics/docs/gdata/dimsmets/dimsmets.html
query <- QueryBuilder() query$Init(start.date = "2011-08-20", end.date = "2012-08-25", dimensions = c("ga:date","ga:hour","ga:dayOfWeek"), metrics = c("ga:visitors","ga:visits","ga:pageviews","ga:timeOnSite"), sort = c("ga:date","ga:hour","ga:dayOfWeek"), table.id = paste(profiles$profile[3,3])) #5. Make a request to get the data from the API ga.data <- ga$GetReportData(query) #6. Look at the returned data str(ga.data) head(ga.data$data)
and we need the lubridate package to create a ymd:hour (time stamp) since GA gives data aggregated at a hourly level at most. Also we need to smoothen the effect of weekend on web analytics data.
#Using package lubridate to convert character dates into time
To be continued-
This is a continuation of the previous post on using Google Analytics .
Now that we have downloaded and plotted the data- we try and fit time series to the website data to forecast future traffic.
Some observations-
1) Google Analytics has 0 predictive analytics, it is just descriptive analytics and data visualization models (including the recent social analytics). However you can very well add in basic TS function using R to the GA API.
Why do people look at Website Analytics? To know today’s traffic and derive insights for the Future
2) Web Data clearly follows a 7 day peak and trough for weekly effects (weekdays and weekends), this is also true for hourly data …and this can be used for smoothing historic web data for future forecast.
3) On an advanced level, any hugely popular viral posts can be called a level shift (not drift) and accoringly dampened.
Test and Control!
Similarly using ARIMAX, we can factor in quantity and tag of posts as X regressor variables.
and now the code-( dont laugh at the simplicity please, I am just tinkering and playing with data here!)
You need to copy and paste the code at the bottom of this post http://www.decisionstats.com/using-google-analytics-with-r/ if you want to download your GA data down first.
Note I am using lubridate ,forecast and timeSeries packages in this section.
#Plotting the Traffic plot(ga.data$data[,2],type="l")
library(timeSeries)
library(forecast)
#Using package lubridate to convert character dates into time library(lubridate) ga.data$data[,1]=ymd(ga.data$data[,1]) ls() dataset1=ga.data$data names(dataset1) <- make.names(names(dataset1)) str(dataset1) head(dataset1) dataset2 <- ts(dataset1$ga.visitors,start=0,frequency = frequency(dataset1$ga.visitors), names=dataset1$ga.date) str(dataset2) head(dataset2) ts.test=dataset2[1:200] ts.control=dataset2[201:275] #Note I am splitting the data into test and control here fitets=ets(ts.test) plot(fitets) testets=ets(ts.control,model=fitets) accuracy(testets) plot(testets) spectrum(ts.test,method='ar') decompose(ts.test) library("TTR") bb=SMA(dataset2,n=7)#We are doing a simple moving average for every 7 days. Note this can be 24 hrs for hourly data, or 30 days for daily data for month # to month comparison or 12 months for annual #We notice that Web Analytics needs sommethening for every 7 thday as there is some relation to traffic on weekedays /weekends /same time last week head(dataset2,40) head(bb,40) par(mfrow=c(2,1)) plot(bb,type="l",main="Using Seven Day Moving Average for Web Visitors") plot(dataset2,main="Original Data")
Created by Pretty R at inside-R.org
Though I still wonder why the R query, gA R code /package could not be on the cloud (why it needs to be downloaded)– cloud computing Gs?
Also how about adding some MORE predictive analytics to Google Analytics, chaps!
To be continued-
auto.arima() and forecasts!!!
cross validations!!!
and adapting the idiosyncratic periods and cycles of web analytics to time series !!
Some code to read in data from Google Analytics data. Some modifications include adding the SSL authentication code and modifying (in bold) the table.id parameter to choose correct website from a GA profile with many websites
The Google Analytics Package files can be downloaded from http://code.google.com/p/r-google-analytics/downloads/list
It provides access to Google Analytics data natively from the R Statistical Computing programming language. You can use this library to retrieve an R data.frame with Google Analytics data. Then perform advanced statistical analysis, like time series analysis and regressions.
> library(XML)
>
> library(RCurl)
Loading required package: bitops
>
> #Change path name in the following to the folder you downloaded the Google Analytics Package
>
> source(“C:/Users/KUs/Desktop/CANADA/R/RGoogleAnalytics/R/RGoogleAnalytics.R”)
>
> source(“C:/Users/KUs/Desktop/CANADA/R/RGoogleAnalytics/R/QueryBuilder.R”)
> # download the file needed for authentication
> download.file(url=”http://curl.haxx.se/ca/cacert.pem”, destfile=”cacert.pem”)
trying URL ‘http://curl.haxx.se/ca/cacert.pem’ Content type ‘text/plain’ length 215993 bytes (210 Kb) opened
URL downloaded 210 Kb
>
> # set the curl options
> curl <- getCurlHandle()
> options(RCurlOptions = list(capath = system.file(“CurlSSL”, “cacert.pem”,
+ package = “RCurl”),
+ ssl.verifypeer = FALSE))
> curlSetOpt(.opts = list(proxy = ‘proxyserver:port’), curl = curl)
An object of class “CURLHandle” Slot “ref”: <pointer: 0000000006AA2B70>
>
> # 1. Create a new Google Analytics API object
>
> ga <- RGoogleAnalytics()
>
> # 2. Authorize the object with your Google Analytics Account Credentials
>
> ga$SetCredentials(“USERNAME”, “PASSWORD”)
>
> # 3. Get the list of different profiles, to help build the query
>
> profiles <- ga$GetProfileData()
>
> profiles #Error Check to See if we get the right website
$profile AccountName ProfileName TableId
1 dudeofdata.com dudeofdata.com ga:44926237
2 knol.google.com knol.google.com ga:45564890
3 decisionstats.com decisionstats.com ga:46751946
$total.results
total.results
1 3
>
> # 4. Build the Data Export API query
>
> #Modify the start.date and end.date parameters based on data requirements
>
> #Modify the table.id at table.id = paste(profiles$profile[X,3]) to get the X th website in your profile
> # 4. Build the Data Export API query
> query <- QueryBuilder() > query$Init(start.date = “2012-01-09”, + end.date = “2012-03-20”, + dimensions = “ga:date”,
+ metrics = “ga:visitors”,
+ sort = “ga:date”,
+ table.id = paste(profiles$profile[3,3]))
>
>
> #5. Make a request to get the data from the API
>
> ga.data <- ga$GetReportData(query)
[1] “Executing query: https://www.google.com/analytics/feeds/data?start-date=2012%2D01%2D09&end-date=2012%2D03%2D20&dimensions=ga%3Adate&metrics=ga%3Avisitors&sort=ga%3Adate&ids=ga%3A46751946”
>
> #6. Look at the returned data
>
> str(ga.data)
List of 3
$ data :’data.frame’: 72 obs. of 2 variables: ..
$ ga:date : chr [1:72] “20120109” “20120110” “20120111” “20120112” … ..
$ ga:visitors: num [1:72] 394 405 381 390 323 47 169 67 94 89 …
$ aggr.totals :’data.frame’: 1 obs. of 1 variable: ..
$ aggregate.totals: num 28348
$ total.results: num 72
>
> head(ga.data$data)
ga:date ga:visitors
1 20120109 394
2 20120110 405
3 20120111 381
4 20120112 390
5 20120113 323
6 20120114 47 >
> #Plotting the Traffic >
> plot(ga.data$data[,2],type=”l”)
Update- Some errors come from pasting Latex directly to WordPress. Here is some code , made pretty-r in case you want to play with the GA api
library(XML) library(RCurl) #Change path name in the following to the folder you downloaded the Google Analytics Package source("C:/Users/KUs/Desktop/CANADA/R/RGoogleAnalytics/R/RGoogleAnalytics.R") source("C:/Users/KUs/Desktop/CANADA/R/RGoogleAnalytics/R/QueryBuilder.R") # download the file needed for authentication download.file(url="http://curl.haxx.se/ca/cacert.pem", destfile="cacert.pem") # set the curl options curl <- getCurlHandle() options(RCurlOptions = list(capath = system.file("CurlSSL", "cacert.pem", package = "RCurl"), ssl.verifypeer = FALSE)) curlSetOpt(.opts = list(proxy = 'proxyserver:port'), curl = curl) # 1. Create a new Google Analytics API object ga <- RGoogleAnalytics() # 2. Authorize the object with your Google Analytics Account Credentials ga$SetCredentials("ohri2007@gmail.com", "XXXXXXX") # 3. Get the list of different profiles, to help build the query profiles <- ga$GetProfileData() profiles #Error Check to See if we get the right website # 4. Build the Data Export API query #Modify the start.date and end.date parameters based on data requirements #Modify the table.id at table.id = paste(profiles$profile[X,3]) to get the X th website in your profile # 4. Build the Data Export API query query <- QueryBuilder() query$Init(start.date = "2012-01-09", end.date = "2012-03-20", dimensions = "ga:date", metrics = "ga:visitors", sort = "ga:date", table.id = paste(profiles$profile[3,3])) #5. Make a request to get the data from the API ga.data <- ga$GetReportData(query) #6. Look at the returned data str(ga.data) head(ga.data$data) #Plotting the Traffic plot(ga.data$data[,2],type="l")
and finally I can download my Google spreadsheet file using-
require(RCurl)
download.file(url=”http://curl.haxx.se/ca/cacert.pem”, destfile=”cacert.pem”)
b <- getURL(url,cainfo=”cacert.pem”)
write.table(b,quote = FALSE, sep = “,”,file=”test.csv”)
Previously (for past 3 5 7 hours)-
The codes at http://blog.revolutionanalytics.com/2011/09/using-google-spreadsheets-with-r-an-update.html dont work thanks to the SSL authentication issue. and the packages at [http://www.omegahat.org/RGoogleDocs/] and [https://r-forge.r-project.org/projects/rgoogledata/] are missing in action.
So I mixed the codes at http://blog.revolutionanalytics.com/2009/09/how-to-use-a-google-spreadsheet-as-data-in-r.html and http://www.brocktibert.com/blog/2012/01/19/358/
and I get this error while using http://thebiobucket.blogspot.in/2012/03/r-function-to-read-data-from-google.html#more
Error in read.table(file = file, header = header, sep = sep, quote = quote, :
more columns than column names
DECISION STATS 