Category: Analytics

Dataset too big for R ?

In case you have a dataset too big for fitting in memory for R, there is a package called biglm .

You install it like this-

install.packages("biglm", dep=TRUE)

 

 

  Information on package ‘biglm’

Description:

Package:       biglm
Type:          Package
Title:         bounded memory linear and generalized linear models
Version:       0.6
Date:          2005-09-24
Author:        Thomas Lumley
Maintainer:    Thomas Lumley <tlumley@u.washington.edu>
Description:   Regression for data too large to fit in memory
License:       GPL
Suggests:      RSQLite, RODBC
Enhances:      leaps
Packaged:      Tue Feb 24 10:47:44 2009; tlumley
Built:         R 2.8.1; i386-pc-mingw32; 2009-02-24 21:35:12; windows

Index:

bigglm                  Bounded memory linear regression
biglm                   Bounded memory linear regression
predict.bigglm          Predictions from a biglm/bigglm

and in case you are the statistical kind of chap who want to know what’s IN the code for these functions

function (formula, data, family = gaussian(), …)
UseMethod("bigglm", data)
<environment: namespace:biglm>

 

R tip of the day – If you want to know what an R Function say procmeans does…..all you need to do is type procmeans at the command prompt , and it will tell you what is inside the code.

If it gives an error most probably you need to

1) Install

and 2) Load the package containing the function

Which are conveniently here

image

credit source –http://www.nabble.com/R-f13819.html

An R Package only for SAS Users

Dear All,

I am doing some research into creating a R Package for SAS language Users.

The name of the beta package is “ Anne”, but I am open to suggestions for the name please.

The basic idea is to enable SAS language Users (especially Windows SAS language  users) to get a feel to try out the R package without getting overwhelmed with the matrix level powerful capabilities as well as command line interface.

Creating new functions is quite easy as the following code shows.

The first R code for the “Anne 1.0” Package is

procunivariate(x) <- function(x) summary(x)

procimportcsv(x) <- function(x) read.table(x,header=TRUE,

                           + sep=”,”, row.names=”id”, na.string=”   “)

libname(x) <-function(x) setwd(x)

 

Note I am tweaking the code as we speak and would be trying to add one proc per week.

But how to put functions in a R Package ?

This is how to create a R package –( To be Continued)

Note- SAS here refers to SAS Language.

 

Learning SAS for SPSS Users

SAS Publishing just came out with a nice and nifty 28 page pdf document “ Coming To SAS FROM SPSS – A programming approach” Its a nice read, very useful for people curious or willing to try  SAS after learning SPSS, and very well written by Susan J Slaughter and Lora D , who have written “The Little SAS Book” , one of the most popular SAS handbooks ever written.

 

You can download it or plainly read it from

http://support.sas.com/publishing/bbu/companion_site/62272.pdf

SPSS of course has very nice menu driven setting, while SAS programmers generally prefer the scripting way of writing code- they do have menus in various products.

Linux for busy people

For people who always wanted to try out Linux but never had the time or the energy ( or courage to risk  moving to a Linux only environment) here is a great application which allows you to keep Linux as well as Windows for double booting environment . You need 256 mb ram and 5 gb hard disk and you are good to go. It is a single click download and install to try out Ubuntu Linux and it preserves your Windows too.

If you wanted to try out R with Linux , then it is an easy way out for you ( and me).

Saves quite a lot of money per desktop per OS and per office productivity software.

Sounds good to be true – well the site is http://wubi-installer.org/

Here are some screenshots courtesy of the site itself-

Ubuntu Desktop Preview

And if you need to install your favorite applications (like R , Subversion) and don’t want to command line your way the solution is quite simple – It is called Synaptic and it is free and downloadable here

http://www.nongnu.org/synaptic/index.html

KNIME and Zementis shake hands

Two very good and very customer centric (and open source ) companies shook hands on a strategic partnership today.

Knime  www.knime.org and Zementis www.zementis.com .

Decision Stats has been covering these companies and both the products are amazing good, synch in very well thanks to the support of the PMML standard and lower costs considerably for the consumer. (http://www.decisionstats.com/2009/02/knime/ ) and http://www.decisionstats.com/2009/02/interview-michael-zeller-ceozementis/ )

While Knime has both a free personal as well as a commercial license , it supports R thanks to the PMML (www.dmg.org initiative ). Knime also supports R very well .

See http://www.knime.org/blog/export-and-convert-r-models-pmml-within-knime

The following example R script learns a decision tree based on the Iris-Data and exports this as PMML and as an R model which is understood by the R Predictor node:

# load the library for learning a tree model
library(rpart);
# load the pmml export library
library(pmml);
# use class column as predicted column to build decision tree
dt <- rpart(class~., R)
# export to PMML
r_pmml <- pmml(dt)
# write the PMML model to an export file
write(toString(r_pmml), file="C:/R.pmml")
# provide the native R model at the out-port
R<-dt

 

Zementis takes the total cost of ownership and total pain of creating scored models to something close to 1$ /hour thanks to using their proprietary ADAPA engine.