Tag: model

JMP and R – #rstats

An amazing example of R being used sucessfully in combination (and not is isolation) with other enterprise software is the add-ins functionality of JMP and it’s R integration.

See the following JMP add-ins which use R

http://support.sas.com/demosdownloads/downarea_t4.jsp?productID=110454&jmpflag=Y

JMP Add-in: Multidimensional Scaling using R

This add-in creates a new menu command under the Add-Ins Menu in the submenu R Add-ins. The script will launch a custom dialog (or prompt for a JMP data table is one is not already open) where you can cast columns into roles for performing MDS on the data table. The analysis results in a data table of MDS dimensions and associated output graphics. MDS is a dimension reduction method that produces coordinates in Euclidean space (usually 2D, 3D) that best represent the structure of a full distance/dissimilarity matrix. MDS requires that input be a symmetric dissimilarity matrix. Input to this application can be data that is already in the form of a symmetric dissimilarity matrix or the dissimilarity matrix can be computed based on the input data (where dissimilarity measures are calculated between rows of the input data table in R).

Submitted by: Kelci Miclaus SAS employee Initiative: All
Application: Add-Ins Analysis: Exploratory Data Analysis

Chernoff Faces Add-in

One way to plot multivariate data is to use Chernoff faces. For each observation in your data table, a face is drawn such that each variable in your data set is represented by a feature in the face. This add-in uses JMP’s R integration functionality to create Chernoff faces. An R install and the TeachingDemos R package are required to use this add-in.

Submitted by: Clay Barker SAS employee Initiative: All
Application: Add-Ins Analysis: Data Visualization

Support Vector Machine for Classification

By simply opening a data table, specifying X, Y variables, selecting a kernel function, and specifying its parameters on the user-friendly dialog, you can build a classification model using Support Vector Machine. Please note that R package ‘e1071’ should be installed before running this dialog. The package can be found from http://cran.r-project.org/web/packages/e1071/index.html.

Submitted by: Jong-Seok Lee SAS employee Initiative: All
Application: Add-Ins Analysis: Exploratory Data Analysis/Mining

Penalized Regression Add-in

This add-in uses JMP’s R integration functionality to provide access to several penalized regression methods. Methods included are the LASSO (least absolutee shrinkage and selection operator, LARS (least angle regression), Forward Stagewise, and the Elastic Net. An R install and the “lars” and “elasticnet” R packages are required to use this add-in.

Submitted by: Clay Barker SAS employee Initiative: All
Application: Add-Ins Analysis: Regression

MP Addin: Univariate Nonparametric Bootstrapping

This script performs simple univariate, nonparametric bootstrap sampling by using the JMP to R Project integration. A JMP Dialog is built by the script where the variable you wish to perform bootstrapping over can be specified. A statistic to compute for each bootstrap sample is chosen and the data are sent to R using new JSL functionality available in JMP 9. The boot package in R is used to call the boot() function and the boot.ci() function to calculate the sample statistic for each bootstrap sample and the basic bootstrap confidence interval. The results are brought back to JMP and displayed using the JMP Distribution platform.

Submitted by: Kelci Miclaus SAS employee Initiative: All
Application: Add-Ins Analysis: Basic Statistics

Google Webinar on Web Analytics

Google webinar on web analytics-

recommended for anyone with anything to do with the WWW

From

http://analytics.blogspot.com/2011/11/webinar-reaching-your-goals-with.html

 

Webinar: Reaching Your Goals with Analytics

 

 

Is your website performing as well as it could be? Do you want to get more out of your digital marketing campaigns, including AdWords and other digital media? Do you feel like you have gaps in your current Google Analytics setup?

We’ve heard from many of our users who want to go deeper into their Analytics — with so much data, it can be hard to know where to look first. If you’d like to move beyond standard “pageview” metrics and visitor statistics, then please join us next Thursday:

Webinar: Reaching Your Goals with Analytics
Date: Thursday, December 1
Time: 11am PST / 2pm EST
Sign up here!

During the webinar, we’ll cover:

  • Key questions to ask for richer insights from your data
  • How to define “success” (for websites, visitors, or campaigns)
  • How to set up and use Goals
  • How to set up and use Ecommerce (for websites with a shopping cart)
  • How to link AdWords to your Google Analytics account

Whatever your online business model — shopping, lead-generation, or pure content — these tools will deliver actionable insights into your buying cycle.

This webinar will be led by Joe Larkin, a technical specialist on the Google Analytics team, and it’s designed for intermediate users of Google Analytics. If you’re comfortable with the basics, but you’d like to do more with your data, then we hope you’ll join us next week!

Interview Zach Goldberg, Google Prediction API

Here is an interview with Zach Goldberg, who is the product manager of Google Prediction API, the next generation machine learning analytics-as-an-api service state of the art cloud computing model building browser app.
Ajay- Describe your journey in science and technology from high school to your current job at Google.

Zach- First, thanks so much for the opportunity to do this interview Ajay!  My personal journey started in college where I worked at a startup named Invite Media.   From there I transferred to the Associate Product Manager (APM) program at Google.  The APM program is a two year rotational program.  I did my first year working in display advertising.  After that I rotated to work on the Prediction API.

Ajay- How does the Google Prediction API help an average business analytics customer who is already using enterprise software , servers to generate his business forecasts. How does Google Prediction API fit in or complement other APIs in the Google API suite.

Zach- The Google Prediction API is a cloud based machine learning API.  We offer the ability for anybody to sign up and within a few minutes have their data uploaded to the cloud, a model built and an API to make predictions from anywhere. Traditionally the task of implementing predictive analytics inside an application required a fair amount of domain knowledge; you had to know a fair bit about machine learning to make it work.  With the Google Prediction API you only need to know how to use an online REST API to get started.

You can learn more about how we help businesses by watching our video and going to our project website.

Ajay-  What are the additional use cases of Google Prediction API that you think traditional enterprise software in business analytics ignore, or are not so strong on.  What use cases would you suggest NOT using Google Prediction API for an enterprise.

Zach- We are living in a world that is changing rapidly thanks to technology.  Storing, accessing, and managing information is much easier and more affordable than it was even a few years ago.  That creates exciting opportunities for companies, and we hope the Prediction API will help them derive value from their data.

The Prediction API focuses on providing predictive solutions to two types of problems: regression and classification. Businesses facing problems where there is sufficient data to describe an underlying pattern in either of these two areas can expect to derive value from using the Prediction API.

Ajay- What are your separate incentives to teach about Google APIs  to academic or researchers in universities globally.

Zach- I’d refer you to our university relations page–

Google thrives on academic curiosity. While we do significant in-house research and engineering, we also maintain strong relations with leading academic institutions world-wide pursuing research in areas of common interest. As part of our mission to build the most advanced and usable methods for information access, we support university research, technological innovation and the teaching and learning experience through a variety of programs.

Ajay- What is the biggest challenge you face while communicating about Google Prediction API to traditional users of enterprise software.

Zach- Businesses often expect that implementing predictive analytics is going to be very expensive and require a lot of resources.  Many have already begun investing heavily in this area.  Quite often we’re faced with surprise, and even skepticism, when they see the simplicity of the Google Prediction API.  We work really hard to provide a very powerful solution and take care of the complexity of building high quality models behind the scenes so businesses can focus more on building their business and less on machine learning.

 

 

Movie Review- Bollywood "Rock Star"

This one is a wee bit different. The music sounds more contemporary and fusion ,East blends West, the direction is both subtle and at times flamboyant, and the acting is notches above the average dance round the tree, laugh like a idiot fare.

Rock Star is the tale of a college dropout (aint they all!) called Jordan / by Ranbir Kapoor  musical prodigy who battles personal demons, lady luck , lady love (in the impossible pout of the American fashion model ,Nargis F), and his own musical ambitions. The breathtaking art moves from Kashmir, Prague, Europe and the streets of Delhi University. This one is a taker, breaker, soul shaker.  Try getting a sub titled DVD if you dont know Hindi, or atleast dekko some songs streaming free at http://www.saavn.com/search/hindi/album%20Rockstar

Imitiaz Ali http://en.wikipedia.org/wiki/Imtiaz_Ali_(director) does all olde Hindu College alumni proud (including this reviewer)
https://www.youtube-nocookie.com/v/cn1jx_JUpi0?version=3&hl=en_US&rel=0

Moving data between Windows and Ubuntu VMWare partition

I use Windows 7 on my laptop (it came pre-installed) and Ubuntu using the VMWare Player. What are the advantages of using VM Player instead of creating a dual-boot system? Well I can quickly shift from Ubuntu to Windows and bakc again without restarting my computer everytime. Using this approach allows me to utilize software that run only on Windows and run software like Rattle, the R data mining GUI, that are much easier installed on Linux.

However if your statistical software is on your Virtual Disk , and your data is on your Windows disk, you need a way to move data from Windows to Ubuntu.

The solution to this as per Ubuntu forums is –http://communities.vmware.com/thread/55242

Open My Computer, browse to the folder you want to share.  Right-click on the folder, select Properties.  Sharing tab.  Select the radio button to “Share this Folder”.  Change the default generated name if you wish; add a description if you wish.  Click the Permissions button to modify the security settings of what users can read/write to the share.

On the Linux side, it depends on the distro, the shell, and the window manager.

Well Ubuntu makes it really easy to configure the Linux steps to move data within Windows and Linux partitions.

 

NEW UPDATE-

VMmare makes it easy to share between your Windows (host) and Linux (guest) OS

 

Step 1

and step 2

Do this

 

and

Start the Wizard

when you finish the wizard and share a drive or folder- hey where do I see my shared ones-

 

see this folder in Linux- /mnt/hgfs (bingo!)

Hacker HW – Make this folder //mnt/hgfs a shortcut in Places your Ubuntu startup

Hacker Hw 2-

Upload using an anon email your VM dark data to Ubuntu one

Delete VM

Purge using software XX

Reinstall VM and bring back backup

 

Note time to do this

 

 

 

-General Sharing in Windows

 

 

Just open the Network tab in Ubuntu- see screenshots below-

Windows will now ask your Ubuntu user for login-

Once Logged in Windows from within Ubuntu Vmware, this is what happens

You see a tab called “users on “windows username”- pc appear on your Ubuntu Desktop  (see top right of the screenshot)

If you double click it- you see your windows path

You can now just click and drag data between your windows and linux partitions , just the way you do it in Windows .

So based on this- if you want to build  decision trees, artifical neural networks, regression models, and even time series models for zero capital expenditure- you can use both Ubuntu/R without compromising on your IT policy of Windows only in your organization (there is a shortage of Ubuntu trained IT administrators in the enterprise world)

Revised Installation Procedure for utilizing both Ubuntu /R/Rattle data mining on your Windows PC.

Using VMWare to build a free data mining system in R, as well as isolate your analytics system (thus using both Linux and Windows without overburdening your machine)

First Time

  1. http://downloads.vmware.com/d/info/desktop_end_user_computing/vmware_player/4_0Download and Install
  2. http://www.ubuntu.com/download/ubuntu/downloadDownload Only
  3. Create New Virtual Image in VM Ware Player
  4. Applications—–Terminal——sudo apt get-install R (to download and install)
  5.                                          sudo R (to open R)
  6. Once R is opened type this  —-install.packages(rattle)—– This will install rattle
  7. library(rattle) will load Rattle—–
  8. rattle() will open the GUI—-
Getting Data from Host to Guest VM
Next Time
  1. Go to VM Player
  2. Open the VM
  3. sudo R in terminal to bring up R
  4. library(rattle) within R
  5. rattle()
At this point even if you dont know any Linux and dont know any R, you can create data mining models using the Rattle GUI (and time series model using E pack in the R Commander GUI) – What can Rattle do in data mining? See this slideshow-http://www.decisionstats.com/data-mining-with-r-gui-rattle-rstats/
If Google Docs is banned as per your enterprise organizational IT policy of having Windows Explorer only- well you can see these screenshots http://rattle.togaware.com/rattle-screenshots.html

The Amazing Microsoft Robotics

Amazing stuff from the makers of Kinetic-

Operating systems of Robots may be the future cash cow of Microsoft , while the pirates of Silicon Valley fight fascinating cloudy wars! 🙂

http://www.microsoft.com/robotics/#Product

 

Product Information

Microsoft Robotics Developer Studio 4 beta (RDS4 beta) provides a wide range of support to help make it easy to develop robot applications. RDS4 beta includes a programming model that helps make it easy to develop asynchronous, state-driven applications. RDS4 beta provides a common programming framework that can be applied to support a wide variety of robots, enabling code and skill transfer.

RDS4 beta includes a lightweight asynchronous services-oriented runtime, a set of visual authoring and simulation tools, as well as templates, tutorials, and sample code to help you get started.

Microsoft Robotics Developer Studio 4 beta Datasheet – English (PDF Format)

Product VideoView the product video on Channel 9!

This release has extensive support for the Kinect sensor hardware throug the Kinect for Windows SDK allowing developers to create Kinect-enabled robots in the Visual Simulation Environment and in real life. Along with this release comes a standardized reference spec for building a Kinect-based robot.

See how Microsoft Robotics Developer Studio 4 beta is being used to bring ideas to life in the Microsoft Robotics@Home competition.

Lightweight Asynchronous ServicesOriented Runtime

Lightweight Asynchronous ServicesOriented Runtime

Concurrency and Coordination Runtime (CCR) helps make it easier to handle asynchronous input and output by eliminating the conventional complexities of manual threading, locks, and semaphores. Lightweight state-oriented Decentralized Software Services (DSS) framework enables you to create program modules that can interoperate on a robot and connected PCs by using a relatively simple, open protocol.

Visual Programming Language (VPL)

Visual Programming Language

Visual Programming Language (VPL) provides a relatively simple drag-and-drop visual programming language tool that helps make it easy to create robotics applications. VPL also provides the ability to take a collection of connected blocks and reuse them as a single block elsewhere in your program. VPL is also capable of generating human-readable C#.

DSS Manifest Editor

DSS Manifest Editor

DSS Manifest Editor (DSSME) provides a relatively simple creation of application configuration and distribution scenarios.

DSS Log Analyzer

DSS Log Analyzer

The DSS Log Analyzer tool allows you to view message flows across multiple DSS services. DSS Log Analyzer also allows you to inspect message details.

Visual Simulation Environment (VSE)

Visual Simulation Environment

Visual Simulation Environment (VSE) provides the ability to simulate and test robotic applications using a 3D physics-based simulation tool. This allows developers to create robotics applications without the hardware. Sample simulation models and environments enable you to test your application in a variety of 3D virtual environments.

Building a Regression Model in R – Use #Rstats

One of the most commonly used uses of Statistical Software is building models, and that too logistic regression models for propensity in marketing of goods and services.

 

If building a model is what you do-here is a brief easy essay on  how to build a model in R.

1) Packages to be used-

For smaller datasets

use these

  1. CAR Package http://cran.r-project.org/web/packages/car/index.html
  2. GVLMA Package http://cran.r-project.org/web/packages/gvlma/index.html
  3. ROCR Package http://rocr.bioinf.mpi-sb.mpg.de/
  4. Relaimpo Package
  5. DAAG package
  6. MASS package
  7. Bootstrap package
  8. Leaps package

Also see

http://cran.r-project.org/web/packages/rms/index.html or RMS package

rms works with almost any regression model, but it was especially written to work with binary or ordinal logistic regression, Cox regression, accelerated failure time models, ordinary linear models, the Buckley-James model, generalized least squares for serially or spatially correlated observations, generalized linear models, and quantile regression.

For bigger datasets also see Biglm http://cran.r-project.org/web/packages/biglm/index.html and RevoScaleR packages.

http://www.revolutionanalytics.com/products/enterprise-big-data.php

2) Syntax

  1. outp=lm(y~x1+x2+xn,data=dataset) Model Eq
  2. summary(outp) Model Summary
  3. par(mfrow=c(2,2)) + plot(outp) Model Graphs
  4. vif(outp) MultiCollinearity
  5. gvlma(outp) Heteroscedasticity using GVLMA package
  6. outlierTest (outp) for Outliers
  7. predicted(outp) Scoring dataset with scores
  8. anova(outp)
  9. > predict(lm.result,data.frame(conc = newconc), level = 0.9, interval = “confidence”)

 

For a Reference Card -Cheat Sheet see

http://cran.r-project.org/doc/contrib/Ricci-refcard-regression.pdf

3) Also read-

http://cran.r-project.org/web/views/Econometrics.html

http://cran.r-project.org/web/views/Robust.html