Churn Analytics Contest at Crowd Analytix

Crowd Analytix- the Bangalore based Indian startup is moving fast in the

data scientist contest space (so watch out Kaggle!! )

Churn (loss of customers to competition) is a problem for telecom companies because it is more expensive to acquire a new customer than to keep your existing one from leaving. This contest is about enabling churn reduction using analytics.

To join, go to –

Book Review- Machine Learning for Hackers

This is review of the fashionably named book Machine Learning for Hackers by Drew Conway and John Myles White (O’Reilly ). The book is about hacking code in R.


The preface introduces the reader to the authors conception of what machine learning and hacking is all about. If the name of the book was machine learning for business analytsts or data miners, I am sure the content would have been unchanged though the popularity (and ambiguity) of the word hacker can often substitute for its usefulness. Indeed the many wise and learned Professors of statistics departments through out the civilized world would be mildly surprised and bemused by their day to day activities as hacking or teaching hackers. The book follows a case study and example based approach and uses the GGPLOT2 package within R programming almost to the point of ignoring any other native graphics system based in R. It can be quite useful for the aspiring reader who wishes to understand and join the booming market for skilled talent in statistical computing.

Chapter 1 has a very useful set of functions for data cleansing and formatting. It walks you through the basics of formatting based on dates and conditions, missing value and outlier treatment and using ggplot package in R for graphical analysis. The case study used is an Infochimps dataset with 60,000 recordings of UFO sightings. The case study is lucid, and done at a extremely helpful pace illustrating the powerful and flexible nature of R functions that can be used for data cleansing.The chapter mentions text editors and IDEs but fails to list them in a tabular format, while listing several other tables like Packages used in the book. It also jumps straight from installation instructions to functions in R without getting into the various kinds of data types within R or specifying where these can be referenced from. It thus assumes a higher level of basic programming understanding for the reader than the average R book.

Chapter 2 discusses data exploration, and has a very clear set of diagrams that explain the various data summary operations that are performed routinely. This is an innovative approach and will help students or newcomers to the field of data analysis. It introduces the reader to type determination functions, as well different kinds of encoding. The introduction to creating functions is quite elegant and simple , and numerical summary methods are explained adequately. While the chapter explains data exploration with the help of various histogram options in ggplot2 , it fails to create a more generic framework for data exploration or rules to assist the reader in visual data exploration in non standard data situations. While the examples are very helpful for a reader , there needs to be slightly more depth to step out of the example and into a framework for visual data exploration (or references for the same). A couple of case studies however elaborately explained cannot do justice to the vast field of data exploration and especially visual data exploration.

Chapter 3 discussed binary classification for the specific purpose for spam filtering using a dataset from SpamAssassin. It introduces the reader to the naïve Bayes classifier and the principles of text mining suing the tm package in R. Some of the example codes could have been better commented for easier readability in the book. Overall it is quite a easy tutorial for creating a naïve Bayes classifier even for beginners.

Chapter 4 discusses the issues in importance ranking and creating recommendation systems specifically in the case of ordering email messages into important and not important. It introduces the useful grepl, gsub, strsplit, strptime ,difftime and strtrim functions for parsing data. The chapter further introduces the reader to the concept of log (and affine) transformations in a lucid and clear way that can help even beginners learn this powerful transformation concept. Again the coding within this chapter is sparsely commented which can cause difficulties to people not used to learn reams of code. ( it may have been part of the code attached with the book, but I am reading an electronic book and I did not find an easy way to go back and forth between the code and the book). The readability of the chapters would be further enhanced by the use of flow charts explaining the path and process followed than overtly verbose textual descriptions running into multiple pages. The chapters are quite clearly written, but a helpful visual summary can help in both revising the concepts and elucidate the approach taken further.A suggestion for the authors could be to compile the list of useful functions they introduce in this book as a sort of reference card (or Ref Card) for R Hackers or atleast have a chapter wise summary of functions, datasets and packages used.

Chapter 5 discusses linear regression , and it is a surprising and not very good explanation of regression theory in the introduction to regression. However the chapter makes up in practical example what it oversimplifies in theory. The chapter on regression is not the finest chapter written in this otherwise excellent book. Part of this is because of relative lack of organization- correlation is explained after linear regression is explained. Once again the lack of a function summary and a process flow diagram hinders readability and a separate section on regression metrics that help make a regression result good or not so good could be a welcome addition. Functions introduced include lm.

Chapter 6 showcases Generalized Additive Model (GAM) and Polynomial Regression, including an introduction to singularity and of over-fitting. Functions included in this chapter are transform, and poly while the package glmnet is also used here. The chapter also introduces the reader formally to the concept of cross validation (though examples of cross validation had been introduced in earlier chapters) and regularization. Logistic regression is also introduced at the end in this chapter.

Chapter 7 is about optimization. It describes error metric in a very easy to understand way. It creates a grid by using nested loops for various values of intercept and slope of a regression equation and computing the sum of square of errors. It then describes the optim function in detail including how it works and it’s various parameters. It introduces the curve function. The chapter then describes ridge regression including definition and hyperparameter lamda. The use of optim function to optimize the error in regression is useful learning for the aspiring hacker. Lastly it describes a case study of breaking codes using the simplistic Caesar cipher, a lexical database and the Metropolis method. Functions introduced in this chapter include .Machine$double.eps .

Chapter 8 deals with Principal Component Analysis and unsupervised learning. It uses the ymd function from lubridate package to convert string to date objects, and the cast function from reshape package to further manipulate the structure of data. Using the princomp functions enables PCA in R.The case study creates a stock market index and compares the results with the Dow Jones index.

Chapter 9 deals with Multidimensional Scaling as well as clustering US senators on the basis of similarity in voting records on legislation .It showcases matrix multiplication using %*% and also the dist function to compute distance matrix.

Chapter 10 has the subject of K Nearest Neighbors for recommendation systems. Packages used include class ,reshape and and functions used include cor, function and log. It also demonstrates creating a custom kNN function for calculating Euclidean distance between center of centroids and data. The case study used is the R package recommendation contest on Kaggle. Overall a simplistic introduction to creating a recommendation system using K nearest neighbors, without getting into any of the prepackaged packages within R that deal with association analysis , clustering or recommendation systems.

Chapter 11 introduces the reader to social network analysis (and elements of graph theory) using the example of Erdos Number as an interesting example of social networks of mathematicians. The example of Social Graph API by Google for hacking are quite new and intriguing (though a bit obsolete by changes, and should be rectified in either the errata or next edition) . However there exists packages within R that should be atleast referenced or used within this chapter (like TwitteR package that use the Twitter API and ROauth package for other social networks). Packages used within this chapter include Rcurl, RJSONIO, and igraph packages of R and functions used include rbind and ifelse. It also introduces the reader to the advanced software Gephi. The last example is to build a recommendation engine for whom to follow in Twitter using R.

Chapter 12 is about model comparison and introduces the concept of Support Vector Machines. It uses the package e1071 and shows the svm function. It also introduces the concept of tuning hyper parameters within default algorithms . A small problem in understanding the concepts is the misalignment of diagram pages with the relevant code. It lastly concludes with using mean square error as a method for comparing models built with different algorithms.


Overall the book is a welcome addition in the library of books based on R programming language, and the refreshing nature of the flow of material and the practicality of it’s case studies make this a recommended addition to both academic and corporate business analysts trying to derive insights by hacking lots of heterogeneous data.

Have a look for yourself at-

Exciting Contest at CrowdANALYTIX

A new contest from a relatively new website. This one is fast and furious and has a decent chunk of money!




Submission Deadline:
Sun, 26 February 2012 05:00 AM UTC
Results Announced by:
Mon, 05 March 2012 05:00 AM UTC
Category: Text Analytics Function: Aerospace & Aviation


Analysis of sentiment and its intensity – feedback from airport guests
ABC (name intentionally obfuscated) is one of the best managed and highly profitable airports  in India. As with all well managed airports, ABC would like to understand what guests feel about their experience when traveling, using or transiting through their airport. ABC has a website in which guests can visit and leave behind a comment, agree or disagree with others’ comments, or respond to a comment confirming or negating the expressed opinion.

The goal of this contest is to create a summarization of the opinions, feelings and sentiments expressed in the comments left behind by guests on the website. This information is being provided as data for solvers. Some understanding of the intensity of the opinion, feeling or sentiment will also be useful. For example, if there is a consistent demand for more spas across guest conversations, it needs to be highlighted.  Consistent positive or negative sentiments and opinions need to be discovered and highlighted.
Guest comments have been crawled and provided to you. The data consists approximately 1000 comments from guests including the timestamp of those comments.  Personal information (name, email etc) have been hidden. This data is publicly available
Solver Expectations:
Participants may submit entries before the deadline. If a participant submits multiple entries, the entry submitted last before the deadline will be considered as the participant’s submission.
The following deliverables are expected to be submitted:
  • a report expressing the results of the opinion and sentiment mining
  • documentation about how you approached the problem, what tools, technologies and languages you used, and what problems you encountered
Timeline and Prizes: 
This contest begin on 16 Feb 2012 and will last for a duration of 9 days.
  • One 1st Prize – $1000
  • One 2nd Prize – $500
  • Two 3rd Prizes – $250

Predictive Analytics World Events in 2012

A new line up of Predictive Analytics World and Text Analytics World conferences and workshops are coming March through July, plus see the save-the-dates and call-for-speakers for events in Sept, Oct, and Nov.

CONFERENCE: Predictive Analytics World – San Francisco

March 4-10, 2012 in San Francisco, CA
Discount Code for $150 off: AJAYBP12

CONFERENCE: Text Analytics World – San Francisco
March 6-7, 2012 in San Francisco, CA
Discount Code for $150 off: AJAYBP12

A plethora of 1-day workshops are held alongside PAW and TAW
For details see:

SEMINAR: Predictive Analytics for Business, Marketing & Web
March 22-23, 2012 in New York City, NY
July 26-27, 2012 in São Paulo, Brazil
Oct 11-12, 2012 in San Francisco
A concentrated training program lead by PAW’s chair, Eric Siegel

CONFERENCE: Predictive Analytics World – Toronto
April 25-26, 2012 in Toronto, Ontario
Discount Code for $150 off: AJAYBP12

CONFERENCE: Predictive Analytics World – Chicago
June 25-26, 2012 in Chicago, IL
Discount Code for $150 off: AJAYBP12


From Ajay-

CONTEST- If you use the discount code AJAYBP12, you will not only get the $150 off, but you will be entered in a contest to get 2 complementary passes like I did last year . Matt Stromberg won that one


see last year results-

Creating Pages on Google Plus for some languages

So I decided to create Pages on Google Plus for my favorite programming languages.

a programming language that lets you work more quickly and integrate your systems more effectively

Add to circles

  –  Comment  –  Share

Ajay Ohri

Ajay Ohri's profile photo

Ajay Ohri  –
Ajay Ohri shared a Google+ page with you.
Structured Query Language
Leading statistical language since 1960’s especially in sociology and market research
The leading statistical language in the world
The leading statistical language since 1970’s



These are in accordance with Google’s Policies  Continue reading “Creating Pages on Google Plus for some languages”

SAP HANA Contest

From SAP, a new contest based on HANA

Become a champion and join our new developer challenge! Explore SAP HANA and share a critical business problem you want to solve on idea place for SAP HANA InnoJam online and describe your idea. Your participation starts here today, 13th September. The first 100 submitters will play on SAP HANA and get a chance to shine and win big prizes! To be prepared for the new challange, please read All sources you might need for your SAP HANA training, first chapters of the SAP HANA Pocketbook (coming soon) and read more on the official SAP HANA webpage.

Phase 1

  • Share a critical business problem you want to solve on Idea Place, the place is open from September 13th, 2011
  • First 100 submitters go to Phase 2

Phase 2

  • Join SAP HANA Champions community, get started with HANA, build teams
  • Access SAP HANA sandbox
  • Use SAP HANA to solve your business problem
  • Get helped by SAP experts if and when needed

Phase 3

  • Share your solution: written description and a short video
  • Top 8 finalists go to Phase 4

Phase 4

  • 8 finalists present their problem/solution/feedback in front of SAP executives and a panel of judges (early January in Palo Alto)
  • Winner prizes will be identified soon
Also suggested reading-

Use R for Business- Competition worth $ 20,000 #rstats

All you contest junkies, R lovers and general change the world people, here’s a new contest to use R in a business application


$20,000 in Prizes for Users Solving Business Problems with R


PALO ALTO, Calif. – September 1, 2011 – Revolution Analytics, the leading commercial provider of R software, services and support, today announced the launch of its “Applications of R in Business” contest to demonstrate real-world uses of applying R to business problems. The competition is open to all R users worldwide and submissions will be accepted through October 31. The Grand Prize winner for the best application using R or Revolution R will receive $10,000.

The bonus-prize winner for the best application using features unique to Revolution R Enterprise – such as itsbig-data analytics capabilities or its Web Services API for R – will receive $5,000. A panel of independent judges drawn from the R and business community will select the grand and bonus prize winners. Revolution Analytics will present five honorable mention prize winners each with $1,000.

“We’ve designed this contest to highlight the most interesting use cases of applying R and Revolution R to solving key business problems, such as Big Data,” said Jeff Erhardt, COO of Revolution Analytics. “The ability to process higher-volume datasets will continue to be a critical need and we encourage the submission of applications using large datasets. Our goal is to grow the collection of online materials describing how to use R for business applications so our customers can better leverage Big Analytics to meet their analytical and organizational needs.”

To enter Revolution Analytics’ “Applications of R in Business” competition Continue reading “Use R for Business- Competition worth $ 20,000 #rstats”