The Comic Water Games (aka Common Wealth Games)

We in Delhi, India are a tough people. With summer temperatures from 46 Degree Celcius (114 Degree Fahrenheit) and Winter temperatures from 2-3 Degree Celcius (just above freezing), high pollution levels, the worst traffic jams (and highest per capita cars)- there is very little that intimidates the Average Delhiite-

But the Return of the British Empire is scaring us- and it is called Common Wealth Games. The Common Wealth is a group of countries that used to be colonized by Britain in her colonial days ( USA is not a member though- as they probably kicked way too much British butt while gaining independence).

And every 4 years they have CommonWealth games (read games for the non US English speaking world). So when our commie neighborhood– the Chinese went and got themselves an Olympics- we decided to get ourselves this CWG games too. Big deal- national pride- rising economic power and all that.

So far the Games has meant the following- lots of roads dug up, lot of stadiums in various degrees of preparation, a total cost of 2 Billion USD, rampant allegations of corruption due to the ten times increase in budget – including rather suspicious looking documents procured by our local press (yes Indian press is free as it is a democracy)

And add divine grace. Delhi has the wettest monsoon since 1978- it rains cats and dogs in September- and we now have a mini dengue malaria epidemic. 4 countries have declared the living quarters for athletes as uninhabitable , some have walked out, the inevitable terrorists injured two Taiwanese tourists this weekend (in a semi ironic email they said they were prepared as the government was prepared- it isn’t)

Today a bridge collapsed-

http://www.nytimes.com/2010/09/22/sports/22iht-GAMES.html?_r=1&hp

On Tuesday afternoon, a bridge next to Jawaharlal Nehru Stadium, the main Games venue, fell apart. The footbridge collapsed into three pieces, taking several workers with it and uprooting one side of the arch that supported it.

A police officer at the scene said that 27 people had been injured, four of them seriously, in the collapse.

“This will not affect the Games,” said Raj Kumar Chauhan, a Delhi minister for development, who spoke on the scene. “We can put the bridge up again, or make a new one.”

and

http://www.nytimes.com/2010/09/20/world/asia/20india.html?ref=sports

“We really need to learn how to plan,” said Vrinda Walavalkar, a public relations executive who is not connected to the Games.

“Maybe we feel we have so many lifetimes to achieve things” that it does not matter if it gets done this time, she said.

Mr. Gupta, the shopkeeper, found a metaphor in Hindu wedding tradition.

The groom’s party, known as the barat, traditionally marches to the bride’s house on horseback with his friends and family, he explained. When the barat appears, the bride has to come to the door, he said.

“If the bride is not ready, you patch her up and try to hide all her defects,” Mr. Gupta said, and then you send her outside.

————————————————————————————————————–

To some this may be shocking. To the average Delhi-ite battling traffic and rain , this is one more episode in the chaotic Capital. As a small solace- Delhi still has the best and cheapest street food this part of the world- with golgappas, tikki and chat. If only you can beat the rain to get them !

Also see http://en.wikipedia.org/wiki/Delhi if you like to know more.

Hearst DataMining Challenge

Check out the Hearst Data Mining Challenge- a new competition-sponsored by DMA, Hearst Magazine, and EXL

THE HEARST CHALLENGE STARTS ON OCTOBER 14TH

CHALLENGE

DESCRIPTION

Over the years, the magazine publishing industry has made significant strides in improving subscription based circulation by developing analytic frameworks that better predict customer response to acquisition and renewal offers. The objective of this contest is to apply the same analytic discipline and effectively predict newsstand locations “response”. Specifically the objective is to predict the number of copies to be placed in each newsstand location to optimize the overall contribution of the newsstand location typically referred to as draw.

Data for the competition is provided by CMG and Experian.

and

RULES

HOW TO ENTER: Beginning October 14th, 2010 at 12:01 AM (ET) throughDecember 3rd, 2010 at 11:59 PM (ET) go to the Hearst Challenge website located at http://www.HearstChallenge.com (the “Site”) and complete and submit the entry form pursuant to the onscreen instructions. Entrants will be provided a historical sample of newsstand location draw, sales and associated location level data to help develop their predictive algorithm. Hearst will in turn hold back two distinct sets of draw/sales data, one to be used as a validation set by the contestant and one to be used as a final contest evaluation set. Entrants may not include any other external variables for the challenge. Additional details will be provided with the data. Entrants will be able to track their performance against the validation set throughout the course of the challenge via a leader tracking board to be made available on the Site. Entries must include the following documentation:

  • Data file with id variables and expected sales values by store and publication
  • The final model/ algorithm code used to score the final data set
  • Any supporting documentation that pertains to the development of the submitted model/algorithm including variable creation. Variables that were used in the model need to be traced through from input to coefficient / node (if using a tree based methodology).

Check out http://www.hearstchallenge.com/index.php for further details.

Where is Waldo? Webcast on Network Intelligence

From the good folks at AsterData, a webcast on a slightly interesting analytics topic

Enterprises and government agencies can become overwhelmed with information. The value of all that data lies in the insights it can reveal. To get the maximum value, you need an analytic platform that lets you analyze terabytes of information rapidly for immediate actionable insights.

Aster Data’s massively parallel database with an integrated analytics engine can quickly reveal hard-to-recognize trends on huge datasets which other systems miss. The secret? A patent-pending SQL-MapReduce framework that enables business analysts and business intelligence (BI) tools to iteratively analyze big data more quickly. This allows you to find anomalies more quickly and stop disasters before they happen.

Discover how you can improve:

  • Network intelligence via graph analysis to understand connectivity among suspects, information propagation, and the flow of goods
  • Security analysis to prevent fraud, bot attacks, and other breaches
  • Geospatial analytics to quickly uncover details about regions and subsets within those communities
  • Visual analytics to derive deeper insights more quickly

September Roundup by Revolution

From the monthly newsletter- which I consider quite useful for keeping updated on application of R

——————————————————————————————————————————————————————————————————–

Revolution News
Every month, we’ll bring you the latest news about Revolution’s products and events in this section.
Follow us on Twitter at @RevolutionR for up-to-the-minute news and updates from Revolution Analytics!

Revolution R Enterprise 4.0 for Windows now available. Based on the latest R 2.11.1 and including the RevoScaleR package for big-data analysis in R, Revolution R Enterprise is now available for download for Windows 32-bit and 64-bit systems. Click here to subscribe, or available free to academia.

New! Integrate R with web applications, BI dashboards and more with web services. RevoDeployR is a new Web Services framework that integrates dynamic R-based computations into applications for business users. It will be available September 30 with Revolution R Enterprise Server on RHEL 5. Click here to learn more.

Free Webinar, September 22: In a joint webinar from Revolution Analytics and Jaspersoft, learn how to use RevoDeployR to integrate advanced analytics on-demand in applications, BI dashboards, and on the web. Register here.

Revolution in the News:
SearchBusinessAnalytics.com previews the forthcoming Revolution R GUI; Channel Register introduces RevoDeployR, while IT Business Edge shows off the Web Services architecture; and ReadWriteWeb.com looks at how RevoScaleR tackles the Big Data explosion.

Inside-R: A new site for the R Community. At www.inside-R.org you’ll find the latest information about R from around the Web, searchable R documentation and packages, hints and tips about R, and more. You can even add a “Download R” badge to your own web-page to help spread the word about R.

R News, Tips and Tricks from the Revolutions blog
The Revolutions blog brings you daily news and tips about R, statistics and open source. Here are some highlights from Revolutions from the past month
.

R’s key role in the oil spill response: Read how NIST’s Division Chief of Statistical Engineering used R to provide critical analysis in real time to the Secretaries of Energy and the Interior, and helped coordinate the government’s response.

Animating data with R and Google Earth: Learn how to use R to create animated visualizations of geographical data with Google Earth, such as this video showing how tuna migrations intersect with the location of the Gulf oil spill.

Are baseball games getting longer? Or is it just Red Sox games? Ryan Elmore uses nonparametric regression in R to find out.

Keynote presentations from useR! 2010: the worldwide R user’s conference was a great success, and there’s a wealth of useful tips and information in the presentations. Video of the keynote presentations are available too: check out in particular Frank Harrell’s talk Information Allergy, and Friedrich Leisch’s talk on reproducible statistical research.

Looking for more R tips and tricks? Check out the monthly round-ups at the Revolutions blog.

Upcoming Events
Every month, we’ll highlight some upcoming events from R Community Calendar.

September 23: The San Diego R User Group has a meetup on BioConductor and microarray data analysis.

September 28: The Sydney Users of R Forum has a meetup on building world-class predictive models in R (with dinner to follow).

September 28: The Los Angeles R User Group presents an introduction to statistical finance with R.

September 28: The Seattle R User Group meets to discuss, “What are you doing with R?”

September 29: The Raleigh-Durham-Chapel Hill R Users Group has its first meeting.

October 7: The NYC R User Group features a presentation by Prof. Andrew Gelman.

There are also new R user groups in SingaporeSeoulDenverBrisbane, and New Jersey.  Please let us know if we’re missing your R user group, or if want to get a new one started.

———————————————————————————————-Editor

David Smith, VP Marketing
david@revolutionanalytics.com
Twitter: @revodavid

subscribe here for Revo’s Monthly newsletter-

IBM Buys Netezza

IBM just bought Netezza (maker of Twin Fin appliance) for handling big data.

http://dealbook.blogs.nytimes.com/2010/09/20/i-b-m-to-buy-analytics-firm-for-1-7-billion/?hpw

The deal values Netezza at $27 a share, a 9.8 percent premium to its closing price on Friday.

Since Netezza was an existing SAS partner, probably it would impact it more if at all, since IBM-SPSS acquisition. Also Netezza was one of the foremost BI companies for both using and expounding R-

See- Using Netezza and R http://www.biecek.pl/WZUR2009/LukaszBartnik2009c.pdf

and http://www.netezza.com/userconference/pce.html#rmftfic

Below a paper on using R on Netezza-

> library(nzr)
> nzconnect(“user”, “password”, “host”, “database”)
> library(rpart)
> data(kyphosis)
# this creates a table out of kyphosis data.frame
# and sends its data to TwinFin
> invisible(as.nz.data.frame(kyphosis))
> nzQuery(“SELECT * FROM kyphosis”)
KYPHOSIS AGE NUMBER START
1 absent 71 3 5
2 absent 158 3 14
3 present 128 4 5
[ cut ]
# now create a nz.data.frame
> k <- nz.data.frame(“kyphosis”)
> as.data.frame(k)
KYPHOSIS AGE NUMBER START
1 absent 71 3 5
2 absent 158 3 14
3 present 128 4 5
[ cut ]
> nzQuery(“SELECT * FROM kyphosis”)
COUNT
1 81

JMP 9 releasing on Oct 12

JMP 9 releases on Oct 12- it is a very good reliable data visualization and analytical tool ( AND available on Mac as well)

AND IT is advertising R Graphics as well (lol- I can visualize the look on some ahem SAS fans in the R Project)

Updated Pricing- note I am not sure why they are charging US academics 495$ when SAS On Demand is free for academics. Shouldnt JMP be free to students- maybe John Sall and his people can do a tradeoff analysis for this given JMP’s graphics are better than Base SAS (which is under some pressure from WPS and R)

http://www.sas.com/govedu/edu/programs/soda-account-setup.html

and http://www.enterpriseinnovation.net/content/sas-delivers-free-data-management-and-analytics-solutions-academe

*Offer good in the U.S. only.

OFFER PRICING DETAILS
New Corporate Customer

$1,595

Save $300.

No special requirements.
ORDER NOW (WIN) ORDER NOW (MAC)
Corporate Upgrade

$795

Save $155.

Complete the form below or call 1-877-594-6567. Requires valid JMP® 8 serial number.
New Academic

$495

Save $100.

Complete the form below or call 1-877-594-6567. Requires campus street address and campus e-mail address.
Academic Upgrade

$250

Save $45.

Complete the form below or call 1-877-594-6567. Requires campus street address and campus e-mail address.

From- the mailer-

Be First in Line for JMP® 9
Save up to $300 when you pre-order a
single-user license by Oct. 11

Pre-Order JMP 9

Make JMP your analytic hub for visual data discovery with this special offer, good through Oct. 11, 2010. Pre-order a single-user license of JMP 9 – for a discount of up to $300 – and get ready for a leap in data interactivity.

Order now and enjoy the compelling new features of JMP 9 when the software is released Oct. 12. New capabilities in JMP 9 let you:

  • Optimize and simulate using your Microsoft Excel spreadsheets.
  • Use maps to find patterns in your geographic data.
  • Enjoy the updated look and flexibility of JMP 9 on Microsoft Windows.
  • Create and share custom add-ins that extend JMP.
  • Leverage an expanded array of advanced statistical methodologies.
  • Display analytic results from R using interactive graphics.

PRE-ORDER JMP 9

What if I already have a JMP 8 single-user license?
Great news! You can upgrade to JMP 9 for less than half the regular price.

What if I’m an annual license customer?
Don’t worry, we’ve got you covered. Annual license customers enjoy priority access to all the latest JMP releases as soon as they become available. JMP 9 will be shipped to you automatically.

What if I work or study in the academic world?
Call 1-877-594-6567 to learn about significant discounts for students and professors through the JMP Academic Program.

Please feel free to forward this offer to interested colleagues.


Got two or more users?
A JMP® annual license is the way to go. Call for details.
1-877-594-6567

Remember: Act by Oct. 11!

JMP runs on Macintosh and Windows

Rattle Re-Introduced

Latest version of Rattle just went online-

Here is the change log- Dr Graham Williams is also coming out with a book on using Rattle- the R GUI devoted to data mining.

Source-http://cran.r-project.org/web/packages/rattle/index.html

rattle (2.5.42) unstable; urgency=low

  * Update rattle.info() to recursively identify all dependencies,
 report
    their version number and any updates available from CRAN and generate
    command to update packages that have updates available. See
    ?rattle.info for the options.

  * Fix bug causing R Dataset option of the Evaluate window to always
    revert to the first named dataset.

  * Fix bug in transforms where weights were not being handled in
    refreshing of the Data tab.

  * Fix a bug in box plots when trying to label outliers when there aren't
    any.

 -- Graham Williams <Graham.Williams@togaware.com>  Sun, 
19 Sep 2010 05:01:51 +1000

rattle (2.5.41) unstable; urgency=low

  * Use GtkBuilder for Export dialog.

  * Test use of glade vs GtkBuilder on multiple platforms.

  * Rename rattle.info to rattle.version.

  * Add weight column to data tab.

  * Support weights for nnet, multinom, survival.

  * Add weights information to PMML as a PMML Extension.

  * Ensure GtkFrame is available as a data type whilst waiting for 
updated
    RGtk2.

  * Bug fix to packageIsAvailable not reruning any result.

  * Replace destroy with withdraw for plot window as the former has
    started crashing R.

  * Improve Log formatting for various model build commands.

  * Be sure to include the car package for Anova for multinom models.

  * Release pmml 1.2.24: Bug fix glm binomial regression - note as
    classification model.

 -- Graham Williams <Graham.Williams@togaware.com>  Wed, 15 Sep 2010 
14:56:09 +1000
And a video I did of exploring various Rattle options using Camtasia,
 a very useful software for screen capture and video tutorials
from http://www.techsmith.com/download/camtasiatrial.asp
Updated- my video skils being quite bad- I replaced it with another video. 
However Camtasia is the best screen capture video tool
Also , an update Analyticdroid is on hold for now. see- for more details http://rattle.togaware.com/