Dude, Where’s my Water!

A recent extract from the “independent” Times of India – privately owned and indeed the World’s largest newspaper in English

http://timesofindia.indiatimes.com/india/West-uses-glacier-theory-to-flog-India-on-climate-change/articleshow/5482652.cms

NEW DELHI: IPCC’s admission of getting its facts on Himalayan glaciers completely wrong has again brought out concerns about the use of science,

Twitter Facebook Share
Email Print Save Comment

and pseudo-science, to put pressure on India to take stronger action on climate change or to put greater responsibility for the climate crisis on it.

The ‘2035 demise’ date drawn by IPCC in its fourth assessment report for Himalayan glaciers was used very often to demand that India should take greater action to reduce its emissions in order to protect people from catastrophes like glacial melts and floods. Similarly, a ‘premature’ release of information on the so-called Asian Brown Cloud was used by several western NGOs and governments to pin the blame on the melting of glaciers and other climate change impacts on pollution from burning firewood and cow dung in India.

I had earlier pointed out the same based on my proximity to Oakridge , TN and some data ( see here-

https://decisionstats.wordpress.com/2010/01/05/climate-die-oxide/

on January 5

1) What is the expected date of melting of glaciers in Himalayas thus affecting sacred rivers like Ganges and also causing floods in densely populated Asia. How would nation states with shareable resources like Water react on the disputes, dams , hydro electricity and floods.

2) How would you count per capita CO2 consumption- Assume a Factory in China makes 3 tonnes of C02 every year but exports all its products to USA on Indian Cargo ship. Travel contributes another 1 tonne of C02 including air travel, visits etc.

As of now this will be counted as 3 tonne for China, 1 Tonne for India, X tonne for USA ? What is wrong in these assumptions

Indeed I gave a presentation ro senior Times Group People on using data which is available on my Linkedin profile with the Google Docs presentation at

http://linkedin.com/in/ajayohri

Who is correct? The Indians or the Cowboys see NYT article

http://www.nytimes.com/2010/01/05/science/earth/05satellite.html

The nation’s top scientists and spies are collaborating on an effort to use the federal government’s intelligence assets — including spy satellites and other classified sensors — to assess the hidden complexities of environmental change. They seek insights from natural phenomena like clouds and glaciers, deserts and tropical forests.

Not a coincidence this comes close on the National Security Function in India coming totally revamped

http://timesofindia.indiatimes.com/india/Narayanans-exit-gives-full-control-of-internal-security-to-Chidambaram/articleshow/5474408.cms

The exit of M K Narayanan as national security advisor has set the stage for a significant re-ordering of UPA-2’s power structure with

home minister P Chidambaram set to gain fuller control of internal security reducing the role of the next NSA to foreign policy.

Debate and discussion between the freest and largest democracy are welcome steps.

But who is right?

Is climate change negotiations also a proxy for negotiation on terror co operation- as pointed out by me the Sikhs and Indians remain the only forces to be in Kabul (respectively the Sikhs  in recent (late 18th-19th Century) Source- A Brief History of Sikhs and ancient history ( 8 th Century AD) while Churchill’s memoirs in Young Winston talk of the stellar role of the Indian Army in Afghanistan or NWFP. Remember we have been here before- the Bush Administration negotiated and failed to get Indian troops in Iraq in 2004 over lack of monetary negotiations- the Indians turned to be right on true costs!

Are the Chinese or the Americans using India’s insecurities as a proxy?

ps- on Movies Why was Shekhar Kapur’s ( The Oscar nomianted director of Elizabeth ) documentary Paani stopped due to funding issues?

How can ice melting in North Pole lead to lack of water. Do water projections measure that rainwater harvesting has been low in India and ancient Indian religion is okay with Saraswati as one dis appeared river. If the Ganges dries up- the people in India may riot or may just blame it on sin and build smaller rain water dams.

Dude, Where’s my water? When is it gonna go ?

R for Stats : Updated

Here is the new website for statistical analysis using the free analytical software called R (which is enabled for cloud computing as well : see here http://bit.ly/OhriCloud

or http://rgrossman.com/2009/05/17/running-r-on-amazons-ec2/

for the R tutorial on running it on Amazon’s EC2 pay per demand RAM.

It is called R 4 stats or simply http://www.r4stats.com/

Hosted on Google’s Updated Google Sites Platform- it offers a preview to Bob’s earlier run away hit R for SAS and SPSS users updation as well as his upcoming work R for Stata Users.

In Bob’s words himself –

I have substantially expanded the table that compares SAS and SPSS
add-on modules to somewhat equivalent R packages. This new version is
at:
http://r4stats.com/add-on-modules
and I would very much appreciate any feedback you might have on it.

The site http://r4stats.com is the replacement to
http://RforSASandSPSSusers.com and includes the support files for both
“R for SAS and SPSS Users” and the new “R for Stata Users”, due out in
March from Springer.

Topic SAS Product SPSS Product R Package
Advanced Models
SAS/STAT IBM SPSS Advanced Statistics
R, MASS, many others
Association Analysis
Enterprise Miner
IBM SPSS Association
arules, arulesNBMiner, arulesSequences
Basics Base SAS
IBM SPSS Statistics Base
R
Bootstrapping
SAS/STAT
IBM SPSS Bootstrapping
BootCL, BootPR, boot, bootRes, BootStepAIC, bootspecdens, bootstrap, FRB, gPdtest, meboot, multtest, pvclust, rqmcmb2, scaleboot, simpleboot
Classification Analysis
Enterprise Miner
IBM SPSS Classification
rattle, see the neural networks and trees entries in this table.
Conjoint Analysis
SAS/STAT: PROC TRANSREG
IBM SPSS Conjoint
homals, psychoR, bayesm
Correspondence Analysis
SAS/STAT: PROC CORRESP
IBM SPSS Categories
ade4, cocorresp, FactoMineR, homals, made4, MASS, psychoR, PTAk, vegan
Custom Tables
Base SAS, PROC REPORT, PROC SQL, PROC TABULATE, Enterprise Reporter
IBM SPSS Custom Tables
reshape
Data Access
SAS/ACCESS
SPSS Data Access Pack
DBI, foreign, Hmisc: sas.get, sasxport.get, RODBC
Data Collection
SAS/FSP
IBM SPSS Data Collection Family
RSQLite, and the other open source programs MySQL or PostgreSQL are popular among R users for this purpose.
Data Mining
Enterprise Miner
IBM SPSS Modeler
(formerly Clementine)
arules, FactoMineR, rattle, various functions
Data Mining, In-database Processing
SAS In-Database Initiative with Teradata
IBM SPSS Modeler
PL/R
Data Preparation
Various procedures
IBM SPSS Data Preparation, various commands
dprep, plyr, reshape, sqldf, various functions
Developer Tools
SAS/AF, SAS/FSP, SAS Integration Technologies, SAS/TOOLKIT IBM SPSS Statistics Developer, IBM SPSS Statistics Programmability Extension
StatET, R links to most popular compilers, scripting languages, and databases.
Direct Marketing
Nothing quite like it
IBM SPSS Direct Marketing
Nothing quite like it
Exact Tests
SAS/STAT various
IBM SPSS Exact Tests
coin, elrm, exactLoglinTest, exactmaxsel, and options in many others
Excel Integration
SAS Enterprise BI Server IBM SPSS Advantage for Excel 2007
RExcel
Forecasting
SAS/ETS
IBM SPSS Forecasting
Over 40 packages that do time series are described at the Task View link above under Time Series.
Forecasting, Automated
Forecast Server IBM SPSS Forecasting
forecast
Genetics JMP Genomics
None http://www.bioconductor.org
Geographic Information Systems
SAS/GIS, SAS/GRAPH
None (Maps is defunct)
maps, mapdata, mapproj, GRASS via spgrass6, RColorBrewer, see Spatial in Task Views at link at top
Graphical user interfaces
Enterprise Guide, IML Studio, SAS/ASSIST, Analyst, Insight
IBM SPSS Statistics Base Deducer, JGR, R Commander, pmg, rattle, many others at http://www.sciviews.org/_rgui/
Graphics, Interactive
SAS/IML Studio, SAS/INSIGHT, JMP
None
GGobi via rggobi, iPlots, latticist, playwith
Graphics, Static
SAS/GRAPH
SPSS Base, Graphics Production Language
ggplot2, gplots, graphics, grid, gridBase, hexbin, lattice, plotrix, scatterplot3d, vcd, vioplot, geneplotter, Rgraphics
Graphics, Template Builder
Doesn’t use Grammar of Graphics model that forms the core of IBM SPSS Viz Designer or R’s ggplot2
IBM SPSS Viz Designer
Doesn’t use templates, but this GUI for ggplot2 http://www.stat.ucla.edu/~jeroen/ggplot2.html works similarly to IBM SPSS Viz Designer.
Guided Analytics
SAS/LAB
None
None
Matrix/linear Algebra
SAS/IML Studio
IBM SPSS Matrix
R, matlab, Matrix, sparseM
Missing Values Imputation
SAS/STAT: PROC MI
IBM SPSS Missing Values
amelia, Hmisc: aregImpute, EMV, rms (replaces Design): fit.mult.impute, mice, mitools, mvnmle, VIM
Neural Networks
Enterprise Miner
IBM SPSS Neural Networks
AMORE, grnnR, neuralnet, nnet, rattle
Operations Research
SAS/OR
None
glpk, linprog, LowRankQP, TSP
Power Analysis
SAS Power and Sample Size Application, SAS/STAT:
PROC POWER,
PROC GLMPOWER
SamplePower
asypow, powerpkg, pwr, MBESS
Quality Control
SAS/QC
IBM SPSS Statistics Base qcc, spc
Regression Models
SAS/STAT
IBM SPSS Regression
R, Hmisc, lasso, VGAM, pda, rms (replaces Design)
Sampling, Complex
SAS/STAT: PROC SURVEY SELECT, SURVEYMEANS, etc.
IBM SPSS Complex Samples
pps, sampfling, sampling, spsurvey, survey
Segmentation Analysis
Enterprise Miner
IBM Modeler Segmentation
cluster, rattle, som, see CRAN Task Views under Cluster for over 70 packages
Server Version
SAS for your particular server IBM SPSS Statistics Server,
IBM SPSS Modeler Server
rapache, R(D)COM Server, Rserve, StatET
Structural Equation Modeling
SAS/STAT: PROC CALIS
Amos OpenMX, sem
Text Analysis/Mining
Text Miner
IBM SPSS Text Analytics,
IBM SPSS Text Analysis for Surveys
Rstem, las, tm
Trees, Decision, Classification or Regression
Enterprise Miner
IBM SPSS Decision Trees, IBM SPSS AnswerTree, IBM SPSS Modeler (formerly Clementine)
ada, adabag, BayesTree, boost, GAMboost, gbev, gbm, maptree, mboost, mvpart, party, pinktoe,
quantregForest, rpart,rpart.permutation, randomForest, rattle, tree

All SAS and SPSS product names are registered trademarks of their respective companies.

Disclaimer- Bob Muenchen and I work for the same University. While we do have interesting conflicts often, his interview was one of the earliest where this blog began.

See- http://sites.google.com/site/r4statistics/interview

3 Idiots: Insight to Indian Engineer Campus Life

Ever wondered what makes Indian engineers so ahem hard working. Or Just in the mood to sample a BollyWood Movie. Here is 2009’s best movie – an all time grosser from the Oscar Nominated Aamir Khan.

It’s called 3 Idiots and loosely based on the adventures of 3-5 engineering students as they face academic and peer pressure challenges. Awesome. Loosely based on Chetan Bhagat’s book of 3 IIT friends.

Here is a preview of the video-

(Note the students praying for good grades).

A Noisy Algorithm

Here is something I created while having sea food at Pier 39 in San Fransisco-

Creating an algorithm for distorting predictive models by generating random noise ( either amplified or reduced sample).

Applications-

“If you can not convince them, confuse them”

  1. Generating white noise like signals to fake and distort noise and signal ratios
  2. Aggressive merger and acquisitions negotiations
  3. Media and Entertainment _                                     (Create Marketing Buzz/ Tabloid /Hype/ Fear , Uncertainty Doubt)
  4. National Security -( Kill _all_ the Terrorists with Love –                        black,brown,yellow,olive,white,blue,red …)
  5. Dating                                                                 (as in u2’s sweetest thing- Brown Eyed Boy meets Blue Eyed Girl)

The 0 1-1 1R 1 Algorithm

  1. Define Initial Position (i.e Use 6 sigma Define step)
  2. Take ANY Step 1 (i.e take a walk, make a phone call)
  3. Repeat ANY Step 1 again
  4. Do ANY Step 2 which is an opposite to ANY Step 1 in directional and /or  magnitude ( maybe time, or x,y,z and T ) vector to Any Step 1
  5. Return to Initial Position
  6. Loop the above 5 steps R times.

A detailed work flow would be followed by a simple diagram.

An earlier attempt to mash creativity with science as far back as July 2008 was the now redundant Ohri Framework

at https://decisionstats.wordpress.com/?s=ohri+framework (note WordPress timestamps can be manipulated so Google cache remains the true source of time series analysis of posts except when affected by black hat SEO )

New Edition of SAS.com Magazine q 1 2010

As always a great edition of an excellent online magazine.

The cover story of GE on stopping service fraud is great ( I am an ex GE alumnus- DIS claimer)

Click the screenshot for the real thing itself.

ps-
As my friends used to say, a magazine is something that can shoot multiple times.

New R Journal Edition

With special articles by my two favorite GUI creators ,
Dr John Fox (Basic Stats and DoE) and Dr Graham Williams (Rattle- Advanced Data Mining)

Notice : The look in the revised scribd is much better than the slideshare.net chaps

Cloud MapReduce

Apparently claimed to be much faster than Hadoop, here is the cloud OS flavor for MapReduce.

http://code.google.com/p/cloudmapreduce/

Cloud MapReduce was initially developed at Accenture Technology Labs. It is a MapReduce implementation on top of the Amazon Cloud OS.

By exploiting a cloud OS’s scalability, Cloud MapReduce achieves three primary advantages over other MapReduce implementations built on a traditional OS:

  • It is faster than other implementations (e.g., 60 times faster than Hadoop in one case. Speedup depends on the application and data.).
  • It is more scalable and more failure resistant because it has no single point of bottleneck.
  • It is dramatically simpler with only 3,000 lines of code (e.g., two orders of magnitude simpler than Hadoop).