Category: Analytics

Big Data Evil Empire

  1. Much more progress has been made in data storage , data querying and data analysis of huge amounts of personally identifiable information , than in encrypting such information
  2. Big Data has as much dual use usage for governments and corporations as uranium has for building bombs or power plants.
  3. There is as much lucre and potential revenue for encrypted data streams in the cloud era – as there for anti virus software in the PC era
  4. Tracking citizens totally is evil- the total costs of such programs is unjustified given the thwarted terrorism plots by Big Data ‘s Cyber Spying. At best I can understand governments spying on citizen’s of other countries to gain advantages in trade
  5. The American dominance of cyber spying and big data threaten to unravel and undermine it’s credibility as de facto leader of the Internet. It proves China’s vision of a walled off internet makes sense and that is a dangerous precedent which could lead to the break up of the internet along national boundaries of electronic fire walls.

2013 in review

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The Louvre Museum has 8.5 million visitors per year. This blog was viewed about 150,000 times in 2013. If it were an exhibit at the Louvre Museum, it would take about 6 days for that many people to see it.

Click here to see the complete report.

2013 Thank You Note

I would like to write a thank you note to  some of the people who helped make Decisionstats.com possible . We had a total of 150,644 views this year.For that, I have to thank you dear readers for putting up with me- it is now our seventh year.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total
13,940 12,153 12,948 13,371 12,778  12,085  12,894  11,934  9,914  14,764  12,907  10,956  150,644

I would like to thank Chris  (of Mashape) for helping me with some of the interviews I wrote here .I did 26 interviews this year for Programmable Web and a total of 30+ articles including the interviews in 2013.

Of course- we have now reached 116 excellent interviews on Decisionstats.com alone ( see http://goo.gl/V6UsCG )I would like to thank each one of the interviewees who took precious time to fill out the questions.

Sponsors- I would like to thank Dr Eric Siegel ( individually as an author and as founder chair of www.pawcon.com ) , Nadja and Ingo (for Rapid-Miner) , Dr Jonathan ( for Datamind) , Chris M (for Statace.com ) , Gergely ( Author) and many more during all these six years who have kept us afloat and the servers warm in these days of cold reflection, including Gregory (of KDNuggets.com) and erstwhile AsterData founders.

Training Partners- I would like to thank Lovleen Bhatia ( of Edureka  for giving me the opportunity to make http://www.edureka.in/r-for-analytics which now has 1721 learners as per http://www.edureka.in/)

I would also specially say Thank you to Jigsaw Academy for giving me the opportunity to create
the first affordable and quality R course in Asia http://analyticstraining.com/2013/jigsaw-completes-training-of-300-students-on-r/

These training courses including those by Datamind and Coursera remain a formidable and affordable alternative to many others catching up in the analytics education game in India ( an issue I wrote here)

Each and Everyone of my students (past and present) and Everyone in the #rstats  and SAS-L community, including people who may have been left out.

Thank you sir, for helping me and Decisionstats.com !

Wish each one of you a very happy and Joyous Happy New Year and a great and prosperous 2014!

Misconceptions and Fallacies in Analytics Education in India

  1.  Teaching a software and labeling it as analytics education- Some examples are Teaching Analytics with MS Excel (a spreadsheet software) , or Teaching a Statistics or Optimization syllabus and tagging it as Business Analytics.
  2. Promise to teach language X but use cheaper software Y– Examples can be offering to teach SPSS language but using the open source equivalent PSPP
  3. Overcharge for a day or two’s workshop- Albert Einstein could not learn a computer language in 3 days he could just get the basics. Anything priced above 500 $ and less than 4 days training is a simple effort to fool you you are getting your much more than your money’s worth.
  4. Extend training to more than 2 months and then overcharge– This is a failure unless done by an accredited college
  5. Freebies– There is no free lunch. Overcharging and giving a discount is a standard marketing malpractice.
  6. Brand Associations– Brand X is well known but has no credentials in Analytics. So it ties up with a couple of analytics consultants and launches a certificate or certification or diploma program in analytics. Unfortunately this extends to the very very best of Indian education.
  7. Hidden costs also known as We are cheap because we are in India-  Analytics software costs almost the same through out the world ( I did propose a PPP method for pricing software differently). Anyone offering discount because of geography is selling you a bridge in Nigeria or a million dollars in Iraq.
  8. Self Paced Learning-Learn Online for Fee- or Free- No, learning needs interaction and instructors- otherwise all universities in the worlds would have moved the professors to research (?) and offered videos to the students for self learning
  9. Better Much Better Support- Some analytics providers aim to distinguish themselves by saying we give better support. Yet their support team is hidden and mostly the instructor giving support. The best solution is to publish members of support team names as is done in support services industry.

These are personal observations and may or may not be true to every organization. All opinions are mine only.

Top 7 Business Strategy Models

UPDATED POST- Some Models I use for Business Strategy- to analyze the huge reams of qualitative and uncertain data that business generates. I have added a bonus the Business canvas Model (number 2)

  1. Porters 5 forces Model-To analyze industries
  2. Business Canvas
  3. BCG Matrix- To analyze Product Portfolios
  4. Porters Diamond Model- To analyze locations
  5. McKinsey 7 S Model-To analyze teams
  6. Gernier Theory- To analyze growth of organization
  7. Herzberg Hygiene Theory- To analyze soft aspects of individuals
  8. Marketing Mix Model- To analyze marketing mix.

Continue reading “Top 7 Business Strategy Models”

Readings Lots of CSV Files in #Rstats

Assign Names to Objects Based on Names of Files in a Directory using R for a large number of input csv files

getwd()

setwd(“C:/Users/KUs/Desktop/delhi crimes”)

a=dir()

for(i in 1:length(a)){

assign(paste(a[i]),(read.csv(a[i],header=T,sep=“,”)))

}

ls()

#Rstats continues its march in data mining

From the famous KARL REXER ANNUAL DATA MINING SURVEY

—

HIGHLIGHTS from the 2013 Data Miner Survey:
  • SURVEY & PARTICIPANTS:  68-item survey conducted online in 2013.  Participants: 1,259 analytic professionals from 75 countries.  This is the 6th Data Miner Survey.
  • FOCUS ON CRM:  In the past few years, there has been an increase among data miners in the already substantial area of customer-focused analytics.  Respondents are looking for a better understanding of customers and seeking to improve the customer experience.  This can be seen in their goals, analyses, big data endeavors, and in the focus of their text mining.
  • BIG DATA:  Many in the field are talking about the phenomena of Big Data.  There are clearly some areas in which the volume and sources of data have grown.  However it is unclear how much Big Data has impacted the typical data miner.  While data miners believe that the size of their datasets have increased over the past year, data from previous surveys indicate that the size of datasets have been fairly consistent over time.
  • THE ASCENDANCE OF R:  The proportion of data miners using R is rapidly growing, and since 2010, R has been the most-used data mining tool.  While R is frequently used along with other tools, an increasing number of data miners also select R as their primary tool.
  • CHALLENGES IN THE USE OF ANALYTICS:  Data miners continue to report challenges at each level of the analytic process.  Companies often are not using analytics to their fullest and have continuing issues in the areas of deployment and performance measurement.
  • ENGAGEMENT & JOB SATISFACTION:  The Data Miners in our survey are highly engaged with the analytic community: consuming and producing content, entering competitions and searching for education and growth within their jobs.  All of these activities lead to high job satisfaction, which has been increasing over time.
  • ANALYTIC SOFTWARE:  Data miners are a diverse group who are looking for different things from their data mining tools.  Ease-of-use and cost are two distinguishing dimensions.  Software packages vary in their strengths and features.  STATISTICA, KNIME, SAS JMP and IBM SPSS Modeler all receive high satisfaction ratings.
  • OTHER FINDINGS include the labels analytic professionals use to describe themselves (Data Scientist is #1), the algorithms being used (regression, decision trees, and cluster analysis continue to be the triad of core algorithms), and computing environments (cloud computing is increasing).