The Hack of the Century

  • Fake news propogated by social media just before the US General Election
  • Hacking of Democratic Party by hackers of East European Russian origin /affiliation coordinating publication of leaked emails
  • Leaked emails by hackers causing FBI chief to make fateful statement days before election
  • Recanting of statement by FBI chief further undermining Clintonian credibility

I really think- given the margin of victory- that general elections 2016 in USA was the hack of the century. Cyber players influenced enough to tip the scales slightly.

So folks, there would be no Khaleesi after all, yall. Thanks for watching. We will be back, say in 4 years

14543535534530

 

PESTLE Analysis of Barack H Obama’s Presidency

On Nov , 2008 I wrote the article on using PESTLE for analyzing 43 POTUS.

PESTLE Analysis of George W Bush's Presidency

 

The article tries to use the PESTLE ( details below ) analysis framework for analyzing the 43 President of United States. It may be a bit biased , as all articles written by men are. It will try its best to be faithful to the facts , and not to the opinions surrounding this President.

The PESTLE framework stands for –

  • Political
  • Economic
  • Social
  • Technological
  • Legal
  • Environmental.
It will rate the Presidency as a Presidency, means it will evaluate his tenure as leader of the United States, not of the free world or anything else he was not elected or declared by the Supreme Court to be ( pun unintended). Scores are from 1 to 10, but you are free to comment with your own rating scores.
 I am writing this on July 30, 2016 for analyzing Barack H Obama‘s Presidency using PESTLE framework.
Political-5
 A change from 8 years of Republican to 8 years of Democratic politics was made with a relatively inexperienced senator following a relatively experienced governor to be POTUS
The United States retreated from the preemptive doctrine to eliminate hostile nations by September 11.It used special forces and drones to be selective in military action and used social media for enabling change. It retreated but not fully from a successful war in Afghanistan . The inability to deal with ISIS with a coalition is a big failure. The breach with Europe over the Iraq war has largely been remedied new threats from ISIS. Overall the United States gained diplomatic leverage in 8 years but lost political leverage
Both on  North Korean , and ISIS it failed. No tangible progress on climate change was made. Isolationism gained ground as political capital was spent getting healthcare passed but the initiative was lost in terms of current domestic politics. Cuba detente and Arab revolutions were a change. POTUS 44 promised hope and change but he did deliver change.
Economic-8
In terms of job creation, and economic growth POTUS 44 presided over the best period of growth and the best period of spin on this front since Bill Clinton. Remarkably economic growth happened as financial crisis eased but war driven spending eased. Keynesian spending on bailouts was corrupt and unethical but proved effective for the financial sector bailout
Social- 7
The Obama era was categorized by even more partisan politics amplified by social media, as the first black POTUS failed to prevent black men from being legally killed leading to Black Lives Matter movement and the killings of police by black men. Gun control debates were remarkable partisan and enhanced, the gains in gay equality were almost  lost in a nighclub in Orlando as repeated ISIS attempts tested the great melting pot.
The  standards of health care for the average American citizen remain a concern compared to rest of developing world, and immigration reform is now a partisan matter., and the rising income  rhetoric this as one of the most under performing sector of the 44 rd presidency. While the Bush presidency is marked for both generous aid to Africa, the Obama era proved prone to a few charges of hypocritical elitism in its defensive way of not defending black America against social tensions. The dreams of the mother will take a village to give back the audacity of hope to one of largest incarcerated societies in the world.
Technology-9
American innovation chugged on, but FB and Google both came up despite and inspite Bush. Today Uber, Air BNB, Whatsapp and Facebook Live offer innovation and  changes to the world not just America. On a disappointing note, climate change innovation was largely either absent or not shared openly.
Legal-3
Astonishingly POTUS 44 chose legal nihilism for selective targeting for US citizens. This included cyber whistle blowers like Snowden and propaganda from clerics. Rather than shutting down the ability to speak, POTUS 44 enhanced the ability to kill legally without involving Congress
Environmental-7
Environmental record by a President was better thanks to a good call on delaying the Canadian pipeline and more investment in alternative sources.  Not much was achieved in effective environmental change that addresses climate change

In summary ,

the Obama presidency would be judged much later for how much good or bad it actually did versus how much it got criticized for, but it did became one of the most historic presidencies
Conclusion- POTUS 44 disappointed  domestic audiences on the expectations it raised at start in 2008 and this shall be an important factor for evaluation in terms of historical legacy)
TO BE EXPANDED AND UPDATED

Rexer Analytics Data Mining Survey Results

Latest Results from Rexer Analytics

HIGHLIGHTS from the 2015 Data Science Survey:
 
    SURVEY & PARTICIPANTS:  59-item online survey conducted in 2015.  Participants: 1,220 analytic professionals from 72 countries.  This is the 7th survey in this ongoing research program.
 
    CORE ALGORITHM TRIAD:  Regression, Decision Trees, and Cluster analysis remain the most commonly used algorithms in the field.
 
    THE ASCENDANCE OF R:  76% of respondents report using R.  This is up dramatically from just 23% in 2007.  More than a third of respondents (36%) identify R as their primary tool.
 
    JOB SATISFACTION:  Job satisfaction in the field remains high, but has slipped since the 2013 survey. A number of factors predict Data Scientist job satisfaction levels.
 
    DEPLOYMENT:  Deployment continues to be a challenge for organizations, with less than two thirds of respondents indicating that their models are deployed most or all of the time.  Getting organizational buy-in is the largest barrier to deployment, with real-time scoring and other technology issues also causing significant deployment problems.
 
    TERMINOLOGY:  The term “Data Scientist” has surged in popularity with over 30% of us describing ourselves as data scientists now compared to only 17% in 2013.
 
Please contact us if you have any questions about the attached report or this research program.  Summary reports for all 7 surveys (2007-2015) are free and available by contacting DataMinerSurvey@RexerAnalytics.com.  Additional information about the surveys is available at www.RexerAnalytics.com, including verbatim best practices and insights shared by respondents in previous surveys — see the Data Miner Survey links in the top right corner of our home page.
 
Information about Rexer Analytics and our consulting services is also available at www.RexerAnalytics.com.

Interview Kiran Rama India’s Number One Data Scientist

Here is an interview with Kiran Rama. He is currently Director, Data Sciences & Advanced Analytics at VMWare. I have chosen Kiran as India’s number one data scientist for the following reasons

  1. He has both an impeccable academic record as well as steady work experience across multiple companies
  2. He has demonstrated his expertise in competitions like Kaggle and KDD cup (which is tougher)
  3. He spends more time doing and expanding data science in India

12976970_10154173725606579_4175448674223012627_o

Here is the interview with Kiran Rama, India’s Number One Data Scientist as per 2016 as per Decisionstats.com

Ajay- Describe your career as a data scientist from corporate job, entrepreneurial work experience, winning competitions, patents and finally back to industry
Kiran- I always had a flair for programming being a computer science engineer and winning inter-college on-the-spot programming and debugging fests. Post computer science engineering graduation, my interesting work was at Motorola using C/Linux for developing features for protocol of Multimedia Messaging Service on several mobile phones. I also owned protocol analytics that involved debugging log files looking for null pointer errors!
Post a couple of years of work, I pursued my post-graduation in management from Indian Institute of Management Kozhikode (IIMK) where I finished in the top 5 of the batch majoring in Information Technology & Systems. I was confused what to do after engineering getting 99 percentile+ in both CAT and GATE. I did not want to stop programming or lose my technical orientation and therefore a role in analytics/data sciences seemed like a natural fit as it involved both technical and business stuff.
I was one of the first hires of the e-business analytics team in Dell in 2006.  I got certified in Base SAS and SAS Enterprise Miner and used SAS primarily for data sciences while I used Omniture tools, Excel, SQL for analytics. At Dell Global Analytics, I took on diverse responsibilities and grew from the equivalent of a senior analyst to a Senior Analytics Manager. I touched all parts of e-commerce and e-business.
Some of my achievements in Dell included:
  • “2012 India Innovator of the Year” Award from Michael Dell
  • 3 patents filed at US PTO on various aspects of e-commerce and marketing analytics
  • World Quality Day Finalist in 2010
  • Won the Best Project Award in Global Consumer & Small Business Analytics for 4 consecutive quarters
While at Dell, in 2010 or so, knowing SAS well I was frustrated that I could not freelance using SAS owing to the high cost. At that time I picked up R and it is the best decision I made in my career as I took to R like a fish to water owing to it’s many similarities with C.
At Dell, I started participating in data mining competitions on Kaggle.com and had several top finishes.  I was a “Master Data Miner” on kaggle. I had great results in the Amazon Employee Access Challenge, Merck Competition to predict Molecular activity, GEFcom competition on load forecasting, on wind forecasting….etc. My Kaggle pursuits was one of the reason I was recruited by Amazon.
I worked as a Senior Business Analytics Lead at Amazon in their Bangalore office as a Level-6 individual contributor. Level-6 in Amazon in those days was one of the senior-most individual contributor position that they had in Bangalore in the engineering teams and very difficult to get laterally on the technical side. However the role was not to my liking and I decided to leave to head marketing & Customer analytics at Flipkart.
I freelanced for several US startups as part of part-time proprietorship “Chaotic Experiments”. Some projects included:
  • Software Errors: Predict which line in software code is likely to be an error for a US based startup
  • Accident evaluation analysis for a US semi sized startup
  • Predict which music label to recommend to a startup
  • Trying to predict futures prices in the stock market for a US Startup
  • HLA Imputation of Genomic data
At Flipkart, I had the good opportunity of leading several data sciences & analytics projects for Flipkart of which the below ones I am proud of:
  • Leveraging Data Sciences to come up with customer segments for Flipkart’s digital properties
  • Coming up with an email rules engine to determine the best customers to target per category
  • Setting up mobile app analytics at Flipkart
I worked closely at Flipkart with the CTO on data scientist hiring and helped in hiring data scientists being the decision maker for the “data sciences depth” round for data scientists at Flipkart
I continued my hand at Kaggle while at Flipkart and at one time for over a year, I was ranked amongst the top 10 data miners in the world on Kaggle. My top rank was 7 out of some 300K data miners competing in data sciences competitions for sport and the icing on the cake came when I finished in the top 3 in KDD Cup 2014 winning the competition to predict which essay was likely to  get a donation on donorschoose.org
Post Flipkart, I was hired into VMW, where I play the role of “Director, Data Sciences & Advanced Analytics”. I play dual roles of functional (where I lead the data sciences innovations team for VMW globally working closely with digital analytics, Digital store/e-commerce, Professional Services, Sales, Marketing, Partner, Pricing, Support,… verticals) and dotted-line (where I represent the equivalent of the Enterprise Information Management in India comprising of Master Data Management, Business Intelligence & Advanced Analytics).
At VMW, have had a unique experience driving B2B data sciences with industry leading projects like:
  • First ever digital buyer journey data sciences project at VMW
  • “Propensity to Buy” models for several products of VMW, for the Technical Account Manager organization,..
  • “Propensity to Sell” models for the partner organization of VMW
  • “Propensity to Respond” models
  • Deployment models
I currently am at VMW and have been here since almost 2.5 years and loving every moment of it.
Ajay- What are the key things you want to say to someone with no work experience and who wants to work as a data scientist ? What would you say to someone who has a few years work experience and wants to switch to data science
Kiran- For both (freshers and experienced), I would say the following are key in the order of priority:
  • Debugging Skills: You cannot give up as a data scientist and should be a person who can sit at one place and continuously debug for hours. Data Science techniques usage will involve installations, OS issues, nitty-gritty aspects of the code,… etc
  • Programming Skills: You cannot be a data scientist if you cannot program. You need to be good at programming. Comments like code is available on the net and I will copy-paste do not work. I judge a data scientist by different parameters and one of the most important ones is the quality of the code!
  • Knowledge of a Programming Language that has a machine learning library (R or Python are an example. R has access to many of the libraries on the CRAN repository while Python has the world beating scikit-learn package)
  • Strong understanding of the mathematical and computer science and statistical background of the data sciences techniques behind the techniques
  •  Ability to translate a business problem into a data sciences problem. This involves key decisions like which is the target, is this a prediction or classification problem, what is the right cross-validation technique, what algorithms to use for data mining, what should be the right evaluation criteria, how the model will likely be deployed,…
  • Strong business/domain understanding can lead to great feature engineering and great success while deployment.
  • Ability to present the results to stakeholders and get buy-in for implementation is very important as well
There is a lot of misconception that everyone should do data sciences. Not everyone is suited for this. If you cannot sit in a place for a stretch and code for 5-6 hours in R or Python and SQL, this is not the right job for you. If you do, this is the best thing to do.
Ajay- you have used many tools like SAS Python and R. How would advise a new data scientist on which tool to learn and how to structure their tools training
Kiran-I would suggest a new data scientist to use Python > R > SAS mainly because:
  • Python and R have better and wider machine learning libraries than SAS
  • Most of the academic work and latest advancements are in Python & R
  • Python is better than R because there are more things you can do in Python including software development. Trust me – there is no money in machine learning libraries. There is money only in applications and closer you are to software development + machine learning, the better
  • Most of the high paying startups and young firms use Python/R and not SAS
  • It is easier to learn Python/R and then if you happen to work for an old behemoth that is a SAS shop, pick up SAS as well
  • Python/R are actual programming languages and better than SAS. SAS uses macros and not functions. SAS uses proprietary dataset format that is largely inefficient. SAS requires you to know different syntax for different methods and also different types of plots. On the other hand, the interface to call any function in R or Python is the same. Example: predict function in R. Since everything is returned as an object in R & Python it is easier to examine them (contrast looking at the object sub-objects to running multiple commands in SAS to find the output datasets – the infamous “ods trace on” in SAS,………etc)
A good data scientist should know both R & Python but better to start out in one and master for a year.
Great books to learn R and Python are:
  • “The Art of R Programming” by Norman Matloff
  • “Python for Data Analysis” by Wes Mc Kinney
For Machine learning fundamentals would recommend:
  • Learning from Data by Mostafa
  • Applied Data Mining by Paolo Giudici
  • Machine Learning by Tom Mitchell

13725095_10154440244451579_3409709718217659176_o

Ajay- What are some key best practises you want to tell to people preparing for data science competitions
Kiran- Here is the link to my kaggle interview on winning the KDD 2014 cup: http://blog.kaggle.com/2014/08/05/3rd-place-interview-from-the-kdd-cup-2014/
Here is the link to my code repository for the winning solution in KDD 2014 cup:https://github.com/rkirana/kdd2014
Best practices I suggest are:
  • Build your own repository of functions and methods that you can re-use
  • Understand what the winners of prior competitions did. For example: my code above
  • Keep yourself current with the latest techniques. For example: xgboost
  • Choose the right cross-validation technique. Else, you will overfit
  • Be paranoid about leakage and look for ways to fix leakage in everything including data preparation, feature engineering and modeling
  • Feature Engineering is the key. Even with lesser data, better features will do better than big data
  • Try different methods that are varied. Example: one learner can be tree-based, one bagging, one boosting based, one neural network….etc
  • Always ensemble. It can give 2-5% lift
Last mile optimization is difficult. While you can get a 0.85  AUC easily, taking it to 0.88 AUC can be an uphill task
Ajay- What are some of the key algorithms that a data scientist should know?
Kiran- Some of the algorithms that one should know are:
  • Regularized Logistic Regression (glmnet in R)
  • Bagging technique: Random Forest
  • Boosting Technique: Gradient Boosting Machine, Extreme Gradient Boosting
  • Collaborative Filtering Techniques: LIBFM
  • Non linear learners like Neural Networks
  • Bayesian Methods like BayesTree, bartMachine
  • Support Vector Machines – LIBSVM library
  • Fast learners like Vowpal Wabbit
One should always ensemble multiple techniques in order to get better results
Ajay- Describe your favourite online learning resources for learning data science languages, algorithms etc
Kiran- kaggle.com – nothing beats it
cran.r-project.org – all the vignettes there
Ajay- How do you keep yourself updated on data science knowledge 
Kiran- I have not participated in data science competitions for last 2 years – participating was a way in which I kept myself and pushed myself to be updated
I am very keen on making some original contributions to data sciences research and teaching. I am pursuing a part time doctoral program (PhD) at IIM Lucknow while I do my full-time job. Spend a lot of time on scholar.google.com these days to understand the existing contributions to data science theory and how I can make original contributions to the same
I also drive industry-academia interaction with the Data Center and Analytics Lab at IIM Bangalore where I represent VMW on the DCAL board. I am at the forefront of organizing industry events on data sciences to share knowledge and learn about the latest in the industry.
I am thankful to my leaders and my direct team at VMW for giving me so many interesting business problems to solve using data sciences and that pushes me and drives me to keep myself updated

About-

Kiran is a Data Sciences Leader with more than 12 years of experience across marketing, digital (web/mobile), retail, pricing, partner, sales. Experience across B2C, e-commerce & B2B data sciences. One of the Top 10 Player on Kaggle – data mining competition platform – in 2013 and half of 2014 world-wide, Kiran is also KDD 2014 Prize Winner and Holder of 3 US patents. 2012 Innovator of the Year award from Michael Dell.

You can read about him here https://www.linkedin.com/in/rkirana