Month: August 2016

How do I shift to a data science career

I get this question a  lot – How do I shift to a data science career. I have been doing data analysis since 2004 (in SAS) when we used to call it business analytics , and since 2007 in R, Since 2014 in Python,  by when we re branded business analytics as data science. So here are a few basics to people trying to SHIFT to data science.

My answer is learn coding, learn math, and most importantly know when to use what for insights. Data scientists are as good as the insights they create or miss not the code they write.

See this first

A slideshare I put forward last year for Summer School

Do this self examination-

  1. What are you good at – programming , stats, or business
  2. What are you bad at- programming , stats or business
  3. What can you learn and at what proficiency

Learning Programming

Learning R, SAS, Python is easy but there is a confusing clutter of resources out there on the internet.

SAS Language -should be learnt from SAS University Edition and for the SAS Certification Exam.

Dont wanna be SAS Certified (its just 100$ psst)

Here is some free SAS Training by Decisionstats

There is no certification in R or Python, though Hadoop has it just like SAS has it.

For R- learn R and RStudio till you can master some of the code here

http://rpubs.com/ajaydecis

Screenshot from 2016-08-23 10-48-29

or see all the R packages here at CRAN VIEWS https://cran.r-project.org/web/views/

For Python-

A shorter tutorial on Python by the author is here

http://www.statisticsviews.com/details/feature/8868901/A-Tutorial-on-Python.html

Learn PANDAS and SCIKIT-LEARN  example https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks

 

Learning Statistics and Techniques

 

Data Mining in R

http://rattle.togaware.com/rattle-examples.html

Where to learn machine learning

http://scikit-learn.org/stable/tutorial/basic/tutorial.html

Learning Business

This comes with experiences and domain research and study.

 

I hope this helps. I will follow with specific answers to specific career questions in data science soon.

 

 

Some thoughts on the Revolution Sellout

The revolution will not be televised, brother –Gil Scott-Heron

 

Veteran R Community members must recall R founder’s Ross Ihaka ‘s warning against Revolution Analytics not being truly open source,

and the sale to Microsoft will be keeping Revolution R open source in the time being ,

it did proved Ross Ihaka was right.

How do you help create an open source revolution in statistics by selling a company to Microsoft beats me.

And how do you just take 6000 packages for free from open source community, add 6-9 packages of your own and then repackage the bundle as a new innovation?

Even though Revolution analytics created 3 CEO JOBs,including SPSS founder Norman Nie, and 1 name change  (from computing to analytics) and  1 mass firing ( with a 50% layoff they wont be winning the best employer award),  in the end what drives software is lots of sales and not lots of blogs

(quoting Larry Ellison‘s purchase of Sun ).

In addition

love for computing and not hypocrisy on love for money should drive science.

A potato is a potato.

In Australia or Seattle or San Fransisco

SAS and Jupyter work well together now

 

While  R community continues to move ahead with  RStudio (open source still),  and other interfaces,

SAS is moving forward to embrace Jupyter in it’s free University Edition. The word Jupyter itself is made from Julia, Python and R. Note whether you are a R fan or Py fan or a SAS fan, you should compare and contrast the quality of blogs, the documentation and the interface on your own. As a blogger and data scientist (?) I actually love all science

Screenshot from 2016-08-12 19-24-19

Using Jupyter and SAS together with SAS University Edition

A few months ago I shared the news about Jupyter notebook support for SAS. If you have SAS for Linux, you can install a free open-source project called sas-kernel and begin running SAS code within your Jupyter notebooks. In my post, I hinted that support for this might be coming in the SAS University Edition. I’m pleased to say that this is one time where my crystal ball actually worked — Jupyter support has arrived!

(Need to learn more about SAS and Jupyter? Watch this 7-minute video from SAS Global Forum.)

https://support.sas.com/software/products/university-edition/faq/jn_runvirtualbox.htm

 

How do I run Jupyter Notebook in SAS University Edition using VirtualBox?

In order to run Jupyter Notebook in SAS University Edition, you must first add the SAS University Edition vApp to VirtualBox. When you specify the URL to run Jupyter Notebook, you must specify the port number for Jupyter Notebook.

  1. Follow the steps to add the SAS University Edition vApp to VirtualBox.
    Note If you want to access files from or save files to your local computer from Jupyter Notebook in SAS University Edition, you must also set up a shared folder. For more information, see the following topics:
  2. If you downloaded a new version in July 2016, the additional port is automatically added for you. Skip this step and proceed to step 3.

A Healthy Routine for Hackers and Tech Workers

  • Meet people for dinner
  • Sleep
  • Do until

 

Heuristics and Occam’s razor for Counter Terrorism

When overworked analysts use shortcuts to search huge noisy dirty databases, they create trails which can be mined for actual heuristics

Heuristics –

A heuristic technique (/hjᵿˈrɪstᵻk/; Ancient Greek: εὑρίσκω, “find” or “discover”), often called simply a heuristic, is any approach to problem solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goals. Where finding an optimal solution is impossible or impractical, heuristic methods can be used to speed up the process of finding a satisfactory solution

 

 

Example- A Police chief in Chicago may adopt different heuristics than in New York than in New Orleans for allocating human resources

Solution- Make a database of heuristics as actually in practice for that particular domain

Additional Solution- Search Companies to partner not just in giving data but also training and in some case search algorithms for database analysis and database design reviews of Homeland Security

Occam’s Razor-

Occam’s razor (also written as Ockham’s razor, and lex parsimoniae in Latin, which means law of parsimony) is a problem-solving principle attributed to William of Ockham (c. 1287–1347), who was an English Franciscan friar, scholastic philosopher and theologian. The principle can be interpreted as stating Among competing hypotheses, the one with the fewest assumptions should be selected.

https://en.wikipedia.org/wiki/Occam%27s_razor

https://en.wikipedia.org/wiki/Heuristic

Related – How to amplify noise in social media using other algorithms

https://decisionstats.com/2010/01/19/a-noisy-algorithm/

https://decisionstats.com/2015/04/14/random-thoughts-on-cryptography/

https://decisionstats.com/2013/12/14/play-color-cipher-and-visual-cryptography/

 

https://decisionstats.com/2010/11/25/increasing-views-to-youtube-videos/

Click to access shmat_oak09.pdf

De-anonymizing Social Networks
Arvind Narayanan and Vitaly Shmatikov
The University of Texas at Austin
Abstract
Operators of online social networks are increasingly sharing potentially sensitive information about users and their relationships with advertisers, application developers,
and data-mining researchers. Privacy is typically protected by anonymization, i.e., removing names, addresses, etc.
We present a framework for analyzing privacy and anonymity in social networks and develop a new re-identification algorithm targeting anonymized social- network graphs. To demonstrate its effectiveness on real-
world networks, we show that a third of the users who can be verified to have accounts on both Twitter, a popular microblogging service, and Flickr, an online photo-sharing site, can be re-identified in the anonymous Twitter graph with only a 12% error rate.