Data Science Apps for Plug and Play Data Science

I was reading the 12 factor App and was struck by how much data science practitioners could use these principles too, for example when making a Shiny Dashboard App

Also I hope we can have more plug and play data science for mobile data or data generated by mobile apps (which is increasing)

Screenshot from 2016-05-11 23:11:17

An example is this app here https://gallery.shinyapps.io/CampaignPlanner_v3/ which can possible modified to add integration with Google Web Analytics API (etc).

This approach can make R more enterprise ready for production environments where it currently lags behind Python in terms of both appeal as well as trained people.

http://12factor.net/

The Twelve Factors

I. Codebase

One codebase tracked in revision control, many deploys

II. Dependencies

Explicitly declare and isolate dependencies

III. Config

Store config in the environment

IV. Backing services

Treat backing services as attached resources

V. Build, release, run

Strictly separate build and run stages

VI. Processes

Execute the app as one or more stateless processes

VII. Port binding

Export services via port binding

VIII. Concurrency

Scale out via the process model

IX. Disposability

Maximize robustness with fast startup and graceful shutdown

X. Dev/prod parity

Keep development, staging, and production as similar as possible

XI. Logs

Treat logs as event streams

XII. Admin processes

Run admin/management tasks as one-off processes

Because it is Friday really

Timestamps can be manipulated leading to confusing meta data. cognition can be manipulated using images.

cog·ni·tion
ˌkäɡˈniSH(ə)n/
noun
 
  1. the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses.

 

Which tool to learn for a better data science career

Some questions I get from new data scientists

I like R a lot, so should I work towards being better at just that or should I learn excel and python and sas as well (Like a jack of all master of none)?

I like R so much I wrote two books on it. Then I started writing a book on Python and now I am on writers block.

  1. You need to be good at many things (Python, R, SAS, Excel, SQL)
  2. You need to be really really good at one thing ( I prefer Python, but R or SAS could do. SAS people work in large corporations a lot, R people are more statistically driven, Python people are more Silicon Valley /IT driven. I would go with Python)
  3. You should know how data is stored ( in RDBMS and in NoSQL)
  4. You should know how data is processed (cloud computing, server)
  5. You should know how data is visualized ( GGPLOT, Qlikview, Tableau)
  6. You have a limited time to learn all this and again many choices. So try going for more education and more training!

Is it a thumb rule to know advanced analytics with Excel before actually aiming at R?

There are 8 fingers but two thumbs. Thumb rules are shortcuts. They save time (for instructor to explain). Yes I would learn how to analyze data in a spreadsheet too since a lot of employers use spreadsheets. Spreadsheets ( not juts Excel but OpenOffice and Google Docs) are used more than data science tools in analyzing data. Basic principles remain the same.

If I choose to begin with a job so I can get a feel of the industry and get to know it better they ask for all these tools with?

Of course industry wants people with 2 year work experience. Why should they pay you to learn? So learn skills before you expect jobs.

I found out that internships, kaggle competitions, certifications,case studies could help, but not yet in India is what I’ve come to see.. Is that true sir? Am I judging it wrong?

Wrong judging. Judging itself is wrong. Stop judging 1.2 billion people and millions of sq km.

I know many Meetups in India ( I founded one in New Delhi). Kaggle compeitors are great in Mumbai. Bangalore is great for coneferences. So yes, you are comparing to USA ( not fair, its better here because people just are more disciplined in organizing). if you compare to Pakistan we are better.

Best is to stop asking for help, and just go out and attend. Maybe create a Meetup group yourself ( takes only 10$ a month but again someone has to pay it!). Maybe create a desi meetup. Maybe create a Meetup only for Women. or only for New Comers. Or Ask your online education provider. Stop judging countries. Start volunteering. People like http://www.venturesity.com/ are doing a great job for hackthons in India. Meetups in Bangalore https://goo.gl/1wPta5

http://www.ushistory.org/documents/ask-not.htm

ask not what your country can do for you — ask what you can do for your country. more meetups in data science. more hackathons. yup.  that is good enough work to start doing for your country.

images

 

career advice for data science newcomers

Someone I dont know asked this question-

I had a few questions sir. Could I ask them to you? Like mainly based on the direction in which I should work or learn. I don’t mean to bother you. But it’s hard to find the right people who can guide new comers like me.

I first explained why people don’t really give advice for free. I am using principles I learnt from Reader’s Digest about something known as a Fermi Problem. Fermi Problems are common in tech interviews.

  1. there are 3000 newcomers to every such right person (who gives free advice to newcomers he does not know).
  2. out of them only 300 will get the courage to ask the right people.
  3. out of them 250 will write a badly written email.

so by the time the right person has gotten 250 spam emails, he is not responding to the 50 out of the 3000 who

  1. write well, and
  2. are passionate about learning more.

that is an example how you can use mathematical thinking to understand why things work.

Then I gave in and gave her some free advice on what direction a data science newcomer should put efforts in

which direction should you work?-

  1. interest/passion/quality – do something you are good at, because then only it will sustain your interest and you will be put up the 10000 hours to be great at it. and
  2. greed  (higher salary) versus fear (different skills)– it should make you money but you make more money if you create your own niche. so should you be like thousands of analysts in credit card analytics (easy route) or should you do analysis on videos ( tougher).
  3. networking– why dont you atleast go to data science meetups, and try to take part in a few kaggle competitions. also, have you stopped reading r-bloggers.com or kdnuggets.com.Do this for three months and you will find enough opportunities or data. take decisions based on data not from anecdotal advice from experts.

I hope I was able to be useful. What do you think?

Portrai
Adding a 2.4 mb file slows page load on mobile devices, but that is a small cost to learn about this great Italian American – Enrico Fermi. 

Enrico Fermi, Italian-American physicist, received the 1938 Nobel Prize in physics for identifying new elements and discovering nuclear reactions by his method of nuclear irradiation and bombardment. The Fermi technique is named after physicist Enrico Fermi as he was known for his ability to make good approximate calculations with little or no actual data. Fermi problems typically involve making justified guesses about quantities and their variance or lower and upper bounds. Probably you can use it for Big Data Analysis about online chatter when your machine learning is not able to process videos (Youtube) or Images ( Instagram) as efficiently as it analyzes text.