Category: Analytics

I created a list of questions and answers I have seen for data science interviews. Since everyone claims to be an expert in data science, let me assure I am obediently learning new things with bemused humility after 12 years.

Interview Questions for Budding Data Scientists

Background– I have been working into analytics since February 2004. From 2004 to 2007 I worked only in SAS language. From 2008 onwards I started working with R and SAS languages. From 2013 I started working with Python. Since around 2009, we had a term called Big Data thanks to Hadoop and since 2013 we had a term called data science. What used to be called just analytics is now called data science, with added variants of Big Data Analytics and Business Analytics to refer to the same. For techniques in building models, we have used the terms predictive analytics, data mining and machine learning to mean roughy the same thing ( but actually they might be different). I had a MBA for business training and have written two books on R by now. I am currently writing my third book but on Python for Wiley.
During this journey I have taught, trained, mentored hundreds of budding data scientists and also interviewed a few of them, while giving a few interviews myself.  Based on this experience, here are a few questions to help you clear an entry level interview for data science roles and become data scientists.  Since data science itself is an intersection of business perspective, statistics and coding, I have accordingly labeled them in specific sections. This will not give you a sure shot chance of clearing an interview but learning a few of these questions and answers will definitely help increase the probability of you clearing the interview process.

http://www.edureka.co/blog/top-data-science-interview-questions-for-budding-data-scientists

I helped create Edureka’s R course  in 2013 (before I did it for Collabera in 2015 and after I did it for Jigsaw in 2011)- you can see a video of the initial class here which has gotten 100,000 views.

Edureka remains one of the true believers in customer centered online education without fooling young people of too much money, by a mass market, mass course approach with actual teacher student interaction than the cold robotic automation of MOOCs.

(related-

http://analyticstraining.com/author/ajay-ohri/

http://www.collaberatact.com/online-training-courses/analytics-with-r-certification/

I am writing a new book on a new language for me (python) for a new publisher ( Wiley)

This book is the first of its kind to provide a reference that enables students and practitioners to easily learn to code in Python if they are familiar with R and vice versa, even if they are beginners in the second language. It also provides a detailed introduction and overview of each language to the reader who might be unfamiliar with the other. While R has better statistical and graphical tools, Python has good machine learning tools and proves to be more useful software for the analysis of Big Data. A unique feature of this book is how it provides a command-by-command translation between R and Python for many mathematical, visualization and machine learning techniques. The intended audience is statistical practitioners and data scientists trying to learn one of R or Python or both, as well as students that are familiar with one of the languages.

http://www.amazon.co.uk/Python-R-Users-Ajay-Ohri/dp/1119126762

Despite my visa blues ( see more at https://todayilearnedinamerica.wordpress.com/2016/02/15/night-13-make-epic-shit/ ) I am still hanging on and traveling on in the United States of America. I might have posted some blog posts here by mistake but I corrected that today since I am traveling by an Amtrac pass and I dont have wifi there.

I am also going to TWO of the best conferences I have never attended despite being a blog Partner since past three years.

Predictive Analytics World San Francisco – April 3-7, 2016 – http://www.predictiveanalyticsworld.com/sanfrancisco/2016/

Predictive Analytics World for Workforce – April 3-6, 2016 – http://www.predictiveanalyticsworld.com/workforce/2016/

Do you want to go to San Fransisco for this conference ?

MAGIC CODE TO USE- You can get a 15% off the price of registration for 2 Day and Combo passes:    AJAYBP16

Does the NSA track Git requests. I mean can’t the terrorists just be talking to each other by Visual Cryptography of Arabic through Git Repo requests.

Basically increase the cost of decryption.

This is Visual Cryptography. Now Imagine using a one time pad codebook of just emojis and talking through mobile and Kik.

Etherpad is a highly customizable Open Source online editor providing collaborative editing in really real-time

Think of it as Kik for laptops.

Unfortunately NSA killed Waste Again

What is “WASTE again”?

“WASTE again” enables you to create a decentralized and secure private mesh network using an unsecure network, such as the internet. Once the public encryption keys are exchanged, sending messages, creating groupchats and transferring files is easy and secure.

Creating a mesh

To create a mesh you need at least two computers with “WASTE again” installed. During installation, a unique pair of public and private keys for each computer is being generated. Before the first connection can be established, you need to exchange these public keys. These keys enable “WASTE again” to authenticate every connection to other “WASTE again” clients.

After exchanging the keys, you simply type in the computers IP address to connect to. If that computer is located behind a firewall or a NAT-router, you have to create a portmap first to enable incoming connections.

At least one computer in your mesh has to be able to accept incoming connections, making it a “public node”. If no direct connection between two firewalled computers can be made, “WASTE again” automatically routes your traffic through one or more of the available public nodes.

Every new node simply has to exchange keys with one of the connected nodes and then connect to it. All the other nodes will exchange their keys automatically over the mesh.

Google Drive safety for government as well as corporate espionage.

Post Snowden- we know that that company cofounded by a Russian Jewish immigrant went back on those very core values and helped create a benign version of the KGB. So you run away from Russia because hey the State is monitoring everything and then you refuse to stop your company from helping the new country’s government to do exactly the same. Google’s post snowden loss of credibility should make businesses think whether Microsoft Exchange servers are more secure than Google’s. Atleast Microsoft offers no – Dont be Evil Hypocricy”

Chinese hackers  are just a Google bogey. Pakistani Intelligence ISI helping ISIS with ideas and consulting is what is the real threat. Too bad, we were too busy creating algorithms to click more ads.

The TOR- ONION Project is more like the ONION comedy thanks to its proliferation of NSA nodes.

The Tree of Liberty needs to be watered with the blood of patriotic cryptographers not accepting easy money but staying true to the ideals of a public free from espionage from Government with the fourth amendment rights secured digitally as well.

I was in New York for past two days. New York is very pretty, very cold and the trains are very confusing. So I ended up walking and going back and forth.

Later on, when I reached home, heat, food, bed and jet lag, I decided to analyze where and what did I see in the visual ephiphany tour.

This was my route

This was my R code

library(jsonlite)
a=fromJSON("/home/ajay/Desktop/Takeout/Location History/LocationHistory.json")
b=as.data.frame(a)
 
mygoog=NULL
mygoog$latitude=b$locations.latitudeE7/10000000
mygoog$longitude=b$locations.longitudeE7/10000000
mygoog$time=as.POSIXct(as.numeric(b$locations.timestampMs)/1000 , origin="1970-01-01")
 
 
mygoog=as.data.frame(mygoog)
head(mygoog)
nrow(mygoog)
#Clearly that is over the API limit for free usage
length(unique(mygoog$longitude))
library(magrittr) #to make code easier to read
mygoog$longitude%>%unique%>%length
unique(mygoog$latitude)
mygoog$latitude%>%unique%>%length
 
fivenum(mygoog$latitude) #tukey
 
#or using Dr H over Tukey
library(Hmisc)
describe(mygoog$latitude)
describe(mygoog$longitude)
 
#deleting Non NY data
mygoog2=mygoog[mygoog$longitude<0,]
describe(mygoog2$longitude)
rm(mygoog2)
mygoog2=mygoog[mygoog$latitude<48,]
describe(mygoog2$latitude)
rm(mygoog2)
 
mygoog=mygoog[mygoog$longitude<0&mygoog$latitude<48,]
 
 
library(ggmap)=
#Starting Point
revgeocode(c(mygoog$longitude[1],mygoog$latitude[1]))
#Starting Time
mygoog$time[1]
------
  #Median Location
a1=median(mygoog$longitude)
print(a1)
a2=median((mygoog$latitude))
print(a2)
revgeocode(c(a1,a2))
 
#Lingering Location
Mode <- function(x) {
  ux <- unique(x)
  ux[which.max(tabulate(match(x, ux)))]
}
b1=Mode(mygoog$longitude)
b1
b2=Mode(mygoog$latitude)
b2
 
revgeocode(c(b1,b2))
#Creating New Fields to minimize API calls to Google Maps
unique(mygoog$longitude)
unique(mygoog$latitude)
 mygoog2=mygoog[!duplicated(mygoog[c("longitude", "latitude")]),]
nrow(mygoog2)
 
result <- do.call(rbind,
                  lapply(1:nrow(mygoog2),
                         function(i)revgeocode(as.numeric(mygoog2[i,1:2]))))
mygoog2 <- cbind(mygoog2,result)
 
library(stringr)
mygoog2$zipcode <- substr(str_extract(mygoog2$result," [0-9]{5}, .+"),2,6)
mygoog2[,-4]
 
#merge(x, y, by=c("k1","k2")) # NA's match
 
#Cleaning up workspace
             #rm(a1)
             #rm(a2)
             #gc()
 
Map <- get_googlemap(center = c(lon = median(mygoog$longitude), lat = median(mygoog$latitude)),
                     zoom = 13, 
                     size = c(640, 640), 
                     scale = 2, maptype = c("terrain"), 
                     color = "color")
 
plot1 <- ggmap(Map) + 
  geom_path(data = mygoog, aes(x = longitude, y = latitude
  ), 
  alpha = I(0.9), 
  size = 1.8)
suppressWarnings(print(plot1))

Code contains Easter Eggs created by Pretty R at inside-R.org

Dr Eric Siegel has just released the updated version of his very popular book on Predictive Analytics, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die

at http://www.predictiveanalyticsworld.com/book/overview.php

The book which is a bestseller in many categories on Amazon has met with overwhelming praise from industry. One of the reasons is that it is chockablock with real case studies that make it much more easy to learn and execute predictive analytics. I frequently recommend it as an additional book when I am teaching data science online. Here is the link to my 2013 review of the earlier edition

https://decisionstats.com/2013/02/25/book-review-predictive-analytics-the-power-to-predict-who-will-click-buy-lie-or-die/

Blogging Conflict of Interest Disclaimer – The book author is also founder of Predictive Analytics Conference, a sponsor of this site since many years.

Wireshark is the world’s foremost network protocol analyzer. It lets you see what’s happening on your network at a microscopic level. It is the de facto (and often de jure) standard across many industries and educational institutions.

INSTALL 

First, we install Wireshark from the terminal.

Source-

http://www.dickson.me.uk/2012/09/17/installing-wireshark-on-ubuntu-12-04-lts/

CAPTURE

Type wireshark from terminal.

Start capture by looking at Capture Tab and interfaces

Export data as a csv

ANALYZE

Import file in R to analyze

(from http://www.statmethods.net/input/importingdata.html )