Workflows in R compared to Workflows in Python

A workflow consists of an orchestrated and repeatable pattern of business activity enabled by the systematic organization of resources into processes that transform materials, provide services, or process information.

Both R and Python have similar workflows but slightly different syntax. one of the biggest difference is how they refer to parts of object ( $ [] in R while [] in Python) as well as how they apply functions ( fun(object) in R while object.fun() in Python)

 

a workflow in Python

http://nbviewer.ipython.org//gist/decisionstats/4142e98375445c5e4174

Screenshot from 2015-10-24 08:31:22

a workflow in R

http://rpubs.com/ajaydecis/rworkflow

Screenshot from 2015-10-24 08:31:07

Garbage Collection in a technology startup

Garbage collection (GC) is a form of automatic memory management. The garbage collector, or justcollector, attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program. Thats garbage collection from Jimmypedia. When you dont do enough garbage collection in a program you can end up with Stack Overflow.

Tech Startups have garbage collections too. A garbage collector looks only for garbage. In the edges, on the floor, below the carpet, under ths stairs. In meetings, in team discussions, in stock options, in business plans, in cash burn projections. These are negative anti social emotionally dysfunctional people. Their over abundant IQ (Intelligence Quotient)is balanced by their teenager like EQ ( Emotional Quotient). To balance the cyncial Einstein, you generally need a shiny eyed startup founder who has dreams of ringing the NASDAQ bell every night.

Tech Startups also have unicorn catterpillars. These are people who think they will shed their legs, wrap them in a silky cocoon and become unicorn butterfliess with wings. The shiny eyed founder can become an unicorn butterfly very fast, till the garbage is collected from the cocoon and the oyster returns to being an oyster than turning into a pearl.

Tech Startups have over caffienated engineers and under caffienated salesmen. I wonder how the industry would react if they introduce mandatory drug testing for startups. Maybe we will all migrate to Canada under a Treudian utopia of hemp and grass.

 

Web Analytics is funny statistics

I have a simple question for my Web Analytics software. I want to know who is reading what, and how much are they being impacted ?

In return my Web Analytics gives me dashboards that can be line charts, bar plots, path diagrams (including Google Analytics).

  • Some questions for my Web Analytics to answer-
  • Will it count 500 CEOs reading my blog as less significant as 5000 college students. Thats not a problem if I am on a social network or is it?
  • I get 15000 unique viewers every month . How many people is that? Does that mean the same 500 people visited every day. Does it mean every day a different 500 people visited. Yes I know Google Analytics has some kind of pie chart (horrible) split and returning and new users- but HOW MANY PEOPLE DID I reach?
  • What did they do after the read my blog? Where did they go? Google shares Adsense revenue. Can it share data too- lets call it DataSense. Even create a new internet data bureau (like we have credit data bureau for financial data)
  • How can I use the web analytics software to give me a forecast of future traffic ( by a time series plot with an added regressor of number of posts per category type ?)
  • How can I get some ANALYTICS to take a decision from the web analytics- (A Siri for Web Analytics?-  You last posted X days ago. Please consider posting. Please consider delaying posting to a more appropriate time?)
  • Is there more to life for a blogger than views and visitors. Is there some way we can measure satisfaction?
  • Is there a SEO penalty for boasting on blog traffic boasting when meeting another blogger. Is there a SEO incentive for openly sharing your web statistics
  • Can Google Analytics give a big data dump for open data analytics (sigh). Can you use custom JS libraries for making your own dashboard with GA

Screenshot from 2015-10-22 08:01:27

Choices

You wake up every day with a bank balance of 12 hours of productive work time. every night as you go to sleep the balance goes to zero. you wake up every day with a finite energy balance of a few kilojoules willing to be expended. the balance is upto you but it cannot be carried over the next day.

You wake up with choices and you go to sleep with having made the decisions on the choices. You can focus on what lies forward and stay positive.OR . You can swallow the negativity and be swallowed in it’s swamp.

Intelligent men can make bad choices. Choices that you make today will be with you in the future.

repair the world

The phrase “Tikkun Olam” literally means “world repair.” It is commonly used to refer to the pursuit of social action and social justice.  The concept behind it originate in kabbalah, in the teachings of the 16th century mystic Isaac Luria.

One of the ways you can repair the world is by building good products to replace the ones that are currently in it.  Facebook replaced Orkut and MySpace and its easy user interface and user experience enabled grandmoms to see the pictures of their children from a distance, for old friends to reunite, for global entrepreneurship to be energized. Google Search repaired the world, by pioneering Online Advertising, by  investing in free video sharing, by investing and mentoring and inspiring numerous startups and of course with their free and open source Android system that lowered the cost of smartphone ownership.

Hollywood tries to repair the world by repeatedly portraying certain values. With an immense audience, Hollywood has great soft power beyond glamor or superficial entertainment. Recently it has started portraying repair the world science and global cooperation rather than American dominance.

The Walk is a great movie. Robin (Joseph Gordon-Levitt) takes in help from his accomplices  Gandhi (Ben Kingsley), a George Carlin lookalike, a mathematician (yay!!), a pretty french girl fulfills his childhood dream. He audaciously walks the tallest buildings in the world , the Twin Towers at the World Trade Center. Based on a real-life story, The Walk shows an inspired French Man was instrumental in the World Trade Center’s birth as the center of the world’s greatest city.  The movie is more of a poem of the architecture of the buildings too and it’s haunting to think why the actions of 19 angry men led by a stupid ideology of hatred and destruction killed that dream. I have seen the city of New York once in 2009 but it was between trade towers at the moment. Bin Laden wanted to repair the Muslim World’s dominance of corrupted dynasties supported by the West. The West reacted by unleashing war, and then squandering the entire goodwill by extending it to Iraq, a mess that endures today. In spite of everything,  people are truly good at heart -that is a concept that only an idealistic teenager can have.

To repair the world, you need to assist the ones trying to heal it and you need to thwart the ones trying to steal it. Repairing the world means stopping the ones sapping the climate, the environment and the planet of precious resources.

No force can stop an idea whose time can come, but as time changes the ideas to repair the world change. What have you done to repair the world today?

How Jupyter Ipython threaten the dominance of RStudio for data science developers

RStudio is the clear market leader in IDE used by developers for R data science.

R is the clear market leader for data science.

Python can do with more wrappers for R like packages.

But Jupyter is awesome (once you get it working!)

Screenshot from 2015-10-20 14:29:53

Hopefully, multi core stuff and cloud hosted stuff should be easy too. Google Cloud Data Labs with hosted Jupyter is just the first step. see https://cloud.google.com/datalab/

One of the best things I like about Jupyter OVER RStudio’s interface is the ability to divide code blocks in cells. In addition the ability to install new packages from with RSTUDIO really helps me over the Jupyter. The syntax prompt in latest version of RSTUDIO is something I wish JUPYTER really worrks on.

Can we have a RSTUDIO like interface to working with Python. Yes Yhat made one and called it RODEO. This is because the interface is based on the ACE editor ( yes esseentially RStdudio the company married ACE Editor to Hadley Wickham to get RSTUDIO the product 😉  . Shiny was wonderful but for scalable data science Python and Java help me just as much as R does for BIg DATA ANALYSIS) Scalability is the key here! Rpubs isnt as popular as NBviewer is and now we can wrap markdown within a Jupyter notebook

Screenshot from 2015-10-20 14:28:03

Screenshot from 2015-10-20 14:27:36

Screenshot from 2015-10-15 18:37:41

can Jupyter help in my data science work more than RStudio? These are early days but I prefer a cross platform cross language ( Julia, Python and R) solution anyday. Provided it works just as seamlessly than the established market leader RStudio.

BIG DATA ANALYTICS is where I clearly see JUPYTER help data scientists more than RStdudio as you can use the IRKERNEL. I am especially hoping to see the Spark Kernel , JS Kernel  https://www.npmjs.com/package/ijavascript  and others be  more production ready for business enterprises.

https://github.com/ibm-et/spark-kernel

A version of the Spark Kernel is deployed as part of the Try Jupyter! site. Select Scala 2.10.4 (Spark 1.4.1) under the New dropdown. Note that this version only supports Scala.

https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages

Python/Jupyter kernels:

The Kernel Zero, is of course IPython, which you can get though ipykernel, and still comes (for now) as a dependency of jupyter. The IPython kernel can be thought as a reference implementation, here are other available kernels:

Name Link Jupyter/IPython Version Language(s) Version 3rd party dependencies
ICSharp https://github.com/zabirauf/icsharp Jupyter 4.0 C# 4.0+ scriptcs
IRKernel http://irkernel.github.io/ IPython 3.0 R 3.2 rzmq
SageMath http://www.sagemath.org/ IPython 3.2 Any

Screenshot 2015-10-20 14.07.08 (1)

Screenshot 2015-10-17 18.10.10

 

Screenshot 2015-10-20 11.17.52

The God of Big Data

An Omniscient God should be someone who can analyze all data all the time in any form. Think of a twitter created by the alpha waves of the thoughts of your brain. God is following you. Like Tom of MySpace , HE is friends to all of us. Or thats what the definition of GOD is supposed to be

super small sensitive sensors generate data that are analyzed by algorithms that present results for decision making by angels. sensors embedded in your brain your mobile your watch your laptop.

What do humans do with this data? Do they give you a personalized learning program on what to read today. Do they give you a personalized exericse and food program on how to stay healthy today? or do they just ask you to click click click, so you can maybe convert a bt, and then buy buy buy.

Buying generates debt and economic activity and puts pressure on the buyer to keep working harder so he can come to play the buying game in conspicuous consumption. Does the product you buy list your carbon impact. No that will lower sales. Does it give you a rush of chemicals approximating happiness. Of course retail therapy is the best psycho therapy.

A merciful loving God that has access to all data would intervene not only yo keep us buying and working, but to keep us informed of choices we make and choices we disdain. On how to be better educated, better informed, better health and better impact on government and enironment.

Google failed us as the God of all Data. Alphabet is just a primal soup right now.

The God of Big Data? Now accepting applications.