As you can see, my poem on Michael Jackson got more page views
than any code I wrote or any interview I did. which reminds me to be
more humble than be humbled :)) More Details are on Advertise page above.
As you can see, my poem on Michael Jackson got more page views
than any code I wrote or any interview I did. which reminds me to be
more humble than be humbled :)) More Details are on Advertise page above.
Here is a nice video from Cloudera on a HIVE tutorial. I wonder what would happen if they put a real analytical system and not just basic analytics and reporting … like R or SPSS or JMP or SAS on big database system like Hadoop (including some text mined data from legacy company documents)
Unlike Oracle or other data base systems, Hadoop is free now and in reasonable future (like MySQL used to be before acquired by big fish Sun acquired by bigger Oracle).
Citation-
http://wiki.apache.org/hadoop/Hive
Hive is a data warehouse infrastructure built on top of Hadoop that provides tools to enable easy data summarization, adhoc querying and analysis of large datasets data stored in Hadoop files
Hive is based on Hadoop which is a batch processing system. Accordingly, this system does not and cannot promise low latencies on queries. The paradigm here is strictly of submitting jobs and being notified when the jobs are completed as opposed to real time queries. As a result it should not be compared with systems like Oracle where analysis is done on a significantly smaller amount of data but the analysis proceeds much more iteratively with the response times between iterations being less than a few minutes. For Hive queries response times for even the smallest jobs can be of the order of 5-10 minutes and for larger jobs this may even run into hours.
If your input data is small you can execute a query in a short time. For example, if a table has 100 rows you can ‘set mapred.reduce.tasks=1’ and ‘set mapred.map.tasks=1’ and the query time will be ~15 seconds.
The business intelligence -business analytics- data mining industry ( or as James Taylor would say Decision Management Industry) have some reactions on IBM – SPSS ( which was NOT a surprise to many including me). Really.
From SAS Institute, Anne Milley
http://blogs.sas.com/sascom/index.php?/archives/557-Analytics-is-still-our-middle-name.html
Besides SAS, SPSS was one of the last independent analytic software companies. A colleague says, “It’s the end of the analytics cold war.”
I’ve been saying all along that analytics is required for success. Yes, data integration, data quality, and query & reporting are important too but, as W. Edwards Deming says, “The object of taking data is to provide a basis for action.”
The end of the analytics cold war- hmm. We all know what the end of real cold war brought us- Google, Cloud Computing, and other non technical issues.
From KXEN, Roger Hadaad
“The price paid for SPSS of four times revenues and 25 times earnings shows just how valuable this sector really is,” says Haddad. “But the deal has also created a tremendous opportunity for the sector’s remaining independent vendors that
KXEN is well placed to capitalize on. “There is no For Sale sign hanging in our window,” continues Haddad. “We launched KXEN in 1998 to democratize the benefits of data mining and predictive analytics, making them practical and affordable across the whole enterprise and not just the exclusive preserve of a few specialists. It’s going to take up to two years for the dust to settle following the IBM
“Former SPSS partners, systems integrators and distributors will face uncertainty.”
I think the PE multiple was still low- SPSS was worth more if you count the client base, active community, brand itself in the valuation. Tremendous cross sell opportunities and IBM with it’s nice research and development is a good supporter of pure science. Yes, next two years would be facing increasing consolidation and more “surprising” news. At 4 times earnings, anyone can be bought in the present market if it is a public listed company. 😉
From the rather subdued voices on SPSS list, some subjective and non quantitative ‘strategic” forecasts.
http://www.listserv.uga.edu/cgi-bin/wa?A2=ind0907&L=spssx-l&F=&S=&P=36324
I think the Ancient Chinese said it best “May you live in interesting times”.
Having worked with some flavors of Cognos and SPSS, I think there could be areas for technical integration for querying and GUI based forecasting as well, apart from financial mergers and administrative re adjustments. I mean people pull data not just to report it, but to estimate what comes next as well.
This could also spell the end of uni platform skilled analysts. You now need to learn atleast two different platforms like SAS,SPSS or KXEN, R or Cognos, Business Objects to hedge your chances of getting offshored (Note- I worked in offshoring for almost 4 years in India in data analytics).
Answering what IBM will do with SPSS and it’s open source commitment to R and consequences for employees, customers, vendors,partners who have more choices now than ever.
…. well it depends. Who is John Galt?
SPSS Inc maker of PASW series of analytics softwares is being bought by IBM ( unless Oracle spikes this deal too). IBM is seeking a play in the rapidly growing analytics market and is also a strategic partner to WPS ( who makes the Base SAS alternative SAS language software).
In a personal note- I just entered University of Tennessee as a statistics student.
Citation-
http://en.oreilly.com/oscon2009/public/schedule/detail/10432
StackOverflow Flash Mob for the R User Community
Moderated by: Michael E. Driscoll
7:00pm Wednesday, 07/22/2009
Location: Ballroom A2In concert with users online across the country, this session will lead a flashmob to populate StackOverflow with R language content.
R, the open source statistical language, has a notoriously steep learning curve. The same technical questions tend be asked repeatedly on the R-help mailing lists, to the detriment of both R experts (who tire of repeating themselves) and the learners (who often receive a technically correct, but terse response).
We have developed a list of the most common 100 technical R questions, based on an analysis of (i) queries sent to the RSeek.org web portal, and (ii) an examination of the R-help list archives, and (iii) a survey of members of R Users Groups in San Francisco, LA, and New York City.
In the first hour, participants will pair up to claim a question, formulate it on StackOverflow, and provide a comprehensive answer. In the second hour, participants will rate, review, and comment on the set of submitted questions and answers.
While Stackoverflow currently lacks content for the R language, we believe this effort will provide the spark to attract more R users, and emerge as a valuable resource to the growing R community.
This is an interesting example of a statistical software community using twitter for a tech help event. I hope this trend/ event gets replicated again and again-
Statisticians worldwide unite in the language of maths !!!
Please follow @rstatsmob to participate. See you at 7 PM PST!
The Europe based Rapid Miner came out with version 4.5 of their data mining tool ( also known as Yale) with a much promising “Script” tool.
Also, Rapid Miner came in 1st in open source data mining tools in a poll by Industry benchmark www.kdnuggets.com
They have a brilliant video here for people who just want to have a look at the new Rapid Miner
http://rapid-i.com/videos/rapidminer_tour_3_4_en.html
Citation-
http://rapid-i.com/content/view/147/1/
If you have more than 100 followers, or people following you on Twitter and want to have some kind of Outlook like manager for managing so much info- this is a great tool from www.refollow.com
Added benefits-
1) Secure Login using Twitter Authorization
2) Visual Click and Easy Follow- Unfollow Blocking based on activities
3) Segmenting groups of people basede on behavior
4) Hidden insights on who all suddenly have un-followed me ( after I accidentally revealed the spoiler end of latest Harry Potter movie- Dumbeldore will sleep with the fishes)
The Screenshot (of my refollow page) below shows you most of the properties-
