Author: ajayohri

Turning the Internet into a Super Computer (Uodated)

If you are a fan of distributed computing or parallel computation- you may notice a strange thing

All parallel computation basically involves tieing a lot of desktops and servers together- which is very profitable for the companies that make them.

However the biggest and most idle source of processing power is the Internet- millions of web servers.


So is it possible to create a parallel computer which ties in the web servers. Imagine if we take the R Web package , add it to R- Hadoop Streaming package AND SNOW Phase 3 and create a wordpress plugin for the web.

Eye%20of%20the%20Storm
Would it work? Could it work? or is this a craxy thought

TO add some more fun- add in bit torrenting BUT on web servers through the wordpress or a javascript plugin. OR a cloud operating system ( call it K.U.S.H)

Are you having fun yet?

So if we have this resource, then it basically helps reduce the digital divide a bit.

Think about it——————————————————————————–
Ajay- Note I am working on this project in my spare time and I call it K.U.S.H

Kinematically User Shared Hosts ( KUSH is also my son). Thus if you pursue it, prior ..*.art is claimed.

pps I was kidding on the prior *art , some people didnt get the *.

3 Billion Asian and Africans stay without a computer because someone has a patent to each and everything that goes with it.

Art Credit- “http://www.sgeier.net/fractals/fractals/02/Eye%20of%20the%20Storm.jpg”

Interview Timo Elliott SAP

Here is an interview with Timo Elliott, Senior Product Director SAP Business Objects.

Ajay- Describe your career in science from school to Senior Director in SAP to blogger/speaker. How do you think we can convince students of the benefits of learning science and maths.

Timo- I studied economics with statistics in the UK, but I had always been a closet geek and had dabbled with computers ever since I was a kid, starting with Z80 assembler code. I started my career doing low-level computer consulting in Hong Kong, and worked on a series of basic business intelligence projects at Shell in New Zealand, cobbling together a solution based on a mainframe HR system, floppy-disk transfers, and Lotus 1-2-3 macros. When I returned to Europe, I stumbled across a small French startup that provided exactly the “decision support systems” that I had been looking for, and enthusiastically joined the company.

Over the last eighteen years, I’ve worked with hundreds of companies around the world on their BI strategy and my job today is to help evangelize what works and what doesn’t, to help organizations avoid the mistakes that others have made.

When it comes to BI initiatives, I see the results of one fundamental problem almost on a daily basis: 75% of project success depends on people, process, organization, culture, and leadership, but we typically spend 92% of our time on data and technology.

BI is NOT about technology – it’s about helping people do their jobs. So when it comes to education, we need to teach our technologists more about people, not science!

Ajay- You were the 8th employee of SAP Business Objects. What are the key turning points or transition stages in the BI industry that you remember seeing in the past 18 years, and how has SAP Business objects responded to them.

Timo- Executive information systems and multidimensional databases have been around since at least the 1970s, but modern business intelligence dates from the early 1990s, driven by the widespread use of relational databases, graphical user interfaces, and the invention of the “semantic layer”, pioneered by BusinessObjects, that separated business terms from technical logic. For the first time, non-expert business people had self-service access to data.

This was followed by a period of rapid expansion, as leading vendors combined reporting, multidimensional, and dashboard approaches into fully-fledged suites. During this period, BusinessObjects acquired a series of related technology companies to complete the existing offer (such as the leader in operational reporting, Crystal Reports) and extend into enterprise information management and financial performance management.

Finally, the theme of the last few years has clearly been consolidation – according to Gartner, the top four “megavendors” (SAP, IBM, Microsoft, and Oracle) now make up almost two-thirds of the market, and accounted for fully 83% of the growth since last year. Perhaps as a result, user deployments are accelerating, with usage growth rates doubling last year.

Ajay- How do you think Business Intelligence would be affected by the following

a) Predictive Analytics.

Timo- Predictive analytics has been the “next big thing in BI” for at least a decade. It has been extremely important in some key areas, such as fraud detection, but the dream of “no longer managing by looking out of the rear-view mirror” has proved hard to achieve, notably because business conditions are forever changing.

We offer predictive analytics with our Predictive Workbench product – but I think the real opportunity for this technology in the future is “power analytics”, rather than “prediction”. For example, helping business people automatically cluster similar values, spot outliers, determine causal factors, and detect trend inflection points, using the data that they already have access to with traditional BI.

b) Cloud Computing.

Timo- In terms of architecture, it’s clearly not about on-demand OR on-premise: it’s about having a flexible approach that combines both approaches. You can compare information to money: today, we tend to keep our money in the bank rather than under our own mattress, because it’s safer, more convenient, and more cost-efficient. At the same time, there are situations where the convenience of cash is still essential.

Companies should be able to choose a BI strategy, and decide how to deploy it later. This is what we offer with our BI on-demand solutions, which use the same technology as on-premise. You can start to build on-premise and move it to on-demand, or vice-versa, or have a mix of both.

In terms of data, “cloud intelligence” is still a work in progress. As with modern financial instruments, we can expect to see the growth of new information services, such as our “information on-demand” product that provide data feeds from Reuters, Thompson Financial, and other providers to augment internal information systems. Looking further into the future, we can imagine new information marketplaces that would pay us “interest” to store our data in the cloud, where it can be adapted, aggregated and sold to others.

c) Social Media.

Timo- Conversations and collaboration are an essential part of effective business intelligence. We often talk about the notion of a “single view of the truth” in this industry, but that’s like saying we can have “a single view of politics” – while it’s vital to try to give everybody access to the same data, there will always be plenty of room for interpretation and discussion. BI platforms need to support this collaborative decision-making.

In particular, there are many, many studies that show up our all-too-human limitations when it comes to analyzing data. For example, did you know that children with bigger feet have better handwriting?

It’s absolutely true — because the children are older! Mixing up correlation and causality is a common issue in business intelligence, and one answer to the problem is to add more people: the more reviewers there are of the decision-making process, the better the decisions will be.

Analysis is also critical to the development of social media, such as analyzing sentiment trends in Twitter — a functionality we offer with SAP CRM — or tracking social communities. For example, Jive, the leader in Enterprise 2.0 platforms, offers our BI products as part of their solution, to help their customers analyze and optimize use of the system. Administrators can track if usage is trailing off in a particular department, for example.

d) Social Network Analysis.

Timo- Over the last twenty years, partly as a result of extensive automation of operational tasks with systems such as SAP, there’s has been a huge shift from “routine” to “non-routine” work. Today, fully 90% of business users say that their work involves decision making, problem solving, and the creation of new analysis and insight.

To help support this new creativity, organizations are becoming more porous as we work closer with our ecosystem of customers, partners, and suppliers, and we work in ever-more matrixed environments and cross-functional teams.

We’ve developed a Social Network Analyzer prototype that combines BI and social networking to create a “single view of relationships”. It can gather information from multiple different systems, such as HR, CRM, email distribution lists, project teams, Twitter, etc., to create a multi-layered view of how people are connected, across and beyond the enterprise. For more information, see the SAP Web 2.0 blog post, and you can try it yourself on our ondemand.com web site.

Ajay- What is the area that SAP BusinessObjects is very good at (strength). What are the key areas that you are currently seeking to improve ( opportunities)

Timo- Companies evaluating BI solutions should look at four things: product functionality for their users’ needs, fit with the overall IT architecture, the vendor’s reputation and ecosystem, and (of course) price. SAP BusinessObjects is the clear leader in the BI industry, and I’d say that SAP BusinessObjects has the best overall solution if you’re a large organization (or looking to become one) with a variety of user needs, multiple data sources, and a heterogeneous IT infrastructure.

In terms of opportunities, we have high expectations for new interfaces for casual users, and in-memory processing, which we have combined in our SAP BusinessObjects Explorer product. Initial customer feedback has been excellent, with quotes such as “finding information is as easy as using the internet” and “if you can use a computer, you can use Explorer”.

In terms of future directions, we’re taking a very transparent, Web 2.0 approach. The SAP Business Objects innovation center is modeled on Google Labs and we share our prototypes (including the Social Network Analyzer mentioned above) with anybody who’s interested, and let our customers give us early feedback on what directions we should go.

Ajay- What does Timo Elliott do for work life balance when not writing, talking, and evangelizing about Business Intelligence?

Timo- I’m a keen amateur photographer – see www.timoelliott.com/personal for more!

Biography- http://timoelliott.com/blog/about

Timo Elliott is Senior Director of Strategic Marketing for SAP BusinessObjects. For the last twenty years he has been a thought leader and conference speaker in business intelligence and performance management.

A popular and engaging speaker, Elliott presents regularly to IT and business audiences at international conferences, drawing on his experience working with enterprise customers around the globe. Topics include the latest developments in BI/PM technology, how best to suceed with BI/PM projects, and future trends in the industry. 

Prior to Business Objects, Elliott was a computer consultant in Hong Kong and led analytics projects for Shell in New Zealand. He holds a first-class honors degree in Economics with Statistics from Bristol University, England.

Additional websites: http://www.sapweb20.com —  web 2.0 technology by, with, and at SAP

Email: telliott@timoelliott.com or timo.elliott@sap.com

LinkedIn: http://www.linkedin.com/in/timoelliott

Twitter: http://twitter.com/timoelliott

Flickr: http://www.flickr.com/photos/timoelliott/

Facebook: http://www.facebook.com/people/Timo-Elliott/544744135

For an earlier interview with Oracle Data Mining Product Management, Charlie Berger see https://decisionstats.wordpress.com/2009/09/02/oracle/

Adobe gulps Omniture

Another analytics takeover. Adobe needing to do something exciting and cash generating made a smart play with a 50 % premium for Omniture- with the amount of web traffic that adobe is embedded into (from documents ,graphics and videos especially) Adding in analytics can only mean better growth prospects for both given the pressure they are likely to face soon from competing products ( MS Silverlight and Yahoo Index Tools, Google Analytics respectively).

From the Press Release (note the cute diagram)

Adobe to Acquire Omniture

On Sept. 15, 2009, Adobe Systems Incorporated (Nasdaq:ADBE) and Omniture, Inc.                 (Nasdaq:OMTR) announced the two companies have entered into a definitive agreement                               for Adobe to acquire Omniture in a transaction valued at approximately $1.8 billion on a                          fully diluted equity-value basis. Under the terms of the agreement, Adobe will commence                                a tender offer to acquire all of the outstanding common stock of Omniture for $21.50   

per share in cash.

Adobe’s acquisition of Omniture furthers its mission to revolutionize the way the world engages             with ideas and information. By combining Adobe’s content creation tools and ubiquitous                     clients with Omniture’s Web analytics, measurement and optimization technologies, Adobe will be well positioned to deliver solutions that can transform the future of engaging experiences and e-commerce across all   digital content, platforms and devices.

Adobe and Omniture

The combination of the two companies will increase the value Adobe delivers to customers.                     For designers, developers and online marketers, an integrated workflow—with optimization   capabilities embedded in the creation tools—will streamline the creation and delivery of                relevant content and applications. This optimization will enable advertisers and advertising       agencies, publishers,  and e-tailers to realize greater ROI from their digital media investments                  and improve their end users’ experience

And the official fact sheet

ADOBE

  1. FOUNDED: 1982
  2. PRESIDENT & CEO: Shantanu Narayen
  3. MARKET CAP: $18.19 billion (as of 9/11/09)
  4. FY 08 REVENUE: US $3.58 billion (FYE Nov. 28, 2008)

Omniture

  1. CO-FOUNDER & CEO: Josh James
  2. FOUNDED: 1996
  3. MARKET CAP: $1.29 billion (as of 9/11/09)
  4. FY 08 REVENUE: US $295.6 million (FYE Dec. 31, 2008)

From-

http://www.adobe.com/aboutadobe/invrelations/adobeandomniture.html

The Return of PAWS

Bigger Wins, Broader Capabilities at Predictive Analytics World

PAW’s October 20-21 program includes 25 sessions divulging how more than 20 brand name companies across a dozen industry verticals apply predictive analytics to over 10 business uses by employing the hottest advanced analytical methods. See the entire conference agenda at a glance, or check out an agenda overview.

Workshop Benefits

Sign up for one of PAW’s leading-edge Workshops and receive these great benefits:

  • PAW’s hands-on workshop, lead by Dean Abbott, has chosen its tool: SAS Enterprise Miner. So attend Hands-On Predictive Analytics for direct experience applying the industry best practices process, and ramp up on this leading, industry-leading tool to boot.
  • A free copy of Dr. John Elder’s just released Handbook of Statistical Analysis and Data Mining Applications book is yours if you attend John’s Predictive Modeling Methods and Common Data Mining Mistakes workshop.
  • A free copy of James Taylor’s book Smart Enough Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions is yours if you attend James’s Putting Predictive Analytics to Work workshop.

Don’t miss the free BetterManagement webinar Sept 23, featuring conference chair Eric Siegel, on Driving Decisions with Predictive Analytics: The Top Business Applications.

Register Now

For inquiries, call (866) 223-2579 or e-mail registration@predictiveanalyticsworld.com.

Sign up for the PAW group on LinkedIn.

Predictive Analytics World
October 20-21 – Washington DC
Location |   Contact Us

Disclaimer- As a student and also I am a blog partner to PAWS and get access to interviews, books, discounts but not financial renumeration for these.

Interview Professor John Fox Creator R Commander

Here is an interview with Prof John Fox, creator of the very popular R language based GUI, RCmdr.

Ajay- Describe your career in science from your high school days to the science books you have written. What do you think can be done to increase interest in science in young people.

John Fox- I’m a sociologist and social statistician, so I don’t have a career in science, as that term is generally understood. I was interested in science as a child, however: I attended a science high school in New York City (Brooklyn Tech), and when I began university in 1964 at New York’s City College, I started in engineering. I moved subsequently through majors in philosophy and psychology, before finishing in sociology — had I not graduated in 1968 I probably would have moved on to something else. I took a statistics course during my last year as an undergraduate and found it fascinating. I enrolled in the sociology graduate program at the University of Michigan, where I specialized in social psychology and demography, and finished with a PhD in 1972 when I was 24 years old. I became interested in computers during my first year in graduate school, where I initially learned to program in Fortran. I also took quite a few courses in statistics and math.

I haven’t written any science books, but I have written and edited a number of books on social statistics, including, most recently, Applied Regression Analysis and Generalized Linear Models, Second Edition (Sage, 2008).

I’m afraid that I don’t know how to interest young people in science. Science seemed intrinsically interesting to me when I was young, and still does.

Ajay- What prompted you to R Commander. How would you describe R Commander as a tool, say for a user of other languages and who want to learn R, but get afraid of the syntax.

John- I originally programmed the R Commander so that I could use R to teach introductory statistics courses to sociology undergraduates. I previously taught this course with Minitab or SPSS, which were programs that I never used for my own work. I waited for someone to come up with a simple, portable, easily installed point-and-click interface to R, but nothing appeared on the horizon, and so I decided to give it a try myself.

I suppose that the R Commander can ease users into writing commands, inasmuch as the commands are displayed, but I suspect that most users don’t look at them. I think that serious prospective users of R should be encouraged to use the command-line interface along with a script editor of some sort. I wouldn’t exaggerate the difficulty of learning R: I came to R — actually S then — after having programmed in perhaps a dozen other languages, most recently at that point Lisp, and found the S language particularly easy to pick up.

Ajay- I particularly like the R Cmdr plugins. Is it possible for anyone to increase R Commander with a customized package- plugin.

John- That’s the basic idea, though the plug-in author has to be able to program in R and must learn a little Tcl/Tk.

Ajay- Have you thought of using the R Commander GUI on an Amazon EC2 and thus making R high performance computing say available on demand ( similar to Zementis model deployment using Amazon Ec2). What are you views on the future of statistical computing

John- I’m not sure whether or how an interface like the Rcmdr, which is Tcl/Tk-based, can be adapted to cloud computing. I also don’t feel qualified to predict the future of statistical computing.

I think that R is where the action is for the near future.

Ajay-What are the best ways for using R Commander as a teaching tool ( I noticed the help is a bit outdated).

John- Is the help a bit outdated? My intention is that the R Commander should be largely self-explanatory. Most people know how to use point-and-click interfaces. In the basic courses for which it is principally designed, my goals are to teach the essential ideas of statistical reasoning and some skills in data analysis. In this kind of course, statistical software should facilitate the basic goals of the course.

As I said, for serious data analysis, I believe that it’s a good idea to encourage use of the command-line interface.

Ajay- What are your views on R being recognized by SAS Institute for it’s IML product. Do you think there can be a middle way for open source and proprietary software to exist.

John- I imagine that R is a challenge for producers of proprietary software like SAS, partly because R development moves more quickly, but also because R is giving away something that SAS and other vendors of proprietary statistical software are selling. For example, I once used SAS quite a bit but don’t anymore. I also have the sense that for some time SAS has directed its energies more toward business uses of its software than toward purely statistical applications.

Ajay- Do people in R Core team recognize the importance of GUI? What does the rest of R community feel? What has the feedback of users ben to you. Any plans to corporate sponsors for R Commander ( Rattle , an R language data mining GUI has a version called Rstat at http://www.informationbuilders.com/products/webfocus/predictivemodeling.html while the free version and code is at rattle.togaware.com)

John- I feel that the R Commander GUI has been generally positively received, both by members of R Core who have said something about it to me and by others in the R community. Of course, a nice feature of the R package system is that people can simply ignore packages in which they have no interest. I noticed recently that a Journal of Statistical Software paper that I wrote several years ago on the Rcmdr package has been downloaded nearly 35,000 times.

Because I wouldn’t expect many students using the Rcmdr package in a course to read that paper, I expect that the package is being used fairly widely.

Ajay- What does John Fox do for fun or as a hobby?

John- I’m tempted to say that much of my work is fun — particularly doing research, writing programs, and writing papers and books. I used to be quite a serious photographer, but I haven’t done that in years, and the technology of photography has changed a great deal. I run and swim for exercise, but that’s not really fun. I like to read and to travel, but who doesn’t?

Biography-

Prof John Fox is a giant in his chosen fields and has edited/authored 13 books and written chapters for 12 more books. He has also written and been published in almost 49 Journal articles. He is also editor in chief for R News newsletter. You can read more about Dr Fox at http://socserv.mcmaster.ca/jfox/

On R Cmdr-

R Cmdr has substantially decreased the hygiene factor for people wanting to learn R- they begin with the GUI and then later transition to customization using command line. It is so simple in its design that even under graduates have started basic data analysis with R Cmdr after just a class.You can read more on it here at http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/Getting-Started-with-the-Rcmdr.pdf

Backedup or Hacked Up

Decisionstats.com was undergoing some hacking attacks this past two weeks.

Backing up WordPress Blog-

  • use export feature in wordpress to create files.
  • If your number of articles is more than 70, then create multiple usernames, export using bulk apply to seperate wordpress xml files.
  • then in wordpress.com site, import the various xml files ( note multiple file method avoid corruption and using bulk apply -change author – makes it very fast)

Hackedup

  1. once your wordpress.com blog is updated, use a wild card redirect so as to preserve your search engine traffic. Your Backup is now online even if your original site is hacked
  2. Use the server to access your .htaccess file to check whether rouge redirects happened.
  3. Use logs of server access ( painful but true) to pinpoint IP addresses of hack attacks ( note hackers WOULD use relay servers to disguise IP addresses)
  4. To prevent domain name hijacking, make sure your who.is information is private
  5. Change your email passwords, security questions, server passwords. Use random password generators to create secure passwords.
  6. To prevent rogue malware from infecting your laptop create a dual boot Ubuntu/Windows laptop using a 10 minute tutorial. Use the Ubuntu Linux boot to do all the above operations.
  7. Inform the Federal authorities in cyber crime division with the server logs and a SPECIFIC complaint ( no rambling sob stories)
  8. Pray to God, Matt ( both Cutts and Mullenweg), and if all above steps fail ask Donncha O Caoimh at http://ocaoimh.ie/about/ to step in. 

Switching to WordPress.com

Some of you may have noticed that the domain name http://www.decisionstats.com now redirects to http://www.decisionstats.wordpress.com

Note ALL Content has been moved- some images may be not showing up but that would be transitioned shortly.

This is done because of the following-

1) Get more traffic from http://www.wordpress.com blogs thanks to the related blogs/posts feature.

2) I no longer have time and money to keep fiddling with my self hosted blog. Let Matt Muelberg worry about it.

3) It is more secure ( touch wood and cross fingers) to have a wordpress.com than a self hosted wordpress site. This is true due to recnt hacking attacks on wordpress sites.

4) I plan to launch a new site http://www.dudeofdata.com which is aimed at making analytics and BI fun for undergraduate and high school students.

Decisionstats now has 600 RSS readers, 170 Facebook fans, 880 Linkedin members , 1100 Twitter followers and more than 8000 page views /5-6000 unique visitors a month WITHOUT any advertising support or vendor affiliation.

More importantly it’s interviews are read and re read by many people. (My inspiration for interviews was Larry King -the longest serving media journalist)

Thanks for your support ( even the occasional fellow techie brother spammer/hackers) and I hope to add more features shortly (like Videos- Tutorials etc)

5780_253683715470_575840470_8308099_6809171_s

Ajay Ohri