Category: Analytics

Introducing Radoop

Thats Right- This is Radoop and it is

Hadoop meats Rapid Miner=Radoop

 

 

http://prezi.com/bin/preziloader.swf

http://prezi.com/dxx7m50le5hr/radoop-presentation-at-rcomm-2011/

 

Radoop presentation at RCOMM 2011 on Prezi

What about Hive and Mahout?

Hive is a data warehouse infrastructure built on top of Hadoop, i.e. it uses the distributed file system of Hadoop and the efficient access technologies. Hive was initially developed by Facebook and is now used and developed by many other companies for their distributed data warehouse.

Mahout is a machine learning library already offering many scalable machine learning libraries implemented as well on top of Hadoop and its map & reduce paradigm. Hence, Mahout is one of the first distributed data analytics framework making use of the power of Hadoop.

You will see below that both frameworks will be tightly integrated with RapidMiner.

What can RapidMiner bring into the game?

Hadoop is great for large scale analytics, but it lacks an easy-to-use graphical interface. RapidMiner is an excellent tool for data analytics, but unless the analyst is not performing some nasty tricks, the data size is limited by the memory available. So we have the algorithms, the support for analytical process design, the user interface, and of course the community with a demand for large-scale analytics.

RapidMiner + Hadoop = Radoop

Radoop combines the strengths of RapidMiner and Hadoop. The result is a RapidMiner extension for editing and running ETL, data analytics and machine learning processes over Hadoop. The developers have closely integrated the highly optimized data analytics capabilities of Hive and Mahout, and the user-friendly interface of RapidMiner to form a powerful and easy-to-use data analytics solution for Hadoop.

Source-https://rapid-i.com/component/option,com_myblog/show,Big-data-analytics-made-easy-Radoop.html/Itemid,172/

 

and http://blog.radoop.eu/

Interesting? Sign up here- http://radoop.eu/z1sxe

Cloud Computing with #Rstats and CloudNumbers.com

Some of you know that I am due to finish “R for Business Analytics” for Springer by Dec 2011 and “R for Cloud Computing” by Dec 2012. Accordingly while I am busy crunching out ” R for Business Analytics” which is a corporate business analyst\s view on using #Rstats, I am gathering material for the cloud computing book too.

I have been waiting for someone like CloudNumbers.com for some time now, and I like their initial pricing structure.  As scale picks up, this should only get better. As a business Intelligence analyst, I wonder if they can help set up a dedicated or private cloud too for someone who wants a data mart solution to be done.The best thing I like about this- they have a referral scheme so if someone you know wants to test it out, well it gives you some freebies too in the form of an invitation code.

 

 

 

 

  • I read the instructions

  • I reviewed the pricing plan and click back to the dashboard 

 

  • I clicked on start new session

  • I click next
  • Choosing R from a very convenient interface design
  • Choosing all the applications I may need
  • This is a really nice feature in enabling to choose packages for R
  • Finally I can choose ONLY 7.5 gb RAM in the free version

I name the session in case I want to start multiple sessions

After waiting 15 minutes, my instance is up and I type R to get the following

Note I can also see the desktop- which is a great improvement over EC2 interface for R Cloud computing on Linux. Also it shuts down on its own if I leave it running (as of now after 180 minutes) so i click shut down session

 

You can click this link to try and get your own cloud in the sky for free -10 hours are free for you

https://my.cloudnumbers.com/register/65E97A

 

CloudNumbers.com – #Rstats gets real in the cloud

I came across Cloudnumbers.com . Awesome name , I didnt know groovy domain names existed anymore.

What is cloudnumbers.com – The website which looks like the salesforce.com website in style and design-

says-

Things are still very raw here- but its an awesome concept. With 68 GB of Memory, I am sure R can blow away everything out of the water.

Probably the competition needs to ahem launch that private cloud soon, before they lose the momentum.

and you Get 2GB of storage, 2GB of traffic and 10h computation cost per month for free! I think this German startup has hit the nail on the head and it would be interesting to see what the future holds.

 

Check out http://cloudnumbers.com/product yourself and/or see the video

https://www.youtube.com/v/0ZNEpR_ElV0?version=3&hl=en_US&hd=1

 

Contribution to #Rstats by Revolution

I have been watching for Revolution Analytics product almost since the inception of the company. It has managed to sail over storms, naysayers and critics with simple and effective strategy of launching good software, making good partnerships and keeping up media visibility with white papers, joint webinars, blogs, conferences and events.

However this is a listing of all technical contributions made by Revolution Analytics products to the #rstats project.

1) Useful Packages mostly in parallel processing or more efficient computing like

 

2) RevoScaler package to beat R’s memory problem (this is probably the best in my opinion as it is yet to be replicated by the open source version and is a clear cut reason for going in for the paid version)

http://www.revolutionanalytics.com/products/enterprise-big-data.php

  • Efficient XDF File Format designed to efficiently handle huge data sets.
  • Data Step Functionality to quickly clean, transform, explore, and visualize huge data sets.
  • Data selection functionality to store huge data sets out of memory, and select subsets of rows and columns for in-memory operation with all R functions.
  • Visualize Large Data sets with line plots and histograms.
  • Built-in Statistical Algorithms for direct analysis of huge data sets:
    • Summary Statistics
    • Linear Regression
    • Logistic Regression
    • Crosstabulation
  • On-the-fly data transformations to include derived variables in models without writing new data files.
  • Extend Existing Analyses by writing user- defined R functions to “chunk” through huge data sets.
  • Direct import of fixed-format text data files and SAS data sets into .xdf format

 

3) RevoDeploy R for  API based R solution – I somehow think this feature will get more important as time goes on but it seems a lower visibility offering right now.

http://www.revolutionanalytics.com/products/enterprise-deployment.php

  • Collection of Web services implemented as a RESTful API.
  • JavaScript and Java client libraries, allowing users to easily build custom Web applications on top of R.
  • .NET Client library — includes a COM interoperability to call R from VBA
  • Management Console for securely administrating servers, scripts and users through HTTP and HTTPS.
  • XML and JSON format for data exchange.
  • Built-in security model for authenticated or anonymous invocation of R Scripts.
  • Repository for storing R objects and R Script execution artifacts.

 

4) Revolutions IDE (or Productivity Environment) for a faster coding environment than command line. The GUI by Revolution Analytics is in the works. – Having used this- only the Code Snippets function is a clear differentiator from newer IDE and GUI. The code snippets is awesome though and even someone who doesnt know much R can get analysis set up quite fast and accurately.

http://www.revolutionanalytics.com/products/enterprise-productivity.php

  • Full-featured Visual Debugger for debugging R scripts, with call stack window and step-in, step-over, and step-out capability.
  • Enhanced Script Editor with hover-over help, word completion, find-across-files capability, automatic syntax checking, bookmarks, and navigation buttons.
  • Run Selection, Run to Line and Run to Cursor evaluation
  • R Code Snippets to automatically generate fill-in-the-blank sections of R code with tooltip help.
  • Object Browser showing available data and function objects (including those in packages), with context menus for plotting and editing data.
  • Solution Explorer for organizing, viewing, adding, removing, rearranging, and sourcing R scripts.
  • Customizable Workspace with dockable, floating, and tabbed tool windows.
  • Version Control Plug-in available for the open source Subversion version control software.

 

Marketing contributions from Revolution Analytics-

1) Sponsoring R sessions and user meets

2) Evangelizing R at conferences  and partnering with corporate partners including JasperSoft, Microsoft , IBM and others at http://www.revolutionanalytics.com/partners/

3) Helping with online initiatives like http://www.inside-r.org/ (which is curiously dormant and now largely superseded by R-Bloggers.com) and the syntax highlighting tool at http://www.inside-r.org/pretty-r. In addition Revolution has been proactive in reaching out to the community

4) Helping pioneer blogging about R and Twitter Hash tag discussions , and contributing to Stack Overflow discussions. Within a short while, #rstats online community has overtaken a lot more established names- partly due to decentralized nature of its working.

 

Did I miss something out? yes , they share their code by GPL.

 

Let me know by feedback

Analytics 2011 Conference

From http://www.sas.com/events/analytics/us/

The Analytics 2011 Conference Series combines the power of SAS’s M2010 Data Mining Conference and F2010 Business Forecasting Conference into one conference covering the latest trends and techniques in the field of analytics. Analytics 2011 Conference Series brings the brightest minds in the field of analytics together with hundreds of analytics practitioners. Join us as these leading conferences change names and locations. At Analytics 2011, you’ll learn through a series of case studies, technical presentations and hands-on training. If you are in the field of analytics, this is one conference you can’t afford to miss.

Conference Details

October 24-25, 2011
Grande Lakes Resort
Orlando, FL

Analytics 2011 topic areas include:

Workflows and MyExperiment.org

Here is a great website for sharing workflows – it is called MyExperiment.org and it can also include Work flows from many software.

myExperiment currently has 4742 members, 270 groups, 1842 workflows, 423 files and 173 packs

Could it also include workflow from Red-R from #rstats or Enterprise Miner

Continue reading “Workflows and MyExperiment.org”

Machine Learning Contest

New Contest at http://www.ecmlpkdd2011.org/dcOverview.php

 

 

Discovery Challenge Overview

Organization | Overview | Task and Datasets | Timeline

 

General description: tasks and dataset

VideoLectures.net is a free and open access multimedia repository of video lectures, mainly of research and educational character. The lectures are given by distinguished scholars and scientists at the most important and prominent events like conferences, summer schools, workshops and science promotional events from many fields of Science. The portal is aimed at promoting science, exchanging ideas and fostering knowledge sharing by providing high quality didactic contents not only to the scientific community but also to the general public. All lectures, accompanying documents, information and links are systematically selected and classified through the editorial process taking into account also users’ comments.

The ECML-PKDD 2011 Discovery Challenge is organized in order to improve the website’s current recommender system. The challenge consists of two main tasks and a “side-by” contest. The provided data is for both of the tasks, and it is up to the contestants how it will be used for learning (building up) a recommender.

Due to the nature of the problem, each of the tasks has its own merit: task 1 simulates new-user and new- item recommendation (cold-start mode), task 2 simulates clickstream based recommendation (normal mode). Continue reading “Machine Learning Contest”