Category: Analytics

Here is a nice SAS to Java compiler. It basically cuts away at the problem of executing legacy SAS code, SAS training and focusses on executing the tasks in Java thus making them much faster.

It’s available at http://dullesopen.com/

And its free for personal use.And academic use.

I quote from the website "

Carolina Benefits

Converting Base SAS® to Java with Carolina provides two main benefits to enterprises:

• Savings on license fees. Carolina costs about 70% less than SAS.
• Performance gains. Carolina-converted code runs significantly faster than the native SAS program.

Additional Benefits

• Greater flexibility. Java is an industry-standard environment that runs on all platforms. It is much easier to support than the legacy SAS environment it replaces.
• Better integration. Carolina, as a Java application, supports web services through true J2EE integration.
• Flawless automated conversion. Eliminate time-consuming, error-prone manual conversion.
• Simpler contracts. Carolina is licensed in a simple, straightforward fashion.
• Reduced training costs. Carolina-converted programs can be understood by analysts without training in SAS, and SAS-trained analysts don’t need to learn a new programming language."

My presentation at the Timesgroup office went well. I have uploaded a copy on to the Google Docs and it can be viewed here http://docs.google.com/Presentation?docid=dcvss358_324cfhpg7cv&hl=en

Some questions that came out of the discussion were –

 

1) Businesses are aligned in types of products that can be sold ? 

What are the 5 top segments of customers ( irrespective of product) that come to your website

2) Cross Sell- What are the top 10 products ( irrespective of whether you own them or not) that can be sold to your customer database

3) Who are the top dis satisfied customers of your competitors ?

4) Businesses change every year . How many times are reporting systems over hauled completely to reflect the change – say once in six months?

5) How many employees go in for self funded  training once a year ?

Overall , a great discussion and hopefully it helps you answer the above questions too.

Here is a very good ,in fact brilliant post from Joe Hellerstein, a Professor of Computer Science at UC Berkeley at http://radar.oreilly.com/2008/11/the-commoditization-of-massive.html

It explains the difference between the two databases type.

 

Enterprise IT camp tends to favor relational databases and the SQL language, while the web upstarts have rallied around the MapReduce programming model popularized at Google, and cloned in open source as Apache Hadoop. Hadoop is in wide use at companies like Yahoo! and Facebook, and gets a lot of attention in tech blogs as the next big open source project. But if you mention Hadoop in a corporate IT shop you are often met with blank stares — SQL is ubiquitous in those environments

 

Setting aside the trash talk, the usual cases made for the two technologies can be summarized as follows:

Relational Databases

• multipurpose: useful for analysis and data update, batch and interactive tasks
• high data integrity via ACID transactions
• lots of compatible tools, e.g. for loading, management, reporting, data visualization and mining
• support for SQL, the most widely-used language for data analysis
• automatic SQL query optimization, which can radically improve performance
• integration of SQL with familiar programming languages via connectivity protocols, mapping layers and user-defined functions

MapReduce (Hadoop)

• designed for large clusters: 1000+ computers
• very high availability, keeping long jobs running efficiently even when individual computers break or slow down
• data is accessed in "native format" from a filesystem — no need to transform data into tables at load time
• no special query language; programmers use familiar languages like Java, Python, and Perl
• programmers retain control over performance, rather than counting on a query optimizer
• the open-source Hadoop implementation is funded by corporate donors, and will mature over time as Linux and Apache did

Hadoop is still relatively young, and by all reports much slower and more resource intensive than Google’s MapReduce implementation.

What I liked about the article was explaining Hadoop in simple terms to corporate SQL types like me.

It’s interesting how Hadoop would be configured on the NVidia Tesla supercomputer ( at 10000 USD)

 

– Update – Mathematica is already being modified for the GPU versus CPU system, and there was an interesting discussion in R _help list today on this.

Mathematica is launching a version working with Nvidia GPUs. It is claimed that it’d make it
~10-100x faster!
http://www.physorg.com/news146247669.html

The One Laptop per Child project is having its annual give get one promotion. Basically you pay 400 $ , you get one XO laptop free and another XO laptop is donated in your name in a devleoping country. For the technically minded ,here is a great review of the XO laptop at http://www.bunniestudios.com/blog/?p=218

 

On a slightly different scale is the NVidia GPU ( as opposed to CPU powered computers). They are available here at http://www.nvidia.com/object/wtb_notebooks.html

These and the forthcoming series of NVidia powered GPus are going to give extremely high speeds within a price range of 10,000 USD. How high is the speed ?

Well here is a case study from the NYT "

Techniscan Medical Systems of Salt Lake City has turned to Nvidia’s graphics processors to speed up a three-dimensional breast scanning device that could be used for cancer detection if the machine received regulatory approval. Techniscan must turn tens of gigabytes of raw data generated by transmitting pulses of energy through a breast submerged in water into medical image files that consume just 100 megabytes. This whole process used to take a couple of hours using Intel’s processors and now takes just 15 minutes with Nvidia’s hardware."

 

And here is finally your desktop supercomputer, the Tesla from Nvidia.

http://www.nvidia.com/object/personal_computing.html

"Get your own supercomputer. Experience cluster level computing performance—up to 250 times faster than standard PCs and workstations—right at your desk. The NVIDIA® Tesla™ Personal Supercomputer is based on the revolutionary NVIDIA® CUDA™ parallel computing architecture and powered by up to 960 parallel processing cores."

Now data mining and analytics people love processing power. With this much processing power  it can be quite a lot of fun !

So if you deal with data more than 1 Gb at a time or have more than 10 Pcs, or 2 servers, try the Tesla.

Tesla Architecture

• Massively-parallel many-core architecture
• 240 scalar processor cores per GPU
• Integer, single-precision and double-precision floating point operations
• Hardware Thread Execution Manager enables thousands of concurrent threads per GPU
• Parallel shared memory enables processor cores to collaborate on shared information at local cache performance
• Ultra-fast GPU memory access with 102 GB/s peak bandwidth per GPU
• IEEE 754 single-precision and double-precision floating point
• Each Tesla C1060 GPU delivers 933 GFlops Single Precision and 78 GFlops Double Precision performance

Software Development Tools

• C language compiler, debugger, profiler, and emulation mode for debugging
• Standard numerical libraries for FFT (Fast Fourier Transform), BLAS (Basic Linear Algebra Subroutines), and CuDPP (CUDA Data Parallel Primitives)

Product Details

• 3 or 4 Tesla C1060 Computing Processors with 4GB of dedicated memory per GPU
• 2.33 GHz+ Quad-core AMD Phenom or Opteron, — OR — Quad-core Intel Core 2 or Xeon
• Minimum system memory: 12 GB for 3 Tesla C1060s and 16 GB for 4 Tesla C1060s (at least 4GB per Tesla C1060)
• 12GB+ system memory (at least 4GB per Tesla C1060)
• 1200-1350 Watt Power supply
• Acoustics < 45dbA

Supported Platforms

• Microsoft® Windows® XP 64-bit and 32-bit (64-bit recommended)
• Linux® 64-bit and 32-bit (64-bit recommended)
  • Red Hat Enterprise Linux 4 and 5
  • SUSE 10.1, 10.2 and 10.3

Source-http://www.nvidia.com/object/tesla_supercomputer_tech_specs.html

 

REVIEW

Introduction- Even though R is a very powerful tool and is free, people with SAS and SPSS background have trouble adapting to R language. That is because all data objects in SAS, SPSS are in fixed rectangular layout, and the programmer just needs to write a series of pre given functions to give results. In R the flexibility of functions, and the sheer diversity of it can confuse and confound the SAS and SPSS programmer wanting to learn it. Note that most SAS and SPSS programmers are corporate users, thus they pay for licenses only by just signing the approved email, and they have a paucity of time.

 

 

 

 

The technical review-

The book is lucid, exhaustive and lists down all reasons for and against R in an objective scientific manner. It goes in great detail, has ready datasets and offers the earlier reference sheet from its websites (http://RforSASandSPSSusers.com). At 75  $ it is not expensive considering the cost of other textbooks in this domain. Having both SAS and SPSS can be a distraction as many SPSS users actually use its click and point interface rather than write raw syntax- perhaps those screenshots should be included.

 It thus gives you the side effect of teaching you twice the languages you wanted to learn, but that’s a good thing. Of course you can choose to ignore the second language if you don’t want to learn it.Maybe this book can be split into R for SAS users and R for SPSS users separately (unless Muenchen is trying an agenda of unifying the whole analytics world – a common theme in November 2008 in the USA 😉 )

The book is very easy to understand, providing a step by step way of learning R thoroughly. In addition it uses screenshots extensively to make the point. However this sometimes slows the pace down of the book as it resorts to oversimplification for an advanced SAS users . It does have a good reference guide for people who just need R for some functions like graphics to be used in combination with WPS or Base SAS. You can simply pick and choose.

However it would have been great if there was a CD version.Perhaps it would be in the next edition.

An additional point is that it tries to briefly explain advanced statistical functions (or SAS/STAT equivalents). Some more depth in this section especially to the very popular logistic and linear regression techniques would have helped.(I.e a Chapter on how to build, validate and test a scoring risk model using R).

However you can use the existing chapters to get started on the modeling and iteratively come to its solutions. Some more coverage of Graphical User Interface would have helped, even though that that would have helped many people to simply go to the easy way of using click and get results of the RCmdr and Rattle GUI.

 

Overall- a great textbook, you would need to spend an hour a day for as much as three weeks to master the book and once you do it would be worth it.

I highly recommend this book for both technical and business users, and for libraries as a reference guide. Both corporate and academic users would relate to the readability of the book and Amazon is also launching a kindle version soon.

 

 

The business case review-

 

In these troubled economic times, if you have to choose between cutting down your software costs or your supervisor having to choose all his software AND employee costs, would you take the time and 50 $ to learn two additional analytics platforms. Because that is what this book by Muenchen does, it can teach SAS to R programmers, and SPSS to both. It’s almost a triple pack combo.

 

Muenchen spent two years creating a textbook and off the shelf manual for all three languages. Given the fact that the Base SAS language is also supported by WPS ( a software that costs only 600 dollars a year and reads/writes SAS codes and datasets), this book deals with both SAS and WPS languages

Wait a minute ! That’s what all advertisers say…but this is an honest review.(Gandhian Economics-If price of everything was demand and supply, then price of honesty would be infinite- because the supply is so low and the demand so high.)

Summary-

Buying this book for yourself or a friend could thus save your organizations a lot of money, give you additional skills on your resume and give you bragging rights at the cooler for knowing three languages (SAS, SPSS and R) – all for the price of $ 75 .

 

 

 

 

 

 

• use more contractors. use company alumni rather than out of town contractors as this saves the time for them to get up to speed.

 

• use more skype rather than phones. get over the headphones discomfort. It is better than the unemployment discomfort.

 

• use more WebEX meetings than flying people around. Money you save for your company will help you in retaining others, and yourselves. Most of the airlines money goes to the foreign oil providers anyways.

 

• shop for cheaper open source software than blindly approve annual license fees.  this can save you more money than you think and is less complex. Dont go for 100 % either -or solution. Example replace 20 % of your desktops with open office , rather than microsoft office renewals.

 

• use more variable linked bonus than flat fees for salaries. this helps separates the grain from the chaff. And improve revenue and cut costs rather than employees.

 

 

Each unemployed person is a drag on the economy that will take social security that YOU will pay anyways. Employed people spend money on groceries , movies and provide even more jobs or potential markets.

 

Mahatma Gandhi once said – Be Indian , Buy Indian. It helped India get rid of the British Empire.

Once in a while, think about paying a small premium for being American , buying American