Save the Data

Breakdown of political party representation in...
Image via Wikipedia

I just read an online cause here-

http://sunlightfoundation.com/savethedata/

Some of the most important technology programs that keep Washington accountable are in danger of being eliminated. Data.gov, USASpending.gov, the IT Dashboard and other federal data transparency and government accountability programs are facing a massive budget cut, despite only being a tiny fraction of the national budget. Help save the data and make sure that Congress doesn’t leave the American people in the dark.

I wonder why the federal government/ non profit agencies can help create a SPARQL database, and in days of cloud computing, why a tech major cannot donate storage space to it, after all despite US corporate tax rate being high, US technological companies do end up paying a lower rate thanks to tax breaks/routing overseas revenue.

In the new age data is power, and the US has led in its mission to use technology to further its own values even especially in Middle East. The datasets should be made public and transitioned to the private sector/academia for research and re designing for data augmentation with out straining the massive deficit /borrowing/ fighting 3 wars. Of particular interest would be datasets of campaign finances  and donors especially given large number of retail/small donors/internet marketing in elections as it will also help serve as an example of democracy and change. Even countries like China can create a corruption/expense efficiency tracking internal dashboard with restricted rights to help with rural and urban governance.

The impact of currency fluctuations on outsourcing businesses globally

 

The impact of currency fluctuations on outsourcing businesses globally.

If you have a current offshore team in a different country/currency zone then you may find that the significant cost savings from outsourcing have vanished due to currency fluctuations that occur for reasons like earthquakes, war or oil- something which is outside the core competency of your business corporation. As off shoring companies incur cost in local currencies but gain revenue in American Dollars and Euro (mostly), they pass on these fluctuating costs to their customers but rarely pass along discounts on existing contracts. Sometimes the offshoring contract actually gains from currency fluctuations.The Indian rupee has fluctuated from  43.62 Rupees per USD (04-01-2005) to 48.58 (12-31-2008) to the current value of 44.65.This makes for a volatility component of almost 10 percentage points to the revenue and profit margins of an off shoring vendor. Inflation in India has been growing at 8.5 % and the annual increase in salaries has been around 10-15 % for the past few years. Offshoring vendors have been known to cut back on quality in recruitment when costs have risen historically, and the current attrition rate in Indian ITES is almost 17%.
This raises important questions for companies going for global bids for the offshoring contracts. Should macroeconomic indicators like currency fluctuations, wage-inflation be part of the request for proposal process (RFP). Would vendors be comfortable in disclosing the ratio of salary costs to billing revenue. Should dips in service quality be penalized by customer. Most importantly, while going in for a multi year contract, the projection of fore-casted savings may vary greatly due to extraneous factors.
(this article was originally written for and published by http://www.indiasoftwarebrief.com/ in their daily newsletter and their socail media channel- see http://www.linkedin.com/groups/impact-currency-fluctuations-on-outsourcing-3825591.S.48411960)

 

 

KDNuggets Survey on R

CRISP-DM
Image via Wikipedia

From http://www.kdnuggets.com/2011/03/new-poll-r-in-analytics-data-mining-work.html?k11n07

A new poll/survey on actual usage of R in Data Mining

R has been steadily growing in popularity among data miners and analytic professionals.

In KDnuggets 2010 Data Mining / Analytic Tools Poll, R was used by 30% of respondents.
In 2010 Rexer Analytics Data Miner SurveyR was the most popular tool, used by 43% of the data miners.

Another aspect of tool usefulness is how much does it help with the entire data mining process from data preparation and cleaning, modeling, evaluation, visualization and presentation (excluding deployment).

New KDnuggets Poll is asking:
What part of your analytics / data mining work in the past 12 months was done in R?

http://www.kdnuggets.com/2011/03/new-poll-r-in-analytics-data-mining-work.html?k11n07

 

IBM and Revolution team to create new in-database R

From the Press Release at http://www.revolutionanalytics.com/news-events/news-room/2011/revolution-analytics-netezza-partnership.php

Under the terms of the agreement, the companies will work together to create a version of Revolution’s software that takes advantage of IBM Netezza’s i-class technology so that Revolution R Enterprise can run in-database in an optimal fashion.

About IBM

For information about IBM Netezza, please visit: http://www.netezza.com.
For Information on IBM Information Management, please visit: http://www.ibm.com/software/data/information-on-demand/
For information on IBM Business Analytics, please visit the online press kit: http://www.ibm.com/press/us/en/presskit/27163.wss
Follow IBM and Analytics on Twitter: http://twitter.com/ibmbizanalytics
Follow IBM analytics on Tumblr: http://smarterplanet.tumblr.com/tagged/new_intelligence
IBM YouTube Analytics Channel: http://www.youtube.com/user/ibmbusinessanalytics
For information on IBM Smarter Systems: http://www-03.ibm.com/systems/smarter/

About Revolution Analytics

Revolution Analytics is the leading commercial provider of software and services based on the open source R project for statistical computing.  Led by predictive analytics pioneer Norman Nie, the company brings high performance, productivity and enterprise readiness to R, the most powerful statistics language in the world. The company’s flagship Revolution R product is designed to meet the production needs of large organizations in industries such as finance, life sciences, retail, manufacturing and media.  Used by over 2 million analysts in academia and at cutting-edge companies such as Google, Bank of America and Acxiom, R has emerged as the standard of innovation in statistical analysis. Revolution Analytics is committed to fostering the continued growth of the R community through sponsorship of the Inside-R.org community site, funding worldwide R user groups and offers free licenses of Revolution R Enterprise to everyone in academia.


Netezza, an IBM Company, is the global leader in data warehouse, analytic and monitoring appliances that dramatically simplify high-performance analytics across an extended enterprise. IBM Netezza’s technology enables organizations to process enormous amounts of captured data at exceptional speed, providing a significant competitive and operational advantage in today’s data-intensive industries, including digital media, energy, financial services, government, health and life sciences, retail and telecommunications.

The IBM Netezza TwinFin® appliance is built specifically to analyze petabytes of detailed data significantly faster than existing data warehouse options, and at a much lower total cost of ownership. It stores, filters and processes terabytes of records within a single unit, analyzing only the relevant information for each query.

Using Revolution R Enterprise & Netezza Together

Revolution Analytics and IBM Netezza have announced a partnership to integrate Revolution R Enterprise and the IBM Netezza TwinFin  Data Warehouse Appliance. For the first time, customers seeking to run high performance and full-scale predictive analytics from within a data warehouse platform will be able to directly leverage the power of the open source R statistics language. The companies are working together to create a version of Revolution’s software that takes advantage of IBM Netezza’s i-class technology so that Revolution R Enterprise can run in-database in an optimal fashion.

This partnership integrates Revolution R Enterprise with IBM Netezza’s high performance data warehouse and advanced analytics platform to help organizations combat the challenges that arise as complexity and the scale of data grow.  By moving the analytics processing next to the data, this integration will minimize data movement – a significant bottleneck, especially when dealing with “Big Data”.  It will deliver high performance on large scale data, while leveraging the latest innovations in analytics.

With Revolution R Enterprise for IBM Netezza, advanced R computations are available for rapid analysis of hundreds of terabyte-class data volumes — and can deliver 10-100x performance improvements at a fraction of the cost compared to traditional analytics vendors.

Additional Resources


Protected: Whats behind that pretty SAS Blog?

This content is password-protected. To view it, please enter the password below.

WPS Version 2.5.1 Released – can still run SAS language/data and R

However this is what Phil Rack the reseller is quoting on http://www.minequest.com/Pricing.html

Windows Desktop Price: $884 on 32-bit Windows and $1,149 on 64-bit Windows.

The Bridge to R is available on the Windows platforms and is available for free to customers who
license WPS through MineQuest,LLC. Companies and organizations outside of North America
may purchase a license for the Bridge to R which starts at $199 per desktop or $599 per server

Windows Server Price: $1,903 per logical CPU for 32-bit and $2,474 for 64-bit.

Note that Linux server versions are available but do not yet support the Eclipse IDE and are
command line only

WPS sure seems going well-but their pricing is no longer fixed and on the home website, you gotta fill a form. Ditt0 for the 30 day free evaluation

http://www.teamwpc.co.uk/products/wps/modules/core

Data File Formats

The table below provides a summary of data formats presently supported by the WPS Core module.

Data File Format Un-Compressed
Data
Compressed
Data
Read Write Read Write
SD2 (SAS version 6 data set)
SAS7BDAT (SAS version 7 data set)
SAS7BDAT (SAS version 8 data set)
SAS7BDAT (SAS version 9 data set)
SASSEQ (SAS version 8/9 sequential file)
V8SEQ (SAS version 8 sequential file)
V9SEQ (SAS version 9 sequential file)
WPD (WPS native data set)
WPDSEQ (WPS native sequential file)
XPORT (transport format)

Additional access to EXCEL, SPSS and dBASE files is supported by utilising the WPS Engine for DB Filesmodule.

and they have a new product release on Valentine Day 2011 (oh these Europeans!)

From the press release at http://www.teamwpc.co.uk/press/wps2_5_1_released

WPS Version 2.5.1 Released 

New language support, new data engines, larger datasets, improved scalability

LONDON, UK – 14 February 2011 – World Programming today released version 2.5.1 of their WPS software for workstations, servers and mainframes.

WPS is a competitively priced, high performance, highly scalable data processing and analytics software product that allows users to execute programs written in the language of SAS. WPS is supported on a wide variety of hardware and operating system platforms and can connect to and work with many types of data with ease. The WPS user interface (Workbench) is frequently praised for its ease of use and flexibility, with the option to include numerous third-party extensions.

This latest version of the software has the ability to manipulate even greater volumes of data, removing the previous 2^31 (2 billion) limit on number of observations.

Complimenting extended data processing capabilities, World Programming has worked hard to boost the performance, scalability and reliability of the WPS software to give users the confidence they need to run heavy workloads whilst delivering maximum value from available computer power.

WPS version 2.5.1 offers additional flexibility with the release of two new data engines for accessing Greenplum and SAND databases. WPS now comes with eleven data engines and can access a huge range of commonly used and industry-standard file-formats and databases.

Support in WPS for the language of SAS continues to expand with more statistical procedures, data step functions, graphing controls and many other language items and options.

WPS version 2.5.1 is available as a free upgrade to all licensed users of WPS.

Summary of Main New Features:

  • Supporting Even Larger Datasets
    WPS is now able to process very large data sets by lifting completely the previous size limit of 2^31 observations.
  • Performance and Scalability Boosted
    Performance and scalability improvements across the board combine to ensure even the most demanding large and concurrent workloads are processed efficiently and reliably.
  • More Language Support
    WPS 2.5.1 continues the expansion of it’s language support with over 70 new language items, including new Procedures, Data Step functions and many other language items and options.
  • Statistical Analysis
    The procedure support in WPS Statistics has been expanded to include PROC CLUSTER and PROC TREE.
  • Graphical Output
    The graphical output from WPS Graphing has been expanded to accommodate more configurable graphics.
  • Hash Tables
    Support is now provided for hash tables.
  • Greenplum®
    A new WPS Engine for Greenplum provides dedicated support for accessing the Greenplum database.
  • SAND®
    A new WPS Engine for SAND provides dedicated support for accessing the SAND database.
  • Oracle®
    Bulk loading support now available in the WPS Engine for Oracle.
  • SQL Server®
    To enhance existing SQL Server database access, a new SQLSERVR (please note spelling) facility in the ODBC engine.

More Information:

Existing Users should visit www.teamwpc.co.uk/support/wps/release where you can download a readme file containing more information about all the new features and fixes in WPS 2.5.1.

New Users should visit www.teamwpc.co.uk/products/wps where you can explore in more detail all the features available in WPS or request a free evaluation.

and from http://www.teamwpc.co.uk/products/wps/data it seems they are going on the BIG DATA submarine as well-

Data Support 

Extremely Large Data Size Handling

WPS is now able to handle extremely large data sets now that the previous limit of 2^31 observations has been lifted.

Access Standard Databases

Use I/O Features in WPS Core

  • CLIPBOARD (Windows only)
  • DDE (Windows only)
  • EMAIL (via SMTP or MAPI)
  • FTP
  • HTTP
  • PIPE (Windows and UNIX only)
  • SOCKET
  • STDIO
  • URL

Use Standard Data File Formats

R for Analytics is now live

Okay, through the weekend I created a website for a few of my favourite things.

It’s on at https://rforanalytics.wordpress.com/

Graphical User Interfaces for R

 

Jerry Rubin said: “Don’t trust anyone over thirty

I dont trust anyone not using atleast one R GUI. Here’s a list of the top 10.

 

Code Enhancers for R

Here is a list of top 5 code enhancers,editors in R

R Commercial Software

A list of companies and software making (and) selling R software (and) services. Hint- it is almost 5 (unless I missed someone)

R Graphs Resources

R’s famous graphing capabilities and equally famous learning curve can be made a bit more humane- using some of these resources.

Internet Browsing

Because that’s what I do (all I do as per my cat) , and I am pretty good at it.

Using R from other Software

R can be used successfully from a lot of analytical software including some surprising ones praising the great 3000 packages library.

(to be continued- as I find more stuff I will keep it there, some ideas- database access from R, prominent R consultants, prominent R packages, famous R interviewees 😉 )

ps- The quote from Jerry Rubin seems funny for a while. I turn 34 this year.