Using R for Time Series in SAS

 

Time series: random data plus trend, with best...
Image via Wikipedia

 

Here is a great paper on using Time Series in R, and it specifically allows you to use just R output in Base SAS.

SAS Code

/* three methods: */

/* 1. Call R directly – Some errors are not reported to log */

x “’C:\Program Files\R\R-2.12.0\bin\r.exe’–no-save –no-restore <“”&rsourcepath\tsdiag.r””>””&rsourcepath\tsdiag.out”””;

/* include the R log in the SAS log */7data _null_;

infile “&rsourcepath\tsdiag.out”;

file log;

input;

put ’R LOG: ’ _infile_;

run;

/* include the image in the sas output.Specify a file if you are not using autogenerated html output */

ods html;

data _null_;

file print;

put “<IMG SRC=’” “&rsourcepath\plot.png” “’ border=’0’>”;

put “<IMG SRC=’” “&rsourcepath\acf.png” “’ border=’0’>”;

put “<IMG SRC=’” “&rsourcepath\pacf.png” “’ border=’0’>”;

put “<IMG SRC=’” “&rsourcepath\spect.png” “’ border=’0’>”;

put “<IMG SRC=’” “&rsourcepath\fcst.png” “’ border=’0’>”;

run;

ods html close;

The R code to create a time series plot is quite elegant though-


library(tseries)

air <- AirPassengers #Datasetname

ts.plot(air)

acf(air)

pacf(air)

plot(decompose(air))

air.fit <- arima(air,order=c(0,1,1), seasonal=list(order=c(0,1,1), period=12) #The ARIMA Model Based on PACF and ACF Graphs

tsdiag(air.fit)

library(forecast)

air.forecast <- forecast(air.fit)

plot.forecast(air.forecast)

You can download the fascinating paper from the Analytics NCSU Website http://analytics.ncsu.edu/sesug/2008/ST-146.pdf

About the Author-

Sam Croker has a MS in Statistics from the University of South Carolina and has over ten years of experience in analytics.   His research interests are in time series analysis and forecasting with focus on stream-flow analysis.  He is currently using SAS, R and other analytical tools for fraud and abuse detection in Medicare and Medicaid data. He also has experience in analyzing, modeling and forecasting in the finance, marketing, hospitality, retail and pharmaceutical industries.

LibreOffice Beta 2 (Office Fork off Oracle) launches!

 

Windows 7, the latest client version in the Mi...
Image via Wikipedia

 

Announcement from Code Ninjas at Document Foundation

10 years after the StarOffice code has been opened as OpenOffice.org, The Document Foundation is proud to announce the availability of LibreOffice Beta 2 for public testing. Please, download the suitable package(s) from

http://www.documentfoundation.org/download/

 

Ajay- Note that first beta was downloaded almost 100,000 times

install them, and start testing! Should you find bugs, please report them to the FreeDesktop Bugzilla:

https://bugs.freedesktop.org

If you want to get involved in this exciting project, you can contribute code:

http://www.documentfoundation.org/develop/

translate LibreOffice to your language:

http://www.freedesktop.org/wiki/Software/LibreOffice/i18n/translating_3.3

or just donate:

http://www.documentfoundation.org/contribution/
A list of known issues with Beta 2 is available in our wiki:

http://wiki.documentfoundation.org/Beta2

Beta Release Notes

This beta release is not intended for production use!

There are a number of known issues being worked on:

  • The Windows build is an International build – you can choose the user interface language that is suitable for you, but the help is always English. We are currently working on improving the delivery mechanism to be able to provide you with the localized help. We are also working on smaller problems like wrong description of several languages.
  • The Linux and MacOSX builds are English builds with the possibility to install language packs. Please browse the archives to get the langugage pack you need for your platform.
  • The LibreOffice branding and renaming is new and work in progress. You may still see old graphics, icons or websites. So please bear with us. This also applies to the BrOffice.org branding – applicable in Brazil.
  • Filters for the legacy StarOffice binary formats are missing.

I tested it- it seems okay enough. Once again Open Source tends to underplay expectations (when was the last time you saw that in enterprise software?)

China biggest threat to Indian Software in 5 years: Indian Tech CEO

The Hall of Prayer for Good Harvest, Temple of...
Image via Wikipedia

An interview with a noted Indian Software CEO, mentions China the possible biggest threat in next 5 years at  http://www.thehindubusinessline.com/2010/10/13/stories/2010101353180700.htm

 

China could be the biggest threat to India in next five years, positioning itself as the lowest-cost manpower supplier in the IT sector by 2015, according to Mr Vineet Nayar, CEO, HCL Technologies.

“I believe it (China) is the biggest threat in the next five years that we are going to face…So India will have to up its game,” he told reporters on sidelines of ‘Directions’, the company’s annual town hall.

Terming China, as both “threat and opportunity”, Mr Nayar said that India will have to find alternate “differentiators” than the ones it currently has. Despite issues of language and the purported inability to scale-up, China has sharpened its technological and innovation edge, he added.

“Look at the technology companies from China…how does that fit in with the assumption that they (China) do not understand English or technology. They are producing cutting edge technology at a price which is lower than everyone else,” he said.

Manpower

By 2015, Mr Nayar said, China will be the lowest cost manpower supplier in IT sector to the world

——————————————————————————————–

I wonder how he did his forecast. Did he do a time series analysis using a software, did he peer into his crystal ball, or did he spend a lot of time brainstorming with his strategic macro economic team on Chinese threat.

China has various advantages over India (and in fact the US)-

1) Big pool of reliable scientific manpower

2) State funded education in higher studies and STEM

3) Increasing exposure with the West-English speaking is no longer an issue. Almost 50 % of Grad Students in the US in STEM and certain sectors are Chinese and they not only retain fraternal ties with the motherland- they often remain un-assimilated with American Culture mainstream. or they have a separate interaction with fellow American Chinese and seperate with American Americans.

Chinese suffer from some disadvantages in software-

1) Communism Perception- Just because the Govt is communist and likes to confront US once a year (and India twice a month)- is no excuse for the hapless Chinese startup guy to lose out on software outsourcing contracts. unfortunately there have been reported cases where sneak codes have been inserted in code deliverables for American partners, just like American companies are forced to work with DoD (especially in software, embedded chips and telecom)

If you have 10000 lines of code delivered by your Chinese partner, how sure are you of going through each line of code for each sub routine or call procedure.

2) English- Chinese accent is like Chinese cooking. Unique- many Chinese are unable to master the different style of English even after years (derived from Latin and Indo European class of languages)

Sales jobs tend to go to American trained Chinese or to Westerners.

In Indian software companies, accent is a lesser problem.

———————————————————————————-

The biggest threat to Indian software in 5 years is actually Indian software itself- Can it evolve and mature to a product based model from a service only model.

Can Indian software partner with Chinese companies and maybe teach the Indian government why friendship is more profitable than envy and suspicion. If the US and China can trade enormously despite annual tensions, why cant Indian services do the same- if they lose this opportunity, US companies will likely bypass them and create the same GE/McKinsey style backoffices that started the Indian offshoring phenomenon.

3) Lastly- what did the poor American grad student do to deserve that even if devotes years to study STEM (and being called a Geek and Nerd) his job will get outsourced to India or China (if not now- in his 30s or worse in his 40s). Talk to any middle aged IT chap in the US who is middle class- and India and China would figure in why he still worries about his overpriced mortgage.

Unless the US wants only Twitter and Facebook as dominant technologies in the 21 st century.

Amen.

 

 

 

Interview John F Moore CEO The Lab

Social Media Landscape

Here is an interview with John F Moore, social media adviser,technologist and founder and CEO of The Lab.

Ajay-  The internet seems to be crowded by social media experts with everyone who spends a lot of time on the internet claiming to be one? How  does a small business owner on a budget distinguish for the correct value proposition that social media can give them. 

John- You’re right.  It seems like everytime I turn around I bump into more social media “experts”.  The majority of these self-proclaimed experts are not adding a great deal of value.  When looking to spend money for help ask the person a few questions about their approach. Things you should be hearing include:

  • The expert should be seeking to fully understand your business, your goals, your available resources, etc..
  • The expert should be seeking to understand current management thinking about social media and related technologies.

If the expert is purely focused on tools they are the wrong person.  Your solution may require tools alone but they cannot know this without first understanding your business.

Ajay- Facebook has 600 million people, with people preferring to play games and connect to old acquaintances rather than use social media for tangible career or business benefit..

John- People are definitely spending time playing games, looking at photos, and catching up with old friends.  However, there are many businesses seeing real value from Facebook (primarily by tying it into their e-mail marketing and using coupons and other incentives).  For example, I recently shared a small case study (http://thejohnfmoore.com/2010/10/07/email-social-media-and-coupons-makes-the-cfo-smile/) where a small pet product company achieved a 22% bump in monthly revenue by combining Facebook and coupons together.  In fact,45% of this bump in revenue came from new clients.  Customer acquisition and increased revenue were accomplished by using Facebook for their business.
Ajay-  How does a new social media convert (individual) go on selecting communities to join (Facebook,Twitter,Linkedin,Ning, Ping,Orkut, Empire Avenue etc etc.
How does a small business owner take the same decision.

John- It always starts with taking the time to define your goals and then determine how much time and effort you are willing to invest.  For example:
  • LinkedIn. A must have for individuals as it is one of the key social networking communities for professional networking.  Individuals should join groups that are relevant to their career and invest an hour a week.  Businesses should ensure they have a business profile completed and up to date.
  • Facebook can be a challenge for anyone trying to walk the personal/professional line.  However, from a business standpoint you should be creating a Facebook page that you can use to compliment your other marketing channels.
  • Twitter.  It is a great network to learn of, to meet, and to interact with people from around the world.  I have met thousands of interesting people, many of which I have had the pleasure to meet with in real life.  Businesses need to invest in listening on twitter to determine if their customers (current or potential) or competitors are already there discussing them, their marketplace, or their offerings.
In all cases I would encourage businesses to setup social media accounts on LinkedIn, Facebook, Twitter, YouTube, and Flickr.  You want to ensure your brand is protected by owning these accounts and ensuring at least the base information is accurate.
Ajay- Name the top 5 points that you think make a social media community successful.  What are the top 5 points for a business to succeed in their social media strategy.

John-
  • Define your goals up front.  Understand why you are building a community and keep this goal in mind.
  • Provide education.  Ideally you want to become a thought leader in your space, the trusted resource that people can turn to even if they are not using your product or services today.
  • Be honest.  We all make mistakes.  When you do, be honest with your community and engage them in any fall-out that may be coming out of your mistake.
  • Listen to them.  Use platforms like BubbleIdeas to gather feedback on what your community is looking for from the relationship.
  • Measure.  Are you on track with your goals?  Do your goals need to change?
Ajay- What is the unique value proposition that “The Lab” offers

John- The Lab understands the strategic importance of leveraging social media, management and leadership best practices, and our understanding of local government and small and medium business to help people in these areas achieve their goals.  Too many consultants come to the table with a predefined solution that really misses the mark as it lacks understanding of the client’s goals.
Ajay-  What is “CityCamp in Boston” all about.

John- CityCamp is a FREE unconference focused on innovation for municipal governments and community organizations (http://www.citycampboston.org/what-is-citycamp-boston/).  It brings together politicians, local municipal employees, citizens, vendors, developers, and journalist to build a common understanding of local government challenges and then works to deliver measurable outcomes following the event.  The key is the focus on change management, driving change as opposed to just in the moment education.
Biography-

John F Moore is the Founder and CEO of The Lab (http://thelabinboston.com).  John has experience working with local governments and small and medium business owners to achieve their goals.  His experience with social media strategies, CRM, and a plethora of other solutions provides immense value to all of our clients.   He has built engineering organizations, learned sales and marketing, run customer service teams, and built and executed strategies for social media thought leadership and branding.  He is also a prolific blogger as you can see by checking out his blog at http://thejohnfmoore.com.

Which software do we buy? -It depends

Software (novel)
Image via Wikipedia

Often I am asked by clients, friends and industry colleagues on the suitability or unsuitability of particular software for analytical needs.  My answer is mostly-

It depends on-

1) Cost of Type 1 error in purchase decision versus Type 2 error in Purchase Decision. (forgive me if I mix up Type 1 with Type 2 error- I do have some weird childhood learning disabilities which crop up now and then)

Here I define Type 1 error as paying more for a software when there were equivalent functionalities available at lower price, or buying components you do need , like SPSS Trends (when only SPSS Base is required) or SAS ETS, when only SAS/Stat would do.

The first kind is of course due to the presence of free tools with GUI like R, R Commander and Deducer (Rattle does have a 500$ commercial version).

The emergence of software vendors like WPS (for SAS language aficionados) which offer similar functionality as Base SAS, as well as the increasing convergence of business analytics (read predictive analytics), business intelligence (read reporting) has led to somewhat brand clutter in which all softwares promise to do everything at all different prices- though they all have specific strengths and weakness. To add to this, there are comparatively fewer business analytics independent analysts than say independent business intelligence analysts.

2) Type 2 Error- In this case the opportunity cost of delayed projects, business models , or lower accuracy – consequences of buying a lower priced software which had lesser functionality than you required.

To compound the magnitude of error 2, you are probably in some kind of vendor lock-in, your software budget is over because of buying too much or inappropriate software and hardware, and still you could do with some added help in business analytics. The fear of making a business critical error is a substantial reason why open source software have to work harder at proving them competent. This is because writing great software is not enough, we need great marketing to sell it, and great customer support to sustain it.

As Business Decisions are decisions made in the constraints of time, information and money- I will try to create a software purchase matrix based on my knowledge of known softwares (and unknown strengths and weakness), pricing (versus budgets), and ranges of data handling. I will add in basically an optimum approach based on known constraints, and add in flexibility for unknown operational constraints.

I will restrain this matrix to analytics software, though you could certainly extend it to other classes of enterprise software including big data databases, infrastructure and computing.

Noted Assumptions- 1) I am vendor neutral and do not suffer from subjective bias or affection for particular software (based on conferences, books, relationships,consulting etc)

2) All software have bugs so all need customer support.

3) All software have particular advantages , strengths and weakness in terms of functionality.

4) Cost includes total cost of ownership and opportunity cost of business analytics enabled decision.

5) All software marketing people will praise their own software- sometimes over-selling and mis-selling product bundles.

Software compared are SPSS, KXEN, R,SAS, WPS, Revolution R, SQL Server,  and various flavors and sub components within this. Optimized approach will include parallel programming, cloud computing, hardware costs, and dependent software costs.

To be continued-

 

 

 

 

Top ten RRReasons R is bad for you ?

This is the original symbol of the Perl progra...
Image via Wikipedia

R stands for programming language based out of www.r-project.org

R is bad for you because –

1) It is slower with bigger datasets than SPSS language and SAS language .If you use bigger datasets, then you should either consider more hardware , or try and wait for some of the ODBC connect packages.

2) It needs more time to learn than SAS language .Much more time to learn how to do much more.

3) R programmers are lesser paid than SAS programmers.They prefer it that way.It equates the satisfaction of creating a package in development with a world wide community with the satisfaction of using a package and earning much more money per hour.

4) It forces you to learn the exact details of what you are doing due to its object oriented structure. Thus you either get no answer or get an exact answer. Your customer pays you by the hour not by the correct answers.

5) You can not push a couple of buttons or refer to a list of top ten most commonly used commands to finish the project.

6) It is free. And open for all. It is socialism expressed in code. Some of the packages are built by university professors. It is free.Free is bad. Who pays for the mortgage of the software programmers if all softwares were free ? Who pays for the Friday picnics. Who pays for the Good Night cruises?

7) It is free. Your organization will not commend you for saving them money- they will question why you did not recommend this before. And why did you approve all those packages that expire in 2011.R is fReeeeee. Customers feel good while spending money.The more software budgets you approve the more your salary is. R thReatens all that.

8) It is impossible to install a package you do not need or want. There is no one calling you on the phone to consider one more package or solution. R can make you lonely.

9) R uses mostly Command line. Command line is from the Seventies. Or the Eighties. The GUI’s RCmdr and Rattle are there but still…..

10) R forces you to learn new stuff by the month. You prefer to only earn by the month. Till the day your job got offshored…

Written by a R user in English language

( which fortunately was not copyrighted otherwise we would be paying Britain for each word)

Ajay- The above post was reprinted by personal request. It was written on Jan 2009- and may not be truly valid now. It is meant to be taken in good humor-not so seriously.

Poem : A Poets Life

I read and wrote and joked and pondered
Inevitably I found myself occasionally wandered
Those who wander are not always lost
And so I consoled my earthly Faust

Sometimes the good guys win and place first
In an uncharectristic improbable late blooming burst

Predicting God‘s plan for yourself
Is like predicting who and if there is a God
No dearth of contradictions,
Self doubting logic brought.

Thanks for reading, hope it was worth your time
I am medium sized poet, with a mediocre chime
And sometimes you yourself may feel blue
Its your own time to waste, and remember that’s true.


Before I leave or you have left
Ponder once more what this moment brought
Summarizing once more a poets life
In an age of blogs, tweets and promotional strife

I am dying of a disease called life
Friends and Family are protective still
My medication helps but will only stall the end
My stubborn body awaits my soul upwards to send

I have lived not too long nor too short it will be
Fought battles some planned some momentarily
Made friends and loved ah so well
On the whole it was rather interesting to dwell