March 2009 – Page 4 – DECISION STATS

A Farewell to Guns

A 17 yr old teenage German shot and killed 15 people at his former high school and was killed himself. In Alabama ten people are dead including a family that was wiped out. A mentally disturbed ethnic Korean student killed 30 people in Virginia tech before shooting himself. Supporters of guns including the NRA, ex Army types , plain rural hunting people continue to cling to guns and will do more so as the economic environment worsens.

These people abuse the semantics of the American Second Amendment which provided for arms for a militia , but my friend ,teenagers do not constitute militia nor do mentally disturbed students.With the worlds most expensive and powerful army do they still need a militia. While applying to and getting bailouts�� from the Government do they still need individual gun rights.

In California an ethic Chinese engineer shot and killed his boss and head of Hr before surrendering. Bad Bosses are nothing new yet this marks a new low in free availability and abuse of weapons.Violence and anger have no creed , or ethnicity.

A gun in the hand of a person untrained in the implications of gun shot wounds is as lethal a weapon like a deadly poison snake whose fangs have been temporarily been trimmed. If you ask a layman to operate surgery without teaching him how to use a scalpel he could do much less damage than semi automatic weapons in immature and still malleable minds , who resort to binges of violence and outbursts.

The gun trade has thrived on the misery of a few victims and the mirth of many supporters. Researchers use billions of dollars to find cures to diseases that kill people, yet the common sense cure for shot gun ,hand gun , automatic domestic guns is missing- Ban these guns. They are of no use in the military at all �hand guns lack effective offensive ranges and they are increasingly killing too many innocent people.

Charlton Heston is dead- and it is time to pry the guns� from his cold dead hands.

A Constitutional Farewell to Ammo

Here is a solution to the latest outbreak of random shootings by young male

shooters –

Dont ban Guns , but regulate the ammunition. Most shooters generally build

up�ammunition before going on shooting binges.

If credit history can electronically track each tiny credit purchase, then why

cant an extra variable ot two be added for number of guns purchased in that

social id, previous medical history, previous ammunition used.

Restricting ammunition to say two per hand gun, and two per shot gun per

month , and making new sale of ammunition �dependent on return of old

ammunition is NOT in breach of the Second Amendment- and the militia men

can still technically have arms , correct ?�

�

Whats the flaw with this plan legally ….?

More Ways to get a Scoring Model wrong

I got the following answer from Linkedin groups�http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&gid=53432&discussionID=1946379&commentID=2213879&goback=.mgr_false_0_DATE.mgr_true_1_DATE.mid_1066685320#commentID_2213879

�

on my Ten Ways to get a Scoring Model Wrong.

�Typo�
Refuse to use central tendency to patch missing values. Instead, assign highest response rate because WOE says so�
Marketing people tell me to force the variable into the model�
�Selection bias�
�Forgot to segment�
Solely rely on data to segment without consulting the biz side�
�Just delete observations with missing values, OK, without studying geometricl boundaries�
�Using oversampling, but refuse to weight it back. That boosts lift, right? Let us do 50-50�
Insist random sampling is sufficient, while stratified sampling is critical�
Binning too much, or two little�
Selecting variables without repeated sampling�
Forgot to exclude numeric customer id from the candidate variables. AND,it pops….Well, both Unica and Kxen accepted it, So I see no problem�
When the same variable is sourced by different vendors, did not look up the scales under the same name. Just combine them�
�Well, SAS Enterprise Miner gave me this model yesterday�
The binary variable is statistically significant, but there are only 27 event=1, out of ~1mm, since only 27 made some purchases..�
Well, I only have 250 events=1. But I think I can use exact logistic to make it up, all right? I got a PHD in Statistics, Trust me, my professor is OK with it. I just called her.�
�Build two-stage model without Heckman adjustment�
Use global mean over the WHOLE customer base to replace missing value on a much smaller universe/subset. So average networth of a high networth client group has 22% worth only 225K�
I just spent the past two days boosting R-square. Now it is 92. Great.�
Forgot to set descending option in proc logistic in SAS�
I think we should hold out missing values when conducting EDA.�
Without proper separation of ‘treatment and control�
Treat business entities and individuals as equal and mix them in the same universe
Runing clustering without validation�
Running discriminant model without validation. So correct classification rate on development is 89% and that over validation is …35%.(no wonder you finished it in two hours and came here to ask me for a raise)�
Disregard link function in multi-nomil models�
I think this is a better variable: xnew=y*y*y*. It is the top variable dominating others.�
Use standardized coefficient to calculate relative importance, because many people are doing and marketing loves it.�
I tried Goolge Analtyics last Friday. It recommends this variable: click stream density over Thanksgivning weekend, on my web portal, on this item�
�Let us treat this matrix as unary so we can apply Euclidean, since that runs faster and has a lot of optimal properties. It makes our life easier�
Let us use score from that model to boost this model and use score from this model to boost it back. Is that what they call neural nets, Jia?�

Enough?

�

31 Ways to get a model wrong – and Hats off to a fellow mate in suffering -Jia

Coming up – One Way to get a scoring model correct

Ten ways to build a wrong scoring model

Some ways to build a wrong scoring model are below- The author doesn’t take any guarantee if your modeling team is using one of these and still getting a correct model.

1) Over fit the model to the sample. This over fitting can be checked by taking a random sample again and fitting the scoring equation and compared predicted conversion rates versus actual conversion rates. The over fit model does not rank order � deciles with lower average probability may show equal or more conversions than deciles with higher probability scores.

2) Choose non random samples for building and validating the scoring equation. Read over fitting above.

3) Use Multicollinearity (http://en.wikipedia.org/wiki/Multicollinearity ) without business judgment to remove variables which may make business sense.Usually happens a few years after you studied and forgot Multicollinearity.

If you don�t know the difference between Multicollinearity , Heteroskedasticity http://en.wikipedia.org/wiki/Heteroskedasticity this could be the real deal breaker for you

4) Using legacy codes for running scoring usually with step wise forward and backward regression .Happens usually on Fridays and when in a hurry to make models.

5) Ignoring signs or magnitude of parameter estimates ( that�s the output or the weightage of the variable in the equation).

6) Not knowing the difference between Type 1 and Type 2 error especially when rejecting variables based on P value. ( Not knowing P value means you may kindly stop reading and click the You Tube video in the right margin )

7) Excessive zeal in removing variables. Why ? Ask yourself this question every time you are removing a variable.

8) Using the wrong causal event (like mailings for loans) for predicting the future with scoring model (for mailings of deposit accounts) . or using the right causal event in the wrong environment ( rapid decline/rise of sales due to factors not present in model like competitor entry/going out of business ,oil prices, credit shocks sob sob sigh)

9) Over fitting

10) Learning about creating models from blogs and not reading and refreshing your old statistics textbooks

Guns and No Glory

Beginning henceforth here is the policy on comments and posts.It is in response of the comments on my �A Farewell to Guns � post in which I projected my Gandhian non violence too far in suggesting the remote possibility of a ban on guns based on Alabama and Germany events.

No more political posts (including on India) or poems (including funny) will be imposed on this blog or unsuspecting readers.*
I use an offline blogging system called Windows Live Writer and do not go to approve the comments online (requires me to login to word press and manually do it). All comments are read via email settings and feedback incorporated.Comments sometimes get deleted within 7 days because of auto settings (like me not going and logging in to WordPress for 7 days) not because I am ignoring anything
You didn’t like the �Guns� article � you have the right to say so on the comments page. You didn’t like the R article or the package�comments page please.
Read Page �Fine Print� . I use a professional analytics tracking system which I pay for every month- it tracks Ip address ,Ip provider, organization, location, time, country, with an integrated Google maps that allows me to see which block the material entered the net.Writing offensive comments from your work computer is not a great idea- not with the angry one.Not if you are��
I never use the analytics system for individuals unless the comments are ghastly. Then I delete the comments and don’t use the analytics system for individuals.
Akismet for spam will catch and has caught multiple attempts at malware linking and spamming.It will do so.
Anonymous comments are not anonymous as explained above.

A blog on Decision Stats should quote political statements only when accompanied with statistics. or better still nothing but the statistics.So it will be.

**They will be posted on* www.iwannacrib.com

Interview- BI Dashboards dMINE Sanjay Patel

If you have ever felt frustrated in knowing business metrics in your or your client organization, negotiated with a host of either legacy applications that don’t talk to each other or good solutions that cost more than the benefit they bring � a young man from India has a solution for you. With a total implementation time range of 1-6 weeks and costs to as low as 10,000 USD for Enterprise WIDE implementation , Dmine promises to shake things up. Here is an interview with the co founder of this startup.

Ajay- Describe your career journey. What advice would you give to new entrepreneurs in this recession.

Sanjay- Geared with an M. Tech from BITS Pilani, and MBA from Jamnalal Bajaj Institute of Management, Mumbai, in 2000 I teamed up with Praveen Wicliff and ventured to start our own Product company with a focus to deliver critical enterprise solutions and the company has now grown to a strong team of 100 innovative minds. My current mission is to make Icicle a strong leader in business driven Software Products across all segments with the best of the delivery capabilities.
Recession is a trying time for most people and this is one phase which brings out the best in everyone as most of the innovative solutions are floated during this phase. My only suggestion would be to move from emotional connect with customers to direct tangible benefits, stay focussed on cash flow, aim high, set your goals & targets and never give up on any of these. Even in the darkest moments, find the faith to keep going.

Ajay- One more Dashboard Solution. How is dMine different from it’s competitors. What are the principal competitors.

Sanjay- If you just plainly look at dMine you would see that dMine is just another Dashboard solution but what makes dMine different from all other competitive products is its intuitiveness and User friendly features which help even the Business users to use the product most effectively.

dMine is positioned as a product for Business Users and not for IT team. Unlike other Dashboard or BI products dMine can create Dashboards in just 3 easy steps:

1- IT team connects to Data-sources & Creates Business Views

2- Users can create Dashboards & Charts with dMine’s Intuitive interface

3- Users can share Dashboards & schedule Emails in PDF or PPT

The potent combination of best-of-class looking Comprehensive Graphical and Analytical reports, Easier representation and Interpretation of Key Business Data, Integrates data from multiple systems on a single chart and / or Dashboard for real-time Analysis, all these, with minimized IT overheads is a unique proposition from us. See dMine-in-action on www.dminebi.com and you will know the difference.

Ajay- What is the area where dMine would not be suited for dashboards.Suppose I have data for 200,000 rows x 40 columns – would dMine work for me .

Sanjay-
dMine is positioned as a pure Dashboard product that does not implement a complete BI stack which requires to work on Transactional Data to create cubes and universe.

We look beyond Data warehouses and Datamarts and emphasize on summarized data to deliver key business performance metrics with high focus on Data Visualization.

The idea here is to target the Business Executives who would see these Dashboards and they are not interested in Transactional Data but the overall performance hence the summarized Data.

Here the summarized Data could be in the form of any RDBMS Database, Flat Files, Spreadsheets, Analytics output, Cubes or Universe. We Support almost 16 Database vendors in the market starting from as small as MS Access to as big as DB2.

Ajay- What is the pricing strategy of dMine . Any other products or complements that you are thing about. Name some customer case studies or big wins.

Sanjay- dMine pricing strategy is very simple and is based keeping in mind that the product can be used by customers in the SME/SMB segment or even at the Enterprise level.

Currently we are offering dMine in two forms

firstly On-Premise and
second one as a Hosted service.

In an on-premise version, dMine has a Product License fee and per user license fees issued separately.

At additional cost you can have loads of Add-on goodies catering to various needs of the customers. All the cost mentioned above are just one time.

The Hosted version, is on a monthly subscription model where the cost is decided based on the various parameters like the usage of Bandwidth, Server configuration, Disk space etc. Very soon we will be enabled to the Amazon cloud service.

Typically for an on-premise version a smaller implementation just at a corporate level with Dashboard access to only few top executives would cost anywhere between 10,000 USD to 16,000 USD plus the implementation cost which is on actuals plus Applicable taxes.

For larger implementation like for e.g in BFSI segment where you need to roll-out user licenses to all the branch managers in addition to the top executives the total User licenses goes to a few 100 licenses. Usually the implementation period is as low as a week and not more than 6 weeks.

Few of the Customers using dMine are some of the Marketing Analytics companies that use the product for submitting the final report of their Marketing / Customer Analysis to their end customers.

Case Study � Summary

We recently implemented dMine at a leading FM Radio channel to monitor the performance across its radio stations spread all over India.

The client being a major player in the media and communication space has to constantly monitor all their stations for their entire operations like Revenues from sponsors, Peak and Non-Peak time Inventory & Sales, Market Share & Channel Ranking, P&L, Forecasts and other critical informations.

Currently the client uses multiple Applications for supporting these business functions which capture data in different databases and sources. The MIS-reports were manually created by extracting data from multiple sources in spreadsheets. These spreadsheets are distributed among the management, with a turn around time of about 15 days.

The dMine solution implemented, collates information from multiple data sources like ERP and Sales systems including lots of Spreadsheets. More than 70 Metrics (KPI’s) and Analytics are defined for the Client and now the management has access to these information whenever they require.

All these metrics are identified as critical and are categorized under 5 Dashboards – Organizational KPI, Financial Dashboard, Sales Dashboard, Market Share
& Metrics Dashboard and Operational Dashboard. The Metrics are parameterized and drill-downs allows the management to get the source of issue/problem rapidly.

The implementation of dMine Business Dashboard product helped the client in effectively monitoring business operations, KPIs, and organizational performance.Making actionable and Real-time information available on-demand for the decision Makers and Operational Managers, has also helped in taking any timely critical business decisions, all these while minimizing IT overhead cost. The short implementation time lines also allows the users to see the benefits quite quickly and achieve the ROI within 1 to 3 months time. The detailed case study is readily available for your reference at our website www.dminebi.com .

Ajay- Do you read or write blogs. What do you think about the Web 2.0 paradigm for social and community marketing.

Sanjay –I do read and write lots of blogs and am myself a member with quite a few groups that share interest in the virtual community. Web 2.0 provides a platform of many-to-many communications and in its social sense is based on the principles of collaboration & sharing, information & content putting social interaction at heart of it all.

A recent study by Fox Interactive Media reveals that 40% of social network users rely on social media outlets to learn more about brands and products. Whether you�re a freelancer promoting your own brand or part of a company, social media marketing is an essential component of an integrated campaign. If you are looking to startup new business, launch a new product & services or even expand your presence you cannot miss-out on the eMarketing process focusing on three prong strategy i.e. social networking sites, your own website, and the blogosphere these will help empower your brand and positively convey your message.

Ajay � Sanjay Patel is an experienced entrepreneur with Icicle Technologies and the Dmine dashboard is currently winning rave reviews ( see http://www.dminebi.com/ibm-nominates-icicle-as-isv-on-ibm-smart-business-platform/ ) . Here�s wishing luck to Mr Patel for the summarized data dashboard that can be a game changer at www.dminebi.com.