Interview Thomas C. Redman Author Data Driven

Here is an interview with Tom Redman, author of Data Driven. Among the first to recognize the need for high-quality data in the information age, Dr. Redman established the AT&T Bell Laboratories Data Quality Lab in 1987 and led it until 1995. He is the author of four books, two patents and leads his own consulting group. In many respects the “Data Doc’ as his nickname is- is also the father of Data Quality Evangelism.

tom redman

Ajay- Describe your career as a science student to an author of science and strategy books.
Redman: I took the usual biology, chemistry, and physics classes in college.  And I worked closely with oceanographers in graduate school.  More importantly, I learned directly from two masters.  First, was Dr. Basu, who was at Florida State when I was.  He thought more deeply and clearly about the nature of data and what we can learn from them than anyone I’ve met since.  And second is people in the Bell Labs’ community who were passionate about making communications better. What I learned there was you don’t always need “scientific proof” to mover forward.


Ajay- What kind of bailout do you think the Government can give to the importance of science education in this country.

Redman: I don’t think the government should bail science education per se. Science departments should compete for students just like the English and anthropology departments do.  At the same time, I do think the government should support some audacious goals, such as slowing global warming or energy independence.  These could well have the effect of increasing demand for scientists and science education.

Ajay- Describe your motivations for writing your book Data Driven-Profiting from your most important business asset.

Redman: Frankly I was frustrated.  I’ve spent the last twenty years on data quality and organizations that improve gain enormous benefit.  But so few do.  I set out to figure out why that was and what to do about it.

Ajay- What can various segments of readers learn from this book-
a college student, a manager, a CTO, a financial investor and a business intelligence vendor.

Redman: I narrowed my focus to the business leader and I want him or her to take away three points.  First, data should be managed as aggressively and professionally as your other assets.  Second, they are unlike other assets in some really important ways and you’ll have to learn how to manage them.  Third, improving quality is a great place to start.

Ajay- Garbage in Garbage out- How much money and time do you believe is given to data quality in data projects.

Redman:   By this I assume you mean data warehouse, BI, and other tech projects.  And the answer is “not near enough.”  And it shows in the low success rate of those projects.

Ajay-Consider a hypothetical scenario- Instead of creating and selling fancy algorithms , a business intelligence vendor uses simple Pareto principle to focus on data quality and design during data projects. How successful do you think that would be?

Redman: I can’t speak to the market, but I do know that if organizations are loaded with problems and opportunities.  They could make great progress on most important ones if could clearly state the problem and bring high-quality data and simple techniques to bear.  But there are a few that require high-powered algorithms.  Unfortunately those require high-quality data as well.

Ajay- How and when did you first earn the nickname “Data Doc”. Who gave it to you and would you rather be known by some other names.

Redman: One of my clients started calling me that about a dozen years ago.  But I felt uncomfortable and didn’t put it on my business card until about five years ago.  I’ve grown to really like it.

Ajay- The pioneering work at AT & T Bell laboratories and at Palo Alto laboratory- who do you think are the 21st century successors of these laboratories. Do you think lab work has become too commercialized even in respected laboratories like Microsoft Research and Google’s research in mathematics.

Redman: I don’t know.  It may be that the circumstances of the 20th century were conducive to such labs and they’ll never happen again.  You have to remember two things about Bell Labs.  First, was the cross-fertilization that stemmed from having leading-edge work in dozens of areas.  Second, the goal is not just invention, but innovation, the end-to-end process which starts with invention and ends with products in the market.  AT&T, Bell Labs’ parent, was quite good at turning invention to product.  These points lead me to think that the commercial aspect of laboratory work is so much the better.

Ajay-What does ” The Data Doc” do to relax and maintain a work life balance. How important do you think is work-life balance for creative people and researchers.

Redman: I think everyone needs a balance, not just creative people.  Two things have made this easier for me.  First, I like what I do.  A lot of days it is hard to distinguish “work” from “play.”  Second is my bride of thirty-three years, Nancy.  She doesn’t let me go overboard too often.

Biography-

Dr. Thomas C. Redman is President of Navesink Consulting Group, based in Little Silver, NJ.  Known by many as “the Data Doc” (though “Tom” works too), Dr. Redman was the first to extend quality principles to data and information.  By advancing the body of knowledge, his innovations have raised the standard of data quality in today’s information-based economy.

Dr. Redman conceived the Data Quality Lab at AT&T Bell Laboratories in 1987 and led it until 1995.  There he and his team developed the first methods for improving data quality and applied them to important business problems, saving AT&T tens of millions of dollars. He started Navesink Consulting Group in 1996 to help other organizations improve their data, while simultaneously lowering operating costs, increasing revenues, and improving customer satisfaction and business relationships.

Since then – armed with proven, repeatable tools, techniques and practical advice – Dr. Redman has helped clients in fields ranging from telecommunications, financial services, and dot coms, to logistics, consumer goods, and government agencies. His work has helped organizations understand the importance of high-quality data, start their data quality programs, and also save millions of dollars per year.

Dr. Redman holds a Ph.D. in statistics from Florida State University.  He is an internationally renowned lecturer and the author of numerous papers, including “Data Quality for Competitive Advantage” (Sloan Management Review, Winter 1995) and “Data as a Resource: Properties, Implications, and Prescriptions” (Sloan Management Review, Fall 1998). He has written four books: Data Driven (Harvard Business School Press, 2008), Data Quality: The Field Guide (Butterworth-Heinemann, 2001), Data Quality for the Information Age (Artech, 1996) and Data Quality: Management and Technology (Bantam, 1992). He was also invited to contribute two chapters to Juran’s Quality Handbook, Fifth Edition (McGraw Hill, 1999). Dr. Redman holds two patents.

About Navesink Consulting Group (http://www.dataqualitysolutions.com/ )

Navesink Consulting Group was formed in 1996 and was the first company to focus on data quality.  Led by Dr. Thomas Redman, “the Data Doc” and former AT&T Bell Labs director, we have helped clients understand the importance of high-quality data, start their data quality programs, and save millions of dollars per year.

Our approach is not a cobbling together of ill-fitting ideas and assertions – it is based on rigorous scientific principles that have been field-tested in many industries, including financial services (see more under “Our clients”).  We offer no silver bullets; we don’t even offer shortcuts. Improving data quality is hard work.

But with a dedicated effort, you should expect order-of-magnitude improvements and, as a direct result, an enormous boost in your ability to manage risk, steer a course through the crisis, and get back on the growth curve.

Ultimately, Navesink Consulting brings tangible, sustainable improvement in your business performance as a result of superior quality data.

Interview Stephen Baker Author The Numerati

Here is an interview with Stephen Baker, the author of the famous and remarkable book The Numerati. Stephen is the senior editor at Businessweek and his remarkable book made the world sit up and pay attention because for the first time, anyone wrote of the increasingly quant driven lives we lead thanks to the internet and the analytical brains that power the stimulus, design and targeting of it. Increasing amounts of data is collected about consumers than at any previous point of time in human history and the number crunchers or the quant jocks are the ones who increasingly help with decision making and decision management. Steve calls these people “The Numerati” or the new math people who help shape our lives.

There will always be lawyers and financiers who make loads of money. But they will have quantitative experts on their teams- Stephen L Baker

Ajay- Describe your career journey from high school to a technology writer to author of The Numerati.

Steve- I was always interested in history and in literature, and in college I fell in love with Spanish. So after college, I moved to Ecuador, taught English, and wrote fiction. I saw early on that I wasn’t going to be able to make a living with fiction. So I went into journalism. My goal was to become a correspondent in Latin America. Through my 20s, I worked in Vermont, Madrid, Argentina, Venezuela, Washington DC, and El Paso, Texas. And I finally got the job I was looking for, bureau chief for the Mexico bureau of BusinessWeek magazine.

After Mexico, my family and I moved to Pittsburgh. It appeared that the magazine was losing interest in heavy industry in the mid-90s, so I began to write about software and robotics coming out of at Carnegie Mellon University. That was my transition into technology. A year later, BusinessWeek offered me a job covering technology in Europe. I moved with my family to Paris, where we lived for four years. I focused largely on mobile communications. It seemed to me that the combination of mobility and the Internet would fundamentally change communications.

I returned to New York in 2002. I focused on big picture stories. One day in 2005, I proposed a story about the decline of the U.S. technology industry. I argued that we were behind in wireless and in broadband, we were graduating fewer scientists and engineers than other great powers, especially in Asia. One editor pointed out that mathematics was critical for these competitive issues. The editor in chief, Steve Adler, called for a cover story on math, and he assigned it to me. I didn’t know much about math at the time, and I still don’t. But this gave me the chance to dive into the world of data analysis. I wrote a cover story, Math will Rock Your Business, and later got the contract to write the Numerati.
Ajay- How do you think the government can motivate more American students to science careers?

Steve- I think focusing on the science that kids find cool–robotics, space and ocean exploration, would help. Funding basic research would be useful. But I don’t think it’s entirely a governmental issue. Parents, companies, universities, they all have to participate.
Ajay- What are the top  tips you would give to aspiring technology writers and bloggers (like myself)?

Steve-
1) Learn about non tech subjects, such as history, literature, art and psychology
2) Work on writing clearly for non experts. Avoid jargon.
3) Do reporting
4) Do more reporting

Ajay-The Numerati portrays a math elite which breaks the stereotype of the lonely, nerdy geek. How important do you think is that common people be more educated in math so they are more aware of marketing operations and credit offers?

Steve- I think it’s important for common people, as you call them, to understand basic statistics. More and more of our lives are going to be analyzed and communicated to us statistically. Those who do not understand this will not know to ask the right questions, and will be easily fooled. This is also true within companies. CEOs can be fooled by numbers, just like anyone else.
Ajay- Asia delivers a disproportionate number of science graduates. Yet one generation ago American and European heritage scientists made the trip to the moon with very basic computers. As our lives get increasingly shaped by the Numerati, how important are geo-cultural influences in its membership?

Steve- Most of the Numerati I met in the United States were born outside the U.S. The US has long relied on foreign brains, especially for its technology industry. As the Numerati study people’s lives, the quantitative experts will increasingly need to work closely with linguists, anthropologists, and psychologists. And they’ll need to understand different global cultures and languages. In this sense, the international nature of the Numerati is an advantage.
Ajay- Do you think the shift in money and influence from lawyers and financiers to scientists and mathematicians is temporary or is it here to stay?

Steve-I think it’s here to stay. There will always be lawyers and financiers who make loads of money. But they will have quantitative experts on their teams.

Ajay- What influenced your decision to be associated with Predictive Analytics world?

Steve- I had the privilege of interviewing Eric Siegel as I was researching the book. We’ve kept in touch since then. I think he’s very bright and does excellent work.
Ajay- What does Stephen Baker do when not writing books or articles or observing the world go around him?

Steve- I like to ride bicycles, I like to travel. I love Spanish and French and baseball and music

Biography-

Stephen L. Baker is the author of The Numerati and a senior writer at BusinessWeek, covering technology. Previously he was a Paris correspondent. Baker joined BusinessWeek in March, 1987, as manager of the Mexico City bureau, where he was responsible for covering Mexico and Latin America. He was named Pittsburgh bureau manager in 1992. Before BusinessWeek, Baker was a reporter for the El Paso Herald-Post. Prior to that, he was chief economic reporter for The Daily Journal in Caracas, Venezuela. Baker holds a bachelor’s degree from the University of Wisconsin and a master’s from the Columbia University Graduate School of Journalism.

You can read more about the Numerati at http://thenumerati.net/index.cfm?catID=18 Stephen L Baker is the keynote speaker at Predictive Analytics World and you can check the details here http://www.predictiveanalyticsworld.com/register.php if you want to listen to  him at the event.

You can follow Steve on twitter at http://twitter.com/stevebaker and follow his blog here http://www.businessweek.com/the_thread/blogspotting/

The Age of the Unthinkable- Book Review

The Age of the Unthinkable is a thought provoking book written by Joshua Cooper Ramos and published by Little, Brown. Anyone who has been surprised by change or the speed of change in recent months in matters economic, political or technology should have a look in at least once of this beautiful, with remarkable case studies  painstaking culled and gathered from all parts of the world and cultures.

The book has an easy to read style, with real life incidents with which we can associate with. It look at creative innovation as a process which is analogous to sand particles piling on to one another, and sudden change being the point at which the sand pile has a flattening avalanche. It learns from examples of how highly centralized systems in Communism collapsed while highly autonomous organizations like Hezbollah flourished as they kept learning and adapting in the face of a bigger enemy. or how creative designers at Nintendo changed the paradigm of video games to invent the Wii Fit to help make video games that help people stay fit, which was un-thought of earlier using inexpensive video chips. And how a little known company in Brazil cut costs by empowering bottoms up cost cutting than top down cost thinking.

The book talks of things like mashup and the speed at which change is unleashed at us. Lastly it offers us lessons in which leaders may help embrace change and thus help themselves or they are changed inevitably by external forces.

Change being a process as sure as death and taxes- it compares and contrast people who change willingly internally to people who are changed externally. An entertaining and informative book- I recommend it (see Amazon link to the right margin) for anyone and everyone who have had a ” Oh, we are  idiots” moment as they were surprised by rising taxes for bailouts, powerful armies that failed to keep them safe or big cash rich corporations that failed to keep them employed.

For the technology or scientifically trained people, this book would be an eye opener.

41k8ugav4FL._SL75_

Interview Jill Dyche Baseline Consulting

Here is an interview with Jill Dyche, co-Founder Baseline Consulting and one of the best Business Intelligence consultants and analysts. Her writing is read by huge portion of the industry and has influenced many paradigms.She is also Author of e-Data, The CRM Handbook, and Customer Data Integration: Reaching a Single Version of the Truth.

BI tools are not recommended when they’re the first topic in a BI discussion.

Jill Dyche, Baseline Consulting

Ajay- What approximate Return of Investment would you give to various vendors within Business Intelligence?

Jill- You don’t kid around do you, Ajay? In general the answer has everything to do with the problem BI is solving for a company. For instance, we’re working on deploying operational BI at a retailer right now. This new program is giving people in the stores more power to make decisions about promotions and in-store events. The projected ROI is $300,000 per store per year—and the retailer has over 1000 stores. In another example, we’re working with an HMO client on a master data management project that helps it reconcile patient data across hospitals, clinics, pharmacies, and home health care. The ROI could be life-saving. So, as they say in the Visa commercials: Priceless.

Ajay- What is impact of third party cloud storage and processing do you think will be there on Business Intelligence consulting?

Jill- There’s a lot of buzz about cloud storage for BI, most of it is coming from the VC community at this point, not from our clients. The trouble with that is that BI systems really need control over their storage. There are companies out there—check out a product called RainStor—that do BI storage in the cloud very well, and are optimized for it. But most “cloud” environments geared to BI are really just hosted offerings that provide clients with infrastructure and processing resources that they don’t have in-house.  Where the cloud really has benefits is when it provides significant processing power to companies that can’t build it easily themselves.

Ajay- What are the top writing tips would you give to young struggling business bloggers especially in this recession.

Jill- I’d advise bloggers to write like they talk, a standard admonishment by many a professor of Business Writing. So much of today’s business writing—especially in blogs—is stilted, overly-formal, and pedantic. I don’t care if your grammar is accurate; if your writing sounds like the Monroe Doctrine, no one will read it. (Just give me one quote from the Monroe Doctrine. See what I mean?) Don’t use the word “leverage” when you can use the word “use.” Be genuine and conversational. And avoid clichés like the plague.

Ajay-  How would you convince young people especially women to join more science careers. Describe your own career journey.

Jill- As much as we need those role models in science, high-tech, and math careers, I’d tell them to only embrace it if they really love it. My career path to high-tech was unconventional and unintentional. I started as a technical writer specializing in relational databases just as they were getting hot. One thing I know for sure is if you want to learn about something interesting, be willing to roll up your sleeves and work with it. My technical writing about databases, and then data warehouses, led to some pretty interesting client work.

Sure I’ve coded SQL in my career, and optimized some pretty hairy WHERE clauses. But the bigger issue is applying that work to business problems. Actually I’m grateful that I wasn’t a very good programmer. I’d still be waiting for that infinite loop to finish running.

Ajay- What are the areas within an enterprise where implementation of BI leads to the most gains. And when are BI tools not recommended?

Jill- The best opportunities for BI are for supporting business growth. And that typically means BI used by sales and marketing. Who’s the next customer and what will they buy? It’s answers to questions like these that can set a company apart competitively and contribute to both the top and bottom lines.

Not to be too heretical, but to answer your second question: BI tools are not recommended when they’re the first topic in a BI discussion. We’ve had several “Don’t go into the light” conversations with clients lately where they are prematurely looking at BI tools rather than examining their overall BI readiness. Companies need to be honest about their development processes, existing skill sets, and their data and platform infrastructures before they start phoning up data visualization vendors. Unfortunately, many people engage BI software vendors way before they’re ready.

Ajay- You and your partner Evan wrote what was really the first book on Master Data Management. But you’d been in the BI and data warehousing world before that. Why MDM?

Jill- We just kept watching what our clients couldn’t pull off with their data warehouses. We saw the effort they were going through to enforce business rules through ETL, and what they were trying to do to match records across different source systems. We also saw the amount of manual effort that went into things like handling survivor records, which leads to a series of conversations about data ownership.

Our book (Customer Data Integration: Reaching a Single Version of the Truth, Wiley) has as much to do with data management and data governance as it does with CDI and MDM. As Evan recently said in his presentation at the TDWI MDM Insight event, “You can’t master your data until you manage your data.” We really believe that, and our clients are starting to put it into practice too.

Ajay- Why did you and Evan choose to focus on customer master data (CDI) rather than a more general book on MDM?

Jill- There were two reasons. The first one was because other master data domains like product and location have their own unique sets of definitions and rules. Even though these domains also need MDM, they’re different and the details around implementing them and choosing vendor products to enable them are different. The second reason was that the vast majority of our clients started their MDM programs with customer data. One of Baseline’s longest legacies is enabling the proverbial “360-degree view” of customers. It’s what we knew.

Ajay- What’s surprised you most about your CDI/MDM clients?

Jill- The extent to which they use CDI and MDM as the context for bringing IT and the business closer together. You’d think BI would be ideal for that, and it is. But it’s interesting how MDM lets companies strip back a lot of the tool discussions and just focus on the raw conversations about definitions and rules for business data. Business people get why data is so important, and IT can help guide them in conversations about streamlining data quality and management. Companies like Dell have used MDM for nothing less than business alignment.

Ajay- Any plan to visit India and China for giving lectures?

Jill- I just turned down a trip to China this fall because I had a schedule conflict, which I’m really bummed about. Far as India is concerned, nothing yet but if you’re looking for houseguests let me know.(Ajay- sure I have a big brand new house just ready- and if I visit USA may I be a house guest too?)

About Jill Dyche-

Jill blogs at http://www.jilldyche.com/. where she takes the perpetual challenge of business-IT alignment head on in her trenchant, irreverent style.

Jill Dyché is a partner and co-founder of Baseline Consulting. Her role at Baseline is a combination of best-practice expert, industry gadfly, key client advisor, and all-around thought leader. She is responsible for key client strategies and market analysis in the areas of data governance, business intelligence, master data management, and customer relationship management. Jill counsels boards of directors on the strategic importance of their information investments.

Author

Jill is the author of three books on the business value of IT. Jill’s first book, e-Data (Addison Wesley, 2000) has been published in eight languages. She is a contributor to Impossible Data Warehouse Situations: Solutions from the Experts (Addison Wesley, 2002), and her book, The CRM Handbook (Addison Wesley, 2002), is the bestseller on the topic.

Jill’s work has been featured in major publications such as Computerworld, Information Week, CIO Magazine, the Wall Street Journal, the Chicago Tribune and Newsweek.com. Jill’s latest book, Customer Data Integration (John Wiley and Sons, 2006) was co-authored with Baseline partner Evan Levy, and shows the business breakthroughs achieved with integrated customer data.

Industry Expert

Jill is a featured speaker at industry conferences, university programs, and vendor events. She serves as a judge for several IT best practice awards. She is a member of the Society of Information Management and Women in Technology, a faculty member of TDWI, and serves as a co-chair for the MDM Insight conference. Jill is a columnist for DM Review, and a blogger for BeyeNETWORK and Baseline Consulting.

R Graphics

A great book for R graphics is here. Its especially useful for people who are new into R and or using graphical function primarily.

http://www.stat.auckland.ac.nz/~paul/RGraphics/chapter2.html

image

This is a good textbook till the new edition November 2008 release of Bob Munchien’s R for SAS and SPSS Users (http://www.springer.com/statistics/computational/book/978-0-387-09417-5 )

The existing free copy is at http://oit.utk.edu/scc/RforSAS&SPSSusers.pdf

 

image

Poetry Books :Online Orderable Anywhere

Using Self Publishing On Demand Website , www.pothi.com (an Indian start up and pioneer in the on demand publishing space) ………………..here are both my poetry books.

You can order them online (including international shipping at

http://pothi.com/pothi/book/ajay-ohri-corporate-poetry

and http://pothi.com/pothi/book/ajay-ohri-case-i-dont-see-you-again

Learning R for SAS and SPSS Users

So you decided to cut down on your Statistical software expenses and decided to get R.

but the problem is you know SAS /SPSS and you need to learn R fast enough to justify switching over …….

the ideal book for you is  http://oit.utk.edu/scc/RforSAS&SPSSusers.pdf

Thanks to the guys who pointed me here. Its a really easy book, you have the SAS Syntax, the corresponding SPSS Syntax and the R Syntax.

 That’s useful for learners in R who got projects to execute, and need to learn either SPSS or R or even switch from SPSS to SAS.