Tag: teradata
Teradata updates Teradata-R
The Teradata add-on package for R
teradataR is a package or library that allows R users to easily connect to Teradata, establish data frames (R data formats) to Teradata and to call in-database analytic functions within Teradata. This allows R users to work within their R console environment while leveraging the in-database functions developed with Teradata Warehouse Miner. This package provides 44 different analytical functions and an additional 20 data connection and R infrastructure functions. In addition, we’ve added a function that will list the stored procedures within Teradata provide the capability to call functions from R.
- 20 Functions to enable R infrastructure to operate with Teradata
- tdConnect – Connect to Teradata via ODBC
- Td.data.frame – Establish data frame connections to a Teradata table
- 44 in-database analytical functions callable from R. Sample of the functions include:
- Descriptive statistics: Overlap, histogram, frequency, statistics, matrix functions, and values analysis
- Reorganization functions: join, merge and samples
- Transformations: bincode, recode, rescale, sigmoid, zscore and null replacement
- K-Means clustering and Score K-Means
- Statistical tests: ks, dagostino.pearson, shapiro.wilk, bionomial, and wilcoxon
- R language features nrow, ncol, min, max, summary, as.dataframe, and dim
- Tool and R functions that allow users to create their own custom analytic functions that’s callable by R.
- Teradata Warehouse Miner can capture any analytic stream including UDFs and create a stored procedure
- Analytic process to create new derived predictive variables can be captured as a stored procedure.
- Entire process to create or update an analytical data set can be captured as a stored procedure.
- R function can list all the stored procedures within Teradata.
- R function can call a stored procedure that runs in-database
TeradataR allows R users to leverage all the benefits of in-database processing with Teradata:
- Eliminate data movement from Teradata to the R framework for key data intensive tasks.
- Leverage the speed of Teradata database’s parallel processing to run analytics against big data.
- Ability to operate within the R console environment.
- Embed your frequently performed tasks to run in-database.
- R and TeradataR are free downloads.
Source- http://developer.teradata.com/applications/articles/in-database-analytics-with-teradata-r
This package allows users of R to interact with a Teradata database. R is an open source language for statistical computing and graphics. R provides a wide variety of statistical (linear and nonlinear modeling, classical statistical tests, time-series analysis, classification, clustering) and graphical techniques, and is highly extensible. Users can use many statistical functions directly against the Teradata system without having to extract the data into memory.
Enhancements included with this new 1.0.1 release include:
- teradataR User Guide
- addition of Mac OS X Package
- addition of Red Hat Linux Package (added 2/23/12)
- summary has been enhanced to run faster
- JDBC support added to allow Windows or Mac users to run the package with JDBC
- td.data.frame enhanced to allow support for manipulation to add columns and expressions
- td.data.frame enhanced to use Teradata 14.0 Fastpath Transform Functions (see Appendix B)
- td.tapply function added to apply a select group of functions to columns of an array
From-http://downloads.teradata.com/download/applications/teradata-r
and
A new R package for Red Hat Linux has been added to the teradataR 1.0.1 release. This new package provides the same functionality as in the previously released Windows and Mac OS X packages, but is built for Red Hat Linux. This version was built and tested on Red Hat Linux 6.2 32-bit. (The R version for Red Hat Linux is 2.14.1)
Installing this package is the same as any normal R package; just extract it into your R library area, or use the install.packages
command with the file path.
from- http://developer.teradata.com/tag/r
and
With plenty of prolific and enthusiastic developers, the number of packages for R is expected to grow tremendously. Statisticians and analysts using these packages will find innovative ways to use data to answer their research and business questions. And as organizations become more willing to rely on open-source software for mission-critical tasks, R is poised to become an essential tool for analyzing our complex world.
Source-http://www.teradatamagazine.com/v09n03/Connections/R-you-ready/
From the user guide-
http://downloads.teradata.com/download/applications/teradata-r
teradataR allows R users to easily connect to Teradata, establish td data frames (virtual R data frames) to
Teradata and to call in-database analytic functions within Teradata. This allows R users to work within their R
console environment while leveraging the in-database functions
A Function List
teradataR-package Allow access to Teradata via R
as.data.frame.td.data.frame Convert td data frame to a data frame
as.td.data.frame Coerce to a td data frame
dim.td.data.frame Dimensions of a td data frame
hist.td.data.frame Histograms
Is.td.data.frame Is an Object a Teradata Data Frame
Is.td.expression Is an Object a Teradata Expression
mean.td.data.frame Arithmetic Mean
median.td.data.frame Median Value
min.td.data.frame Minima
predict.kmeans Kmeans Model Prediction
print.td.data.frame Show contents of a td data frame
sum.td.data.frame Sum of column
summary.td.data.frame Summary of Teradata Data Frame
Td.bincode Create Table of Bincode Values
Td.binomial Binomial Test
Td.binomialsign Binomial Sign Test
Td.call.sp Locate and call stored procedure
Td.cor Correlation Matrix
Td.cov Covariance Matrix
Td.dagostino.pearson D’Agostino Pearson Test
Td.data.frame Teradata Data Frames
Td.f.oneway One way F Test
Td.factanal Factor Analysis
Td.freq Frequency Analysis
Td.hist Histograms
Td.join Join Tables in Teradata
Td.kmeans K-Means Clustering
Td.ks Kolmogorov Smirnov Test
Td.lilliefors Lilliefors Test
Td.merge Merge Rows of Teradata Tables
Td.mode Mode Value of Column
Td.mwnkw Mann-Whitney/Kruskal Wallis Test
Td.nullreplace Replace Null Values
Td.overlap Overlap
Td.quantiles Quantile Values
Td.rank Rank
Td.recode Recode
Td.rescale Rescale Values of Column
Td.sample Sample Rows
Td.shapiro.wilk Shapiro Wilk
Td.sigmoid Sigmoid Transformation
Td.smirnov Smirnov Test
Td.solve Solve a system of equations
Td.stats General Statistics
Td.t.paired T Test Paired
Td.t.unpaired T Test Unpaired
Td.t.unpairedi T Test – Unpaired Indicator
Td.values Values
Td.wilcoxon Wilcoxon Test
Td.zscore Zscore Transformation
tdClose Close connection
tdConnect Connect to Teradata database
tdMetadataDB Set metadata database
tdQuery Query Teradata Database
teradataR Allow access to Teradata via R
[.td.data.frame Extract Teradata Data Frame
[<-.td.data.frame Replace value of Teradata Data Frame
Teradata Analytics
A recent announcement showing Teradata partnering with KXEN and Revolution Analytics for Teradata Analytics.
http://www.teradata.com/News-Releases/2012/Teradata-Expands-Integrated-Analytics-Portfolio/
The Latest in Open Source Emerging Software Technologies
Teradata provides customers with two additional open source technologies – “R” technology from Revolution Analytics for analytics and GeoServer technology for spatial data offered by the OpenGeo organization – both of which are able to leverage the power of Teradata in-database processing for faster, smarter answers to business questions.
In addition to the existing world-class analytic partners, Teradata supports the use of the evolving “R” technology, an open source language for statistical computing and graphics. “R” technology is gaining popularity with data scientists who are exploiting its new and innovative capabilities, which are not readily available. The enhanced “R add-on for Teradata” has a 50 percent performance improvement, it is easier to use, and its capabilities support large data analytics. Users can quickly profile, explore, and analyze larger quantities of data directly in the Teradata Database to deliver faster answers by leveraging embedded analytics.
Teradata has partnered with Revolution Analytics, the leading commercial provider of “R” technology, because of customer interest in high-performing R applications that deliver superior performance for large-scale data. “Our innovative customers understand that big data analytics takes a smart approach to the entire infrastructure and we will enable them to differentiate their business in a cost-effective way,” said David Rich, chief executive officer, Revolution Analytics. “We are excited to partner with Teradata, because we see great affinity between Teradata and Revolution Analytics – we embrace parallel computing and the high performance offered by multi-core and multi-processor hardware.”
and
The Teradata Data Lab empowers business users and leading analytic partners to start building new analytics in less than five minutes, as compared to waiting several weeks for the IT department’s assistance.
“The Data Lab within the Teradata database provides the perfect foundation to enable self-service predictive analytics with KXEN InfiniteInsight,” said John Ball, chief executive officer, KXEN. “Teradata technologies, combined with KXEN’s automated modeling capabilities and in-database scoring, put the power of predictive analytics and data mining directly into the hands of business users. This powerful combination helps our joint customers accelerate insight by delivering top-quality models in orders of magnitude faster than traditional approaches.”
Read more at
http://www.sacbee.com/2012/03/06/4315500/teradata-expands-integrated-analytics.html
Interview Eberhard Miethke and Dr. Mamdouh Refaat, Angoss Software
Here is an interview with Eberhard Miethke and Dr. Mamdouh Refaat, of Angoss Software. Angoss is a global leader in delivering business intelligence software and predictive analytics solutions that help businesses capitalize on their data by uncovering new opportunities to increase sales and profitability and to reduce risk.
Ajay- Describe your personal journey in software. How can we guide young students to pursue more useful software development than just gaming applications.
Mamdouh- I started using computers long time ago when they were programmed using punched cards! First in Fortran, then C, later C++, and then the rest. Computers and software were viewed as technical/engineering tools, and that’s why we can still see the heavy technical orientation of command languages such as Unix shells and even in the windows Command shell. However, with the introduction of database systems and Microsoft office apps, it was clear that business will be the primary user and field of application for software. My personal trip in software started with scientific applications, then business and database systems, and finally statistical software – which you can think of it as returning to the more scientific orientation. However, with the wide acceptance of businesses of the application of statistical methods in different fields such as marketing and risk management, it is a fast growing field that in need of a lot of innovation.
Ajay – Angoss makes multiple data mining and analytics products. could you please introduce us to your product portfolio and what specific data analytics need they serve.
a- Attached please find our main product flyers for KnowledgeSTUDIO and KnowledgeSEEKER. We have a 3rd product called “strategy builder” which is an add-on to the decision tree modules. This is also described in the flyer.
(see- Angoss Knowledge Studio Product Guide April2011 and http://www.scribd.com/doc/63176430/Angoss-Knowledge-Seeker-Product-Guide-April2011 )
Ajay- The trend in analytics is for big data and cloud computing- with hadoop enabling processing of massive data sets on scalable infrastructure. What are your plans for cloud computing, tablet based as well as mobile based computing.
a- This is an area where the plan is still being figured out in all organizations. The current explosion of data collected from mobile phones, text messages, and social websites will need radically new applications that can utilize the data from these sources. Current applications are based on the relational database paradigm designed in the 70’s through the 90’s of the 20th century.
But data sources are generating data in volumes and formats that are challenging this paradigm and will need a set of new tools and possibly programming languages to fit these needs. The cloud computing, tablet based and mobile computing (which are the same thing in my opinion, just different sizes of the device) are also two technologies that have not been explored in analytics yet.
The approach taken so far by most companies, including Angoss, is to rely on new xml-based standards to represent data structures for the particular models. In this case, it is the PMML (predictive modelling mark-up language) standard, in order to allow the interoperability between analytics applications. Standardizing on the representation of models is viewed as the first step in order to allow the implementation of these models to emerging platforms, being that the cloud or mobile, or social networking websites.
The second challenge cited above is the rapidly increasing size of the data to be analyzed. Angoss has already identified this challenge early on and is currently offering in-database analytics drivers for several database engines: Netezza, Teradata and SQL Server.
These drivers allow our analytics products to translate their routines into efficient SQL-based scripts that run in the database engine to exploit its performance as well as the powerful hardware on which it runs. Thus, instead of copying the data to a staging format for analytics, these drivers allow the data to be analyzed “in-place” within the database without moving it.
Thus offering performance, security and integrity. The performance is improved because of the use of the well tuned database engines running on powerful hardware.
Extra security is achieved by not copying the data to other platforms, which could be less secure. And finally, the integrity of the results are vastly improved by making sure that the results are always obtained by analyzing the up-to-date data residing in the database rather than an older copy of the data which could be obsolete by the time the analysis is concluded.
Ajay- What are the principal competing products to your offerings, and what makes your products special or differentiated in value to them (for each customer segment).
a- There are two major players in today’s market that we usually encounter as competitors, they are: SAS and IBM.
SAS offers a data mining workbench in the form of SAS Enterprise Miner, which is closely tied to SAS data mining methodology known as SEMMA.
On the other hand, IBM has recently acquired SPSS, which offered its Clementine data mining software. IBM has now rebranded Clementine as IBM SPSS Modeller.
In comparison to these products, our KnowledgeSTUDIO and KnowledgeSEEKER offer three main advantages: ease of use; affordability; and ease of integration into existing BI environments.
Angoss products were designed to look-and-feel-like popular Microsoft office applications. This makes the learning curve indeed very steep. Typically, an intermediate level analyst needs only 2-3 days of training to become proficient in the use of the software with all its advanced features.
Another important feature of Angoss software products is their integration with SAS/base product, and SQL-based database engines. All predictive models generated by Angoss can be automatically translated to SAS and SQL scripts. This allows the generation of scoring code for these common platforms. While the software interface simplifies all the tasks to allow business users to take advantage of the value added by predictive models, the software includes advanced options to allow experienced statisticians to fine-tune their models by adjusting all model parameters as needed.
In addition, Angoss offers a unique product called StrategyBuilder, which allows the analyst to add key performance indicators (KPI’s) to predictive models. KPI’s such as profitability, market share, and loyalty are usually required to be calculated in conjunction with any sales and marketing campaign. Therefore, StrategyBuilder was designed to integrate such KPI’s with the results of a predictive model in order to render the appropriate treatment for each customer segment. These results are all integrated into a deployment strategy that can also be translated into an execution code in SQL or SAS.
The above competitive features offered by the software products of Angoss is behind its success in serving over 4000 users from over 500 clients worldwide.
Ajay -Describe a major case study where using Angoss software helped save a big amount of revenue/costs by innovative data mining.
a-Rogers Telecommunications Inc. is one of the largest Canadian telecommunications providers, serving over 8.5 million customers and a revenue of 11.1 Billion Canadian Dollars (2009). In 2008, Rogers engaged Angoss in order to help with the problem of ballooning accounts receivable for a period of 18 months.
The problem was approached by improving the efficiency of the call centre serving the collections process by a set of predictive models. The first set of models were designed to find accounts likely to default ahead of time in order to take preventative measures. A second set of models were designed to optimize the call centre resources to focus on delinquent accounts likely to pay back most of the outstanding balance. Accounts that were identified as not likely to pack quickly were good candidates for “Early-out” treatment, by forwarding them directly to collection agencies. Angoss hosted Rogers’ data and provided on a regular interval the lists of accounts for each treatment to be deployed by the call centre dialler. As a result of this Rogers estimated an improvement of 10% of the collected sums.
Biography-
Mamdouh has been active in consulting, research, and training in various areas of information technology and software development for the last 20 years. He has worked on numerous projects with major organizations in North America and Europe in the areas of data mining, business analytics, business analysis, and engineering analysis. He has held several consulting positions for solution providers including Predict AG in Basel, Switzerland, and as ANGOSS Corp. Mamdouh is the Director of Professional services for EMEA region of ANGOSS Software. Mamdouh received his PhD in engineering from the University of Toronto and his MBA from the University of Leeds, UK.
Mamdouh is the author of:
"Credit Risk Scorecards: Development and Implmentation using SAS" "Data Preparation for Data Mining Using SAS", (The Morgan Kaufmann Series in Data Management Systems) (Paperback) and co-author of "Data Mining: Know it all",Morgan Kaufmann Eberhard Miethke works as a senior sales executive for Angoss
Angoss is a global leader in delivering business intelligence software and predictive analytics to businesses looking to improve performance across sales, marketing and risk. With a suite of desktop, client-server and in-database software products and Software-as-a-Service solutions, Angoss delivers powerful approaches to turn information into actionable business decisions and competitive advantage.
Angoss software products and solutions are user-friendly and agile, making predictive analytics accessible and easy to use.
New Community for Analytics- All Analytics
Here is a brand new community for analytics called allanalytics.com or http://www.allanalytics.com/
It seems to be managed by http://www.ubmtechweb.com/about/ who have a lot of other internet properties as well.One feature I disliked was the lengthy registration form which seems anarchic in these days of OAuth
Currently the community seems to be sponsored by SAS Institute, and it is basically a curated blog aggregator, like TeraData has SmartDataCollective , but much different than SAS Institutes’s affiliated sascommunity.org which is more for coders and consultants in SAS Language.
The community talks about analytics as a paradigm and way of getting business down. Have a look, notable names include Thomas C Redman and SAS’s very own Gary Cokins at http://www.allanalytics.com/bloggers.asp. The list of bloggers seems to be much lesser here, a kind of trade-off for high profile bloggers and thinkers. Worth a dekko or RSS subscription at least IMHO
Also see-
https://decisionstats.com/interview-thomas-c-redman-author-data-driven/
https://decisionstats.com/interview-gary-cokins-sas-institute/
#SAS 9.3 and #Rstats 2.13.1 Released
A bit early but the latest editions of both SAS and R were released last week.
SAS 9.3 is clearly a major release with multiple enhancements to make SAS both relevant and pertinent in enterprise software in the age of big data. Also many more R specific, JMP specific and partners like Teradata specific enhancements.
http://support.sas.com/software/93/index.html
Features
Data management
- Enhanced manageability for improved performance
- In-database processing (EL-T pushdown)
- Enhanced performance for loading oracle data
- New ET-L transforms
- Data access
Data quality
- SAS® Data Integration Server includes DataFlux® Data Management Platform for enhanced data quality
- Master Data Management (DataFlux® qMDM)
- Provides support for master hub of trusted entity data.
Analytics
- SAS® Enterprise Miner™
- New survival analysis predicts when an event will happen, not just if it will happen.
- New rate making capability for insurance predicts optimal insurance premium for individuals based on attributes known at application time.
- Time Series Data Mining node (experimental) applies data mining techniques to transactional, time-stamped data.
- Support Vector Machines node (experimental) provides a supervised machine learning method for prediction and classification.
- SAS® Forecast Server
- SAS Forecast Server is integrated with the SAP APO Demand Planning module to provide SAP users with access to a superior forecasting engine and automatic forecasting capabilities.
- SAS® Model Manager
- Seamless integration of R models with the ability to register and manage R models in SAS Model Manager.
- Ability to perform champion/challenger side-by-side comparisons between SAS and R models to see which model performs best for a specific need.
- SAS/OR® and SAS® Simulation Studio
- Optimization
- Simulation
- Automatic input distribution fitting using JMP with SAS Simulation Studio.
Text analytics
- SAS® Text Miner
- SAS® Enterprise Content Categorization
- SAS® Sentiment Analysis
Scalability and high-performance
- SAS® Analytics Accelerator for Teradata (new product)
- SAS® Grid Manager
LICENCE:
• No parts of R are now licensed solely under GPL-2. The licences for packages rpart and survival have been changed, which means that the licence terms for R as distributed are GPL-2 | GPL-3.
This is a maintenance release to consolidate various minor fixes to 2.13.0.
CHANGES IN R VERSION 2.13.1: NEW FEATURES: • iconv() no longer translates NA strings as "NA". • persp(box = TRUE) now warns if the surface extends outside the box (since occlusion for the box and axes is computed assuming the box is a bounding box). (PR#202.) • RShowDoc() can now display the licences shipped with R, e.g. RShowDoc("GPL-3"). • New wrapper function showNonASCIIfile() in package tools. • nobs() now has a "mle" method in package stats4. • trace() now deals correctly with S4 reference classes and corresponding reference methods (e.g., $trace()) have been added. • xz has been updated to 5.0.3 (very minor bugfix release). • tools::compactPDF() gets more compression (usually a little, sometimes a lot) by using the compressed object streams of PDF 1.5. • cairo_ps(onefile = TRUE) generates encapsulated EPS on platforms with cairo >= 1.6. • Binary reads (e.g. by readChar() and readBin()) are now supported on clipboard connections. (Wish of PR#14593.) • as.POSIXlt.factor() now passes ... to the character method (suggestion of Joshua Ulrich). [Intended for R 2.13.0 but accidentally removed before release.] • vector() and its wrappers such as integer() and double() now warn if called with a length argument of more than one element. This helps track down user errors such as calling double(x) instead of as.double(x). INSTALLATION: • Building the vignette PDFs in packages grid and utils is now part of running make from an SVN checkout on a Unix-alike: a separate make vignettes step is no longer required. These vignettes are now made with keep.source = TRUE and hence will be laid out differently. • make install-strip failed under some configuration options. • Packages can customize non-standard installation of compiled code via a src/install.libs.R script. This allows packages that have architecture-specific binaries (beyond the package's shared objects/DLLs) to be installed in a multi-architecture setting. SWEAVE & VIGNETTES: • Sweave() and Stangle() gain an encoding argument to specify the encoding of the vignette sources if the latter do not contain a \usepackage[]{inputenc} statement specifying a single input encoding. • There is a new Sweave option figs.only = TRUE to run each figure chunk only for each selected graphics device, and not first using the default graphics device. This will become the default in R 2.14.0. • Sweave custom graphics devices can have a custom function foo.off() to shut them down. • Warnings are issued when non-portable filenames are found for graphics files (and chunks if split = TRUE). Portable names are regarded as alphanumeric plus hyphen, underscore, plus and hash (periods cause problems with recognizing file extensions). • The Rtangle() driver has a new option show.line.nos which is by default false; if true it annotates code chunks with a comment giving the line number of the first line in the sources (the behaviour of R >= 2.12.0). • Package installation tangles the vignette sources: this step now converts the vignette sources from the vignette/package encoding to the current encoding, and records the encoding (if not ASCII) in a comment line at the top of the installed .R file. DEPRECATED AND DEFUNCT: • The internal functions .readRDS() and .saveRDS() are now deprecated in favour of the public functions readRDS() and saveRDS() introduced in R 2.13.0. • Switching off lazy-loading of code _via_ the LazyLoad field of the DESCRIPTION file is now deprecated. In future all packages will be lazy-loaded. • The off-line help() types "postscript" and "ps" are deprecated. UTILITIES: • R CMD check on a multi-architecture installation now skips the user's .Renviron file for the architecture-specific tests (which do read the architecture-specific Renviron.site files). This is consistent with single-architecture checks, which use --no-environ. • R CMD build now looks for DESCRIPTION fields BuildResaveData and BuildKeepEmpty for per-package overrides. See ‘Writing R Extensions’. BUG FIXES: • plot.lm(which = 5) was intended to order factor levels in increasing order of mean standardized residual. It ordered the factor labels correctly, but could plot the wrong group of residuals against the label. (PR#14545) • mosaicplot() could clip the factor labels, and could overlap them with the cells if a non-default value of cex.axis was used. (Related to PR#14550.) • dataframe[[row,col]] now dispatches on [[ methods for the selected column (spotted by Bill Dunlap). • sort.int() would strip the class of an object, but leave its object bit set. (Reported by Bill Dunlap.) • pbirthday() and qbirthday() did not implement the algorithm exactly as given in their reference and so were unnecessarily inaccurate. pbirthday() now solves the approximate formula analytically rather than using uniroot() on a discontinuous function. The description of the problem was inaccurate: the probability is a tail probablity (‘2 _or more_ people share a birthday’) • Complex arithmetic sometimes warned incorrectly about producing NAs when there were NaNs in the input. • seek(origin = "current") incorrectly reported it was not implemented for a gzfile() connection. • c(), unlist(), cbind() and rbind() could silently overflow the maximum vector length and cause a segfault. (PR#14571) • The fonts argument to X11(type = "Xlib") was being ignored. • Reading (e.g. with readBin()) from a raw connection was not advancing the pointer, so successive reads would read the same value. (Spotted by Bill Dunlap.) • Parsed text containing embedded newlines was printed incorrectly by as.character.srcref(). (Reported by Hadley Wickham.) • decompose() used with a series of a non-integer number of periods returned a seasonal component shorter than the original series. (Reported by Rob Hyndman.) • fields = list() failed for setRefClass(). (Reported by Michael Lawrence.) • Reference classes could not redefine an inherited field which had class "ANY". (Reported by Janko Thyson.) • Methods that override previously loaded versions will now be installed and called. (Reported by Iago Mosqueira.) • addmargins() called numeric(apos) rather than numeric(length(apos)). • The HTML help search sometimes produced bad links. (PR#14608) • Command completion will no longer be broken if tail.default() is redefined by the user. (Problem reported by Henrik Bengtsson.) • LaTeX rendering of markup in titles of help pages has been improved; in particular, \eqn{} may be used there. • isClass() used its own namespace as the default of the where argument inadvertently. • Rd conversion to latex mis-handled multi-line titles (including cases where there was a blank line in the \title section).
Why open source companies dont dance?
I have been pondering on this seemingly logical paradox for some time now-
1) Why are open source solutions considered technically better but not customer friendly.
2) Why do startups and app creators in social media or mobile get much more press coverage than
profitable startups in enterprise software.
3) How does tech journalism differ in covering open source projects in enterprise versus retail software.
4) What are the hidden rules of the game of enterprise software.
Some observations-
1) Open source companies often focus much more on technical community management and crowd sourcing code. Traditional software companies focus much more on managing the marketing community of customers and influencers. Accordingly the balance of power is skewed in favor of techies and R and D in open source companies, and in favor of marketing and analyst relations in traditional software companies.
Traditional companies also spend much more on hiring top notch press release/public relationship agencies, while open source companies are both financially and sometimes ideologically opposed to older methods of marketing software. The reverse of this is you are much more likely to see Videos and Tutorials by an open source company than a traditional company. You can compare the websites of Cloudera, DataStax, Hadapt ,Appistry and Mapr and contrast that with Teradata or Oracle (which has a much bigger and much more different marketing strategy.
Social media for marketing is also more efficiently utilized by smaller companies (open source) while bigger companies continue to pay influential analysts for expensive white papers that help present the brand.
Lack of budgets is a major factor that limits access to influential marketing for open source companies particularly in enterprise software.
2 and 3) Retail software is priced at 2-100$ and sells by volume. Accordingly technology coverage of these software is based on volume.
Enterprise software is much more expensively priced and has much more discreet volume or sales points. Accordingly the technology coverage of enterprise software is more discreet, in terms of a white paper coming every quarter, a webinar every month and a press release every week. Retail software is covered non stop , but these journalists typically do not charge for “briefings”.
Journalists covering retail software generally earn money by ads or hosting conferences. So they have an interest in covering new stuff or interesting disruptive stuff. Journalists or analysts covering enterprise software generally earn money by white papers, webinars, attending than hosting conferences, writing books. They thus have a much stronger economic incentive to cover existing landscape and technologies than smaller startups.
4) What are the hidden rules of the game of enterprise software.
- It is mostly a white man’s world. this can be proved by statistical demographic analysis
- There is incestuous intermingling between influencers, marketers, and PR people. This can be proved by simple social network analysis of who talks to who and how much. A simple time series between sponsorship and analysts coverage also will prove this (I am working on quantifying this ).
- There are much larger switching costs to enterprise software than retail software. This leads to legacy shoddy software getting much chances than would have been allowed in an efficient marketplace.
- Enterprise software is a less efficient marketplace than retail software in all definitions of the term “efficient markets”
- Cloud computing, and SaaS and Open source threatens to disrupt the jobs and careers of a large number of people. In the long term, they will create many more jobs, but in the short term, people used to comfortable living of enterprise software (making,selling,or writing) will actively and passively resist these changes to the paradigms in the current software status quo.
- Open source companies dont dance and dont play ball. They prefer to hire 4 more college grads than commission 2 more white papers.
and the following with slight changes from a comment I made on a fellow blog-
- While the paradigm on how to create new software has evolved from primarily silo-driven R and D departments to a broader collaborative effort, the biggest drawback is software marketing has not evolved.
- If you want your own version of the open source community editions to be more popular, some standardization is necessary for the corporate decision makers, and we need better marketing paradigms.
- While code creation is crowdsourced, solution implementation cannot be crowdsourced. Customers want solutions to a problem not code.
- Just as open source as a production and licensing paradigm threatens to disrupt enterprise software, it will lead to newer ways to marketing software given the hostility of existing status quo.