Home » Posts tagged 'usa'
Tag Archives: usa
Ajay- Why did you choose Rapid Miner and R? What were the other software alternatives you considered and discarded?
Analyst- We considered most of the other major players in statistics/data mining or enterprise BI. However, we found that the value proposition for an open source solution was too compelling to justify the premium pricing that the commercial solutions would have required. The widespread adoption of R and the variety of packages and algorithms available for it, made it an easy choice. We liked RapidMiner as a way to design structured, repeatable processes, and the ability to optimize learner parameters in a systematic way. It also handled large data sets better than R on 32-bit Windows did. The GUI, particularly when 5.0 was released, made it more usable than R for analysts who weren’t experienced programmers.
Ajay- What analytics do you do think Rapid Miner and R are best suited for?
Analyst- We use RM+R mainly for sports analysis so far, rather than for more traditional business applications. It has been quite suitable for that, and I can easily see how it would be used for other types of applications.
Ajay- Any experiences as an enterprise customer? How was the installation process? How good is the enterprise level support?
Analyst- Rapid-I has been one of the most responsive tech companies I’ve dealt with, either in my current role or with previous employers. They are small enough to be able to respond quickly to requests, and in more than one case, have fixed a problem, or added a small feature we needed within a matter of days. In other cases, we have contracted with them to add larger pieces of specific functionality we needed at reasonable consulting rates. Those features are added to the mainline product, and become fully supported through regular channels. The longer consulting projects have typically had a turnaround of just a few weeks.
Ajay- What challenges if any did you face in executing a pure open source analytics bundle ?
Analyst- As Rapid-I is a smaller company based in Europe, the availability of training and consulting in the USA isn’t as extensive as for the major enterprise software players, and the time zone differences sometimes slow down the communications cycle. There were times where we were the first customer to attempt a specific integration point in our technical environment, and with no prior experiences to fall back on, we had to work with Rapid-I to figure out how to do it. Compared to the what traditional software vendors provide, both R and RM tend to have sparse, terse, occasionally incomplete documentation. The situation is getting better, but still lags behind what the traditional enterprise software vendors provide.
Ajay- What are the things you can do in R ,and what are the things you prefer to do in Rapid Miner (comparison for technical synergies)
Analyst- Our experience has been that RM is superior to R at writing and maintaining structured processes, better at handling larger amounts of data, and more flexible at fine-tuning model parameters automatically. The biggest limitation we’ve had with RM compared to R is that R has a larger library of user-contributed packages for additional data mining algorithms. Sometimes we opted to use R because RM hadn’t yet implemented a specific algorithm. The introduction the R extension has allowed us to combine the strengths of both tools in a very logical and productive way.
In particular, extending RapidMiner with R helped address RM’s weakness in the breadth of algorithms, because it brings the entire R ecosystem into RM (similar to how Rapid-I implemented much of the Weka library early on in RM’s development). Further, because the R user community releases packages that implement new techniques faster than the enterprise vendors can, this helps turn a potential weakness into a potential strength. However, R packages tend to be of varying quality, and are more prone to go stale due to lack of support/bug fixes. This depends heavily on the package’s maintainer and its prevalence of use in the R community. So when RapidMiner has a learner with a native implementation, it’s usually better to use it than the R equivalent.
Awesomely informative post on sascom magazine (whose editor I have I interviewed before here at http://www.decisionstats.com/interview-alison-bolen-sas-com/ – )
Great piece by Michael Ames ,SAS Data Integration Product Manager.
Also see SAS’s big data thingys here at
Solutions and Capabilities Using SAS® In-Memory Analytics
- High-Performance Analytics – Get near-real-time insights with appliance-ready analytics software designed to tackle big data and complex problems.
- High-Performance Risk – Faster, better risk management decisions based on the most up-to-date views of your overall risk exposure.
- High-Performance Liquidity Risk Management – Take quick, decisive actions to secure adequate funding, especially in times of volatility.
- High-Performance Stress Testing – Make faster, more precise decisions to protect the health of the firm.
- Visual Analytics – Explore big data using in-memory capabilities to better understand all of your data, discover new patterns and publish reports to the Web and iPad®.
(Ajay- I liked the Visual Analytics piece especially for Big Data )
Western countries are running out of people to fight their wars. This is even more acute given the traditional and current demographic trends in both armed forces and general populations.
A shift to cyber conflict can help the West maintain parity over Eastern methods of assymetrical warfare (by human attrition /cyber conflict).
Declining resources will lead to converging conflicts of interest and dynamics in balance of power in the 21 st century.
The launch of Sputnik by USSR led to the moon shot rush by the US.1960s
The proposed announcement of StarWars by USA led to unsustainable defence expenditure by USSR.1980s
The threat of cyber conflict and espionage by China (and Russian cyber actions in war with Georgia) has led to increasing budgets for cyber conflict research and defense in USA. -2010s
If we do not learn from history, we are condemned to repeat it.
Declining Populations in the West and Rising Populations in the East in the 21 st century. The difference in military age personnel would be even more severe, due to more rapid aging in the west.
Economic output will be proportional to number of people employed as economies reach similar stages of maturity (Factor-Manufacturing-Services-Innovation)
Some wry observations from me on the world on economics-
1) 150 years after humiliating their country in the Opium Wars, Chinese mandarins have somehow convinced their leaders and military to park 2 trillion assets in Anglo Saxon debt. If Greece geting a 50% discount on its loan is the new precedent, when will the USA force its lendors to the negotiation table.
2) Income inequality and protests are something the Arabs and Israelis have in common. Besides being the sons of Abraham of course. Note the Persians are not considered the same as Arabs.
3) Advance knowledge of geo-political events can and ensures Western financial dealers have an edge on the sovereign funds in the other hemisphere. What used to be the playgrounds of Eton has now shifted to the pubs of Boston and So Cal.
4) After spending 1 trillion USD on arms in the past one decade (funded by guys in item 1), the United States military forces is in a much better more advanced position to wage simultaneous war.
5) Can a war in Korean peninsula affect war in the Persian sphere of influence. Just follow the money , baby.
6) Saudi Wahabis continue to fund terror despite losing a lot of money in the economic meltdown in past few years. For every 1 $ increase in Saudi oil revenue, western oil companies ,traders, financiers make more, much more.
7) Demographics is an important key to economics. An aging Japan, and stagnant West is one cause to shift from manpower intensive warfare to cyber warfare. Plus Cyber warfare is good business . Underpopulated Russia and Arabs continue to lack true economic potential.
8) There are new economic incentives to develop tools to disseminate as well as distort information flow in real time in a hyper connected digital world.
This is a ten step program to fight hacking attacks. You may or may not choose to ignore it, laugh at it, or ponder on it.
1) Internet security is a billion dollar business which will only grow in size as cloud computing approaches. Pioneers in providing security will earn considerable revenue like McAffee , Norton did in the PC era. Incidentally it also means the consulting/partner group that is willing to work with virtual workers and virtual payments to offshore consultants.
2) Industrial espionage has existed from the days the West stole Gunpowder and Silk formula from China (and China is now doing the same to its software). The company and country will the best hackers will win. Keep your team motivated mate, or it is very easy for them to defect to the other side of the (cyber) wall.
3) When 2 billion people have access to internet the number of hackers will grow in number and quality much more rapidly than when only 100 million people across the world had access. Thanks to Google Translate, Paypal, Skype video Call, Tor Project, and Google Voice i can and have collaborative with hackers almost in all geographies. You can only imagine what the black hats are doing.
4) Analyzing hackers is like reading Chinese Tea Leaves. If you have experienced analysts, you will slip up. recruit the hackers in the dormitory before China recruits them using Lulz Security as a bogus cover. or USA recruits them as cover for spreading democracy in the Arab countries.
5) get your website audited for security breaches. sponsor a hack my website contest. before someone else does it for you.
6) Fighting hackers was always tough. But now we have part time hackers , people with perfectly respectable jobs who look like Mr Andersen and hack like Neo from the Matrix. Every kid once wanted to be a firefighter. Every geek dreams of the one ultimate hack.
7) if you cant beat hackers, join them.
8) the more machine data is generated, the more you need external experts and newer software interfaces. Investing in open data, datasets is good. Keeping Bradley manning naked in his cell is bad. ignore the bad PR at your own cost.
9) Stop blaming China for every hack attack. You are a techie not a politician
10) Hack hard. Hack well. If someone hacks you, you will need to hack them off offensively unless you just want to be an easy mark for the rest of your lives. Counter -hacking expertise needs to be strengthened and groomed. hacking is an offense not just a defense game.