Big Data : Building Big Brother’s Evil Empire

BackgroundThe Russell–Einstein Manifesto was issued in London on July 9, 1955 by Bertrand Russell in the midst of the Cold War. It highlighted the dangers posed by nuclear weapons and called for world leaders to seek peaceful resolutions to international conflict. The signatories included eleven pre-eminent intellectuals and scientists, including Albert Einstein, who signed it just days before his death on April 18, 1955.

It is time Data Scientists most of whom have tacitly accepted that open source and open societies are movements for good, come together and stop enabling Big Brothers in China, India and United States from accumulating lifetimes of information on the citizens of the Internet. Does the 21st Century need a better equipped and enabled United Nations to deal with climate change, weather weapons and cyber warfare. While the myth of Anglo Saxon unipolar world is dead, renegade elements from the military industrial complex from both the West and East threaten the peace with illegal and unethical spying on citizens and civilians.

As data scientists, we enable them , these Governments with the new cyber nukes to collect, sort, aggregate, visualize , huge amounts of text data. I bet the NSA and China use Hadoop and R for purposes that are both repulsive and illegal. Now you can join the gravy train, and earn big money working for one, or the other, or even both.

Or we can come up with a newer version of Russell- Einstein manifesto for Big Data and Internet. The military industrial complex of the East and West has screwed the real planet enough. Why screw the cyber -world of Internet by these weapons of mass collection? Using Python, R, SAS, JMP, or using regression, k means, map reduce/.  Data scientists should have a conscience before they fire their code.

Al

Big Brother is watching you- George Orwell

Big Data is watching you- George  Bush

Interview Pranay Agrawal Co-Founder Fractal Analytics

Here is an interview with Pranay Agrawal, Executive Vice President- Global Client Development, Fractal Analytics – one of India’s leading analytics services providers and one of the pioneers in analytics services delivery.

Ajay- Describe Fractal Analytics’ journey as a startup to a pioneer in the Predictive Analytics Services industry. What were some of the key turning points in the field of analytics that you have noticed during these times?

IMG_5387

Pranay- In 2000, Fractal Analytics started as a pure-play analytics services company in India with a focus on financial services. Five years later, we spread our operation to the United States and opened new verticals. Today, we have the widest global footprint among analytics providers and have experience handling data and deep understanding of consumer behavior in over 150 counties. We have matured from an analytics service organization to a productized analytics services firm, specializing in consumer goods, retail, financial services, insurance and technology verticals.
We are on the fore-front of a massive inflection point with Big Data Analytics at the center. We have witnessed the transformation of analytics within our clients from a cost center to the most critical division that drives competitive advantage.  Advances are quickly converging in computer science, artificial intelligence, machine learning and game theory, changing the way how analytics is consumed by B2B and B2C companies. Companies that use analytics well are poised to excel in innovation, customer engagement and business performance.

Ajay- What are analytical tools that you use at Fractal Analytics? Are there any trends in analytical software usage that you have observed?

Pranay- We are tools agnostic to serve our clients using whatever platforms they need to ensure they can quickly and effectively operationalize the results we deliver.  We use R, SAS, SPSS, SpotFire, Tableau, Xcelsius, Webfocus, Microstrategy and Qlikview. We are seeing an increase in adoption of open source platform such as R, and specialize tools for dashboard like Tableau/Qlikview, plus an entire spectrum of emerging tools to process manage and extract information from Big Data that support Hadoop and NoSQL data structures

Ajay- What are Fractal Analytics plans for Big Data Analytics?

Pranay- We see our clients being overwhelmed by the increasing complexity of the data. While they are all excited by the possibilities of Big Data, on-the-ground struggle continues to realize its full potential. The analytics paradigm is changing in the context of Big Data. Our solutions focus on how to make it super-simple for our clients combined with analytics sophistication possible with Big Data.
Let’s take our Customer Genomics solution for retailers as an example. Retailers are collecting information about Shopper behaviors through every transaction. Retailers want to transform their business to make it more customer-centric but do not know how to go about it. Our Customer Genomics solution uses advanced machine learning algorithm to label every shopper across more than 80 different dimensions. Retailers use these to identify which products it should deep-discount depending on what price-sensitive shoppers buy. They are transforming the way they plan their assortment, planogram and targeted promotions armed with this intelligence.

We are also building harmonization engines using Concordia to enable real-time update of Customer Genomics based on every direct, social, or shopping transaction. This will further bridge the gap between marketing actions and consumer behavior to drive loyalty, market share and profitability.

Ajay- What are some of the key things that differentiate Fractal Analytics from the rest of the industry? How are you different?

Pranay- We are one of the pioneer pure-play analytics firm with over a decade of experience consulting with Fortune 500 companies. What clients most appreciate about working with us includes:

  • Experience managing structured and unstructured Big Data (volume, variety) with a deep understanding of consumer behavior in more than 150 counties
  • Advanced analytics leveraging supervised machine-learning platforms
  • Proprietary products for example: Concordia for data harmonization, Customer Genomics for consumer insights and personalized marketing, Pincer for pricing optimization, Eavesdrop for social media listening,  Medley for assortment optimization in retail industry and Known Value Item for retail stores
  • Deep industry expertise enables us to leverage cross-industry knowledge to solve a wide range of marketing problems
  • Lowest attrition rates in the industry and very selective hiring process makes us a great place to work

Ajay- What are some of the initiatives that you have taken to ensure employee satisfaction and happiness?

Pranay- We believe happy employees create happy customers. We are building a great place to work by taking a personal interest in grooming people. Our people are highly engaged as evidenced by 33% new hire referrals and the highest Glassdoor ratings in our industry.
We recognize the accomplishments and contributions made through many programs such as:

  1. FractElite – where peers nominate and defend the best of us
  2. Recognition board – where anyone can write a visible thank you
  3. Value cards – where anyone can acknowledge great role model behavior in one or more values
  4. Townhall – a quarterly all hands where we announce anniversaries and FractElite awards, with an open forum to ask questions
  5. Employee engagement surveys – to measure and report out on satisfaction programs
  6. Open access to managers and leadership team – to ensure we understand and appreciate each person’s unique goals and ambitions, coach for high performance, and laud their success

Ajay- How happy are Fractal Analytics customers quantitatively?  What is your retention rate- and what plans do you have for 2013?

Pranay- As consultants, delivering value with great service is critical to our growth, which has nearly doubled in the last year. Most of our clients have been with us for over five years and we are typically considered a strategic partner.
We conduct client satisfaction surveys during and after each project to measure our performance and identify opportunities to serve our clients better. In 2013, we will continue partnering with our clients to define additional process improvements from applying best practice in engagement management to building more advanced analytics and automated services to put high-impact decisions into our clients’ hands faster.

About

Pranay Agrawal -Pranay co-founded Fractal Analytics in 2000 and heads client engagement worldwide. He has a MBA from India Institute of Management (IIM) Ahmedabad, Bachelors in Accounting from Bangalore University, and Certified Financial Risk Manager from GARP. He is is also available online on http://www.linkedin.com/in/pranayfractal

Fractal Analytics is a provider of predictive analytics and decision sciences to financial services, insurance, consumer goods, retail, technology, pharma and telecommunication industries. Fractal Analytics helps companies compete on analytics and in understanding, predicting and influencing consumer behavior. Over 20 fortune 500 financial services, consumer packaged goods, retail and insurance companies partner with Fractal to make better data driven decisions and institutionalize analytics inside their organizations.

Fractal sets up analytical centers of excellence for its clients to tackle tough big data challenges, improve decision management, help understand, predict & influence consumer behavior, increase marketing effectiveness, reduce risk and optimize business results.

 

Possible Digital Disruptions by Cyber Actors in USA Electoral Cycle

Some possible electronic disruptions  that threaten to disrupt the electoral cycle in United States of America currently underway is-

1) Limited Denial of Service Attacks (like for 5-8 minutes) on fund raising websites, trying to fly under the radar of network administrators to deny the targeted  fundraising website for a small percentage of funds . Money remains critical to the world’s most expensive political market. Even a 5% dropdown in online fund-raising capacity can cripple a candidate.

2)  Limited Man of the Middle  Attacks on ground volunteers to disrupt ,intercept and manipulate communication flows. Basically cyber attacks at vulnerable ground volunteers in critical counties /battleground /swing states (like Florida)

3) Electro-Magnetic Disruptions of Electronic Voting Machines in critical counties /swing states (like Florida) to either disrupt, manipulate or create an impression that some manipulation has been done.

4) Use search engine flooding (for search engine de-optimization of rival candidates keywords), and social media flooding for disrupting the listening capabilities of sentiment analysis.

5) Selected leaks (including using digital means to create authetntic, fake or edited collateral) timed to embarrass rivals or influence voters , this can be geo-coded and mass deployed.

6) using Internet communications to selectively spam or influence independent or opinionated voters through emails, short messaging service , chat channels, social media.

7) Disrupt the Hillary for President 2016 campaign by Anonymous-Wikileak sympathetic hacktivists.

 

 

Understanding Indian Govt attitude to Iran and Iraq wars

This is a collection of links for a geo-strategic analysis, and the economics of wars and allies. The author neither condones nor condemns current global dynamics in the balance of power.

nations don’t have friends or enemies…nations only have interests

In 2003

The war in Iraq had a unique Indian angle right at the beginning. Some members of the US administration felt they needed more troops in Iraq, and they started negotiating with India. Those negotiations broke down because the Indians wanted to fight under the UN flag and on MONEY!!

India wanted-

  • More money per soldier deployed,
  • more share in post War Oil Contracts,
  • better diplomatic subtlety
Govt changed in India due to elections in2003 (Muslim voters are critical in any govt forming majority party), and the Iraq war ran its tragic course without any Indian explicit support.
In 26 Nov 2008, Islamic Terrorists killed US, Indian and Israeli citizens in terror strikes in Mumbai Sieze- thus proving that appeasing terrorist nations is just riding a tiger.

http://articles.timesofindia.indiatimes.com/2003-06-13/india/27203305_1_stabilisation-force-indian-troops-pentagon-delegation

NEW DELHI: There will be a lot a Iraq on the menu over the weekend before the Pentagon team arrives here on Monday to talk India into sending troops to the war-torn nation.

http://articles.timesofindia.indiatimes.com/2003-07-28/india/27176989_1_troops-issue-stabilisation-force-defence-policy-group

Jul 28, 2003, 01.28pm IST

NEW DELHI: Chairman of the US Joint Chiefs of Staff Gen Richard B Myers, who is arriving here on Monday evening on a two-day visit, will request India to reconsider its decision on sending troops to Iraq.

and

Jul 29, 2003, 07.00pm IST

NEW DELHI: Though Gen Myers flatly denied his visit had anything to do with persuading India to send troops to Iraq, it is evident that the US desperately wants Delhi to contribute a division-level force of over 15,000 combat soldiers.

http://articles.timesofindia.indiatimes.com/2003-09-10/india/27176101_1_stabilisation-force-force-under-american-control-regional-dialogue

Sep 10, 2003, 05.34pm IST

NEW DELHI: Even as the US-drafted resolution on Iraq is being heatedly debated in many countries, American Assistant Secretary of State for South Asia Christina Rocca held a series of meetings with External Affairs Ministry officials on Wednesday.

Though it was officially called “a regional dialogue”, the US request to contribute a division-level force of over 15,000 combat soldiers to the “stabilisation force” in Iraq is learnt to have figured in the discussions.

The penny wise -pound foolish attitude of then Def Secretary Rumsfield led to break down in negotiations.

“Those who fail to learn from history are doomed to repeat it.” Sir Winston Churchill

In 2012

Indian govt again faces elections and we have 150 million Muslim voters just like other countries have influential lobbies.

and while Israelis are being targeted again in attacks in India-

India is still seeking money-

India has struck a defiant tone over new financial sanctions imposed by the United States and European Union to punish Iran for its nuclear programme, coming up with elaborate trade and barter arrangements to pay for oil supplies.

However, the president of the All India Rice Exporters’ Association, said Monday’s attack on the wife of an Israeli diplomat in the Indian capital will damage trade with Iran and may complicate efforts to resolve an impasse over Iranian defaults on payments for rice imports worth around $150 million.

http://timesofindia.indiatimes.com/india/Unfazed-by-US-sanctions-India-to-step-up-ties-with-Iran/articleshow/11887691.cms

India buys $ 5  billion worth of oil from Iran. Annually. Clearly it is a critical financial trading partner to Iran.

It has now gotten extra sops from Iran to continue trading-and is now waiting for a sweeter monetary offer from US and/or Israel to even consider thinking about going through the pain of unchanging the inertia of ties with Iran.

There are some aspects of political corruption as well, as Indian political establishment  is notoriously prone to corruption by lobbyists (apparently there   is a global war on lobbyists that needs to happen)

http://timesofindia.indiatimes.com/india/Unfazed-by-US-sanctions-India-to-step-up-ties-with-Iran/articleshow/11887691.cms

 Feb 14, 2012, 05.54PM ISTUnfazed by US sanctions, India to step up ties with Iran
India is set to ramp up its energy and business ties with Iran. (AFP Photo)
NEW DELHI: Unfazed by US sanctions and Israel linking Tehran to the attack on an Israeli embassy car here, India is set to ramp up its energy and business ties with Iran, with a commerce ministry team heading to Tehran to explore fresh business opportunities. 

The team is expected to go to Tehran later this month to discuss steps to expand India’s trade with Iran, part of a larger strategy to pay for Iranian oil, said highly-placed sources. 

Despite the US and European Union sanctions on Iran, India recently sealed a payment mechanism under which Indian companies will pay for 45 percent of their crude oil imports from Iran in rupees. 

So diplomats with argue over money in Israel, Indian and US while terrorists will kill.

Against Stupidity- The Gods Themselves -Contend in Vain

Funny Economics

Some wry observations from me  on the world on economics-

1) 150 years after humiliating their country in the Opium Wars, Chinese mandarins have somehow convinced their leaders and military to park 2 trillion assets in Anglo Saxon debt. If Greece geting a 50% discount on its loan is the new precedent, when will the USA force its lendors to the negotiation table.

2) Income inequality and protests are something the Arabs and Israelis have in common. Besides being the sons of Abraham of course. Note the Persians are not considered the same as Arabs.

3) Advance knowledge of geo-political events can and ensures Western financial dealers have an edge on the sovereign funds in the other hemisphere.  What used to be the playgrounds of Eton has now shifted to the pubs of Boston and So Cal.

4) After spending 1 trillion USD on arms in the past one decade (funded by guys in item 1), the United States military forces is in a much better more advanced position to wage simultaneous war.

5) Can a war in Korean peninsula affect war in the Persian sphere of influence. Just follow the money , baby.

6) Saudi Wahabis continue to fund terror despite losing a lot of money in the economic meltdown in past few years. For every 1 $ increase in Saudi oil revenue, western oil companies ,traders, financiers make more, much more.

7) Demographics is an important key to economics. An aging Japan, and stagnant West is one cause to shift from manpower intensive warfare to cyber warfare. Plus Cyber warfare is good business . Underpopulated Russia and Arabs continue to lack true economic potential.

8) There are new economic incentives to develop tools to disseminate as well as distort information flow in real time in a hyper connected digital world.

 

Credit Downgrade of USA and Triple A Whining

As a person trained , deployed and often asked to comment on macroeconomic shenanigans- I have the following observations to make on the downgrade of US Debt by S&P

1) Credit rating is both a mathematical exercise of debt versus net worth as well as intention to repay. Given the recent deadlock in United States legislature on debt ceiling, it is natural and correct to assume that holding US debt is slightly more risky in 2011 as compared to 2001. That means if the US debt was AAA in 2001 it sure is slightly more risky in 2011.

2) Politicians are criticized the world over in democracies including India, UK and US. This is natural , healthy and enforced by checks and balances by constitution of each country. At the time of writing this, there are protests in India on corruption, in UK on economic disparities, in US on debt vs tax vs spending, Israel on inflation. It is the maturity of the media as well as average educational level of citizenry that amplifies and inflames or dampens sentiment regarding policy and business.

3) Conspicuous consumption has failed both at an environmental and economic level. Cheap debt to buy things you do not need may have made good macro economic sense as long as the things were made by people locally but that is no longer the case. Outsourcing is not all evil, but it sure is not a perfect solution to economics and competitiveness. Outsourcing is good or outsourcing is bad- well it depends.

4) In 1944 , the US took debt to fight Nazism, build atomic power and generally wage a lot of war and lots of dual use inventions. In 2004-2010 the US took debt to fight wars in Iraq, Afghanistan and bail out banks and automobile companies. Some erosion in the values represented by a free democracy has taken place, much to the delight of authoritarian regimes (who have managed to survive Google and Facebook).

5) A Double A rating is still quite a good rating. Noone is moving out of the US Treasuries- I mean seriously what are your alternative financial resources to park your government or central bank assets, euro, gold, oil, rare earth futures, metals or yen??

6) Income disparity as a trigger for social unrest in UK, France and other parts is an ominous looming threat that may lead to more action than the poor maths of S &P. It has been some time since riots occured in the United States and I believe in time series and cycles especially given the rising Gini coefficients .

Gini indices for the United States at various times, according to the US Census Bureau:[8][9][10]

  • 1929: 45.0 (estimated)
  • 1947: 37.6 (estimated)
  • 1967: 39.7 (first year reported)
  • 1968: 38.6 (lowest index reported)
  • 1970: 39.4
  • 1980: 40.3
  • 1990: 42.8
    • (Recalculations made in 1992 added a significant upward shift for later values)
  • 2000: 46.2
  • 2005: 46.9
  • 2006: 47.0 (highest index reported)
  • 2007: 46.3
  • 2008: 46.69
  • 2009: 46.8

7) Again I am slightly suspicious of an American Corporation downgrading the American Governmental debt when it failed to reconcile numbers by 2 trillion and famously managed to avoid downgrading Lehman Brothers.  What are the political affiliations of the S &P board. What are their backgrounds. Check the facts, Watson.

The Chinese government should be concerned if it is holding >1000 tonnes of Gold and >1 trillion plus of US treasuries lest we have a third opium war (as either Gold or US Treasuries will burst)

. Opium in 1850 like the US Treasuries in 2010 have no inherent value except for those addicted to them.

8   ) Ron Paul and Paul Krugman are the two extremes of economic ideology in the US.

Reminds me of the old saying- Robbing Peter to pay Paul. Both the Pauls seem equally unhappy and biased.

I have to read both WSJ and NYT to make sense of what actually is happening in the US as opinionated journalism has managed to elbow out fact based journalism. Do we need analytics in journalism education/ reporting?

9) Panic buying and selling would lead to short term arbitrage positions. People like W Buffet made more money in the crash of 2008 than people did in the boom years of 2006-7

If stocks are cheap- buy. on the dips. Acquire companies before they go for IPOs. Go buy your own stock if you are sitting on  a pile of cash. Buy some technology patents in cloud , mobile, tablet and statistical computing if you have a lot of cash and need to buy some long term assets.

10) Follow all advice above at own risk and no liability to this author 😉