Who won the war of big data?

Happy 2017 everybody! I am going to say that 2017 will be the “Year of data” (huh…?). We just entered a year and an era that will re-carve into our thinking, a truth we knew all along: “Information is power”. Data, tools and techniques to process them, and the skills to comprehend and interpret them -like it or not, is going to run every aspect of our lives. If you have been online for the past 5 years, there are nearly 30,000 pieces of information about you out there, enough to characterize you deeper than the classical demographics could; race and ethnicity, age, level of education, gender, political views, marital status, and how much you make. Behavioral data analytics will construct your ambitions, the food preferences, your dating habits, what you wear, what you fear, if you like white or red wine -or none, what your worries are, and even how long your subconscious mind tells you how long you would live. This is not necessarily a bad thing and I won’t insult your intelligence trying to elaborate. Almost all data are qualitative. Nearly almost always, we need to quantify them to be able to analyze them, particularly when there is a lot of data and the data is complicated. This is why the data science is the next -not the “big thing”, not the “Huge thing”, not even the “enormous thing”, but “everything” that is going to be about our lives.

I looked at the Federal Election Commission (FEC) filings to understand who used data science how, leading up to the 2016 US presidential elections. Like I said, I could say it all in three words and a single punctuation: “Oh My God!” This is why. Let’s code the Clinton campaign is BLUE and the Trump campaign is RED for our discussion.

RED spent significantly more than BLUE did on the election campaign.

RED spent $349,487,816 while BLUE spent $601,770,250. This is better visualized in graphs:

Money speaks? Not quite. What speaks is “How money is spent”.

RED relied heftily on data-science, BLUE dismally less

Out of all campaign spending, RED spent 4.5% on data science while BLUE spent 0.24%. That is if I am to be generous towards the BLUE campaign. If we look at the type of services paid for and how the money was spent, the BLUE campaign actually spent a dismal 0.025% of their money on hard data analytics. Graphically this is what it looks like:

RED campaign spent their money on big data, and very importantly big data analytics. CAMBRIDGE ANALYTICA, LLC peeled layers of conventional assumptive thinking to use behavioral Micro-targeting to help RED gauge who would vote, how they will vote, and what majority of potential voters wanted to hear. DIRECT RESPONSE, LLC helped RED meter if their thinking was right. BLUE relied more on the marketing aspect –how to get the message across, cared less what the market wanted. Let’s go back to our previous discussion about selling long or short sleeved shirts this summer. If we were making shirts in a factory in Dhaka, what should we do to maximize our profit margins in my 2017 Income statement? Should we get inside consumer’s mind to understand what they would prefer and make more of them OR, try convincing them that they should buy long sleeved shirts despite the 2017 summer predicted to be warmer than usual and keep making long sleeved?

How much did RED spend on data Science compared to BLUE? RED Spent 10,621% (or over 100 times) the BLUE did on campaign external resources (yes- I checked the numbers)

Let’s compare one-on-one, without relatives, to know exactly who used the state of art in Data-science to leverage consumer (voter) thinking to deliver the products they had the appetite for.

Hope you are “OH-MY-GOD’ing” with me at this point. Here comes the dessert and cognac, and the cigars: Even the data-related expenses by the Hillary campaign heavily gravitated towards campaigning and campaign fundraising, not understanding the American voter. There was data monitoring, but NOT the all-important analysis of them.

Largest single spending

BLUE’s major spending went towards NGP VAN INC. and LEXIS NEXIS who do campaigning, fundraising, and risk management. RED spent heavily on CAMBRIDGE ANALYTICA, LLC and DIRECT RESPONSE, LLC. who specialize in microtargetng. Major disbursement towards a single entity indicates focus and resolved intent. Scattered spending suggests non-reliance on a particular angle of approach, or the self-reliance to achieve the objective. Here we compare the single largest disbursements by the two campaigns.

CAMBRIDGE ANALYTICA does behavioral microtargeting using mass people data, a great use of big-data. Here is a direct extract from the company website that they say they do best.

NGP VAN is known for managing campaign logistics. It includes “Targeted email” among 15 serviced provided that includes website management, event Management, social fundraising, online contributions, and phones. CAMBRIDGE ANALYTICA does behavioral data science.

To me, the Clinton campaign ran like a mom-and-pup sandwich shop. Books were kept great, toilets were clean and they flushed, everything was kept toted and boxed, and they would say what’s good for you. The trump campaign ran the show like a modern day commercial enterprise. They found out what people thought, what they wanted, who they were and where they lived, and they delivered it. What matters to you more at a beer festival: The number of craftsman brewers present or the quality of toilet paper in the bathrooms? You got it.

All data are from Federal Election Commission filings and publically available web data.

Picture: https://cambridgeanalytica.org

All analyses and graphics are by the author