Just like the London Mayoral Campaign and Brexit, Twitter Voices Signaled the Result
London, UK & West Palm Beach, FL, US November 18, 2016: As it did with the London Mayoral Race and the EU Referendum, start-up Essencient’s patented AI technology could sense the momentum on Twitter in the US Presidential Election well before the pundits knew what was happening. Before the polls closed, their tech identified a huge groundswell of engagement on Twitter and it was all for Trump. Essencient say they have once again successfully applied their AI technology to measure the “engagement current” on social media and have demonstrated its potential to predict outcomes.
Whilst Essencient acknowledge they were not the only company to demonstrate the prediction opportunities AI presents when combined with social media, they believe they are the only company doing it without significant cost and time overhead when compared to other approaches.
“Our technology requires no training to produce results, so it is incredibly flexible and allows all sorts of scenario planning very easily and quickly – the ‘what-if’ options allow us to test multiple hypotheses really rapidly.” explained Rob Lancashire, Co-founder of Essencient, “For example, in the US election we examined the data in multiple ways, and every view we took showed us the same thing. Twitter users overwhelmingly engaged actively and positively for Trump over Clinton in the last days. We reported that fact to our shareholders a full day ahead of the election, and then once we concluded that the momentum for Trump was unmistakable, to the public via LinkedIn on the 8th November at 6.58 EST (23.58 GMT). That was at least an hour before the first polls closed and anyone was even considering a win for Trump. It was like Brexit all over again, and in truth some of the team doubted our own predictions.” Lancashire added, on a light note, “Some of our shareholders who had absolute faith in the technology actually backed our predictions with their cash at the bookies, and are riding high. Even without relying on Essencient for betting tips I am now confident that won’t be the last cash pile we help them make!”
While the Essencient engine uses hugely sophisticated AI, it is in practice really simple to use. The team applied a simple Boolean query to gather some 17 million tweets relevant to the election. They then fed these into the engine to filter out those that demonstrated no meaning or feeling of any kind, leaving just those that had “real engagement” towards the candidates. Multiple factors, many of which are unique to the Essencient engine, were taken into account in filtering the results to segment tweets expressing, for example, strongly held opinions, real intent to vote and active advocacy.
“An effort of this scale relies heavily on the ability to automate the process of eliminating ‘noise’, the uninformative conversations, from the ones that actually mean something – right up front. With other systems, this first level of filtering is typically done by humans every time, which is just not feasible in terms of time or cost in a commercial scenario”, explained Mike Petit, Co-founder of Essencient. “We ran two distinct processes. The first, our Noise Floor technology, reduced the 17 million tweets to 9 million. With the ‘noise’ eliminated, the second stage of Essencient’s technology – Engagement Metrics – was applied. The Engagement Metrics, which use Essencient’s patented Natural Language Processing capabilities, allowed us to further filter the truly meaningful tweets based on the strength of engagement. Engagement Metrics reduced the 9 million tweets to 4 million highly qualified ones. Although complicated to explain, the Essencient technology can produce this high value data in seconds. Having identified the high value tweets, conventional analysis techniques were then applied to identify the twitter users’ attitudes and intentions over specified periods.”
Lancashire added, “Interestingly, for months prior to Election Day the tweets reflected the traditional ‘scientific’ polling, but several days before the polling stations opened, a clear swing towards Trump began to emerge despite the traditional polls saying otherwise”. He continued, “It was puzzling at first, but then we began to understand. It wasn’t so much that the tweets were greater in number for one candidate or another. It was, rather, that Trump supporters were significantly more expressive of their feelings than Clinton supporters. They spoke in stronger terms, expressed much more certainty about taking action, spoke more negatively of their candidate’s detractors, and offered more reasons why others should support their candidate. In short, the level of passion was much higher. That trend intensified right up to the end, which we believe may well have influenced those voters who were undecided up to the point of voting. Although doubtlessly some of the traffic was generated by bots, I believe no candidate on the planet, even with the current state of the art in bots, could artificially create the level of characteristically human feeling and engagement we were seeing. “
Petit added, “Although we’re certainly not claiming to have definitively predicted the outcome of the election using this limited data set, we did feel we could say with some confidence that engagement was higher for Trump, which allowed us to predict in the period leading up to the polls finally closing that an unmistakable level of support for Trump was building.”
Did that enthusiasm reflect itself in voter turnout? “We believe it did”, responded Petit, “Clinton tweeters simply did not demonstrate the same level of passion and advocacy on Twitter. Although quantifying the impact of that engagement gap would require far more analysis, the fact of that gap is now obvious to all; therefore it makes a compelling case for this method of analysis.”
Essencient is on to something big. Prior analyses of the London Mayoral campaign and the Brexit battle demonstrated the existence of this “engagement current” on social media in general, and its correlation with the outcome. In the case of Brexit, the Remain and Leave percentages Essencient calculated closely mirrored the final results and – as with Trump – ran counter to what the polls were predicting.
“We shared these US Election predictions only with our shareholders and the LinkedIn network. We can make this kind of social intelligence available to anyone through our website – www.essencient.com – whether they be someone looking for intelligence on political futures or a brand looking for social intelligence on their products”, says Lancashire. He adds, with a chuckle, “I am confident in saying we will have others joining us for a celebratory drink in the future when they win on the back of our predictions.”