Predicting markets using social media sentiment — is it possible?
Oraclum has, in collaboration with The Macro Mail, started a prediction competition on Twitter where we look for traders and other enthusiastic market followers to give us their predictions on the weekly movements of asset prices. Specifically, we have asked about predicting six things: two indices (the Dow and the S&P500), two cryptocurrencies (Bitcoin and Dogecoin), the price of oil (WTI crude), and the yield of the 10-Year T-bill. In other words a healthy mixture of indicators useful in determining market movements and directions. We run a weekly survey, open for 24 hours only (Tuesday at 8am EST to Wednesday 8am EST), where we ask users to predict the values of those six tickers for the end of trading day Friday (4pm EST) of the same week.
A quick recap of our prediction method, called the BASON (Bayesian Adjusted Social Network Survey) (watch a video of it here, or read more here, here, here or here). We poll people on Twitter, asking them what they think the prices of the aforementioned indicators will be by the end of the week, and what their friends think. We aim to find the best observers of their environments and through their responses and the responses of their friends estimate the group bias of each individual prediction. We then place a weight on each individual’s predictions based on their group’s bias and draw patterns of behavior. This methodology has enabled us to accurately predict not only election outcomes (like Brexit, Trump in 2016 or Biden in 2020), but also consumer sentiment and demand, market trends, optimal pricing, and even the economic consequences of the COVID-19 pandemic.
Results
After the first four weeks of making these predictions the results are very encouraging. The graphs and tables below illustrate the precision of our BASON predictions on a weekly basis. The red triangles in the graphs represent our predictions, while the blue line is the actual value of the index. The grey area is the day when the predictions were being made.
The table below gives a more detailed insight into the precision of our predictions. The first three rows show the actual Tuesday value (the day the prediction was being made), the actual Friday value, and the BASON Friday prediction. Then we calculate the precision of the Friday prediction with respect to the actual value, include a calculation of the weekly volatility for each indicator, and the predicted direction of each indicator. The dark green color depicts really accurate precision of a given indicator for a given week (below or around 1 percentage point), while the light green and light red indicate hits and misses of the direction suggested by the prediction.
Notice first how the most accurate predictions each week were those for the Dow and the S&P (also shown graphically), not just in terms of direction, but also on the value of the indices, on average being within 1% accurate (significantly lower than their average weekly volatility of 3%). In fact, every week it delivered a prediction that was three to four times more precise than the size of weekly volatility. This means that the predictions were not just a case of good anchoring on the day before, but that they actually overcame the volatility and can be even better in terms of margin of error.
The BASON method was correct in anticipating both good and bad times for the two indices. For example, in weeks 2 and 3 the prediction accuracy for the Dow and the S&P was within 1% for both, estimating they would bounce back and remain roughly similar to their Tuesday morning values, even after the decline that started already on the same day or the day before.
Oil prices and the T-bill yield predictions were also precise, in the right direction, and lower than their weekly volatility, particularly after we introduced the ability to make predictions up to the second decimal point (after week 1). In week 1 we started with the EUR/USD exchange rate but decided to switch it with Dogecoin from week 2, primarily due to much greater interest, and very granular weekly volatility of the EUR/USD exchange rate.
The prediction precision of cryptocurrencies has, however, been much lower than for other indicators. This is primarily because of high volatility that happened in the crypto market during weeks 2 and 3 (BTC, for example, had 21% volatility in week 2 and 34% in week 3, while Dogecoin had 40% in week 2 and 68% in week 3). For example, on the Wednesday of week 2, Elon Musk tweeted about Tesla moving out of BTC, which was the first big (and unexpected) shock. On Wednesday of week 3, just as our users estimated a slight rebound of BTC, the entire crypto market once again crashed (Dogecoin lost half of its value on that day alone). We will still continue making predictions about crypto and see what happens in the weeks ahead, but so far, we can carefully conclude that with highly volatile assets, prediction precision is much lower, even in terms of its final direction.
Awarding the winners
The rules of the competition specify that each month we will award the top three performers with a $100 each. Below is our current leaderboard featuring the best performers. Most of our users choose to be anonymous and we respect that. We only really need an email to get in touch and pay out the prize.
Congratulations to the top 3 players, each of which will receive a wire transfer of $100 during the following week.
Next steps
In order to answer the question from the title, we still need more time and more predictions. After a few months, and after a dozen more winners being awarded their $100 prize, we will the examine the full commercial value of these predictions. In its current form it is still nothing more than a fun game for traders and investors.
For full transparency we will publish weekly predictions made during Tuesday (for the Dow and S&P) so that the users themselves can see if these predictions will hold for the end of trading day Friday. This information will first be sent to the users of the survey on their email. They will benefit from having the first access to information on our predictions. If you wish to do the same, follow us on Twitter where we publish our new survey each Tuesday. Give your predictions, and wait for the final result on Wednesday after the survey ends. This way you may profit from being the best predictor and earning $100 but also from knowing in advance where the markets might end up on Friday of that week.
Thanks for participating and good luck!
