Stock market prediction has attracted a lot of attention from scholars as well as businessmen over the years. But can the stock market really be predicted? Many expert traders claim to know how to predict whether it will rise or fall at any particular time, but there are few - if any - who can actually do it successfully.

Why? Because conventional economic theory, otherwise known as Efficient Market Theory, states that the movement of prices in a market follow a random pattern, making it impossible to predict with an accuracy greater than 50 per cent. This hypothesis, however, no longer holds much value, since numerous studies have shown that stock market prices are not entirely random, and may actually be linked to mass psychology and other related factors. If this is true, then there may essentially be a way to assess the mass consciousness for predictive signs.

Behavioural economics tells us that emotions can profoundly affect individual behaviour and decision-making. But does this apply to societies at large; can societies experience mood states that affect their collective decision making? Moreover, is the public mood correlated or even predictive of economic indicators? Researchers believe so, and have stated that they have found such a predictor buried in the seemingly endless chatter that springs from Twitterverse.

The recent excitement around the subject of Twitter trading stems from a paper by academics Johan Bollen and Huina Mao of Indiana University, and Xiao-Jun Zeng of the University of Manchester. The question they asked is whether any human states correlate with stock market prices, since it is not unfeasible that the rise and fall of stock market prices may be influenced by the public mood. Bollen and his team took 9.7 million tweets posted by 2.7 million tweeters between March and December 2008 and looked for correlations between the GPOMS indices and whether Dow Jones Industrial Average rose or fell each day. Their illuminating conclusion is that there truly is a correlation: calmness. The calmness index appears to be a good predictor of whether the Dow Jones Industrial Average goes up or down between 2 and 6 days later. According to the report: “We find an accuracy of 87.6 percent in predicting the daily up and down changes in the closing values of the Dow Jones industrial average.” Their report clearly found that gauging the investing public’s mood can be a startlingly predictive mechanism for the stock market. In order words, Twitter mood can predict the stock market.

How can it be explained? “One idea is that the stream of thought is representative of the mental state of humankind at any instant. Various groups have devised algorithms to analyse this data stream hoping to use it to take the temperature of various human states,” reports MIT Technology Review. Given the amount of irrelevant nonsense on Twitter, it’s natural to be highly sceptical of the strategy. But if you think through the logic, analysing Twitter data isn’t such a strange idea. A basic premise of behavioural economics is that the markets aren’t perfectly rational machines, but are expressions of human emotions like greed and fear. If you agree with that premise, and are looking for an immediate way to measure and keep track of those human sentiments, then Twitter is one of the greatest tools ever invented.

A decade ago, anyone who suggested using social media to predict market movement would have been looked upon as if they had landed from outer space. Richard Peterson, managing director of Santa Monica-based MarketPsych, experienced just such a reaction eight years ago when he stated that social media could be mined for data about what people are thinking and feeling, and that this could translate into powerful investment ideas. “People would say to me, ‘You’re crazy,’” says Peterson, who has a post doctorate in neuro-economics from Stanford University.

But with usage of social media like Twitter exploding in recent years, analysts now have a real-time reflection of popular sentiment. As a result, Peterson’s MarketPsych serves up data to hedge funds and research firms like Titan Trading Analytics. “The importance of social media aggregation, and how that might influence the price of a stock, cannot be ignored,” said the latter company’s CEO John Coulter. “We’ve chosen to use it as one of many indicators, providing traders with alerts on events and by flagging socially expressed emotions which haven’t been picked up upon by traditional news outlets.”

Nobody is laughing at Peterson today. Although he is glad that investment managers are now finally taking him seriously, on the downside there is more competition as more and more people are trying to unlock the secret of this trading strategy from the billions of tweets out there. The trick is how to sift through that data effectively and make some sense of the hundreds of millions of tweets generated every day. Peterson, for example, filters the data using 1,500 different factors and keywords to track global moods. His take on the markets: if the public is overly bullish, it’s time to be cautious. If it is extremely gloomy, on the other hand, it might be time to snap up a bargain.

If you feel inspired, take a moment, however, before you start examining your own Twitter feed for brilliant investment ideas. Unless you are armed with highly complex mathematical models and teams of analysts who can dive into social-media data and asses the global mood from billions of tweets, you are going to find it tricky, as an individual investor, to make any sense of the collective tweets of 100 million active Twitter users.

Nevertheless, social-media sentiment analysis is immediate and ongoing, and the Twitter-analysis trend seems to be just gearing up. “It won’t make you a millionaire overnight, but it does work,” says Richard Gardner, president and CEO of Scottsdale, Arizona-based Modulus Financial Engineering, which collects historic Twitter data for hedge funds and research firms to crunch. “The markets are moved by emotion, and I think this is going to be the future of trading. You can actually see global moods moving up and down in real time.”