Tuesday, May 29, 2007

How accurate are bloggers?

I did some scratch work analysis of the Ticker Sense Blogger Sentiment Poll to see how accurate individual bloggers were on calling the S&P 30-days out. It is a shortened analysis based on 2007 performance (starting from December 2006 to allow for a January call, ending on April 23rd - the most recent recordable projection). Not all bloggers which contributed an opinion are included chiefly because I prepared my spreadsheet from recent opinion givers and worked backwards - so some bloggers who were regular providers in the early part of the year but no longer are (in recent months) are excluded.

The data covers 20 polls. The S&P was measured from the open of the start of the week to the close 1-month later. Because the Ticker Sense options are bullish/bearish/neutral I defined a neutral outlook as anything less than a 1% gain or loss; bullish been >1% gain and bearish >1% loss.

Based on these parameters the S&P scored 12 Bullish periods, 4 Bearish periods and 4 Neutral periods [for an average score of 40% net bullish bias based on a 100% (20 bull periods) to -100% (20 bear periods) range].

The most accurate blogger was the Daily Dose of Optimism at 61% correct calls, followed by Carl Futia and Quant Investor at 60% correct calls. Although Daily Dose nicked the overall edge honors by providing 11 correct calls out of 18, compared to 12 for both Carl Futia and Quant Investor out of a maximum 20.

In terms of the net opinion of bloggers, both Carl Futia and Daily Dose of Optimism were the most bullish with all of their reported polls on the plus side. In contrast: Crowder Blog, Random Roger, Learning Curve and Millionaire Now were bearish for every provided opinion.

Interestingly, given the overall bullish performance of the S&P it was not the most bearish of individuals which gave the poorest accurate forecasts, but those in the middle ground who suffered most. The fitted line shows an increase in call accuracy as opinion turned more bullish - which in a bullish market is to be expected. But if one was to look at the accuracy of bloggers who were 6% to 27% bearish (all below the line) and compared it to the accuracy of bloggers who were 35% to 74% bearish - even those who were completely bearish, which are all above the line - there is a considerable difference in accuracy. Part of this may be attributed to the arbitary +/- 1% range of a neutral call making it tougher to be accurate compared to the infinite range of been bullish or bearish. On the flip side, it might be a case of less is more - trying to time the market is a more difficult task than been consistently wrong, if that makes sense!

There are plenty of caveats. This is not a conclusive result as the second half of 2006 was excluded from the analysis and I know from my own provided opinion that I was predominantly (and incorrectly) bearish for the bulk of that period - so my % correct would be lower and I wouldn't be sitting in the middle of the back as regard to net bull/bear opinion. Also, not all bloggers gave an opinion over the 20 polls and there is some sample bias in this regard (e.g. Knight Trader only made 1 call which was correct over the 20 polls - his blog was excluded from the data). I also haven't back checked to ensure I attributed the correct call to each blogger. Finally, Ticker Sense may not have published all opinions - I know I am missing two which I am confident I gave, but may have missed the deadline for submission.