This weekend's Barron's has an in-depth piece by Bill Alpert who examines the results provided by David Greenblatt that are the core of his runaway best-selling book, "The Little Book That Beats the Market."
We noted the book in an earlier post. At that time we were more interested in examining the general philosophy behind portfolios populated with individual securities for individual investors.
The basis of the Barron's piece is an attempt to replicate the seemingly extraordinary returns claimed by Greenblatt in his book. The author is very careful to praise the book as an introduction to value investing. However the bottom line is that any backtested system has its own unique complications.
Greenblatt's strategy does make sense and probably would beat the market by some margin. But don't count on 30%-plus: Investment formulas that look great in a historical "back test" often prove less powerful in the real world. Trading commissions and other irksome frictions hurt performance, of course, but there are more fundamental limits to the predictive power of the kind of study The Little Book crows about. Simply put, the magic formula's jumbo returns were probably limited to the data Greenblatt used in his test.
For the sake of brevity we will encourage readers to get their hands on the Barron's article themselves. We believe the article raises two important caveats for investors interested in creating back-tested systems or employing a manager who does likewise.
The first point is one that is well-covered by the Alpert article. The article uses a number of means to replicate Greenblatt's results. They are never able to precisely match Greenblatt's results, but they get in the same general ballpark. Whether the system is fundamentally based, as in this case, or technical, i.e. price-based, there are always data issues and ambiguities involved in model construction. Even when the purveyor of the system is completely above board, as is presumed in this case, there are seemingly always technical issues. Back-tested systems are dependent on the database used and the time period covered. There is no guarantee that what worked in the past will be replicated in the future.
The second point we want to cover is not really touched on in the Barron's article. More important than the careful construction of an unbiased system is the actual implementation of the system in real-time. Even if you have carefully considered the costs of implementation, i.e. commissions, market impact etc., there is still an psychological component. Every (realistic) system devised will have periods of time in which it has drawdowns or periods of underperformance. The challenge for the investor is to either stick with the system during that time or realize that the system no longer works. While this problem is not all that different than it is for discretionary investors it is often downplayed by systems traders.
Back-testing can be an important tool that helps provide investors with insight into the fundamental workings of their ideas and the market in general. The difficulty with the Greenblatt book is that the historical returns are so high that they may blind investors into believing the system is "fool-proof." In short, no system is fool-proof.
The investing ideas in the Greenblatt book may be quite sound. However most investors, and especially those novice investors who may be reading the Greenblatt book, need to know the caveats involved in any back-tested system. There is a good reason that the phrase, "Past performance is no guarantee of future results" is a popular one with lawyers.
Controlled Greed has their own take on the Barron's piece. They are skeptical of "back-tested" results in general, feeling more comfortable selecting securities with a human touch. That said they also praise the Greenblatt book as a "sound investment guide."
Speaking of Barron's, the folks at Seeking Alpha have a interesting post up this week that is new to us. The "One Page Barron's Summary" is a nice compilation of the current week's editioni.
