Financial Backtesting: A Cautionary Tale
The biggest problem with back-testing in finance that I&#x27;ve come across is that markets change.<p>Nowadays markets change every 5-10 years,this is  my own personal belief here, which means you can&#x27;t do too many meaningful comparisons over longer time periods.  Couple that with the fact that most bull markets last for 5-10 years and this means you can&#x27;t really tell how your active portfolio will perform.<p>Or put another way, strategies that work on a macro level tend not to work very well.  Anything of the flavor of &quot;if the market does X for past 5 days, then buy Y and hold until Z happens tend not to be reproducible or testable in a meaningful way.&quot;<p>What does work, again from personal experience, is arbitrage.   Arbitrage is everywhere in the financial markets.<p>- 5 year bonds that are 2 years old look just like 3 year bonds that were just issued, assuming all terms are the same.<p>- The A and B shares of a single company tend to follow a constant pattern<p>- 2 different ETF&#x27;s that follow the same index(say SPY) tend to move together,<p>- exchanges that tend to cross at midpoint( most dark pools) and price via the SIP tend to offer a half penny latency arbitrage opportunities<p>- when two companies merge the price of the target will converge on the deal price at the closing.<p>- low latency arb where stock prices follow some commodities price, banks vs currency rates, oil produces vs WTI, etc<p>These are all examples of things that you can meaningfully back-test and model.<p>Jim Simmons of Renaissance Capital once remarked that there is no signal in the market data at a macro level,  I tend to agree with him and the author.