What are the problems with backtesting?

March 3rd, 2010, 09:44 AM

Hey,

I am relatively new to Ninjatrader and am using it for backtesting some strategies.

Besides for the obvious fact that curve fitting is evil and the past is not the future, what are the problems that can come out of backtesting?

In other words, if you run a backtest (with default filling) and it shows profit, then what are the reasons that it would not work in a live trading environment?

This is assuming a strategy is run with all limit orders off the market, so slippage should be negligible. I currently have a backtest for a winning strategy but it shows avg time in market = 0.0......how is this possible?

Thanks
-Jeff

March 3rd, 2010, 01:14 PM

One thing I have seen is that when back testing, both with market replay and with backtest in the strategy analyzer, when using limit orders, Ninja has to "guess" if you bid or offer is hit if price just touched your price. It looks to me to be somewhat random and evens out 50% with you and 50% against you. However in live trading, whether or not your bid or offer gets hit is more likely to be in the opposite direction of what you want. So in short, during back testing you get a 50/50 chance of your bid/offer getting hit. during live trading, it will be perhaps 80/20 against you.

March 3rd, 2010, 01:39 PM

Isnt that Default vs Liberal fill? I thought that default is worst case scenario where you are only filled if it completely trades through your price?

Also, I am having a problem where my entry and exit is executed at the same second and is always a 2 tick winner......Is this something in my code or a bug?

March 5th, 2010, 05:08 PM

jeffbg123

I am having a problem where my entry and exit is executed at the same second and is always a 2 tick winner......Is this something in my code or a bug?

Guess that would explain the positive ROI, eh? Could you post the code or a chart?

What are the problems with backtesting?

Discussion in Traders Hideout

What are the problems with backtesting?