Someone correct me if I'm wrong, but the gist of it is...
The issue has to do with "intra-bar granularity". You can see the effect when your strategy only operates on the primary time period/frame (BarsInProgress = 0), but when you simply add a new time period your results change even though your strategy does not interact with this additional time period.
The workaround suggested by NinjaTrader is to make your entries on a higher time period such as a tick or minute chart. For example: EnterLongLimit(1, Close); Which enters on the secondary time period within the strategy.
So what I do is compare the backtesting results of one day to the Market Replay results of the same day. If I get somewhat closer results then I'm a little more confident in the backtesting results as a whole. Next I will sim/paper trade the strategy with live data, which is where I am with my trading journey.
IMHO I find most people expect more than is reasonable from backtesting software. As in most things, it pays to keep things simple. I always make sure that I have very very few trades where the entry and the exit are on the same bar in the backtest results. If your software - NT7, TS, whatever - is hitting exits on the same bar as entries, I think you are asking too much of the software to cope with the situation in a way that reflects real life.
Personally I would programme my own backtesting software to be pessimistic and that if the bar hits both the target and the stop on the same bar, then it chooses the stop.
You can discover what your enemy fears most by observing the means he uses to frighten you.
The following user says Thank You to Adamus for this post: