NT8 - Wildly Differing Backtesting Results

October 7th, 2023, 04:09 AM

I'm a bit frustrated with the NT8 Backtesting process.

There are several different methods that i can use to backtest a strategy.

I use these methods below, listed by order of processing time (Quickest to Slowest)

1. Strategy Analyzer (Tick Replay Off + Order Fill Standard)
2. Strategy Analyzer (Tick Replay Off + Order Fill High 1 Tick)
3. Strategy Analyzer (Tick Replay On+ Order Fill Standard)
4. Market Replay (Playback Connection)

My results are wildly different using all the different methods.
I cant work out which one is accurate, and hence whenever i put strategy i have created running 'Live' it just blows up.

Can anyone shed some light? Or offer some advice?

To highlight my findings, i have done a backtest using all these different methods. Exact same simple single timeframe strategy, exact dates, time frame, instrument etc.

October 7th, 2023, 07:42 AM

Backtesting has never been accurate for me.
I hope others can post examples to show how they backtest.
NT Support created a video about how to compare results of backtesting with several methods.
Perhaps it will help you to discover the differences.

The script used in the video is here
RealtimeReplayHistoricalComparisonsExample_NT8

October 7th, 2023, 08:19 AM

AFAIK Market Replay is the most precise method.

NT8 - Wildly Differing Backtesting Results

Discussion in NinjaTrader

NT8 - Wildly Differing Backtesting Results