Possible to backtest long periods in NT with market replay?

July 28th, 2010, 09:02 PM

Hi all,

Is it possible to use market replay data for backtesting large periods, or do you have to play it live and have your strategy running on each market replay file?

August 3rd, 2010, 06:33 AM

backtesting auto-strategys works with historic datas - not with the recorde Market-replay-datas.

thats something different.

market-replay is more for training, to regognize your signals + execute your trades i would say.

September 13th, 2012, 05:21 PM

Let me preface by saying
1. I�ve read everything I could find on the differences between backtesting, market replay and live trading, so I�m aware of the major issues.
2. My strat uses ExecuteOnBarClose = true.

I've noticed that when I load a strat on a chart in in market replay, it applies that strat immediately to whatever history my chart loads and is available in MR. These results are reportable in Strategy Performance. I noticed also that the P&L from this �historical MR� was identical (day by day) to the same timeframe I had Backtested with the same strat (with a couple of exceptions).

But: when I ran the strat �live� in MR over a day from the same period, the results for that day were slightly off. I discovered that 6 out of 8 executions occurred 1 bar later (and 1 tick away) from the backtested and MR �pre-loaded� history.

So I�m trying to figure out a few things:
1. Are MR �live run� trades executed the same as the MR �pre-loaded� historical executions? I hope the answer is yes, but it looks like �no.�
2. Since my strat executes on bar close, will my Backtest results be as reliable as MR �live run� results?
3. If I forward test my ExecuteOnBarClose strat in live real time, then download the MR data for the day and run the strat in MR, should my results be the same? Seems like they should be close.
4. I was hoping that I could shortcut the MR process by simply downloading the replay data for the time period I need, then in MR �going to� the last day and getting Strategy Performance data for it. This would save much time over running 90 days in MR, even at 500x.

NinjaTrader · September 13th, 2012, 05:28 PM

jacqudy

Let me preface by saying
1. I’ve read everything I could find on the differences between backtesting, market replay and live trading, so I’m aware of the major issues.
2. My strat uses ExecuteOnBarClose = true.

I've noticed that when I load a strat on a chart in in market replay, it applies that strat immediately to whatever history my chart loads and is available in MR. These results are reportable in Strategy Performance. I noticed also that the P&L from this “historical MR” was identical (day by day) to the same timeframe I had Backtested with the same strat (with a couple of exceptions).

But: when I ran the strat “live” in MR over a day from the same period, the results for that day were slightly off. I discovered that 6 out of 8 executions occurred 1 bar later (and 1 tick away) from the backtested and MR “pre-loaded” history.

So I’m trying to figure out a few things:
1. Are MR “live run” trades executed the same as the MR “pre-loaded” historical executions? I hope the answer is yes, but it looks like “no.”
2. Since my strat executes on bar close, will my Backtest results be as reliable as MR “live run” results?
3. If I forward test my ExecuteOnBarClose strat in live real time, then download the MR data for the day and run the strat in MR, should my results be the same? Seems like they should be close.
4. I was hoping that I could shortcut the MR process by simply downloading the replay data for the time period I need, then in MR “going to” the last day and getting Strategy Performance data for it. This would save much time over running 90 days in MR, even at 500x.

The short answer is that historical is not expected to match replay and replay is not expected to match live simulation and live simulation is not expected to match live cash. There are many variables between each scenario that can impact the results not to mention the strategy itself and the types of orders used to enter and exit positions. I suspect you have already read the following but I post it just in case you have not.

Discrepancies: Real-Time vs Backtest

Possible to backtest long periods in NT with market replay?

Discussion in Traders Hideout

Possible to backtest long periods in NT with market replay?