hello, need some opinion/help on how i can solve my problem
i've created and tested a system on market replay, and now i am doing testing on real time data. my system enters on a limit order when XYZ happens. Most of my orders on market replay data is filled, however, on real time (still simulation) my orders are not filled and then cancelled.
Any idea how i can find a solution to this? i do know that market replay data is compiled 1/4 of a second?
The market replay data is Zenfire data and should be more or less identical with Zenfire live data. However, if you use broker feeds (for example Interactive Broker's snapshot data) you will get something entirely different. If you use a different real-time feed such as Kinetick, there also may be minor differences.
Now let us assume that you use the NinjaTrader simulator with Zenfire data. The main difference between market replay and simulation on live data is
market replay -> the simulator settings do NOT apply
replay on live data -> the simulator settings do apply
This means that real-time data simulation takes into account latency, market replay does not. In particular there are three different delay periods
(1) transmission of trade data from the exchange to your PC (this is not simulated but real)
(2) order transmission to exchange (default value is 75 msec)
(3) execution of order and confirmation (default value is 150 msec)
That said, if you trade in Europe and access an exchange in the US, the default values of the order transmission time (2) and the execution and confirmation time (3) need to be increased. But let us assume that the default values for the simulator are sufficient.
If you trade with real money, your limit order will be sitting in an order queue, possibly at the exchange. Whether you get a fill or not when the limit price is touched depends on your rank in the order queue. In a simulation environment, whether market replay or live simulation, it is near-impossible
-> to estimate your rank in the order queue
-> how your orders would have impacted the order book and market prices
This means that there are limitations as to what can be simulated. The impact of your trading on the market is usually neglected. The fill algorithms of the simulation machine will only fill you order once the limit price has been penetrated, not if it has been touched. This is a conservative setting, but better than a too aggressive setting for the simulation, as it should keep you out of trouble once you are going live.
NinjaTrader allows you to change the simulation settings under Tools -> Options -> Simulator. You can check "Enforce immediate fills", and you will get better fills for limit orders. Not sure though that those fills are realistic.
If the system that you have developed does not work with the real-time simulation, then it is probably not worth to develop it any further.
The following 3 users say Thank You to Fat Tails for this post:
Actually the fill algorithms in live simulation and I am pretty sure, in Market Replay both may fill your order on touch. Also, I am pretty sure the simulator settings DO apply to Market Replay. I do find that Market Replay and the live simulation both fill limit orders too easily/quickly on the touch.
An easy solution to get better correlation between Market Reply and Live is to simply add one tick to your profit target orders. Normally, not always of course, you need to move one tick past limit order to fill. So instead of using a 4 tick target on MR, use a 5 tick one. For 8t use 9t. And so on...
My Market Replay does delay its target execution a little, but not back testing so no sweat.