Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I think you and I have crossed paths before on similar threads trying to deconstruct and understand the NT simulation engine. I recently started migrating my code base from 7 to 8 to have another point of comparison and I hit some fairly significant differences in testing between 7 and 8.
I have strategies that beat one or the other or both, but there are some strategies that do very well in one and very bad in another. So I am wondering myself where the line is on what is a realistic representation of the market.
I offered up 2 strategies (one in 7 and one 8) and they show significantly different results. If anyone is interested in just testing and providing feedback on what behavior they think is more realistic I would be glad to get the feedback.
It sounds like from your experience, you favored the behavior of 7 more, is this correct?
Thanks for the question Ian. I say "thanks" because it reminded me that I wanted to test something out and just now I have had the chance to do it.
In this particular case NT support suggested that NT8 is more 'conservative' when filling orders during Market Replay, which is why I had two different experiences between NT7 and NT8. This was back in February.
During last summer, however, I discovered a feature on NT8 called "tick replay" which, as I understand it, replays the trades in the same order as they occurred live.
I first tried that via Market Replay after a live trade I had placed on CL - since Tick Replay was off, I did not get filled. I tried enabling Tick Replay and replayed the CL session, this time being filled.
After having read your question today I tried again on that old trade from February, this time enabling Tick Replay, and this time too I got filled.
So it appears that this option increases accuracy - that's what NT is saying anyway.
You and I were both on this old thread and have both been trying to understand the underlying assumptions and accuracy of the simulation engines for a while now. I have made a little progress in my research since my last post to this thread so I will add a few things that might be helpful to you and others when testing.
1. In NinjaTraader 8 there is a cs file called APQ that has the estimated queue tracking code that NinjaTrader uses. After doing extensive testing, I have found that their fill simulation is more or less the same as their queue tracking. So if you want to peak under the hood and see the code logic, you easily can. But in a nutshell here is how it works... They only improve your position in the queue when the total volume drops below your position. So lets say you are # 500 in the queue and the queue moves up suddenly to 800 and then back down to 500 after some time... How does that delta of 300 (800-500) impact your place in the queue? It has 0 impact. You only improve your position in the queue once the volume drops below 500. Ray confirmed this on an old thread I had with their support staff a while back. It's a fair approach, but a very conservative approach. People that wait in line for a long time will generally get more movement in the queue because you always start off in the back of the line not the front. NT assumes everyone is always at the very front of the line, and unless the volume drops below them, they don't see any improvement.
2. The different in the market replay vs. the Live SIM can be huge for most traders. Virtually everyone has the same experience where the market replay results are significantly better than live SIM. There is a reason for this. Here it is: Market replay assumes 0 latency on order execution. So you move from Initialized > Submitted > Accepted > Working in 0 MS on every trade, every time. All entries, exits and cancels have 0 latency. By contrast Live SIM has a built in 100 to 200 MS of latency (I think it is something like 120) on every execution (entries, exits, cancels). So this alone will put people a few price levels off from where market replay will mark you. Also, (this part is more obvious) the live data feed can lag in real time, but by contrast market replay is just reading the file off your machine and will always be perfectly in sync with 0 lag.
3. This may not be a big one, but I think it needs to be stated. The OnBarUpdate method can front run the level 1 and level 2 feeds, in market replay, Live SIM, and Live Trading. I have a thread on this topic somewhere on their support forum too... But the key thing here is that the SIM engine fills off of the level 1 feed from OnMarketData and not OnBarUpdate. So if you are getting signals from OnBarUpdate, you can actually be front running the SIM engine and seeing into the future a tiny bit. This only applies for HFT types of systems, but you can get false positives due to this front running in some cases depending on how you use your signals. I doubt this would impact any chart traders at all though.
4. There is a strange bug with cancels where you can actually land a cancel after a price level has changed against your position in Simulation (Live SIM and market replay). I attribute this to the front running of the OnBarUpdate vs. the event handler that fills you. (Same as Point 3 above basically)
Overall it's a very good product regarding simulating fills and has more complexity than the other tools on the market. I think the only thing that makes it hard for me personally is the latency in executions... I have to either pick between 0 (Market replay) or (120 MS) live sim. With my particular co-location, in live trading I am in between 0 and 25 MS on every execution... And there is practically no way I can use NT to model for that particular latency scenario. So it's just not accurate at all for my use case as is. I have to take the market replay results and then guess at how bad I will do with real execution latency vs. their 0 latency. I think I put it out there one time asking them to give users an option to address this, but they stated that they are not pursuing features for HF traders at this time.
Anyway... Hope this helps!
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
It sounds like you and I come to this from different angles, i.e. you, if I'm not mistaken, are looking at it from a HF or quasi-HF point of view, and probably by doing a lot of backtesting.
On the other hand, I am a complete backtesting virgin on NT8. I never backtested once on it. I am not familiar with OnBarUpdate. Basically I use Market Replay much in the same way @GruttePier does, i.e. just playing each day as if it were a normal day and trade off it. In fact I don't even look at NT sim anymore. Whenever I need to be on SIM, I use my CQG SIM which to me seems like an excellent SIM engine.
Having said the above, I still tend to get occasionally annoyed when I go and replay on NT8 a trade that I took live on CQG and I don't get filled in Market Replay when I was filled live (using Tick Replay).
Certainly your tip about the queue position code as well as the latency that NT applies are concepts that help broaden the picture, although it sounds like they won't solve the "perfect emulation" of the live environment I was led to believe "tick replay" was going to provide. I intuitively understand why: after all if I am market replaying, I can in no way affect the market dynamics that were taking place at that time: be it placing 1, 10 or 100 lots in Replay wouldn't change what the market would have really done.
I'm sure at some point I will look more closely at the whole HF thing which I find fascinating on a Order Flow level, and when that time comes I'll probably pick your brain.
