Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
All Strategies at loss before June 2016 but profitable after in NT Backtests
Glad to say this is my first post up on Futures.io.
of course being my first post i ran into an issue that i thought some smart dudes from here could help me out with
i dont know if anyone else has ever ran into this or not but if you have any insight would be helpful.
Ive noticed that on ninjatrader's strategy analyser (donno bout other platforms) any script (given its a profitable one) i test from 06/2016 to current date seems to be profitable. but if i test it before 06/2016 it is almost always a straight failure. this happens on any script i test on futures and i dont know why. what is so special about the 06/2016 date...
somethings to keep in mind is that the scripts and codes i use all use bid ask prices for entrys and exits so buys are filled in at ask sells are at bid. i hard coded these in for proper backtesting. what i noticed is that without this hard code the above problem doesnt seem to happen.
so that lead me to look at spreads and sure enough the spread before 06/2016 was on avg 2x what it was after that date. however i have a hard time believing that is the cause of the drastic change in the equity curve.
does anybody know why this happens? and what i steps i should take.
thanks,
Mo
Can you help answer these questions from other members on NexusFi?
Interesting post... It sounds like you have solved it yourself mostly. If you observe this behavior using market orders generically, but when you specify buy ask, sell bid the issue goes away, then I think the spread, or the spread calculation logic in NT is the issue.
As far as next steps.... The strategy analyzer in NT is not that accurate. If you want to test something to see if it is profitable you should use their market replay. Here is kind of my general take on the use cases of the various testing tools NT has.
1. Strategy Analyzer: Good for market order based strategies with small spreads if you make slippage assumptions that are reasonable. This should never be used for limit order based strategies. Why? The SA tool doesn't have the same level of granularity as market replay.
2. Market Replay: This uses level 2 data to estimate your place in the queue and fills. So you can use limit orders with this strategy. But be aware of the following:
A: They assume 0 latency for execution speeds. Your order will hit the exchange immediately.
B: There queue tracking logic is more conservative than the real market. You can check their APQ code if you want to see how they estimate it. See for yourself..
C: Their level 2 and level 1 data feeds have some synchronization issues, so some of the fills you might get could be off on the timing.
Live SIM: Everything about market replay is the same except they have an execution time of around 125 ms for every order. If you are faster in real trading via co-locating, this will understate your speeds considerably. But most retail traders will be around this.
So try market replay, or live sim next IMO.
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
hmmm ya i think you might be right about me answering my own question about the spread being the issue. I am just finding it hard to believe that 1 or 2 ticks is causing such a drastic change. though i know 1 tick in trading can mean life or death... basically what i am thinking is if 1 or 2 ticks of spread effects the strategy so much then maybe it might not be such a good script at all.
and yes this happens on Market orders
however i should have added that this particular pattern i noticed is in the M6E and 6E. so i went to the EURUSD and tried on there as well same thing seems to happen. i think it has something to do with a change in the EURUSD in 2016 but i dont know what.
Yup im with you there on the market replay step already tried it on months and months of data.
so i got a question for you. why dont you think that NT's SA is good enough for Limit order based scripts? because i was going to go in that direction for tweaking my script. you say granularity but what about in scripts where i add tick data for bid and ask as the one i mention? or even adding tick data with SA tick resolution? you think this is still not good enough?
The issue with the SA will not be resolved by using a 1 tick time series. That approach can get you the correct price at a granular enough level, so if you get filled you would get filled on the right bid / ask price.... however the issue is whether or not you would have gotten filled in the first place. The SA does not run the level 2 volumes and sequence your position in the queue, so your fills on limit orders will be determined from the historical fill type algorithm to approximate your fills.
By Contrast the market replay and Live SIM actually marks your estimated place in the queue and then only fills your limit orders when and if the level 2 volume trades beyond your place in the queue. Here is an example.
You enter a buy bid at 2000 and at the time you submit your order the volumes look like this:
Moments later, the price moves up because the entire Ask queue of 100 contracts were filled before the 400 bid contracts got filled. So the question in this scenario is how many of the 400 people in line in the bid queue got filled before the 100 people in the ask queue got filled or canceled and this side broke?
In the real market, only 10%, 25% of the 400 in the bid queue might have gotten filled. The Market replay / Live SIM will attempt to simulate this using queue tracking logic. The SA by contrast just has a very simple algo to predict fills so it will not be as accurate.
In my experience the SA will overstate profitability with respect to limit order based strategies, Market Replay is more conservative but it too assumes 0 latency.... moving into live SIM you hit 150 MS execution speeds which for some depending on their strategy will be vastly different from market replay.
As long as you are aware of the mechanics of each, they all have their uses, and NT does a fair job, but handling limit orders is tricky for any testing tool.
I have actually spend a lot of time picking apart the mechanics of NT's testing tools and extracting the raw data my self and building my own assumption based models. For my use case 150 MS is too high, and 0 latency for obvious reasons is not a good assumption either for execution speeds.
Hope this helps.
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
ok i see what you are saying now. i understood that you ment the price bar granularity but latency and actually fills are the issue that physically can not be replicated in SA. cool cool i get it. i tended to stay away from limits for that exact reason in the beginning. turns out its much more of a pain in the butt than i thought.
so going back to the M6E (eurusd) in june of 2016 having a change of heart, do you know anything that might have caused the implied spread change? still can't pin point it, or have you ran into this issue yourself at all?
yup i double checked all logic a lot. and its not new either so i know its working properly. i also tried deleting all the hist data and redownloading it from nt servers still no change.
so ya i guess it was just the market. but thats very werid that it happens like that.
