All Strategies at loss before June 2016 but profitable after in NT Backtests - futures io
futures io futures trading



All Strategies at loss before June 2016 but profitable after in NT Backtests


Discussion in NinjaTrader

Updated
    1. trending_up 736 views
    2. thumb_up 5 thanks given
    3. group 3 followers
    1. forum 6 posts
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

All Strategies at loss before June 2016 but profitable after in NT Backtests

(login for full post details)
  #1 (permalink)
Tucson arizona
 
 
Posts: 31 since Jun 2018
Thanks: 4 given, 4 received

Hey guys,

Glad to say this is my first post up on Futures.io.

of course being my first post i ran into an issue that i thought some smart dudes from here could help me out with

i dont know if anyone else has ever ran into this or not but if you have any insight would be helpful.

Ive noticed that on ninjatrader's strategy analyser (donno bout other platforms) any script (given its a profitable one) i test from 06/2016 to current date seems to be profitable. but if i test it before 06/2016 it is almost always a straight failure. this happens on any script i test on futures and i dont know why. what is so special about the 06/2016 date...

somethings to keep in mind is that the scripts and codes i use all use bid ask prices for entrys and exits so buys are filled in at ask sells are at bid. i hard coded these in for proper backtesting. what i noticed is that without this hard code the above problem doesnt seem to happen.

so that lead me to look at spreads and sure enough the spread before 06/2016 was on avg 2x what it was after that date. however i have a hard time believing that is the cause of the drastic change in the equity curve.

does anybody know why this happens? and what i steps i should take.


thanks,
Mo

Reply With Quote

Journal Challenge February 2021 results:
Competing for $1500 in prizes from Topstep
looks_oneSBtrader82 's Trading Journalby SBtrader82
(173 thanks from 31 posts)
looks_twoJust BEING a Trader: Letting Go!!by iqgod
(120 thanks from 33 posts)
looks_3Wisdom is Emptinessby Mtype
(68 thanks from 25 posts)
looks_4Deetee’s DAX Trading Journal (time based)by Deetee
(35 thanks from 17 posts)
looks_5Journal for peanuts1956by peanuts1956
(23 thanks from 13 posts)
 
 
(login for full post details)
  #2 (permalink)
charlotte nc
 
Experience: Master
Platform: ninjatrader
Broker: NinjaTrader
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 398 since Jan 2015
Thanks: 76 given, 1,081 received

Interesting post... It sounds like you have solved it yourself mostly. If you observe this behavior using market orders generically, but when you specify buy ask, sell bid the issue goes away, then I think the spread, or the spread calculation logic in NT is the issue.

As far as next steps.... The strategy analyzer in NT is not that accurate. If you want to test something to see if it is profitable you should use their market replay. Here is kind of my general take on the use cases of the various testing tools NT has.

1. Strategy Analyzer: Good for market order based strategies with small spreads if you make slippage assumptions that are reasonable. This should never be used for limit order based strategies. Why? The SA tool doesn't have the same level of granularity as market replay.

2. Market Replay: This uses level 2 data to estimate your place in the queue and fills. So you can use limit orders with this strategy. But be aware of the following:
A: They assume 0 latency for execution speeds. Your order will hit the exchange immediately.
B: There queue tracking logic is more conservative than the real market. You can check their APQ code if you want to see how they estimate it. See for yourself..
C: Their level 2 and level 1 data feeds have some synchronization issues, so some of the fills you might get could be off on the timing.

Live SIM: Everything about market replay is the same except they have an execution time of around 125 ms for every order. If you are faster in real trading via co-locating, this will understate your speeds considerably. But most retail traders will be around this.

So try market replay, or live sim next IMO.

Ian

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to iantg for this post:
 
(login for full post details)
  #3 (permalink)
Tucson arizona
 
 
Posts: 31 since Jun 2018
Thanks: 4 given, 4 received


hmmm ya i think you might be right about me answering my own question about the spread being the issue. I am just finding it hard to believe that 1 or 2 ticks is causing such a drastic change. though i know 1 tick in trading can mean life or death... basically what i am thinking is if 1 or 2 ticks of spread effects the strategy so much then maybe it might not be such a good script at all.

and yes this happens on Market orders

however i should have added that this particular pattern i noticed is in the M6E and 6E. so i went to the EURUSD and tried on there as well same thing seems to happen. i think it has something to do with a change in the EURUSD in 2016 but i dont know what.

Yup im with you there on the market replay step already tried it on months and months of data.


so i got a question for you. why dont you think that NT's SA is good enough for Limit order based scripts? because i was going to go in that direction for tweaking my script. you say granularity but what about in scripts where i add tick data for bid and ask as the one i mention? or even adding tick data with SA tick resolution? you think this is still not good enough?

Reply With Quote
The following user says Thank You to TravelingSalesmen for this post:
 
(login for full post details)
  #4 (permalink)
charlotte nc
 
Experience: Master
Platform: ninjatrader
Broker: NinjaTrader
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 398 since Jan 2015
Thanks: 76 given, 1,081 received

The issue with the SA will not be resolved by using a 1 tick time series. That approach can get you the correct price at a granular enough level, so if you get filled you would get filled on the right bid / ask price.... however the issue is whether or not you would have gotten filled in the first place. The SA does not run the level 2 volumes and sequence your position in the queue, so your fills on limit orders will be determined from the historical fill type algorithm to approximate your fills.

By Contrast the market replay and Live SIM actually marks your estimated place in the queue and then only fills your limit orders when and if the level 2 volume trades beyond your place in the queue. Here is an example.

You enter a buy bid at 2000 and at the time you submit your order the volumes look like this:

Bid Price = 2000
Bid Volume = 400
Ask Price = 2000.25
Ask Volume = 100

Moments later, the price moves up because the entire Ask queue of 100 contracts were filled before the 400 bid contracts got filled. So the question in this scenario is how many of the 400 people in line in the bid queue got filled before the 100 people in the ask queue got filled or canceled and this side broke?

In the real market, only 10%, 25% of the 400 in the bid queue might have gotten filled. The Market replay / Live SIM will attempt to simulate this using queue tracking logic. The SA by contrast just has a very simple algo to predict fills so it will not be as accurate.

In my experience the SA will overstate profitability with respect to limit order based strategies, Market Replay is more conservative but it too assumes 0 latency.... moving into live SIM you hit 150 MS execution speeds which for some depending on their strategy will be vastly different from market replay.

As long as you are aware of the mechanics of each, they all have their uses, and NT does a fair job, but handling limit orders is tricky for any testing tool.

I have actually spend a lot of time picking apart the mechanics of NT's testing tools and extracting the raw data my self and building my own assumption based models. For my use case 150 MS is too high, and 0 latency for obvious reasons is not a good assumption either for execution speeds.


Hope this helps.

Ian



TravelingSalesmen View Post
hmmm

so i got a question for you. why dont you think that NT's SA is good enough for Limit order based scripts? because i was going to go in that direction for tweaking my script. you say granularity but what about in scripts where i add tick data for bid and ask as the one i mention? or even adding tick data with SA tick resolution? you think this is still not good enough?


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Reply With Quote
The following 3 users say Thank You to iantg for this post:
 
(login for full post details)
  #5 (permalink)
Tucson arizona
 
 
Posts: 31 since Jun 2018
Thanks: 4 given, 4 received

ok i see what you are saying now. i understood that you ment the price bar granularity but latency and actually fills are the issue that physically can not be replicated in SA. cool cool i get it. i tended to stay away from limits for that exact reason in the beginning. turns out its much more of a pain in the butt than i thought.

so going back to the M6E (eurusd) in june of 2016 having a change of heart, do you know anything that might have caused the implied spread change? still can't pin point it, or have you ran into this issue yourself at all?

Reply With Quote
 
(login for full post details)
  #6 (permalink)
Houston, TX
 
Experience: Intermediate
Platform: Python
Trading: NQ
 
Posts: 97 since Apr 2016
Thanks: 130 given, 93 received

Have you looked at the price chart and/or individual trades to see if your entry/exit logic is being executed consistently?

Perhaps your historical data is different before/after that certain date?

If everything is working correctly, it could simply be that market conditions changed on 6/16.

Reply With Quote
 
(login for full post details)
  #7 (permalink)
Tucson arizona
 
 
Posts: 31 since Jun 2018
Thanks: 4 given, 4 received


ShatteredX View Post
Have you looked at the price chart and/or individual trades to see if your entry/exit logic is being executed consistently?

Perhaps your historical data is different before/after that certain date?

If everything is working correctly, it could simply be that market conditions changed on 6/16.

yup i double checked all logic a lot. and its not new either so i know its working properly. i also tried deleting all the hist data and redownloading it from nt servers still no change.

so ya i guess it was just the market. but thats very werid that it happens like that.

Reply With Quote


futures io Trading Community Platforms and Indicators NinjaTrader > All Strategies at loss before June 2016 but profitable after in NT Backtests


Last Updated on June 26, 2018


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Battlestations! Show us your trading desk - $1,500 in prizes!

March
 

Call Option Buying: The New Pain Trade? w/Carley Garner

Elite only
 

Importance of Finding Your Own Way w/Adam Grimes

Elite only
 

New Challenge (TBD)

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts