NexusFi: Find Your Edge


Home Menu

 





Trouble with LIVE v. BACKTEST


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one patbateman with 9 posts (0 thanks)
    2. looks_two Fat Tails with 3 posts (4 thanks)
    3. looks_3 TimeTrade with 3 posts (2 thanks)
    4. looks_4 ThatManFromTexas with 2 posts (1 thanks)
      Best Posters
    1. looks_one Big Mike with 2 thanks per post
    2. looks_two Fat Tails with 1.3 thanks per post
    3. looks_3 TimeTrade with 0.7 thanks per post
    4. looks_4 ThatManFromTexas with 0.5 thanks per post
    1. trending_up 8,621 views
    2. thumb_up 10 thanks given
    3. group 5 followers
    1. forum 18 posts
    2. attach_file 9 attachments




 
Search this Thread

Trouble with LIVE v. BACKTEST

  #11 (permalink)
 
ThatManFromTexas's Avatar
 ThatManFromTexas 
Houston,Tx
 
Experience: Advanced
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: TF
Posts: 2,265 since Feb 2010
Thanks Given: 1,206
Thanks Received: 4,348


patbateman View Post
To make things even more amazingly confusing, here is what happened...

I ran the system on a different contract, this time the Dow (YM). The backtest and the replay are 99.999% identical.

I don't know what to make of this, other than there is something in fact going on with the ES contract in replay or backtest. I'll have to investigate tomorrow, it's bedtime now. The NT team probably thinks I'm making this all up to sabotage them... I'm not!

Thank you lornz!

Try running your strategy real time in SIM for a month and at the end of the month back testing the strategy against that same month. Compare the results . When I tried it, I never found any correlation between the two results.

I'm just a simple man trading a simple plan.

My daddy always said, "Every day above ground is a good day!"
Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
Ninja Mobile Trader VPS (ninjamobiletrader.com)
Trading Reviews and Vendors
Online prop firm The Funded Trader (TFT) going under?
Traders Hideout
Deepmoney LLM
Elite Quantitative GenAI/LLM
NexusFi Journal Challenge - April 2024
Feedback and Announcements
Are there any eval firms that allow you to sink to your …
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
44 thanks
NexusFi site changelog and issues/problem reporting
24 thanks
GFIs1 1 DAX trade per day journal
22 thanks
The Program
19 thanks
  #12 (permalink)
 TimeTrade 
Erfurt Germany
 
Experience: Master
Platform: rtMagic, NinjaTrader
Broker: AMP/CQG, IAB
Trading: ES, 6E, FDAX, FGBL
Posts: 338 since Aug 2011
Thanks Given: 54
Thanks Received: 461


patbateman View Post
To make things even more amazingly confusing, here is what happened...

I ran the system on a different contract, this time the Dow (YM). The backtest and the replay are 99.999% identical.

I don't know what to make of this, other than there is something in fact going on with the ES contract in replay or backtest. I'll have to investigate tomorrow, it's bedtime now. The NT team probably thinks I'm making this all up to sabotage them... I'm not!

Thank you lornz!

I mine, you have problem with your non tick based quotes...

Please switch your chrat to !seconds! (1hour = 3600sec)... now NT7 create all bars by using tickdata.
If you have continue wrong results, then juri tools have i internal worng programmed code for backtest

example: one candle = only one tick -> !FirstTickOfBar is equal lastTickOfBar! Any code If(FirstTickOfBar) { /* code */ } else {/* other code*/} well on Backtest never call the "other code".

The most parts of juri tools its give free (full source, no crack)... then you can self check this...

Reply With Quote
Thanked by:
  #13 (permalink)
 
patbateman's Avatar
 patbateman 
NY, NY
 
Experience: None
Platform: T4, NinjaTrader
Trading: All!
Posts: 235 since Mar 2012
Thanks Given: 113
Thanks Received: 200



ThatManFromTexas View Post
Try running your strategy real time in SIM for a month and at the end of the month back testing the strategy against that same month. Compare the results . When I tried it, I never found any correlation between the two results.

Zing!

"A Jedi's strength flows from the force."
-Yoda
Started this thread Reply With Quote
  #14 (permalink)
 
patbateman's Avatar
 patbateman 
NY, NY
 
Experience: None
Platform: T4, NinjaTrader
Trading: All!
Posts: 235 since Mar 2012
Thanks Given: 113
Thanks Received: 200

Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

"A Jedi's strength flows from the force."
-Yoda
Attached Thumbnails
Click image for larger version

Name:	YM Backtest.jpg
Views:	218
Size:	549.1 KB
ID:	71841   Click image for larger version

Name:	YM Replay.jpg
Views:	180
Size:	543.9 KB
ID:	71842  
Started this thread Reply With Quote
  #15 (permalink)
 
Fat Tails's Avatar
 Fat Tails 
Berlin, Europe
Market Wizard
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
Posts: 9,888 since Mar 2010
Thanks Given: 4,242
Thanks Received: 27,102


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.


Do you call that identical?




The backtest was made with setting CalculateOnBarClose = true on historical data. The replay used intra-bar fills as opposed to fills at the bar close. The only thing that one can say is that all the differences and errors cancel out.

Reply With Quote
  #16 (permalink)
 TimeTrade 
Erfurt Germany
 
Experience: Master
Platform: rtMagic, NinjaTrader
Broker: AMP/CQG, IAB
Trading: ES, 6E, FDAX, FGBL
Posts: 338 since Aug 2011
Thanks Given: 54
Thanks Received: 461


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

Without your indicator and strategy settings in continue we make a quiz show here...

Error of NT7?
Error of your used Juri tool?
Error in your settings? (example by Fat Tails: CalcBarOnClose=True on ALL points?)
Error in your strategy?

-> If you don't can or well give more detailed informations, you must self find this problem.

Reply With Quote
Thanked by:
  #17 (permalink)
 
patbateman's Avatar
 patbateman 
NY, NY
 
Experience: None
Platform: T4, NinjaTrader
Trading: All!
Posts: 235 since Mar 2012
Thanks Given: 113
Thanks Received: 200


Fat Tails View Post
Do you call that identical?

No I call it 'practically identical.' All but one of the trades were within the hour. The reason why one of the trades was missing was because the replay had one open position at the end of the replay session (contract rollover day). Good enough for 60min non-intraday IMO, especially since backtests are generalizations anyways. I attached the backtest v. replay results btw which I didn't share above with the trades v. trades post - only one trade off, which was the one that didn't close at the end of the replay.

I got some email going with NT to see if we can figure the problem and how I can fix it... it's looking like my ES tick data or the NT ES replay data is faulty. I'm running the system through NQ and SI, and finding so far that the results are also the same compared to backtest results, just like with YM.


TimeTrade View Post
Without your indicator and strategy settings in continue we make a quiz show here...

Error of NT7?
Error of your used Juri tool?
Error in your settings? (example by Fat Tails: CalcBarOnClose=True on ALL points?)
Error in your strategy?

-> If you don't can or well give more detailed informations, you must self find this problem.

Calc on bar close should be true... I was thorough with the settings (reviewed them again and again) and my code, cross referencing everything. I wouldn't of gone with putting in the effort to make the thread if I knew I could solve it myself. I'm really just wondering if anyone has had a similar experience, perhaps because a problem with Jurik tools or the ES data in particular would be something someone else already came across. Btw, you shouldn't expect me or anyone else frankly to be handing over hard work... indicators are fine, which is why Mark Jurik sells them - but systems are seldom shared, especially in public or in the non-elitecircle section (at least I've never seen it before, on this forum or other places).

I'll be sure to post if I get definitive info from NT on what happened - if there is something incorrect with ES replay data, others will know to be careful of it while it gets fixed.

"A Jedi's strength flows from the force."
-Yoda
Attached Thumbnails
Trouble with LIVE v. BACKTEST-backtest.pdf   Trouble with LIVE v. BACKTEST-replay.pdf  
Started this thread Reply With Quote
  #18 (permalink)
 
Fat Tails's Avatar
 Fat Tails 
Berlin, Europe
Market Wizard
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
Posts: 9,888 since Mar 2010
Thanks Given: 4,242
Thanks Received: 27,102


patbateman View Post
No I call it 'practically identical.' All but one of the trades were within the hour.
Calc on bar close should be true... I was thorough with the settings (reviewed them again and again) and my code, cross referencing everything. I wouldn't of gone with putting in the effort to make the thread if I knew I could solve it myself. I'm really just wondering if anyone has had a similar experience, perhaps because a problem with Jurik tools or the ES data in particular would be something someone else already came across.

The problem is not the Jurik tools, but the data.

If you load historical minute data, you will have one single tick per bar on that data available for a backtest. If a required condition is met, the backtest will enter a position at the open of the next bar. Historical data has no intra-bar price data available for doing otherwise.

In replay, you have set your strategy to CalculateOnBarClose = false, and the strategy will enter intra-bar. This explains why the entry of the replay is systematically earlier than the entry on the backtest.


If you want to make backtest and replay compatible, you have two options

-> either switch the replay to CalculateOnBarClose = true
-> or recode the strategy to work in CalculateOnBarClose = false for the backtest.

As historical data always contains 1 tick per bar, your only option for a real backtest is to load a secondary 1-tick bar series for the backtest and use the information from that secondary bar series to enter and exit your positions.

It just needs to be coded.

Reply With Quote
Thanked by:
  #19 (permalink)
 
ThatManFromTexas's Avatar
 ThatManFromTexas 
Houston,Tx
 
Experience: Advanced
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: TF
Posts: 2,265 since Feb 2010
Thanks Given: 1,206
Thanks Received: 4,348


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

If you believe that your results are as you say they are, why not fire that puppy up with real money... just don't trade ES...

I'm just a simple man trading a simple plan.

My daddy always said, "Every day above ground is a good day!"
Reply With Quote




Last Updated on April 30, 2012


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts