Trouble with LIVE v. BACKTEST - NinjaTrader | futures io social day trading
futures io futures trading


Trouble with LIVE v. BACKTEST
Updated: Views / Replies:5,501 / 18
Created: by patbateman Attachments:9

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 9  
 
Thread Tools Search this Thread
 

Trouble with LIVE v. BACKTEST

  #11 (permalink)
Elite Member
Houston,Tx
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: TF
 
ThatManFromTexas's Avatar
 
Posts: 2,302 since Feb 2010
Thanks: 1,208 given, 4,293 received


patbateman View Post
To make things even more amazingly confusing, here is what happened...

I ran the system on a different contract, this time the Dow (YM). The backtest and the replay are 99.999% identical.

I don't know what to make of this, other than there is something in fact going on with the ES contract in replay or backtest. I'll have to investigate tomorrow, it's bedtime now. The NT team probably thinks I'm making this all up to sabotage them... I'm not!

Thank you lornz!

Try running your strategy real time in SIM for a month and at the end of the month back testing the strategy against that same month. Compare the results . When I tried it, I never found any correlation between the two results.

I'm just a simple man trading a simple plan.

My daddy always said, "Every day above ground is a good day!"
Reply With Quote
The following user says Thank You to ThatManFromTexas for this post:
 
  #12 (permalink)
 Vendor: www.tools4trading.de 
Erfurt Germany
 
Futures Experience: Master
Platform: rtMagic, NinjaTrader
Broker/Data: AMP/CQG, IAB
Favorite Futures: ES, 6E, FDAX, FGBL
 
Posts: 338 since Aug 2011
Thanks: 54 given, 460 received


patbateman View Post
To make things even more amazingly confusing, here is what happened...

I ran the system on a different contract, this time the Dow (YM). The backtest and the replay are 99.999% identical.

I don't know what to make of this, other than there is something in fact going on with the ES contract in replay or backtest. I'll have to investigate tomorrow, it's bedtime now. The NT team probably thinks I'm making this all up to sabotage them... I'm not!

Thank you lornz!

I mine, you have problem with your non tick based quotes...

Please switch your chrat to !seconds! (1hour = 3600sec)... now NT7 create all bars by using tickdata.
If you have continue wrong results, then juri tools have i internal worng programmed code for backtest

example: one candle = only one tick -> !FirstTickOfBar is equal lastTickOfBar! Any code If(FirstTickOfBar) { /* code */ } else {/* other code*/} well on Backtest never call the "other code".

The most parts of juri tools its give free (full source, no crack)... then you can self check this...

Reply With Quote
The following user says Thank You to TimeTrade for this post:
 
  #13 (permalink)
Elite Member
NY, NY
 
Futures Experience: None
Platform: T4, NinjaTrader
Favorite Futures: All!
 
patbateman's Avatar
 
Posts: 235 since Mar 2012
Thanks: 113 given, 193 received



ThatManFromTexas View Post
Try running your strategy real time in SIM for a month and at the end of the month back testing the strategy against that same month. Compare the results . When I tried it, I never found any correlation between the two results.

Zing!

"A Jedi's strength flows from the force."
-Yoda
Reply With Quote
 
  #14 (permalink)
Elite Member
NY, NY
 
Futures Experience: None
Platform: T4, NinjaTrader
Favorite Futures: All!
 
patbateman's Avatar
 
Posts: 235 since Mar 2012
Thanks: 113 given, 193 received

Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

"A Jedi's strength flows from the force."
-Yoda
Attached Thumbnails
Trouble with LIVE v. BACKTEST-ym-backtest.jpg   Trouble with LIVE v. BACKTEST-ym-replay.jpg  
Reply With Quote
 
  #15 (permalink)
Elite Member
Berlin, Europe
 
Futures Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker/Data: Interactive Brokers
Favorite Futures: Keyboard
 
Fat Tails's Avatar
 
Posts: 9,653 since Mar 2010
Thanks: 4,226 given, 25,602 received
Forum Reputation: Legendary


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.


Do you call that identical?

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).



The backtest was made with setting CalculateOnBarClose = true on historical data. The replay used intra-bar fills as opposed to fills at the bar close. The only thing that one can say is that all the differences and errors cancel out.

Reply With Quote
 
  #16 (permalink)
 Vendor: www.tools4trading.de 
Erfurt Germany
 
Futures Experience: Master
Platform: rtMagic, NinjaTrader
Broker/Data: AMP/CQG, IAB
Favorite Futures: ES, 6E, FDAX, FGBL
 
Posts: 338 since Aug 2011
Thanks: 54 given, 460 received


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

Without your indicator and strategy settings in continue we make a quiz show here...

Error of NT7?
Error of your used Juri tool?
Error in your settings? (example by Fat Tails: CalcBarOnClose=True on ALL points?)
Error in your strategy?

-> If you don't can or well give more detailed informations, you must self find this problem.

Reply With Quote
The following user says Thank You to TimeTrade for this post:
 
  #17 (permalink)
Elite Member
NY, NY
 
Futures Experience: None
Platform: T4, NinjaTrader
Favorite Futures: All!
 
patbateman's Avatar
 
Posts: 235 since Mar 2012
Thanks: 113 given, 193 received


Fat Tails View Post
Do you call that identical?

No I call it 'practically identical.' All but one of the trades were within the hour. The reason why one of the trades was missing was because the replay had one open position at the end of the replay session (contract rollover day). Good enough for 60min non-intraday IMO, especially since backtests are generalizations anyways. I attached the backtest v. replay results btw which I didn't share above with the trades v. trades post - only one trade off, which was the one that didn't close at the end of the replay.

I got some email going with NT to see if we can figure the problem and how I can fix it... it's looking like my ES tick data or the NT ES replay data is faulty. I'm running the system through NQ and SI, and finding so far that the results are also the same compared to backtest results, just like with YM.


TimeTrade View Post
Without your indicator and strategy settings in continue we make a quiz show here...

Error of NT7?
Error of your used Juri tool?
Error in your settings? (example by Fat Tails: CalcBarOnClose=True on ALL points?)
Error in your strategy?

-> If you don't can or well give more detailed informations, you must self find this problem.

Calc on bar close should be true... I was thorough with the settings (reviewed them again and again) and my code, cross referencing everything. I wouldn't of gone with putting in the effort to make the thread if I knew I could solve it myself. I'm really just wondering if anyone has had a similar experience, perhaps because a problem with Jurik tools or the ES data in particular would be something someone else already came across. Btw, you shouldn't expect me or anyone else frankly to be handing over hard work... indicators are fine, which is why Mark Jurik sells them - but systems are seldom shared, especially in public or in the non-elitecircle section (at least I've never seen it before, on this forum or other places).

I'll be sure to post if I get definitive info from NT on what happened - if there is something incorrect with ES replay data, others will know to be careful of it while it gets fixed.

"A Jedi's strength flows from the force."
-Yoda
Attached Thumbnails
Trouble with LIVE v. BACKTEST-backtest.pdf   Trouble with LIVE v. BACKTEST-replay.pdf  
Reply With Quote
 
  #18 (permalink)
Elite Member
Berlin, Europe
 
Futures Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker/Data: Interactive Brokers
Favorite Futures: Keyboard
 
Fat Tails's Avatar
 
Posts: 9,653 since Mar 2010
Thanks: 4,226 given, 25,602 received
Forum Reputation: Legendary


patbateman View Post
No I call it 'practically identical.' All but one of the trades were within the hour.
Calc on bar close should be true... I was thorough with the settings (reviewed them again and again) and my code, cross referencing everything. I wouldn't of gone with putting in the effort to make the thread if I knew I could solve it myself. I'm really just wondering if anyone has had a similar experience, perhaps because a problem with Jurik tools or the ES data in particular would be something someone else already came across.

The problem is not the Jurik tools, but the data.

If you load historical minute data, you will have one single tick per bar on that data available for a backtest. If a required condition is met, the backtest will enter a position at the open of the next bar. Historical data has no intra-bar price data available for doing otherwise.

In replay, you have set your strategy to CalculateOnBarClose = false, and the strategy will enter intra-bar. This explains why the entry of the replay is systematically earlier than the entry on the backtest.


If you want to make backtest and replay compatible, you have two options

-> either switch the replay to CalculateOnBarClose = true
-> or recode the strategy to work in CalculateOnBarClose = false for the backtest.

As historical data always contains 1 tick per bar, your only option for a real backtest is to load a secondary 1-tick bar series for the backtest and use the information from that secondary bar series to enter and exit your positions.

It just needs to be coded.


Last edited by Fat Tails; April 30th, 2012 at 05:00 PM.
Reply With Quote
The following 2 users say Thank You to Fat Tails for this post:
 
  #19 (permalink)
Elite Member
Houston,Tx
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: TF
 
ThatManFromTexas's Avatar
 
Posts: 2,302 since Feb 2010
Thanks: 1,208 given, 4,293 received


patbateman View Post
Attached are screenshots of the YM in replay and backtest. They are piratically identical.

However, the ES in backtest v. replay are not the same. I'm going to compare the price data of the ES in replay and backtest. Most likely the replay is the incorrect data, however I'm really not sure.

I'm going to try it on other contracts to confirm that it's working on most everything except for the ES.

Thanks for the great input everyone. I tried the 3600 seconds, with the same results in backtest.

If you believe that your results are as you say they are, why not fire that puppy up with real money... just don't trade ES...

I'm just a simple man trading a simple plan.

My daddy always said, "Every day above ground is a good day!"
Reply With Quote

Reply



futures io > > > > Trouble with LIVE v. BACKTEST

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Renko charts | Live versus Backtest patbateman NinjaTrader Programming 8 July 31st, 2012 01:00 PM
Typical ES slippage from backtest to live frostengine The Elite Circle 4 June 2nd, 2011 04:37 PM
DoubleMAPaint_SharkyPkg trouble desander NinjaTrader 7 February 4th, 2011 06:16 PM
A little trouble with limit orders... BigDog NinjaTrader Programming 8 June 21st, 2009 08:07 PM
Trouble with ChannelThis and DoubleMA lolu NinjaTrader 11 June 21st, 2009 06:38 PM


All times are GMT -4. The time now is 11:21 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-16 in 0.15 seconds with 20 queries on phoenix via your IP 54.226.34.209