Backtests and live results way off - NinjaTrader | futures io social day trading
futures io futures trading


Backtests and live results way off
Updated: Views / Replies:4,131 / 23
Created: by lookOutBelow Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Backtests and live results way off

  #1 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received

Backtests and live results way off

I've been a supporter of NT for a little while now. Mainly because I haven't had the problems others have had. That is starting to change. This latest issue (Version 7) is causing me to re-think whether I want to continue with NT or try to find a new platform to trade from.

I've read that others have had issues with backtesting not being very good with NT. Mostly revolving around CalculateOnBarClose set to false. Well, my problem with a recent strategy is with COBC set to true. Which *should* allow me to backtest stuff and be somewhat reliable.

I spent quite a bit of time writing, then optimizing and walking forward to get good results on a particular strategy (again, COBC set to true) with tick charts. But then, when it came to run the strat in sim, I quickly realized that the results from sim and backtesting were off. And by a long shot in most cases. Not simply slippage, but missing trades, or vastly different entries/exits. So much different that it basically renders backtesting almost useless. I mean. In backtest I could have a loss of $500 on a particular day and in sim mode I've gained $1000 on that same day!!

Am I asking too much for NT to be close (with the exception of expected slippage) with backtesting and sim/live trading? I've been pulling my hair out for days now and can't seem to get the two to come to an agreement for very many trades.

Can anyone offer anything about this? What the heck am I overlooking, or is NT really this flawed? I've invested so much time with it, and have some other strats that work fine and dandy. I'd hate to start over with a new platform. But this is completely unacceptable imo. Might as well not even backtest with the discrepancies I'm getting.

Thanks

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,218 received


Post your code and specific instructions to reproduce such as bar type and size if you want someone to look it over... But, generally speaking, I would not trust NT backtest results very much.

Sent via tapatalk on my DROID X

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #4 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


Big Mike View Post
Post your code and specific instructions to reproduce such as bar type and size if you want someone to look it over... But, generally speaking, I would not trust NT backtest results very much.

Sent via tapatalk on my DROID X

Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

Reply With Quote
 
  #5 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,218 received


lookOutBelow View Post
Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

Well you could create code that duplicates the problem that you are willing to share and post it, if you want feedback from others.

I am extremely happy with MultiCharts. It was built for automation and backtesting from day 1, they are just now adding discretionary in fact.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #6 (permalink)
Membership Revoked
United States of America
 
Futures Experience: Advanced
Platform: NT
 
Posts: 1,149 since Aug 2009
Thanks: 262 given, 471 received


lookOutBelow View Post
Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

The only bars that backtest with integrity to live trading are time based using market orders. COBC=false is a total waste of time, too. This is a fact of life with NT. And, I don't know why you are so surprised. I believe NT discloses this in the documentation.

I'm not sure if things are different with other platforms.

Reply With Quote
The following user says Thank You to Zoethecus for this post:
 
  #7 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


Zoethecus View Post
This is a fact of life with NT. And, I don't know why you are so surprised.

I don't know either. I guess I was under the impression that using COBC=true would make backtesting relevant to live. Live and learn, I guess.

I'd rather not switch from tick to a minute chart, but I'm going to try that this afternoon and see if it gets me any closer to similar backtest/live results.

Reply With Quote
 
  #8 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,218 received

This is , but... Look! There is a smiley just for you!



lol

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #9 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received


lookOutBelow View Post
I don't know either. I guess I was under the impression that using COBC=true would make backtesting relevant to live. Live and learn, I guess.

I'd rather not switch from tick to a minute chart, but I'm going to try that this afternoon and see if it gets me any closer to similar backtest/live results.


Your assumption about COBC=true is as correct as it can be. If you enter at barclose then that's equivalent to COBC=true.

Using anything other than timebased charts can as you noticed produce different results live versus historic. Something I do using nonTimebased charts is to set the Session begins and Session ends parameters of the chart properties (nt6.5). Session begins = 12:00 am and Session ends = 11:50 pm. This means every day starts exactly the same every time and is not dependent on the number days I backfill and further this minimizes the differences you are frustrated with.

Hopefully this will help. It has helped me considerably. Bear in mind that I believe the problem will never go away completely. You can only minimize it's effect.

Mike Winfrey

Reply With Quote
The following 2 users say Thank You to MWinfrey for this post:
 
  #10 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received



MWinfrey View Post
Something I do using nonTimebased charts is to set the Session begins and Session ends parameters of the chart properties (nt6.5). Session begins = 12:00 am and Session ends = 11:50 pm. This means every day starts exactly the same every time and is not dependent on the number days I backfill and further this minimizes the differences you are frustrated with.

Yah, this is something I've already been doing after being confronted with another issue trying walk-forward optimizing. Unfortunately, even when using a session that starts at 2am and ends at 11pm, I get these crazy discrepancies.

When the market is over today, I'm going to do some testing with 1-minute charts and see if that gets me any closer.

Reply With Quote

Reply



futures io > > > > Backtests and live results way off

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
When not to trust optimization results? molecool Traders Hideout 10 September 6th, 2010 06:39 PM
Sim Results leduc NinjaTrader Programming 3 June 2nd, 2010 06:49 AM
Please help me evaluate my system results sharp2be NinjaTrader Programming 6 November 17th, 2009 02:11 PM
Same feed with two different results carlos The Elite Circle 1 October 23rd, 2009 09:33 AM
Different Results T3 vs T3OverUnder HJay NinjaTrader Programming 1 August 23rd, 2009 10:18 AM


All times are GMT -4. The time now is 02:37 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-12 in 0.13 seconds with 19 queries on phoenix via your IP 54.90.207.75