NexusFi: Find Your Edge


Home Menu

 





Backtests and live results way off


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one lookOutBelow with 9 posts (0 thanks)
    2. looks_two MWinfrey with 7 posts (3 thanks)
    3. looks_3 Big Mike with 3 posts (0 thanks)
    4. looks_4 Zoethecus with 2 posts (1 thanks)
    1. trending_up 8,531 views
    2. thumb_up 4 thanks given
    3. group 3 followers
    1. forum 23 posts
    2. attach_file 0 attachments




 
Search this Thread

Backtests and live results way off

  #1 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19

I've been a supporter of NT for a little while now. Mainly because I haven't had the problems others have had. That is starting to change. This latest issue (Version 7) is causing me to re-think whether I want to continue with NT or try to find a new platform to trade from.

I've read that others have had issues with backtesting not being very good with NT. Mostly revolving around CalculateOnBarClose set to false. Well, my problem with a recent strategy is with COBC set to true. Which *should* allow me to backtest stuff and be somewhat reliable.

I spent quite a bit of time writing, then optimizing and walking forward to get good results on a particular strategy (again, COBC set to true) with tick charts. But then, when it came to run the strat in sim, I quickly realized that the results from sim and backtesting were off. And by a long shot in most cases. Not simply slippage, but missing trades, or vastly different entries/exits. So much different that it basically renders backtesting almost useless. I mean. In backtest I could have a loss of $500 on a particular day and in sim mode I've gained $1000 on that same day!!

Am I asking too much for NT to be close (with the exception of expected slippage) with backtesting and sim/live trading? I've been pulling my hair out for days now and can't seem to get the two to come to an agreement for very many trades.

Can anyone offer anything about this? What the heck am I overlooking, or is NT really this flawed? I've invested so much time with it, and have some other strats that work fine and dandy. I'd hate to start over with a new platform. But this is completely unacceptable imo. Might as well not even backtest with the discrepancies I'm getting.

Thanks

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Deepmoney LLM
Elite Quantitative GenAI/LLM
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
Ninja Mobile Trader VPS (ninjamobiletrader.com)
Trading Reviews and Vendors
Better Renko Gaps
The Elite Circle
Online prop firm The Funded Trader (TFT) going under?
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
59 thanks
Funded Trader platforms
37 thanks
NexusFi site changelog and issues/problem reporting
24 thanks
GFIs1 1 DAX trade per day journal
22 thanks
The Program
19 thanks
  #3 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,399 since Jun 2009
Thanks Given: 33,175
Thanks Received: 101,539


Post your code and specific instructions to reproduce such as bar type and size if you want someone to look it over... But, generally speaking, I would not trust NT backtest results very much.

Sent via tapatalk on my DROID X

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #4 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


Big Mike View Post
Post your code and specific instructions to reproduce such as bar type and size if you want someone to look it over... But, generally speaking, I would not trust NT backtest results very much.

Sent via tapatalk on my DROID X

Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

Reply With Quote
  #5 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,399 since Jun 2009
Thanks Given: 33,175
Thanks Received: 101,539


lookOutBelow View Post
Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

Well you could create code that duplicates the problem that you are willing to share and post it, if you want feedback from others.

I am extremely happy with MultiCharts. It was built for automation and backtesting from day 1, they are just now adding discretionary in fact.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #6 (permalink)
 Zoethecus 
United States of America
 
Experience: Advanced
Platform: NT
Posts: 1,145 since Aug 2009


lookOutBelow View Post
Thanks Mike for the response.

As you can probably understand, I'd rather not share my code. Can I ask-- since you moved to MultiCharts, are you having backtesting issues? I'd hate to switch platforms, but this is a big enough deal to make me think long and hard about it.

The only bars that backtest with integrity to live trading are time based using market orders. COBC=false is a total waste of time, too. This is a fact of life with NT. And, I don't know why you are so surprised. I believe NT discloses this in the documentation.

I'm not sure if things are different with other platforms.

Reply With Quote
  #7 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


Zoethecus View Post
This is a fact of life with NT. And, I don't know why you are so surprised.

I don't know either. I guess I was under the impression that using COBC=true would make backtesting relevant to live. Live and learn, I guess.

I'd rather not switch from tick to a minute chart, but I'm going to try that this afternoon and see if it gets me any closer to similar backtest/live results.

Reply With Quote
  #8 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,399 since Jun 2009
Thanks Given: 33,175
Thanks Received: 101,539

This is , but... Look! There is a smiley just for you!



lol

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #9 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335


lookOutBelow View Post
I don't know either. I guess I was under the impression that using COBC=true would make backtesting relevant to live. Live and learn, I guess.

I'd rather not switch from tick to a minute chart, but I'm going to try that this afternoon and see if it gets me any closer to similar backtest/live results.


Your assumption about COBC=true is as correct as it can be. If you enter at barclose then that's equivalent to COBC=true.

Using anything other than timebased charts can as you noticed produce different results live versus historic. Something I do using nonTimebased charts is to set the Session begins and Session ends parameters of the chart properties (nt6.5). Session begins = 12:00 am and Session ends = 11:50 pm. This means every day starts exactly the same every time and is not dependent on the number days I backfill and further this minimizes the differences you are frustrated with.

Hopefully this will help. It has helped me considerably. Bear in mind that I believe the problem will never go away completely. You can only minimize it's effect.

Mike Winfrey

Reply With Quote
Thanked by:
  #10 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19



MWinfrey View Post
Something I do using nonTimebased charts is to set the Session begins and Session ends parameters of the chart properties (nt6.5). Session begins = 12:00 am and Session ends = 11:50 pm. This means every day starts exactly the same every time and is not dependent on the number days I backfill and further this minimizes the differences you are frustrated with.

Yah, this is something I've already been doing after being confronted with another issue trying walk-forward optimizing. Unfortunately, even when using a session that starts at 2am and ends at 11pm, I get these crazy discrepancies.

When the market is over today, I'm going to do some testing with 1-minute charts and see if that gets me any closer.

Reply With Quote




Last Updated on October 22, 2010


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts