NexusFi: Find Your Edge


Home Menu

 





Backtests and live results way off


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one lookOutBelow with 9 posts (0 thanks)
    2. looks_two MWinfrey with 7 posts (3 thanks)
    3. looks_3 Big Mike with 3 posts (0 thanks)
    4. looks_4 Zoethecus with 2 posts (1 thanks)
    1. trending_up 8,636 views
    2. thumb_up 4 thanks given
    3. group 3 followers
    1. forum 23 posts
    2. attach_file 0 attachments




 
Search this Thread

Backtests and live results way off

  #11 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


Big Mike View Post
This is , but... Look! There is a smiley just for you!



lol

Mike

I should put that in a signature!

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
How to apply profiles
Traders Hideout
Pivot Indicator like the old SwingTemp by Big Mike
NinjaTrader
Trade idea based off three indicators.
Traders Hideout
REcommedations for programming help
Sierra Chart
Better Renko Gaps
The Elite Circle
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Just another trading journal: PA, Wyckoff & Trends
31 thanks
Spoo-nalysis ES e-mini futures S&P 500
28 thanks
Tao te Trade: way of the WLD
24 thanks
Bigger Wins or Fewer Losses?
20 thanks
GFIs1 1 DAX trade per day journal
17 thanks
  #12 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335

I read your original post again and remembered that I experienced the same thing at one time. Trades not being entered at the same time in live versus historic or trades entered in live that dont' exist in historic and trades in historic that don't exist in live. Reading your post reminded me of a commercial system I was looking at called Viper Trading Systems which is talked about in a thread on this forum. It was horrible about that kind of thing and they just explained it away. They used tick charts.

I was doing some development around that same time using tick and volume charts and was experiencing the same thing. It was so bad that I gave up on the effort. Eventually I forgot about that and picked up the strategy development again. However, this time I used modified renko bricks and I don't see that same behavior. My entries, aside from expected slippage, are very close to what I see in backtesting. There are times that slippage cause a backtested winner to be a loser just because the move wasn't enough to get my target. However, that is an exception.

As I write this, I'm recalling a possible reason for the problem is the way we get tick data. I think the reason I got was that the timestamp on live tick data isn't the same as historic data for tick based bars. I remember that my broker or maybe NT helped me get to the bottom of that but don't remember for sure. You might ask them about your problem and see if you get anything reasonable. I think I'll do the same and post here if I get anything.

Mike Winfrey

Reply With Quote
Thanked by:
  #13 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335


https://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars

This explains how bars are built in NT. Not sure how enlightening it is but thought I'd provide it anyway.

Mike

Reply With Quote
  #14 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


MWinfrey View Post
I read your original post again and remembered that I experienced the same thing at one time. Trades not being entered at the same time in live versus historic or trades entered in live that dont' exist in historic and trades in historic that don't exist in live. Reading your post reminded me of a commercial system I was looking at called Viper Trading Systems which is talked about in a thread on this forum. It was horrible about that kind of thing and they just explained it away. They used tick charts.

I was doing some development around that same time using tick and volume charts and was experiencing the same thing. It was so bad that I gave up on the effort. Eventually I forgot about that and picked up the strategy development again. However, this time I used modified renko bricks and I don't see that same behavior. My entries, aside from expected slippage, are very close to what I see in backtesting. There are times that slippage cause a backtested winner to be a loser just because the move wasn't enough to get my target. However, that is an exception.

As I write this, I'm recalling a possible reason for the problem is the way we get tick data. I think the reason I got was that the timestamp on live tick data isn't the same as historic data for tick based bars. I remember that my broker or maybe NT helped me get to the bottom of that but don't remember for sure. You might ask them about your problem and see if you get anything reasonable. I think I'll do the same and post here if I get anything.

Mike Winfrey

Hi Mike. Thanks for recalling the issues you had.

I did bring this up on their forums. They can be extremely helpful over there. But, I get the feeling that they think it is my imagination or isn't a big deal. Just part of backtesting that should be accepted. That scares me a little. Mainly because I find backtesting a little better than useless if the backtest results are so far off from live/sim. I accept slippage and all that goes along with it. But what I have been experiencing is another story.


With that in mind. I let my strategies run all day long yesterday. Unlike all the previous days with this problem, I did not edit or compile any code during the trading day. I basically let it trade and didn't tinker with anything. And the backtest vs live/sim results were much closer than they had been on previous days. Far fewer wacked out discrepancies.

From my understanding after asking the question on their forum, editing and compiling code shouldn't affect running strategies. Which I accept, but it is an interesting coincidence. Granted, one day doesn't make a pattern. But, I'll try it again to see what happens.

And before anyone asks the question. My multicore computer is quite beefy. So, the simple task of editing code and compiling shouldn't really affect running strategy calculations that much unless perhaps NT pauses for the 10 or so seconds when it is compiling code. I'd be surprised if that were the case.


I haven't jumped ship from NT yet. But, I have to say, the backtesting capabilities of Multicharts is somewhat intriguing. And if I can't get stable backtest/live results from NT that are in the same ballpark, I won't stick around.

Reply With Quote
  #15 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335

Just for curiosity sake, have you tried running your strategies on timebased or even betterrenko bricks? I'm sure you've written your strategies for tick but just as a test, putting them on betterrenko might be an interesting test as well to see if the same problems occur. I know I don't have those problems when using betterrenko or my own implementation of the renko brick.

Reply With Quote
  #16 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


MWinfrey View Post
Just for curiosity sake, have you tried running your strategies on timebased or even betterrenko bricks? I'm sure you've written your strategies for tick but just as a test, putting them on betterrenko might be an interesting test as well to see if the same problems occur. I know I don't have those problems when using betterrenko or my own implementation of the renko brick.

Hi Mike.

Yes, I tried a 1min chart and the results seemed more stable. But, I haven't really given it a fair chance yet. I plan on doing more today.

As far as using BetterRenko's. I've never touched them. I'm a little afraid of anything that has the word Renko in it. Too many issues with those. What advantage does BetterRenko have over time based or tick based? Also, I'm assuming they don't repaint?

Reply With Quote
  #17 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335


lookOutBelow View Post
Hi Mike.

Yes, I tried a 1min chart and the results seemed more stable. But, I haven't really given it a fair chance yet. I plan on doing more today.

As far as using BetterRenko's. I've never touched them. I'm a little afraid of anything that has the word Renko in it. Too many issues with those. What advantage does BetterRenko have over time based or tick based? Also, I'm assuming they don't repaint?

They are more responsive and less noisy. BetterRenko is superior to the conventional Renko because they have wicks and sticks.

Also, I'm advocating that you change to them except as a troubleshooting measure. I use them but that's neither here nor there.

Reply With Quote
  #18 (permalink)
 MetalTrade 
 
Posts: 1,055 since May 2010

What is the reaction of ninjatrader support team to this issue ?

Reply With Quote
  #19 (permalink)
 
MWinfrey's Avatar
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
Posts: 1,878 since Jul 2009
Thanks Given: 1,450
Thanks Received: 3,335


MetalTrade View Post
What is the reaction of ninjatrader support team to this issue ?

Read this and you'll see what their response is.
[COLOR=#0000ff]http://www.[AUTOLINK]ninjatrader[/AUTOLINK]-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars[/COLOR]

Reply With Quote
  #20 (permalink)
lookOutBelow
San Francisco, CA
 
Posts: 40 since Sep 2010
Thanks Given: 5
Thanks Received: 19


I may of stumbled across the problem. If at the very least, I stumbled across another problem that affects running strategies.

It appears that Ninja Trader FREEZES whenever you do certain backtesting operations. Even compiling a code will freeze quotes. And when I've been heavily backtesting and writing/compiling code over the last weeks, it seems to have been severely affecting my backtest vs live/sim results.

I don't know that this is the sole issue with the problem I've been having. But it certainly must be a serious issue.


It is a sad day if we cannot backtest/compile code/etc while running strategies during market hours.

Reply With Quote




Last Updated on October 22, 2010


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts