Backtests and live results way off - NinjaTrader | futures io social day trading
futures io futures trading


Backtests and live results way off
Updated: Views / Replies:4,142 / 23
Created: by lookOutBelow Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Backtests and live results way off

  #11 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


Big Mike View Post
This is , but... Look! There is a smiley just for you!



lol

Mike

I should put that in a signature!

Reply With Quote
 
  #12 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received

I read your original post again and remembered that I experienced the same thing at one time. Trades not being entered at the same time in live versus historic or trades entered in live that dont' exist in historic and trades in historic that don't exist in live. Reading your post reminded me of a commercial system I was looking at called Viper Trading Systems which is talked about in a thread on this forum. It was horrible about that kind of thing and they just explained it away. They used tick charts.

I was doing some development around that same time using tick and volume charts and was experiencing the same thing. It was so bad that I gave up on the effort. Eventually I forgot about that and picked up the strategy development again. However, this time I used modified renko bricks and I don't see that same behavior. My entries, aside from expected slippage, are very close to what I see in backtesting. There are times that slippage cause a backtested winner to be a loser just because the move wasn't enough to get my target. However, that is an exception.

As I write this, I'm recalling a possible reason for the problem is the way we get tick data. I think the reason I got was that the timestamp on live tick data isn't the same as historic data for tick based bars. I remember that my broker or maybe NT helped me get to the bottom of that but don't remember for sure. You might ask them about your problem and see if you get anything reasonable. I think I'll do the same and post here if I get anything.

Mike Winfrey

Reply With Quote
The following user says Thank You to MWinfrey for this post:
 
  #13 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received


http://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars

This explains how bars are built in NT. Not sure how enlightening it is but thought I'd provide it anyway.

Mike

Reply With Quote
 
  #14 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


MWinfrey View Post
I read your original post again and remembered that I experienced the same thing at one time. Trades not being entered at the same time in live versus historic or trades entered in live that dont' exist in historic and trades in historic that don't exist in live. Reading your post reminded me of a commercial system I was looking at called Viper Trading Systems which is talked about in a thread on this forum. It was horrible about that kind of thing and they just explained it away. They used tick charts.

I was doing some development around that same time using tick and volume charts and was experiencing the same thing. It was so bad that I gave up on the effort. Eventually I forgot about that and picked up the strategy development again. However, this time I used modified renko bricks and I don't see that same behavior. My entries, aside from expected slippage, are very close to what I see in backtesting. There are times that slippage cause a backtested winner to be a loser just because the move wasn't enough to get my target. However, that is an exception.

As I write this, I'm recalling a possible reason for the problem is the way we get tick data. I think the reason I got was that the timestamp on live tick data isn't the same as historic data for tick based bars. I remember that my broker or maybe NT helped me get to the bottom of that but don't remember for sure. You might ask them about your problem and see if you get anything reasonable. I think I'll do the same and post here if I get anything.

Mike Winfrey

Hi Mike. Thanks for recalling the issues you had.

I did bring this up on their forums. They can be extremely helpful over there. But, I get the feeling that they think it is my imagination or isn't a big deal. Just part of backtesting that should be accepted. That scares me a little. Mainly because I find backtesting a little better than useless if the backtest results are so far off from live/sim. I accept slippage and all that goes along with it. But what I have been experiencing is another story.


With that in mind. I let my strategies run all day long yesterday. Unlike all the previous days with this problem, I did not edit or compile any code during the trading day. I basically let it trade and didn't tinker with anything. And the backtest vs live/sim results were much closer than they had been on previous days. Far fewer wacked out discrepancies.

From my understanding after asking the question on their forum, editing and compiling code shouldn't affect running strategies. Which I accept, but it is an interesting coincidence. Granted, one day doesn't make a pattern. But, I'll try it again to see what happens.

And before anyone asks the question. My multicore computer is quite beefy. So, the simple task of editing code and compiling shouldn't really affect running strategy calculations that much unless perhaps NT pauses for the 10 or so seconds when it is compiling code. I'd be surprised if that were the case.


I haven't jumped ship from NT yet. But, I have to say, the backtesting capabilities of Multicharts is somewhat intriguing. And if I can't get stable backtest/live results from NT that are in the same ballpark, I won't stick around.

Reply With Quote
 
  #15 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received

Just for curiosity sake, have you tried running your strategies on timebased or even betterrenko bricks? I'm sure you've written your strategies for tick but just as a test, putting them on betterrenko might be an interesting test as well to see if the same problems occur. I know I don't have those problems when using betterrenko or my own implementation of the renko brick.

Reply With Quote
 
  #16 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


MWinfrey View Post
Just for curiosity sake, have you tried running your strategies on timebased or even betterrenko bricks? I'm sure you've written your strategies for tick but just as a test, putting them on betterrenko might be an interesting test as well to see if the same problems occur. I know I don't have those problems when using betterrenko or my own implementation of the renko brick.

Hi Mike.

Yes, I tried a 1min chart and the results seemed more stable. But, I haven't really given it a fair chance yet. I plan on doing more today.

As far as using BetterRenko's. I've never touched them. I'm a little afraid of anything that has the word Renko in it. Too many issues with those. What advantage does BetterRenko have over time based or tick based? Also, I'm assuming they don't repaint?

Reply With Quote
 
  #17 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received


lookOutBelow View Post
Hi Mike.

Yes, I tried a 1min chart and the results seemed more stable. But, I haven't really given it a fair chance yet. I plan on doing more today.

As far as using BetterRenko's. I've never touched them. I'm a little afraid of anything that has the word Renko in it. Too many issues with those. What advantage does BetterRenko have over time based or tick based? Also, I'm assuming they don't repaint?

They are more responsive and less noisy. BetterRenko is superior to the conventional Renko because they have wicks and sticks.

Also, I'm advocating that you change to them except as a troubleshooting measure. I use them but that's neither here nor there.

Reply With Quote
 
  #18 (permalink)
Membership Revoked
 
Futures Experience: Beginner
 
Posts: 1,081 since May 2010
Thanks: 2,092 given, 595 received

What is the reaction of ninjatrader support team to this issue ?

Reply With Quote
 
  #19 (permalink)
Elite Member
Lubbock TX
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: Stage 5 Trading
Favorite Futures: CL
 
MWinfrey's Avatar
 
Posts: 1,879 since Jul 2009
Thanks: 1,449 given, 3,306 received


MetalTrade View Post
What is the reaction of ninjatrader support team to this issue ?

Read this and you'll see what their response is.
http://www.ninjatrader-support.com/HelpGuideV6/helpguide.html?HowDoesNinjaTraderBuildChartBars

Reply With Quote
 
  #20 (permalink)
Trading Apprentice
San Francisco, CA
 
Futures Experience: Advanced
Platform: inhouse
Favorite Futures: equities
 
lookOutBelow's Avatar
 
Posts: 40 since Sep 2010
Thanks: 5 given, 19 received


I may of stumbled across the problem. If at the very least, I stumbled across another problem that affects running strategies.

It appears that Ninja Trader FREEZES whenever you do certain backtesting operations. Even compiling a code will freeze quotes. And when I've been heavily backtesting and writing/compiling code over the last weeks, it seems to have been severely affecting my backtest vs live/sim results.

I don't know that this is the sole issue with the problem I've been having. But it certainly must be a serious issue.


It is a sad day if we cannot backtest/compile code/etc while running strategies during market hours.

Reply With Quote

Reply



futures io > > > > Backtests and live results way off

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
When not to trust optimization results? molecool Traders Hideout 10 September 6th, 2010 06:39 PM
Sim Results leduc NinjaTrader Programming 3 June 2nd, 2010 06:49 AM
Please help me evaluate my system results sharp2be NinjaTrader Programming 6 November 17th, 2009 02:11 PM
Same feed with two different results carlos The Elite Circle 1 October 23rd, 2009 09:33 AM
Different Results T3 vs T3OverUnder HJay NinjaTrader Programming 1 August 23rd, 2009 10:18 AM


All times are GMT -4. The time now is 11:58 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-17 in 0.17 seconds with 19 queries on phoenix via your IP 54.90.92.204