Backtest not matching Optimization - NinjaTrader Programming | futures io social day trading
futures io futures trading


Backtest not matching Optimization
Updated: Views / Replies:1,162 / 9
Created: by d1g1talfr3ak Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Backtest not matching Optimization

  #1 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

Backtest not matching Optimization

in the past when I develop a strategy I first run the optimizer on a range of historical data to find best settings for parameters. Then I enter the settings the optimizer found as best and run backtest on same time period with same settings and ensure that I do get the value returned by the optimizer (sort of a sanity check on making sure I have settings entered correctly) then I run backtests on out of sample time periods. Only after that do I move on to sim trading.

I have a strategy now though, that when I optimize and get some results and settings, and enter those settings to do backtest... it does not match the results the optimizer claimed to have gotten with the same settings. I have meticulously checked every setting and they are set the same on backtest as on best run from optimizer. The results aren't even close.

Any ideas?

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,240 since Jun 2009
Thanks: 29,356 given, 83,237 received


Are you using exotic bar types?

Are you using genetic optimizer or exhaustive? If genetic, using the NT one or third party?

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #4 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

I am using the Ninjatrader built-in genetic optimizer. I would have thought that each step of the genetic algorithm it would internally run a backtest with selected values... then score value and run genetic routine to choose next generation of parameters. So I am stumped as to why running a backtest with the same settings the optimizer found doesn't match the results it showed. I have verified and re-verified that all settings are the same... including slippage & commisions, etc.

I can optimize other strats and then backtest on same time period with same values and get the same results. At first I thought that it was because this strat used an indicator that repainted past values. I tried telling the backtester and optimizer to not run multithreaded and a few other ideas... no go. I then wrote my own version of the indicator that did not repaint past values and still the same thing.

Thanks for any ideas and help

Reply With Quote
 
  #5 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

Bar type

Also... I am using Better Renko. I am using it on several other strategies and not having the same issue.

Reply With Quote
 
  #6 (permalink)
Elite Member
McKinney, TX
 
Futures Experience: Intermediate
Platform: Think or Swim, NinjaTrade
Favorite Futures: The indexes, ES, YM, NQ & RTY
 
Posts: 152 since Aug 2012
Thanks: 213 given, 125 received

I had problems using Ranko bars. I had a tested strategy that was giving me very good results, then I ran that strategy against live data on the sim account and got a totally different return.

I don't know for sure, but what I think happens is that the Renko bar historical data is truncated so that the bar high/low values are used for the optimization. When running live, it appears that there are ticks outside of the Renko bars, those wide range ticks were hitting my stop loss and changing the results. What I would see is the execution of orders outside of the Renko bar.

Not sure if that is your problem, but when I tried the same strategy with Range bars, I couldn't duplicate the great results in the Analyzer but they matched the results better when I had a live data feed. Because of this, I won't use Renko bars on a strategy.

Reply With Quote
 
  #7 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 102 received

Yes. Ninja does not use any data outside the bars, so you need bars that show true high low or better anther bar in the strat (tick 1) and then issue trades against this one.

Reply With Quote
 
  #8 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,240 since Jun 2009
Thanks: 29,356 given, 83,237 received

Renko bars and other exotic bar types have false opens, which makes backtesting them totally false.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #9 (permalink)
Elite Member
Miami FL, USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Forex
 
Posts: 1 since Apr 2012
Thanks: 5 given, 0 received

I have also had similar experiences with the NinjaTrader built in Genetic Optimizer. In my case, I used a plain old 5 minute chart, ran the optimizer, then backtested the winner and received wildly different results.

Upon inspecting the actual trades in the NinjaTrader Strategy Analyzer, it looks as if the optimizer wasn't using all of the price data available. What I mean is that if the strategy had a rule to close a position on a MA crossover, the optimized strategy did not close the position whereas the backtested strategy (with identical settings) did close the position on the crossover.

Reply With Quote
 
  #10 (permalink)
Membership Temporarily Revoked
Los Angeles, CA
 
Futures Experience: Intermediate
Platform: Ninjatrader7
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: ES
 
grimReaper's Avatar
 
Posts: 50 since Nov 2011
Thanks: 3 given, 12 received



dynoweb View Post
I had problems using Ranko bars. I had a tested strategy that was giving me very good results, then I ran that strategy against live data on the sim account and got a totally different return.

Perhaps overfitting?

Reply With Quote

Reply



futures io > > > > > Backtest not matching Optimization

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Trendlines not matching from Higher Timeframe to lower Timeframe? bluemele NinjaTrader 26 October 8th, 2012 03:35 PM
Trouble matching tick charts across platforms keymoo Platforms and Indicators 2 May 30th, 2012 03:55 PM
ICE matching engines. Address of the building. maxima Elite Automated Trading 5 January 8th, 2012 08:14 PM
How I Backtest GaryD Traders Hideout 3 December 17th, 2011 01:01 PM
Buys and Sells not matching in number richw NinjaTrader Programming 8 February 22nd, 2011 07:15 PM


All times are GMT -4. The time now is 05:14 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-18 in 0.13 seconds with 19 queries on phoenix via your IP 23.22.136.56