Backtest not matching Optimization - NinjaTrader Programming | futures io social day trading
futures io futures trading


Backtest not matching Optimization
Updated: Views / Replies:1,275 / 9
Created: by d1g1talfr3ak Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 100,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Backtest not matching Optimization

  #1 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

Backtest not matching Optimization

in the past when I develop a strategy I first run the optimizer on a range of historical data to find best settings for parameters. Then I enter the settings the optimizer found as best and run backtest on same time period with same settings and ensure that I do get the value returned by the optimizer (sort of a sanity check on making sure I have settings entered correctly) then I run backtests on out of sample time periods. Only after that do I move on to sim trading.

I have a strategy now though, that when I optimize and get some results and settings, and enter those settings to do backtest... it does not match the results the optimizer claimed to have gotten with the same settings. I have meticulously checked every setting and they are set the same on backtest as on best run from optimizer. The results aren't even close.

Any ideas?

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,581 since Jun 2009
Thanks: 29,660 given, 85,195 received


Are you using exotic bar types?

Are you using genetic optimizer or exhaustive? If genetic, using the NT one or third party?

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #4 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

I am using the Ninjatrader built-in genetic optimizer. I would have thought that each step of the genetic algorithm it would internally run a backtest with selected values... then score value and run genetic routine to choose next generation of parameters. So I am stumped as to why running a backtest with the same settings the optimizer found doesn't match the results it showed. I have verified and re-verified that all settings are the same... including slippage & commisions, etc.

I can optimize other strats and then backtest on same time period with same values and get the same results. At first I thought that it was because this strat used an indicator that repainted past values. I tried telling the backtester and optimizer to not run multithreaded and a few other ideas... no go. I then wrote my own version of the indicator that did not repaint past values and still the same thing.

Thanks for any ideas and help

Reply With Quote
 
  #5 (permalink)
Elite Member
Huntsville AL/USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Stocks, Futures
 
d1g1talfr3ak's Avatar
 
Posts: 24 since Nov 2011
Thanks: 31 given, 46 received

Bar type

Also... I am using Better Renko. I am using it on several other strategies and not having the same issue.

Reply With Quote
 
  #6 (permalink)
Elite Member
McKinney, TX
 
Futures Experience: Intermediate
Platform: Think or Swim, NinjaTrade
Favorite Futures: The indexes, ES, YM, NQ & RTY
 
Posts: 154 since Aug 2012
Thanks: 213 given, 127 received

I had problems using Ranko bars. I had a tested strategy that was giving me very good results, then I ran that strategy against live data on the sim account and got a totally different return.

I don't know for sure, but what I think happens is that the Renko bar historical data is truncated so that the bar high/low values are used for the optimization. When running live, it appears that there are ticks outside of the Renko bars, those wide range ticks were hitting my stop loss and changing the results. What I would see is the execution of orders outside of the Renko bar.

Not sure if that is your problem, but when I tried the same strategy with Range bars, I couldn't duplicate the great results in the Analyzer but they matched the results better when I had a live data feed. Because of this, I won't use Renko bars on a strategy.

Reply With Quote
 
  #7 (permalink)
Membership Revoked
Szczecin
 
Futures Experience: Intermediate
Platform: Ninja, writing own now
 
Posts: 212 since Mar 2010
Thanks: 47 given, 103 received

Yes. Ninja does not use any data outside the bars, so you need bars that show true high low or better anther bar in the strat (tick 1) and then issue trades against this one.

Reply With Quote
 
  #8 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,581 since Jun 2009
Thanks: 29,660 given, 85,195 received

Renko bars and other exotic bar types have false opens, which makes backtesting them totally false.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #9 (permalink)
Elite Member
Miami FL, USA
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Forex
 
Posts: 1 since Apr 2012
Thanks: 5 given, 0 received

I have also had similar experiences with the NinjaTrader built in Genetic Optimizer. In my case, I used a plain old 5 minute chart, ran the optimizer, then backtested the winner and received wildly different results.

Upon inspecting the actual trades in the NinjaTrader Strategy Analyzer, it looks as if the optimizer wasn't using all of the price data available. What I mean is that if the strategy had a rule to close a position on a MA crossover, the optimized strategy did not close the position whereas the backtested strategy (with identical settings) did close the position on the crossover.

Reply With Quote
 
  #10 (permalink)
Membership Temporarily Revoked
Los Angeles, CA
 
Futures Experience: Intermediate
Platform: Ninjatrader7
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: ES
 
grimReaper's Avatar
 
Posts: 50 since Nov 2011
Thanks: 3 given, 12 received



dynoweb View Post
I had problems using Ranko bars. I had a tested strategy that was giving me very good results, then I ran that strategy against live data on the sim account and got a totally different return.

Perhaps overfitting?

Reply With Quote

Reply



futures io > > > > > Backtest not matching Optimization

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

September Journal Challenge (sponsored by TopstepTrader)

September

Battlestations! Win $947 in prizes for showing us your trading desk!

October
 

Finding Your Edge & Maximizing Returns w/Richard L Weissman

Oct 16
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Trendlines not matching from Higher Timeframe to lower Timeframe? bluemele NinjaTrader 26 October 8th, 2012 02:35 PM
Trouble matching tick charts across platforms keymoo Platforms and Indicators 2 May 30th, 2012 02:55 PM
ICE matching engines. Address of the building. maxima Elite Automated Trading 5 January 8th, 2012 07:14 PM
How I Backtest GaryD Traders Hideout 3 December 17th, 2011 12:01 PM
Buys and Sells not matching in number richw NinjaTrader Programming 8 February 22nd, 2011 06:15 PM


All times are GMT -4. The time now is 02:36 PM. (this page content is cached, log in for real-time version)

Copyright © 2018 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2018-10-16 in 0.15 seconds with 14 queries on phoenix