Optimization without curve fitting - Traders Hideout | futures io social day trading
futures io futures trading


Optimization without curve fitting
Updated: Views / Replies:4,059 / 17
Created: by cunparis Attachments:7

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 7  
 
Thread Tools Search this Thread
 

Optimization without curve fitting

  #1 (permalink)
Trade with the flow
Paris, France
 
Futures Experience: Advanced
Platform: Market Delta & Ninjatrader
Favorite Futures: ES
 
cunparis's Avatar
 
Posts: 2,563 since Jun 2009
Thanks: 1,157 given, 2,030 received

Optimization without curve fitting

I have developed 3 indicators that each test profitably. I've determined the optimal parameters by optimization (periods, thresholds, etc.). I do not expect to get the same results in the future, but I prefer to use the optimized values rather than some arbitrary values. For example everyone may use 12,26,9 for their MACD but if 10,22,8 give better results in backtesting them it'd be better to use that right?

My question is this: I'm now working on combining these 3 into one signal (short, flat, long). I've tried two different approaches to do this:

1 - I use the optimal parameters that I determined on each indicator individually

2 - I re-optimized all parameters together.

#1 seems to be more realistic, with the acknowledgment that the performance will not be the same as the backtests, due to the performance of each system not being the same. This I know. So the final results will probably not be as good. But this seems robust.

#2 - Seems to be more optimal, with an even stronger acknowledgment that the results will not be as good as the backtest. However there is a greater risk of curve fitting due to the increased rules and degrees of freedom. In defense of the optimization I will say that lots of attempts produced unacceptable results so I believe that if optimization finds something good say PF > 3.0 then it's very likely to be positive in forward testing even though the PF will most likely be less.

I'm curious what people think about these two approaches. I am currently forward testing both #1 & #2 but since they trade on daily charts and not very often, it will take a while to have something meaningful.

I've developed systems that have held up and systems that have fallen apart. I understand the limitations of backtesting and automation. So I prefer not to debate that but focus on which approach would be best.

Reply With Quote
The following user says Thank You to cunparis for this post:
 
  #2 (permalink)
Elite Member
Israel
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: pfg
Favorite Futures: eminis
 
Posts: 323 since Jun 2009
Thanks: 6 given, 206 received

Hi,
The best approach is Walk Forward. You can try your #1 and #2, each optimize on sertain period of time and then back test it on the next period.

Baruch

Reply With Quote
 
  #3 (permalink)
 Vendor: www.innovative-trading-solutions-online.com 
Hartford, CT. USA
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: AMP/CQG, Kinetick
Favorite Futures: TF
 
RJay's Avatar
 
Posts: 677 since Jun 2009
Thanks: 726 given, 740 received


Cunparis,

I am going with multiple independent MAs.

I also believe keeping track of Close[0] in relationship to each MA is very important.

I am using colored bars in the attached image to easily determine what is going on with each MA in regards to Close[0] and who is agreeing with whom.

No eye strain, easily automated if desired, and programmable to display any situation.

My Thoughts,

RJay

Attached Thumbnails
Optimization without curve fitting-multiplema.jpg  
Reply With Quote
 
  #4 (permalink)
Trade with the flow
Paris, France
 
Futures Experience: Advanced
Platform: Market Delta & Ninjatrader
Favorite Futures: ES
 
cunparis's Avatar
 
Posts: 2,563 since Jun 2009
Thanks: 1,157 given, 2,030 received


baruchs View Post
Hi,
The best approach is Walk Forward. You can try your #1 and #2, each optimize on sertain period of time and then back test it on the next period.

Baruch

I've read about this but I've never done it, which is probably why most of my strategies stopped working.

one of my indicators has 500 trades since 2000. I'm going to optimize it for the first 100 trades and then walk it forward the next 100. Then optimize the first 200 and walk it forward the next 100. So I'll have:

0-100 walked 100-200
0-200 walked 200-300
0-300 walked 300-400
0-400 walked 400-500

this will make 4 walk forward tests. I'll compare the results of each walk forward with the same period optimized. I will also compare the optimal values for each test.

Does this sound right?

I'll post the results as I go. Here's the equity curve for the results of the optimization of the entire 9 year period (500 trades).

Attached Thumbnails
Optimization without curve fitting-equity-curve-optimizing-500-trades.jpg  
Reply With Quote
 
  #5 (permalink)
Trade with the flow
Paris, France
 
Futures Experience: Advanced
Platform: Market Delta & Ninjatrader
Favorite Futures: ES
 
cunparis's Avatar
 
Posts: 2,563 since Jun 2009
Thanks: 1,157 given, 2,030 received


RJay View Post
Cunparis,

I am going with multiple independent MAs.

RJay - I think we're talking about different things. I made an indicator, but the indicator uses lookback periods, moving averages, thresholds, etc. It's those that I want to optimize to find the optimal values for my indicator. Just like someone came up with 14,3,3 for stochastic.

Reply With Quote
 
  #6 (permalink)
Elite Member
Austin, TX
 
Futures Experience: Advanced
Platform: NT/Matlab
Broker/Data: Interactive Brokers
Favorite Futures: FX majors
 
sefstrat's Avatar
 
Posts: 285 since Jun 2009
Thanks: 20 given, 749 received

Technically both of the approaches you describe above are curve fitting =)

Global parameter search (method #2) is a much better approach because a disjoint search does not take into consideration the relationships between the different indicators.

You want to find a 'stable region' of parameters, ie a range of numbers near each other which all give similar performance, that is good curve fitting.

Bad curve fitting is when you find the best set of parameters which may be an outlier in an unstable region, ie it may have high profit now but the likelihood of it continuing to work in the future is lower.

See the attached matlab plots for an illustration of this. The colors represent performance of a strategy based on normalized Sharpe ratio, red means highest sharpe ratio blue is lowest.

You might be tempted to pick a value along the sharp red ridge that runs along Param1=20, but actually a better choice for the long run is one of the flatter red bumps at the back (near Param1=50, Param2 = 225)

The sharpness of the first ridge makes it inherently unstable, think of it just like in real life if you were standing on a sharp peak and it moved a little bit, you would fall off (which in this case results in drawdown) whereas if you are standing on a flatter hill you will not be affected as much if it shifts slightly.

(BTW.. i generated these images but they are of a strategy from a mathworks webinar, if any of you are matlab users you can download the code and webinar from their website)

Attached Thumbnails
Optimization without curve fitting-heatsurface.png   Optimization without curve fitting-heatmap.png  

Last edited by sefstrat; August 30th, 2009 at 08:41 AM.
Reply With Quote
The following 6 users say Thank You to sefstrat for this post:
 
  #7 (permalink)
Trade with the flow
Paris, France
 
Futures Experience: Advanced
Platform: Market Delta & Ninjatrader
Favorite Futures: ES
 
cunparis's Avatar
 
Posts: 2,563 since Jun 2009
Thanks: 1,157 given, 2,030 received

I'm sad to report the walk forward tests weren't that good. I started with 2000-2003 to give me 100 trades. Then I walked forward year by year. The first couple years it did well on the walk forward test. But then around 2007-present it didn't. I can explain that because that's when we went from bull to bear market. So if we figure the bear market started roughly end of 2007, the problem is I don't have enough trades during the bear market. And even then things changed in March when we started the bull market (or bear market rally). And the market direction isn't the only thing that changed, volatility has completely changed over the past few years.

For me this is the hardest part of creating a mechanical system on a daily timeframe. I've spent a lot of hours on this but I'm afraid it's back to the drawing board.

Lesson learned: With two moving averages you can come up with a profitable system using just about any data.

I still think there's some predictive power here but I have to find a way to prove it..

Reply With Quote
 
  #8 (permalink)
Elite Member
Israel
 
Futures Experience: Intermediate
Platform: NinjaTrader
Broker/Data: pfg
Favorite Futures: eminis
 
Posts: 323 since Jun 2009
Thanks: 6 given, 206 received

Hi,
Sorry it didn't work, but now you know what to do and hopefully next time you will succeed. Don't blame it on bull or bear market. As I understand you have a trend following system, so it should work on both sides.

Baruch

Reply With Quote
 
  #9 (permalink)
Elite Member
Asia
 
Futures Experience: Beginner
Platform: NinjaTrader, TOS
 
Posts: 798 since Jun 2009
Thanks: 109 given, 786 received

cunparis have you looked at "The Evolution & Optimization of Trading Strategies" by Pardo? Interesting chapters on curve fitting vs. optimization.

Out of curiosity what are you using for historical data and in what timeframe? I'm working on range and tick based charts and it would great if I could test more than the last two-three contracts.

Reply With Quote
 
  #10 (permalink)
Trade with the flow
Paris, France
 
Futures Experience: Advanced
Platform: Market Delta & Ninjatrader
Favorite Futures: ES
 
cunparis's Avatar
 
Posts: 2,563 since Jun 2009
Thanks: 1,157 given, 2,030 received



MXASJ View Post
cunparis have you looked at "The Evolution & Optimization of Trading Strategies" by Pardo? Interesting chapters on curve fitting vs. optimization.

Out of curiosity what are you using for historical data and in what timeframe? I'm working on range and tick based charts and it would great if I could test more than the last two-three contracts.

In this case I'm using minute data from 2000 - present, using Tradestation. This is a big problem with zen-fire not having continuous contracts and long history. I have iqfeed for this purpose for NT.

I have read pardo's book and need to reread the parts about walk forward testing. I worked a lot on my system and when it didn't hold up I was very disappointed. so now I'm focusing on non-mechanical trading. One thing I want to do is try my algorithm with other data sources and that should be interesting. if they can be curve fitted than anything can. if they cannot then maybe my original idea was on to something.

Reply With Quote

Reply



futures io > > > Optimization without curve fitting

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
The battle between optimization and curve-fitting shodson Elite Automated Trading 5 April 7th, 2016 01:10 PM
Curve fitting and money management RM99 Psychology and Money Management 3 July 25th, 2011 03:57 PM
Curve Fitting diverdan Traders Hideout 2 September 13th, 2010 01:14 PM
An experiment on curve fitting shodson Traders Hideout 6 May 31st, 2010 07:34 AM
Using NTDataFeeder to fight off curve fitting Big Mike The Elite Circle 5 September 29th, 2009 03:59 PM


All times are GMT -4. The time now is 07:55 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-12 in 0.14 seconds with 20 queries on phoenix via your IP 54.234.255.29