Is Walk Forward Optimization always a must? - Traders Hideout | futures io social day trading
futures io futures trading


Is Walk Forward Optimization always a must?
Updated: Views / Replies:3,467 / 15
Created: by Albnd Attachments:2

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you dont need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 2  
 
Thread Tools Search this Thread
 

Is Walk Forward Optimization always a must?

  #1 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received

Is Walk Forward Optimization always a must?

There are many threads about walk forward optimization, but something is still unclear to me.

My question is basically as follows: is the Walk Forward Optimization *always* a PLUS in comparison with traditional optimization? In other terms, is there any chance that a WFO is NOT necessary and/or useful?

Just a specific case, to make my point clear.

Im currently testing a (purely automated/mechanical) strategy with very few parameters, and most of them seem to be unrelated to any kind of periodical change in the market Im trading (QM futures contract). I basically started asking myself: is it necessary/useful to include these parameters in the optimization, if they dont seem to be correlated to any specific change in market behavior? (i.e. they basically seem to casually change from one period to another).

One of my mechanical strategies uses a fixed stop loss and evaluates the opening session gap size before taking action. I would usually prefer to set the stop loss just once, by considering the general MAE of the historical backtest. As for the gap size, I usually cannot see any particular change between the sizes of 2002 gaps and the ones related to todays markets. So, does it make sense to include these parameters in the optimization?

What about a strategy that has NO variables/parameters at all?

I would be more than happy to listen to your opinions and clarifications.

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary



Albnd View Post
There are many threads about walk forward optimization, but something is still unclear to me.

My question is basically as follows: is the Walk Forward Optimization *always* a PLUS in comparison with traditional optimization? In other terms, is there any chance that a WFO is NOT necessary and/or useful?

Just a specific case, to make my point clear.

Im currently testing a (purely automated/mechanical) strategy with very few parameters, and most of them seem to be unrelated to any kind of periodical change in the market Im trading (QM futures contract). I basically started asking myself: is it necessary/useful to include these parameters in the optimization, if they dont seem to be correlated to any specific change in market behavior? (i.e. they basically seem to casually change from one period to another).

One of my mechanical strategies uses a fixed stop loss and evaluates the opening session gap size before taking action. I would usually prefer to set the stop loss just once, by considering the general MAE of the historical backtest. As for the gap size, I usually cannot see any particular change between the sizes of 2002 gaps and the ones related to todays markets. So, does it make sense to include these parameters in the optimization?

What about a strategy that has NO variables/parameters at all?

I would be more than happy to listen to your opinions and clarifications.


Here is my experience, take it for what it is worth...

1. Whenever I optimized up to the present day with all data (a traditional backtest), and then started trading it, results were almost always bad.

2. When I do walkforward optimization, results are much better, but just because a strategy passes walkforward doesn't mean it will work going forward, or will work for a specified period of time.

3. Whenever I have done what I thought was "no optimization," I eventually found out that I was doing some type of optimization, just a little bit disguised. So, then it reverted to point 1.


You mention "a strategy that has NO variables/parameters at all." If this was the first and only strategy you have ever tested, and it performs well, I say congratulations, you may just have something wonderful.

If your "NO variables/parameters at all" strategy is your second, third, etc. attempt, then you have optimized to a degree - it just doesn't look or feel like a traditional backtest optimization.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #4 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received


kevinkdog View Post
3. Whenever I have done what I thought was "no optimization," I eventually found out that I was doing some type of optimization, just a little bit disguised. [...]

You mention "a strategy that has NO variables/parameters at all." If this [...] is your second, third, etc. attempt, then you have optimized to a degree - it just doesn't look or feel like a traditional backtest optimization.

Thanks for your reply. Good point indeed.

I was particularly speaking about very simple strategies, like (for example): buy at "x" fib extension bkout and exit at the end of day.
I don't have enough of data to demonstrate this (so I might be wrong), but in some cases, when you take a very simple strategy as it is, it seems to work beautifully for a specific market (and probably only for that one). And even if you would like to optimize and apply a WFO on it... what kind of parameters are you supposed to submit to the testing engine?

As for the specific case I mentioned above: you may think of the "x" variable as a parameter (to be optimized), or simply use one the fib levels like it is usually understood and used by hundreds of traders (and I personally wouldn't think of an optimization for that).

Reply With Quote
 
  #5 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary


Albnd View Post
Thanks for your reply. Good point indeed.

I was particularly speaking about very simple strategies, like (for example): buy at "x" fib extension bkout and exit at the end of day.
I don't have enough of data to demonstrate this (so I might be wrong), but in some cases, when you take a very simple strategy as it is, it seems to work beautifully for a specific market (and probably only for that one). And even if you would like to optimize and apply a WFO on it... what kind of parameters are you supposed to submit to the testing engine?

As for the specific case I mentioned above: you may think of the "x" variable as a parameter (to be optimized), or simply use one the fib levels like it is usually understood and used by hundreds of traders (and I personally wouldn't think of an optimization for that).

If I thought an entry which had no parameters was valid, I might look at different exits which did have parameters, or maybe only take the entry under certain market conditions (bear, bull, flat, volatile, rangebound, etc).

Sometime when you see an indicator work superbly for a short time, it may just be random chance, or a cherry picked example. I love vendors who sell such indicators, because they look like the Holy Grail, based on their example cases. But when put to the test, most indicators are only accurate about 50% of the time. At least in my tests!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #6 (permalink)
Elite Member
Ireland
 
Futures Experience: Intermediate
Platform: Ninja Trader
Favorite Futures: CL GC FDAX
 
Posts: 118 since Jul 2012
Thanks: 85 given, 48 received


Albnd View Post
There are many threads about walk forward optimization, but something is still unclear to me.

My question is basically as follows: is the Walk Forward Optimization *always* a PLUS in comparison with traditional optimization? In other terms, is there any chance that a WFO is NOT necessary and/or useful?

Just a specific case, to make my point clear.

Im currently testing a (purely automated/mechanical) strategy with very few parameters, and most of them seem to be unrelated to any kind of periodical change in the market Im trading (QM futures contract). I basically started asking myself: is it necessary/useful to include these parameters in the optimization, if they dont seem to be correlated to any specific change in market behavior? (i.e. they basically seem to casually change from one period to another).

One of my mechanical strategies uses a fixed stop loss and evaluates the opening session gap size before taking action. I would usually prefer to set the stop loss just once, by considering the general MAE of the historical backtest. As for the gap size, I usually cannot see any particular change between the sizes of 2002 gaps and the ones related to todays markets. So, does it make sense to include these parameters in the optimization?

What about a strategy that has NO variables/parameters at all?

I would be more than happy to listen to your opinions and clarifications.

Can I ask you why you are asking this question? Do you fully understand the NT WFO procedure and all the related stuff that comes with it?

I find myself in a very similar predicament to you with a strategy I have been working on for a long time. My strategy is purely intraday. I have just let it run and run on various instruments for weeks and it seems to do what it says on the tin. I have not used the WFO (yet) and I admit that while I do understand in sample/out of sample testing concept, I dont fully understand the exact NT WFO process/procedure myself. Need to get my head around that though because everyone seems to say its a must for any auto strat.

PM me if you wish but either way i would be very interested in more of your thoughts on this..

Reply With Quote
 
  #7 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received


kevinkdog View Post
If I thought an entry which had no parameters was valid, I might look at different exits which did have parameters, or maybe only take the entry under certain market conditions (bear, bull, flat, volatile, rangebound, etc).

Sometime when you see an indicator work superbly for a short time, it may just be random chance, or a cherry picked example. I love vendors who sell such indicators, because they look like the Holy Grail, based on their example cases. But when put to the test, most indicators are only accurate about 50% of the time. At least in my tests!

You're absolutely right, but this is not always the case. I'm speaking about strategies tested on a 10-12year base, 5min charts, thousands of trades. The accuracy should not be in question.

Also, an objection to your point: adding conditions (like exit rules and so on) wouldn't also mean adding complexities and then decrease the degree of freedom of the overall system?

Reply With Quote
 
  #8 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received


Bosch777 View Post
Can I ask you why you are asking this question? Do you fully understand the NT WFO procedure and all the related stuff that comes with it?

I find myself in a very similar predicament to you with a strategy I have been working on for a long time. My strategy is purely intraday. I have just let it run and run on various instruments for weeks and it seems to do what it says on the tin. I have not used the WFO (yet) and I admit that while I do understand in sample/out of sample testing concept, I dont fully understand the exact NT WFO process/procedure myself. Need to get my head around that though because everyone seems to say its a must for any auto strat.

PM me if you wish but either way i would be very interested in more of your thoughts on this..

Sure.

Yes, I think I understand the WFO process, even though I use it on TradeStation (and not on NT).

The point in my question is: is there any reason (or added value) in adding complexity to a strategy that seems to already work on its own? What's the meaning in adding a parameter, just to make the WFO process run in a meaningful way?

I do think that WFO is a tool, and not a goal in itself.
Also (my impression, but this is exactly what I'm asking your opinion about), I suspect that some unchanging (or randomly changing) market conditions don't require any specific optimization. By applying a WFO in cases like that, you would probably insist on tracking a market adaptation that doesn't really exist. The final outcome would risk to be unrelated to the real reasons that make your stategy work. This is really dangerous: your stategy may produce random results, for better or worse, but definitely out of control.

Reply With Quote
 
  #9 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received

Just to add some more elements to think about:

In my opinion, one of the best books related to the subject is Jaekle - Tomasini, Trading Systems. A new approach to system development and portfolio optimization.

The authors give special emphasis to the fact that in a robust system for every single out-of-sample period the suggested parameters don't change a lot, i.e. their change from one period to the one that follows is smooth and very often it's very small.

Is this also your experience?

Reply With Quote
 
  #10 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary



Albnd View Post
Just to add some more elements to think about:

In my opinion, one of the best books related to the subject is Jaekle - Tomasini, Trading Systems. A new approach to system development and portfolio optimization.

The authors give special emphasis to the fact that in a robust system for every single out-of-sample period the suggested parameters don't change a lot, i.e. their change from one period to the one that follows is smooth and very often it's very small.

Is this also your experience?

I liked the book, but evaluating changes in WFO parameters is not part of my design criteria, so I really can't comment on their position.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote

Reply



futures io > > > Is Walk Forward Optimization always a must?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Walk Forward Testing & Optimization Experiences and Best Practices MrYou NinjaTrader 2 December 29th, 2013 02:31 PM
Synthetic Data Walk Forward Optimization quantarb Elite EasyLanguage Automated Trading 5 December 23rd, 2012 01:52 PM
Include ATM parameters in optimization / walk forward analysis? Krislassen NinjaTrader Programming 3 April 14th, 2012 01:12 PM
Walk Forward Optimization - memory consumption Mondrakete NinjaTrader Programming 2 September 7th, 2011 06:04 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 06:51 PM


All times are GMT -4. The time now is 12:49 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-14 in 0.16 seconds with 20 queries on phoenix via your IP 54.221.93.187