Is Walk Forward Optimization always a must? - Traders Hideout | futures io social day trading
futures io futures trading


Is Walk Forward Optimization always a must?
Updated: Views / Replies:3,504 / 15
Created: by Albnd Attachments:2

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 2  
 
Thread Tools Search this Thread
 

Is Walk Forward Optimization always a must?

  #11 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received


kevinkdog View Post
I liked the book, but evaluating changes in WFO parameters is not part of my design criteria, so I really can't comment on their position.

I see, but if you don't care about their change, you are probably missing a point about the reason WHY your strategy reacts (positively or negatively, winning or losing) to the market. I say this because one of the purposes of WFO is to keep your algo correlated to the market you trade, with some kind of "smart" (not curve-fitted) adaptation. A not appropriate understanding of the way a WFO uses or suggests its values may result in a random exposition to the market. IMHO, to avoid this risk you must be able to make a difference between a correct "identity" of the WFO suggested parameters and a bunch of "magic" numbers a trader may want to use without even understanding why.

Reply With Quote
 
  #12 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary


Albnd View Post
I see, but if you don't care about their change, you are probably missing a point about the reason WHY your strategy reacts (positively or negatively, winning or losing) to the market. I say this because one of the purposes of WFO is to keep your algo correlated to the market you trade, with some kind of "smart" (not curve-fitted) adaptation. A not appropriate understanding of the way a WFO uses or suggests its values may result in a random exposition to the market. IMHO, to avoid this risk you must be able to make a difference between a correct "identity" of the WFO suggested parameters and a bunch of "magic" numbers a trader may want to use without even understanding why.


I'd certainly be interested to see any data you have that supports your contention. Can you please share?

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #13 (permalink)
Elite Member
Ireland
 
Futures Experience: Intermediate
Platform: Ninja Trader
Favorite Futures: CL GC FDAX
 
Posts: 118 since Jul 2012
Thanks: 85 given, 48 received



Albnd View Post
I see, but if you don't care about their change, you are probably missing a point about the reason WHY your strategy reacts (positively or negatively, winning or losing) to the market. I say this because one of the purposes of WFO is to keep your algo correlated to the market you trade, with some kind of "smart" (not curve-fitted) adaptation. A not appropriate understanding of the way a WFO uses or suggests its values may result in a random exposition to the market. IMHO, to avoid this risk you must be able to make a difference between a correct "identity" of the WFO suggested parameters and a bunch of "magic" numbers a trader may want to use without even understanding why.

Yes I do see your point. I too wonder about the amount of importance being put on the WFO for ALL strategy development. After all is this in itself not a form of curve fitting? In sample and out of sample testing is important but with the WFO, the parameters are being constantly adjusted to "fit" the next period of the test. If the WFO then turns round and gives you the best average set of parameters well then what is this only a rake of curve fits just layered over each other. (Mind you, NT do say that the WFO process is their LEAST understood process(outside of NT I assume they mean..LOLLL))

I also think a lot will depend on the type of strategy. An intraday only strategy that works well over say a year of data should not blow up when introduced to another year of data with those same parameters. That is unless there is some seismic change to the behavior of the markets going forward, in which case the WFO would not have been any use anyway as everything is retrospective with its process.
ON THE OTHER HAND..... if your strategy is more of a position type or swing type or something that holds positions for longer periods (days/weeks) then this is a different ball game and I would imagine the WFO starts to become more important/valid....

BTW I think it would be great if people would post a screen shot of what they think are good and bad strategy analyzer results and why they think what they think. This way we could all get more of a feel for whats considered acceptable and not acceptable and indeed to discuss the metrics used a bit more.

So there...!


Reply With Quote
 
  #14 (permalink)
Trading for Fun
Rome, Italy
 
Futures Experience: Advanced
Platform: TradeStation, TWS (IB), MT4
Broker/Data: Tradestation, IB
Favorite Futures: Futures (QM, CL, ES, grains), Stocks, Forex
 
Albnd's Avatar
 
Posts: 62 since Feb 2012
Thanks: 27 given, 26 received


kevinkdog View Post
I'd certainly be interested to see any data you have that supports your contention. Can you please share?


Bosch777
I think it would be great if people would post a screen shot of what they think are good and bad strategy analyzer results and why they think what they think.

As I said, I'm more into asking for your opinion than into demonstrating a thesis. Btw, here is some examples that support my impressions:

Crude Oil contract, excel chart of (daily) opening gap sizes since 2006.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


You may notice that the distribution is more or less homogeneous on the long run, apart from a slightly evident bias to the down side. However, if you "normalize" these data by considering the progressive daily range, you may get a quite steady line of distribution.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).


Now, as Bosch777 correctly noticed, if you take care of the overall scenario, you may suppose that there isn't any real change/evolution in this parameter. Or maybe you might be persuaded that there's a slight bias to a specific direction (red dotted line), and this will widely make room for a WFO test.
However, you may get a very different impression if you give a look to the short-term perspective (the vertical positions of the contiguous blue dots): the change from one day to the other, or from one week to the following one, is totally erratic. This is immediately reflected by the WFO results, that usually take these changes too "seriously" by proposing an adaptation that has no logical or strategical grounding.


Last edited by Albnd; March 25th, 2013 at 01:26 PM. Reason: small editing for clarification
Reply With Quote
 
  #15 (permalink)
Elite Member
San Diego, USA
 
Futures Experience: Intermediate
Platform: MC
Favorite Futures: All
 
Posts: 40 since Aug 2016
Thanks: 7 given, 8 received


Albnd View Post
There are many threads about walk forward optimization, but something is still unclear to me.

My question is basically as follows: is the Walk Forward Optimization *always* a PLUS in comparison with traditional optimization? In other terms, is there any chance that a WFO is NOT necessary and/or useful?

Just a specific case, to make my point clear.

Iím currently testing a (purely automated/mechanical) strategy with very few parameters, and most of them seem to be unrelated to any kind of periodical change in the market IĎm trading (QM futures contract). I basically started asking myself: is it necessary/useful to include these parameters in the optimization, if they donít seem to be correlated to any specific change in market behavior? (i.e. they basically seem to casually change from one period to another).

One of my mechanical strategies uses a fixed stop loss and evaluates the opening session gap size before taking action. I would usually prefer to set the stop loss just once, by considering the general MAE of the historical backtest. As for the gap size, I usually cannot see any particular change between the sizes of 2002 gaps and the ones related to todayís markets. So, does it make sense to include these parameters in the optimization?

What about a strategy that has NO variables/parameters at all?

I would be more than happy to listen to your opinions and clarifications.




I was thinking about WF along similar lines. I was not sure if WF was the ultimate tester for robustness of the strategy because of limitations of in sample windows and market conditions that exit then and following change in the market conditions in out of sample results. And that is what i noticed that many models suffered during WF at the time then the market conditions have been changing - for example going into 2008-09. So, my concern where was that the data that i have been in sampling before 2008-09 did not have market conditions as in these years. As you roll over, you start to account for the these years and they take bigger part of your in sample optimized parameters when you come out from 2008-09 and thereafter when the market normalizes back again and you model gives still quite volatile out of sample results following such roll for the next couple of yours. And thereafter the results normalize.
So, i was not and am not sure whether to use WF or all in sample data and just leave 2-3 latest year as out of sample. Any suggestions are much appreciated.

Reply With Quote
 
  #16 (permalink)
Elite Member
San Diego, USA
 
Futures Experience: Intermediate
Platform: MC
Favorite Futures: All
 
Posts: 40 since Aug 2016
Thanks: 7 given, 8 received


Albnd View Post
Just to add some more elements to think about:

In my opinion, one of the best books related to the subject is Jaekle - Tomasini, Trading Systems. A new approach to system development and portfolio optimization.

The authors give special emphasis to the fact that in a robust system for every single out-of-sample period the suggested parameters don't change a lot, i.e. their change from one period to the one that follows is smooth and very often it's very small.

Is this also your experience?



Recently i have developed a strategy that does not have a single best optimum parameter for the whole data: I was testing on ES, NQ. It performs well most of the time but then has extended period of flat performance if you optimize for the all the data.
BUT, when I plugged in the model into WF, the OS results came nearly perfectly upward slopping. The model has only two variables and both of them did not fluctuate much at all, just a bit to accommodate for changing market conditions - just like as you mentioned above. I guess such models are better used with regular WF re-optimization as their robustness is prone to changing market conditions (not sure if that is not data fitting in itself)).

Reply With Quote

Reply



futures io > > > Is Walk Forward Optimization always a must?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Walk Forward Testing & Optimization Experiences and Best Practices MrYou NinjaTrader 2 December 29th, 2013 02:31 PM
Synthetic Data Walk Forward Optimization quantarb Elite EasyLanguage Automated Trading 5 December 23rd, 2012 01:52 PM
Include ATM parameters in optimization / walk forward analysis? Krislassen NinjaTrader Programming 3 April 14th, 2012 01:12 PM
Walk Forward Optimization - memory consumption Mondrakete NinjaTrader Programming 2 September 7th, 2011 06:04 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 06:51 PM


All times are GMT -4. The time now is 11:52 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-15 in 0.15 seconds with 20 queries on phoenix via your IP 54.163.210.170