WARNING: MC 8.5 walk forward optimizer is FLAWED - MultiCharts | futures io social day trading
futures io futures trading


WARNING: MC 8.5 walk forward optimizer is FLAWED
Updated: Views / Replies:8,275 / 46
Created: by treydog999 Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

WARNING: MC 8.5 walk forward optimizer is FLAWED

  #1 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

WARNING: MC 8.5 walk forward optimizer is FLAWED

I have been banging my head against the wall because of the WFO results I have been getting. So i started to look deeper into it. Turns out there is overlapping time periods in the WFO engine, instead of true shifting of time windows. Also you can not replicate the WFO results inside a strategy using Date and Time flags. I have been working with tech support for over a week, and they finally today decided to acknowledge the flaw. So anyone out there developing using it, be very very careful.

Reply With Quote
The following 4 users say Thank You to treydog999 for this post:
 
  #2 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,139 given, 4,387 received
Forum Reputation: Legendary


treydog999 View Post
I have been banging my head against the wall because of the WFO results I have been getting. So i started to look deeper into it. Turns out there is overlapping time periods in the WFO engine, instead of true shifting of time windows. Also you can not replicate the WFO results inside a strategy using Date and Time flags. I have been working with tech support for over a week, and they finally today decided to acknowledge the flaw. So anyone out there developing using it, be very very careful.

Yikes, that is a bit scary!

Advice for people new to WF, not for Treydog: For anyone doing walkforward, I always recommend doing a case by hand when first starting out with it. It is tedious and time consuming, but at least for me I understood the concept and application so much better afterwards. Try a simple 5 year in period, and 1 year out period, and walk it by hand.

I'm guessing Trey did something like this, to uncover this issue. Thanks for alerting everyone Trey!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
 
  #3 (permalink)
Elite Member
Amsterdam, Europe
 
Futures Experience: Intermediate
Platform: Various
Favorite Futures: ES
 
teyano's Avatar
 
Posts: 22 since Feb 2013
Thanks: 67 given, 14 received

MC's WFO



treydog999 View Post
I have been banging my head against the wall because of the WFO results I have been getting. So i started to look deeper into it. Turns out there is overlapping time periods in the WFO engine, instead of true shifting of time windows. Also you can not replicate the WFO results inside a strategy using Date and Time flags. I have been working with tech support for over a week, and they finally today decided to acknowledge the flaw. So anyone out there developing using it, be very very careful.

@treydog999 Did Multicharts solved the issue ?

Reply With Quote
 
  #4 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

They sent me a hot fix a few days ago. But I haven't had time to test it. There is no general fix out there that i know of. They did not fix the ability to recreate results using date and time flags as I understand it but i would need to test it before I say anything conclusive.

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
  #5 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

Alright, I just did some manual tests vs the WFO engine. The date and times no longer overlap which is good. however the trade results are still incorrect. So i would say its still broken. I have again submitted my issues to multicharts, i been working with Andrew and Henry. But they take forever, took them weeks to come up with the hotfix. So basically do it by hand if you have to. I feel like they should be paying me to be a debugger or something. Seriously this is ridiculous. To bad I have the lifetime license

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
  #6 (permalink)
Elite Member
Amsterdam, Europe
 
Futures Experience: Intermediate
Platform: Various
Favorite Futures: ES
 
teyano's Avatar
 
Posts: 22 since Feb 2013
Thanks: 67 given, 14 received

WFO

This is really quite pathetic for @MultiCharts If they consider this as a no priority issue (it seems though) for a backtester to be flawed in a software that is designed for algo trading than they need to seriously reevaluate their business. Really.

Reply With Quote
 
  #7 (permalink)
 Vendor: tradingcode.net 
The Netherlands
 
Futures Experience: None
Platform: MultiCharts, TradingView
Favorite Futures: ...
 
Jura's Avatar
 
Posts: 774 since Apr 2010
Thanks: 2,347 given, 661 received


teyano View Post
This is really quite pathetic for @MultiCharts If they consider this as a no priority issue (it seems though) for a backtester to be flawed in a software that is designed for algo trading than they need to seriously reevaluate their business. Really.

Well, we don't know how much work these issues are and how many programming hours have been assigned to this already, and if they are really errors or just 'design choices'. So I think that your 'really quite pathetic' is a little bit overdone - even treydog999 was unassuming when he said "i would say its still broken".

That is, of course, not to say that if there is an error in the WFO, this needs to be fixed (obviously). Can you keep us updated about this Treydog999?

Reply With Quote
 
  #8 (permalink)
Elite Member
Amsterdam, Europe
 
Futures Experience: Intermediate
Platform: Various
Favorite Futures: ES
 
teyano's Avatar
 
Posts: 22 since Feb 2013
Thanks: 67 given, 14 received


Jura View Post
Well, we don't know how much work these issues are and how many programming hours have been assigned to this already, and if they are really errors or just 'design choices'. So I think that your 'really quite pathetic' is a little bit overdone - even treydog999 was unassuming when he said "i would say its still broken".

Thats true, I admit my statement was overdone, but considering the fact the this thread is here almost one month and it's on the very top of the whole Multicharts side it would be desirable if someone could make a comment form Multicharts regarding this. What is more a work around that makes "the trade results are still incorrect" is also not the best case scenario.

Reply With Quote
 
  #9 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


Jura View Post
Well, we don't know how much work these issues are and how many programming hours have been assigned to this already, and if they are really errors or just 'design choices'. So I think that your 'really quite pathetic' is a little bit overdone - even treydog999 was unassuming when he said "i would say its still broken".

That is, of course, not to say that if there is an error in the WFO, this needs to be fixed (obviously). Can you keep us updated about this Treydog999?

I will definitely keep this thread updated with new developments between me and the developers/engineers. the outlook does look grim though. They told me when i first came to them about it that it was "too complicated" at first. Then they sent me the hot fix for the easiest part of the problem, but its like putting new paint on a lemon. Looks a lot better but still under the hood is broken. Supposedly I will get in update next week to see how it goes with the programmers. They have extracted so much data from my including my strategies, workspaces and data stream to isolate it. But upon their own admission it is systemic. So its just going to be a waiting game at this point.

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #10 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


Ok so after working with the developers at multicharts, I am able to use a work around to independently verify the walk forward tests. You are required to get the hot fix they gave me, or as them to post it on the forum or something. Without it this is useless. This does not work with time and date flags inside the code itself, you have to manually adjust adjust the look back window in the instrument settings area. They key aspect is to incorporate a lead time of X bars back for however many bars back your walk forward used. SO if you have a 200 bars back on a daily, you need to move your starting window 200 days earlier in the instrument settings. I tried this on the worksheet and strategy optimization they sent me as an example. I haven't done it on any of my strats so far because they are not daily bar based, so figuring out how many bars back you need is going to be a pain. However at least you can verify that now the walk forward does in fact give you proper results post hotfix.

I will continue to work on this and see if i can verify the results with intraday bars and on more complicated strategies.

Reply With Quote
The following 3 users say Thank You to treydog999 for this post:

Reply



futures io > > > > WARNING: MC 8.5 walk forward optimizer is FLAWED

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Is Walk Forward Optimization always a must? Albnd Traders Hideout 15 October 28th, 2016 10:00 AM
Synthetic Data Walk Forward Optimization quantarb Elite EasyLanguage Automated Trading 5 December 23rd, 2012 01:52 PM
Walk Forward Experiment Silver Dragon Elite Automated Trading 327 September 9th, 2012 12:56 AM
Walk Forward Optimization - memory consumption Mondrakete NinjaTrader Programming 2 September 7th, 2011 06:04 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 06:51 PM


All times are GMT -4. The time now is 05:54 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-15 in 0.15 seconds with 19 queries on phoenix via your IP 54.145.16.43