WARNING: MC 8.5 walk forward optimizer is FLAWED - MultiCharts | futures io social day trading
futures io futures trading


WARNING: MC 8.5 walk forward optimizer is FLAWED
Updated: Views / Replies:8,294 / 46
Created: by treydog999 Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

WARNING: MC 8.5 walk forward optimizer is FLAWED

  #21 (permalink)
Elite Member
Wayne, NJ USA
 
Futures Experience: Intermediate
Platform: Multicharts, Deltix DCS
Broker/Data: Interactive Brokers/IQ Feed
Favorite Futures: ETF Options
 
Posts: 41 since Oct 2012
Thanks: 7 given, 36 received


arnie View Post
Hi @Jura

Since HFT and algorithmic trading is the hottest thing around these days, I thought "algo" guys would love the idea of having milliseconds available to them, giving the extra granularity to program their strategies, hence - the most important version release in the last years.
Sorry for that, apparently my assumption was wrong
A good friend of mine always told me that my assumptions will be the death of me

Please don't misinterpret me, bugs should have always precedence but there are bugs, and I have no idea if this is such a case, but I assume (again, I'm assuming) it's not since several fixes have been made available and none solved the issue, that will be easily fixed with some new feature that will be released in the next version.
So if some bug will be easily fix when that new feature becomes available, why lose time trying to fix something now if in a couple of months it might stop working again since with that new feature, the way that tool works will change again?

I share treydog pain, heck, we all share his pain, we all had our share of working with something that was only half working and had to wait for the next version to see the problem solved but the only thing I could do was nagging support about it, so they could move faster in solving the problem.

I hope I'm not putting words in his mouth but I believe treydog said he was losing his time testing the fixes MC sends to him.
He discovered the bug. The tool is not working accordingly to his settings and knowledge.
Apparently, developers thought the tool was working fine, accordingly to their knowledge.
Assuming treydog is more knowledged on the matter, since he's a trader, he's the most qualified to be testing this, telling developers that the tool continues to be faulty, which he is.

Helping MC developers with this treydog is helping himself on the long haul because he will have a tool set accordingly to his settings.

Personally I consider this to be a very useful and friendly relationship between developer and final user. Both helping each other in building a better platform.

The ultra high frequency guys aren't using Multicharts. They usually write their own program or use institutional products.

Reply With Quote
 
  #22 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

Alright after working with the guys at MC, I have gotten some more files to work with. I tried to recreate the example they gave but since they used something that moved instead of a set date it was impossible. So i decided to try my own small test, it seems like it works now. I need to go more indepth but basically here is the run down. You can only do 1 window at a time (this is all i have tested), the trades must be closed at the end of the window, slippage and commision was simply $1 per contract (this was a major issue before), You must have enough lead time in your window to accommodate your max bars back number. If you have these things then your strategy performance report should match any 1 given window of the WFO.

Caveats: I have not tried multiple windows at once, I have not tried complicated multi data stream strategies, I have no tried tiered commissioned systems. Basically i used a preloaded strat and just filled in some blanks. But its a promising start. After I do some more testing I will be able to come to a more concrete answer. But until then at least i am able to replicate to under certain conditions what the WFO is generating.


edit: you still require the hotfix that was sent to me personally from MC. I don't know if they have it publicly available but without it you cant do anything. They should probably post something thats hard coded update released publicly but you can ask.

Reply With Quote
The following 3 users say Thank You to treydog999 for this post:
 
  #23 (permalink)
Elite Member
Hannover
 
Futures Experience: Beginner
Platform: MultiCharts
Favorite Futures: EUR/USD
 
Posts: 5 since Jan 2012
Thanks: 1 given, 1 received



treydog999 View Post
Alright after working with the guys at MC, I have gotten some more files to work with. I tried to recreate the example they gave but since they used something that moved instead of a set date it was impossible. So i decided to try my own small test, it seems like it works now. I need to go more indepth but basically here is the run down. You can only do 1 window at a time (this is all i have tested), the trades must be closed at the end of the window, slippage and commision was simply $1 per contract (this was a major issue before), You must have enough lead time in your window to accommodate your max bars back number. If you have these things then your strategy performance report should match any 1 given window of the WFO.

Caveats: I have not tried multiple windows at once, I have not tried complicated multi data stream strategies, I have no tried tiered commissioned systems. Basically i used a preloaded strat and just filled in some blanks. But its a promising start. After I do some more testing I will be able to come to a more concrete answer. But until then at least i am able to replicate to under certain conditions what the WFO is generating.


edit: you still require the hotfix that was sent to me personally from MC. I don't know if they have it publicly available but without it you cant do anything. They should probably post something thats hard coded update released publicly but you can ask.

It sounds good, that MC is on the way to help to resolve this issue.
Will this fix be a part of the next MC release ?

Reply With Quote
 
  #24 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


wbailey View Post
It sounds good, that MC is on the way to help to resolve this issue.
Will this fix be a part of the next MC release ?

no idea

Reply With Quote
 
  #25 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

Alright, honestly I am getting fed up with this work.

So basically they sent me a cherry picked example where I could replicate. How did i determine this, I started testing other strategies on other time frames. This is using the production release 8.5. I was able to get very close but again, if there are trades that are crossing the period switching barrier then you have yourself a problem. Multicharts does not explain how they deal with it. are they closed at the end of the period, are they held over? I have tried both methods and haven't come up with a way to replicate exactly.

I put 8.7 beta on another machine to try it out, figuring they possibly fixed something. Its actually worse performing than my hotfixed 8.5. just adding commision without slippage creates problems with replication.

Bottom line: Still broken multicharts

GET YOUR ACT TOGETHER!!!!!!

Reply With Quote
The following 3 users say Thank You to treydog999 for this post:
 
  #26 (permalink)
Elite Member
Toronto, Ontario, Canada
 
Futures Experience: Intermediate
Platform: TradeStation
Favorite Futures: forex, gold
 
Posts: 4 since Jul 2012
Thanks: 9 given, 0 received


treydog999 View Post
Alright, honestly I am getting fed up with this work.

I put 8.7 beta on another machine to try it out, figuring they possibly fixed something. Its actually worse performing than my hotfixed 8.5. just adding commision without slippage creates problems with replication.

Bottom line: Still broken multicharts


Is this issue fixed in Multicharts yet?

I appreciate if someone can please provide an update on this.

Reply With Quote
 
  #27 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


Corinne View Post
Is this issue fixed in Multicharts yet?

I appreciate if someone can please provide an update on this.

Alright, so far its been a mixed bag. I have been able to verify individual walk forward windows under some very specific conditions. They include, there are no trades that pass from 1 window to the next, you add maxbarsback (the exact amount) which can be extremely difficult to get down, and using the same start date and end date which i set inside the code itself.

However, if you try to string together your WFOs like Kevin Davey does, you will not be able to sum up your WFO windows to the same answer. Also if any 1 of your windows has a trade that passes through to the next window, you won't be able to validate the entire series (extremely likely). So does it work, kinda. But its highly limited and not reproducible in a window by window approach. I have worked with Multicharts for several weeks and they have basically told me, this is how it is supposed to function. So I dont expect any upgrades or modifications to make it more robust, results reproducible or verifiable in anyway.

If your worried about the results, you can always walk forward the old fashioned way. window by window using the date/max bars back function and be sure you get solid results.

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
  #28 (permalink)
 Vendor: tradingcode.net 
The Netherlands
 
Futures Experience: None
Platform: MultiCharts, TradingView
Favorite Futures: ...
 
Jura's Avatar
 
Posts: 774 since Apr 2010
Thanks: 2,347 given, 661 received


treydog999 View Post
(...)
If your worried about the results, you can always walk forward the old fashioned way. window by window using the date/max bars back function and be sure you get solid results.

You mean with closing open positions at the end of every WF window, or do I misunderstand you?

(Thanks for the update btw)

Reply With Quote
 
  #29 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


Jura View Post
You mean with closing open positions at the end of every WF window, or do I misunderstand you?

(Thanks for the update btw)

I mean setting your date/time settings and then running the strategy there and saving the performance report. But like you mentioned close any currently open trades during that , then manually set the date/time to new settings in order to see the next window. Run a Normal optimization on each of these windows. Basically you are doing the job of the auto WFO by hand. Its the only way to guarantee a clean wf.

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
  #30 (permalink)
Elite Member
San Jose
 
Futures Experience: Intermediate
Platform: NinjaTrader, Esignal
Favorite Futures: Stocks
 
Posts: 107 since Jan 2012
Thanks: 40 given, 21 received

Would this help?


Hi,

I am reading this thread as I was about to embark on some walk forward analysis, and now I am pretty concerned if its worth the effort on the MC platform.

Is Ninja any better?

I did a google search and found a product by codefortraders and another one called Diamond Backtsesting that is some sort of add in that does walk forward, would this be worth getting?

PS I have no affiliation of w/ CodeForTraders nor Diamond -- I am not even a customer of any of these companies, I surely am interested in a good backtesting solution, and if MC is flawed I would like to know which platform is good at backtesting and Walk Forward Analysis.

I do have a copy of AMIBroker but am having a lot of concern regarding using Tick Data with that platform. So to be frank I am pretty unsure what to do at this point in testing my strategies.

Any suggestions are appreciated.

Reply With Quote

Reply



futures io > > > > WARNING: MC 8.5 walk forward optimizer is FLAWED

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Is Walk Forward Optimization always a must? Albnd Traders Hideout 15 October 28th, 2016 10:00 AM
Synthetic Data Walk Forward Optimization quantarb Elite EasyLanguage Automated Trading 5 December 23rd, 2012 01:52 PM
Walk Forward Experiment Silver Dragon Elite Automated Trading 327 September 9th, 2012 12:56 AM
Walk Forward Optimization - memory consumption Mondrakete NinjaTrader Programming 2 September 7th, 2011 06:04 PM
Walk Forward caprica NinjaTrader 4 August 31st, 2009 06:51 PM


All times are GMT -4. The time now is 01:51 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-17 in 0.16 seconds with 19 queries on phoenix via your IP 54.226.113.250