arnie 

Hi @Jura Since HFT and algorithmic trading is the hottest thing around these days, I thought "algo" guys would love the idea of having milliseconds available to them, giving the extra granularity to program their strategies, hence - the most important version release in the last years. Sorry for that, apparently my assumption was wrong A good friend of mine always told me that my assumptions will be the death of me Please don't misinterpret me, bugs should have always precedence but there are bugs, and I have no idea if this is such a case, but I assume (again, I'm assuming) it's not since several fixes have been made available and none solved the issue, that will be easily fixed with some new feature that will be released in the next version. So if some bug will be easily fix when that new feature becomes available, why lose time trying to fix something now if in a couple of months it might stop working again since with that new feature, the way that tool works will change again? I share treydog pain, heck, we all share his pain, we all had our share of working with something that was only half working and had to wait for the next version to see the problem solved but the only thing I could do was nagging support about it, so they could move faster in solving the problem. I hope I'm not putting words in his mouth but I believe treydog said he was losing his time testing the fixes MC sends to him. He discovered the bug. The tool is not working accordingly to his settings and knowledge. Apparently, developers thought the tool was working fine, accordingly to their knowledge. Assuming treydog is more knowledged on the matter, since he's a trader, he's the most qualified to be testing this, telling developers that the tool continues to be faulty, which he is. Helping MC developers with this treydog is helping himself on the long haul because he will have a tool set accordingly to his settings. Personally I consider this to be a very useful and friendly relationship between developer and final user. Both helping each other in building a better platform.

The ultra high frequency guys aren't using Multicharts. They usually write their own program or use institutional products.

Alright after working with the guys at MC, I have gotten some more files to work with. I tried to recreate the example they gave but since they used something that moved instead of a set date it was impossible. So i decided to try my own small test, it seems like it works now. I need to go more indepth but basically here is the run down. You can only do 1 window at a time (this is all i have tested), the trades must be closed at the end of the window, slippage and commision was simply $1 per contract (this was a major issue before), You must have enough lead time in your window to accommodate your max bars back number. If you have these things then your strategy performance report should match any 1 given window of the WFO.

Caveats: I have not tried multiple windows at once, I have not tried complicated multi data stream strategies, I have no tried tiered commissioned systems. Basically i used a preloaded strat and just filled in some blanks. But its a promising start. After I do some more testing I will be able to come to a more concrete answer. But until then at least i am able to replicate to under certain conditions what the WFO is generating.


edit: you still require the hotfix that was sent to me personally from MC. I don't know if they have it publicly available but without it you cant do anything. They should probably post something thats hard coded update released publicly but you can ask.

treydog999 

Alright after working with the guys at MC, I have gotten some more files to work with. I tried to recreate the example they gave but since they used something that moved instead of a set date it was impossible. So i decided to try my own small test, it seems like it works now. I need to go more indepth but basically here is the run down. You can only do 1 window at a time (this is all i have tested), the trades must be closed at the end of the window, slippage and commision was simply $1 per contract (this was a major issue before), You must have enough lead time in your window to accommodate your max bars back number. If you have these things then your strategy performance report should match any 1 given window of the WFO. Caveats: I have not tried multiple windows at once, I have not tried complicated multi data stream strategies, I have no tried tiered commissioned systems. Basically i used a preloaded strat and just filled in some blanks. But its a promising start. After I do some more testing I will be able to come to a more concrete answer. But until then at least i am able to replicate to under certain conditions what the WFO is generating. edit: you still require the hotfix that was sent to me personally from MC. I don't know if they have it publicly available but without it you cant do anything. They should probably post something thats hard coded update released publicly but you can ask.

It sounds good, that MC is on the way to help to resolve this issue.
Will this fix be a part of the next MC release ?

wbailey 

It sounds good, that MC is on the way to help to resolve this issue. Will this fix be a part of the next MC release ?

no idea

Alright, honestly I am getting fed up with this work.

So basically they sent me a cherry picked example where I could replicate. How did i determine this, I started testing other strategies on other time frames. This is using the production release 8.5. I was able to get very close but again, if there are trades that are crossing the period switching barrier then you have yourself a problem. Multicharts does not explain how they deal with it. are they closed at the end of the period, are they held over? I have tried both methods and haven't come up with a way to replicate exactly.

I put 8.7 beta on another machine to try it out, figuring they possibly fixed something. Its actually worse performing than my hotfixed 8.5. just adding commision without slippage creates problems with replication.

Bottom line: Still broken multicharts

GET YOUR ACT TOGETHER!!!!!!

treydog999 

Alright, honestly I am getting fed up with this work. I put 8.7 beta on another machine to try it out, figuring they possibly fixed something. Its actually worse performing than my hotfixed 8.5. just adding commision without slippage creates problems with replication. Bottom line: Still broken multicharts

Is this issue fixed in Multicharts yet?

I appreciate if someone can please provide an update on this.

Corinne 

Is this issue fixed in Multicharts yet? I appreciate if someone can please provide an update on this.

Alright, so far its been a mixed bag. I have been able to verify individual walk forward windows under some very specific conditions. They include, there are no trades that pass from 1 window to the next, you add maxbarsback (the exact amount) which can be extremely difficult to get down, and using the same start date and end date which i set inside the code itself.

However, if you try to string together your WFOs like Kevin Davey does, you will not be able to sum up your WFO windows to the same answer. Also if any 1 of your windows has a trade that passes through to the next window, you won't be able to validate the entire series (extremely likely). So does it work, kinda. But its highly limited and not reproducible in a window by window approach. I have worked with Multicharts for several weeks and they have basically told me, this is how it is supposed to function. So I dont expect any upgrades or modifications to make it more robust, results reproducible or verifiable in anyway.

If your worried about the results, you can always walk forward the old fashioned way. window by window using the date/max bars back function and be sure you get solid results.

treydog999 

(...) If your worried about the results, you can always walk forward the old fashioned way. window by window using the date/max bars back function and be sure you get solid results.

You mean with closing open positions at the end of every WF window, or do I misunderstand you?

(Thanks for the update btw)

Jura 

You mean with closing open positions at the end of every WF window, or do I misunderstand you? (Thanks for the update btw)

I mean setting your date/time settings and then running the strategy there and saving the performance report. But like you mentioned close any currently open trades during that , then manually set the date/time to new settings in order to see the next window. Run a Normal optimization on each of these windows. Basically you are doing the job of the auto WFO by hand. Its the only way to guarantee a clean wf.

Hi,

I am reading this thread as I was about to embark on some walk forward analysis, and now I am pretty concerned if its worth the effort on the MC platform.

Is Ninja any better?

I did a google search and found a product by codefortraders and another one called Diamond Backtsesting that is some sort of add in that does walk forward, would this be worth getting?

PS I have no affiliation of w/ CodeForTraders nor Diamond -- I am not even a customer of any of these companies, I surely am interested in a good backtesting solution, and if MC is flawed I would like to know which platform is good at backtesting and Walk Forward Analysis.

I do have a copy of AMIBroker but am having a lot of concern regarding using Tick Data with that platform. So to be frank I am pretty unsure what to do at this point in testing my strategies.

Any suggestions are appreciated.