Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I bought a "winning program" last year to try it. Turned out to be a waste of time and money. Adjusting these programs is key, unfortunately that is easier said than done.
Indeed adjustments are key; I make them almost weekly. I'd be willing to share my EA and oscillator with you if you'd like to try it out and offer any advice / insight. I haven't really kept track of it's performance because I haven't found an accurate way to test it, other than a live account "test". It does make money, but as I said, I make changes almost weekly. I have traded it manually for a couple years and have doubled my money but it takes an enormous of time, which is why I switched to automation. At any rate, let me know if you'd be interested in trying it out; I'll gladly send you the files.
I'm not convinced frequent changes are that necessary. I think it just depends on what you're doing. I have several systems that I designed, optimized, forward tested, and one is trading live (real money). Haven't changed them at all since they passed forward testing. I optimized them with over 2 years of data which translates to over 1600 trades, then forward tested them for 6 months which added almost 400 trades. I attempted walk forward optimization on each of these systems and that didn't work well at all. So, I think my point is to not get locked in to a specific way of thinking nor should you think conventional wisdom is best.
No worries....I understand life presents itself. I agree about realizing that you "have" to change or "should" change this or that. Because I learn more about automation every day I see opportunities for improvement in my code and thus the "weekly changes". I am certain the changes will slowly diminish as I am convinced that I have articulated via the code how I intend for the program to trade. I believe that is why we are all here @ futures.io (formerly BMT) to share and discuss and overcome the experiential and suggestive influences that have shaped our knowledge and thus decisions. If you ever decide to experiment with the EA or oscillator I'd appreciate any epiphanies you may have from running it.
I've also found walk forward optimization to be a not too effective method for optimizing and testing the potential performance of my automated strategies, despite it being conventional wisdom around these parts that it is best practice.
I'm curious about your experiences with walk forward optimization and why it didn't work for you. What I found is the following:
- caused overfitting: the parameters that resulted from the walk-forward test were fit to recent market conditions and had the potential to fail (sometimes catestrophically) with any slight change in market behaviour when running live.
- took forever: needing to run a complete optimization on each optimization window on my fairly computationally intensive strategy just took too long.
- optimizing the optimizer: never could get a balance I was happy with between the size of each in-sample period and the size of the out-of-sample period. And because each full run took so long, it was hard to quickly experiment with different combinations to see what (if any) were more effective.
So, instead of walk-forward optimizations, I prefer to run a single optimization over almost my entire data set (usually 6 years and > 1000 trades worth) and use the parameters that work best over that entire period. My logic is that the parameters that work well over 6 years worth of data, with all the varying market conditions therein, should have a greater chance of success live than parameters derived from a smaller section of that as would be the case using WFO.
Your experience sounds like mine. I'm not a statistician nor do I have advanced degrees in anything nor do I have good comprehension skills when it comes to reading and studying. Also have never been one to just automatically do what others say is conventional wisdom. I do have a basic understanding of many things as opposed to in depth knowledge of anything. In other words, I view myself as a jack of all trades but a master of none so to speak. That can be a blessing as well as a curse.
When I first started strategy development I didn't know anything about walk forward optimization. All I knew was I had almost 2 years of data and wondered how my idea for a system would work when run against all that data. I was aware of curve fitting and felt that the more data I optimized against and was successful the more likely my system would be going forward. Something else that I was concerned about is how the exclusion of a trade from a set could effect the overall system performance. What I mean by that is excluding a trade could open up the potential for other trades to be taken that otherwise would not have been. I knew from common sense that trying to program a system to hit a moving target was much more complex and I'm not up to that. In my opinion, that's the sort of thing that most people fail to account for and why curve fitting is as big of an issue as it is. So, I figured I must define a context for the market. That's what I did. I defined a trend and my definition of a trend does not change. That trend definition is the system's anchor. The maximum number of trades my systems can have are equal to the number of trends there are. Some trends are only a few bars long and others are much longer. There are no variables in this trend definition. Further, the only variables in the number of trades exist the reduce the number of trades. Never will there be more than the number of trends. There are trade management issues with this approach but that's another story. The bottom line is my systems are bounded, good or bad.
Walk forward as I see it can be useful if you have enough in sample data to be significant. Unfortunately, when you have 3 years of data which in my case translates to about 2000 trades then you break that down into walk forward increments, there just aren't enough trades to do anything meaningful. So, what that means to me is that you curve fit for a much smaller sample size. I tried the walk forward optimization method on several of my systems and found it kind of worked but negatively effected performance to the point that I would not want to trade it.
I think I'll quit there. I think I've said enough and probably way more than you asked for and probably just enough for the smarter people to poke holes in. I invite that because I may just learn something I didn't know and didn't have to read a book to get it.
Oh yes...I should mention that I do what I call a roll forward optimization as well. Remember that I initially optimized against about 2 years of data. Now I have more data. So, ever once in a while I take the same system an optimize against ALL the data I have to see how it has changed. Currently that's about 3 years of data. Funny thing is, I still haven't changed the settings from the original optimization and the system is still working.
@MWinfrey, don't underestimate what you've achieved or the validity of your hard won knowledge. Despite not having formal training related to trading or algo development, you're clearly a smart guy and I think coming at things from a unique perspective and not accepting standard practice just because it is standard practice is a huge advantage, so long as your own opinions are formed on the basis of solid testing and experience (which I think they are).
The way you describe your idea of what constitutes a 'trend' and the idea of this being a static, unchanging definition resonates strongly with the direction I've been moving myself of late. I'm a software engineer by trade, so am fortunate that the coding side of the equation isn't a headache for me, but I've only been trading (forex) for around 12 months, so my experience of how the markets work is still very a work in progress. From what you said, it seems my journey has mirrored yours to an extent.
Anyway, cheers for your response and best of luck with the continued success of your methods and strategies.
