I've spent -- literally -- the past week or so doing nothing but back-testing and optimizing, and have really come away from this with a radically different grasp of our financial markets. A few basic -- yet important -- things have changed in my own head, and I thought I'd just toss them out here, in case anybody cares. It's a pretty random mish-mash of thoughts.
1. The value of trailing stops is profound. I'm not kidding -- a barely profitable "always long or short" strategy can become, simply by adding a trailing stop, incredibly profitable.
2. When back-testing (or optimizing), I think there is some real value in the idea of throwing away the best trade (or, sometimes, the worst trade) as an outlier. Back-tests and optimizations can be badly skewed by one single trade -- I've thought of asking the people over at Ninja if they would consider adding an option to their software allowing a back-test (or an optimization) to throw away the best or worst (or both!) trades in a back-test (or an optimization) as outliers -- for now, doing it by hand (or in a spreadsheet) has become a mandatory part of my number-crunching.
3. All of your indicators are lagging. All of them.
4. The essence of what we are trying to do is to pick entry points and exit points. That's it. That's the whole game. It's easy to lose sight of that. All of what we are doing is picking entry points and exit points. Nothing more.
5. The "optimal" past entry and past exit points can generally be seen clearly on a chart -- most of us can use a "zig-zag" plot to show where these are. You'll find that "optimal" trades are generally at longer time periods -- and what this means is that the value of longer-term moving averages is profound -- it really isn't the tick-by-tick or second-by-second activity that matters -- longer-term trades are really where the money is at. Longer-term moving averages are an incredibly valuable tool -- try and create a Jurik that approximates a zig-zag and you'll learn a lot about what constitutes a true grasp of market dynamics.
6. There are absolutely times when the best trade is none. Just as in poker, where the single best thing you can do at the moment is to not play the game, the market has times where your single best strategy is to sit on your hands.
7. Stop optimizing for things like maximum net profit, and start optimizing for things like SQN and expectancy. You'll be both shocked at how horrible your old favorite ideas were, and at how much merit there actually is in ideas that you had previously dismissed as having no value.
8. The market is always right. It doesn't matter how much effort you've put into your ideas and into coding them -- you're theories and efforts mean nothing if they don't apply to the real world. I can probably talk all day about how I think the market "should" behave -- yet that entire line of thinking is wrong: the market doesn't have to behave in any particular way, and approaching the market with any sort of pre-conceived idea is an error.
9. Value investing is a form of arbitrage -- Warren Buffett is a guy who simply arbitrages time.
10 Your first job is to beat the tax-equivalent yield of the risk-free rate. If the 10-year Treasury is paying 5%, then your first job is to beat that -- after taxes. Observe that any of us can simply buy Treasuries and go lay on the beach -- with no risk. If you're going to expose your capital to risk, you need to be obtaining a reward that is acceptable to you, given your own personal context -- but that reward must be greater than the tax-equivalent yield of the risk-free rate. To accept risk for a smaller return than the TEY of the RFR is absurd.
11. Trading is not a "hobby." It's a job. It's a grind. It is, to paraphrase that old saw from poker, the "hardest easy living you can make." Trading is painful; it is downright heart-wrenching. That's what you get paid to do, by the way: you're getting paid to deal with situations that most people find too painful. 99% of the population can not sit and watch large amounts of money vanish into thin air and then come back for more. The pain involved in trading is significant, and it is real -- that's what you're getting paid to deal with.
The following 8 users say Thank You to AynRandFan for this post:
You are right on the money. I agree with your thoughts and analysis. Trading is basically "sophisticated gambling", but yet we keep going back for more like an addict. We all believe that we can beat the market, and some days or even most days we do, however there comes a time when the market literally kills us and if we do not have good discipline or great money management, then we end up leaving this market and looking for something else to do.
Having said this, it is possible to trade successfully and make money from the market, but it takes time, discilpine and patience to master.
These are all good observations but I think the most important issue is that you can find a combination of just about any data to make a profitable strategy. For example you can optimize for two moving average periods in a moving average crossover strategy. You'll find a value that would be profitable over the backtest period. But if you forward test it, it'll fail.
This issue means that we cannot trust any results of optimization. Just because a strategy is profitable when optimized does not mean it has any predictive value.
After learning this the hard way I stopped optimizing.
AynRandFan - Your opening post had me nodding in agreement on many of the points you make.
I am into Year 14 of trading for a living, but the greatest advance I made was about 3 years in, when I discarded so much of what I had been endeavouring to master. I had started by constructing my own 'workshop manual' in which I filed explanations and examples of the numerous indicators which one can employ. I trialled and backtested each one in turn, and combined them in so many ways. The manual was in a lever-arch ring binder that by then was 3 inches thick.
"A lot of people try to out-think the market. They try and intellectualise and try to get too smart. They try to figure out all these scenarios. You shouldn't think too much during the trading day.. Keep it real simple"
"If the market is not doing much, don't try to force anything out of it."
"You can sit there and play all kinds of little tricks with oscillators if you want. But any oscillator is going to highlight what's there on the bar charts anyway......You can train yourself to see everything in a bar chart that an oscillator will tell you."
Admittedly she is a specific kind of trader whose territory differs from mine - so I didn't attempt to mimic her style. But i did begin to subtract indicators from my armoury instead of adding more of them - until eventually I took a perverse delight in dismissing almost all of them as non-essential. In a way, it was useful that i studied them all - so at least I know what people are on about when they refer to them or excitedly recommend using them. I still look at charts - and it may be that I am unconsciously taking onboard some aspects of what I studied back when. But I'm not intentionally employing indicators to any serious degree. Nor do I study or understand company accounts. I rely on glancing at a chart, checking for impactful news, and shamelessly eavesdropping on discussions among respected fellow traders. And jumping back into safe mode (ie, cash) a lot of the time, pouncing on best opportunities only, rather than holding a portfolio that includes second best.
I don't go along with some folk's notion that stock prices and market indices might be totally random - but I do allow for there being a degree of random, capricious, mercurial behaviour which is always capable of catching out anyone who thinks they've got everything mastered. And that money/stake management is the best way of accommodating (or exploiting) unpredicted anomolies. In other words, accept that the market is a wild beast and work with its behavioural oddities rather than attempting to iron them out or overpower them. I grin when it catches me out, rather than curse or scream.
The automated trailing stoploss is a lovely invention. I use it a lot. Like many traders, I am still toying with the cleverest positioning of it on a variety of stocks that require individually tailored settings.