I don't remember the book I read it in, but it said something like Your biggest drawdowns occur after great periods of success. Or something like that. That after a great period of success you will have a period of drawdown, and how you handle that drawdown will determine if the overall period is profitable.
Last edited by tturner86; February 23rd, 2014 at 07:07 PM.
The following 2 users say Thank You to tturner86 for this post:
I think it's worth the time to get a good spreadsheet going. It's not that hard when you use a few helper columns in excel. Ton's of ways to do it, but some suggestions:
- Put your backtest data on one sheet - with points won/lost.
- On the summary sheet have all your settings - Start equity, points risked per trade, Equity Ruin Level, annual trades, commission, slippage per contract etc
- On the calculation sheet have a number of different Columns. I use - "Trade #", "Random outcome", "Contracts", "P&L", "Cum. P&L", "Max Risk".
The "random outcome" uses the RAND() with simple VLOOKUP to grab one of the backtest data points.
"Contracts" just has a few IF AND nested formulas to calc the contracts based on Cum P&L, checking that it's not above Max Risk, or below the Ruin threshold
"P&L" calculates the outcome and adds on slippage/commissions based on the number of contracts.
You probably already have the VBA to modify so you can loop through x simulations.
Modify it to copy from the Calc page to a Results page taking with it some summary numbers from each simulation. I like to capture number of full wins / losses, max DD etc.
Once all the data is on the Result page you can just manually sort the final data into ranks, and manually graph the 10% lines - it can get pretty tricky with VBA unless you have some experience.
I chart the bottom 10% and 1%simulation - and pick the worst performing of each case - based on its performance 25% of the way through the # of annual trades (there could be a number of trades in the 10 percentile - all with varying paths to the final P&L). I also chart the 50th percentile using 2 lines - with the best and worst performing simulations at the 25% mark. They can travel some funky paths to the same final P&L. Quite often the 1%,10% and bottom 50% travel similar paths for a lot of the period, before diverging right at the end. Performance near these bands is a cause for concern. It actually demonstrates your position quite well. You won't know if you are on that outlier 50% - with some big wins right around the corner, or whether you are on that 1% line, about to drop of just a little bit more. And this is where the analysis of the market data comes in. To try and ascertain if the underlying reason for the original trade is still valid in the current data. This is something I personally need to come to grips with to aid in my trade decisions.
The following 6 users say Thank You to sixtyseven for this post:
Broker/Data: Mirus (Broker), Continuum (Data), Dorman (Clearing)
Favorite Futures: Futures
Posts: 202 since Mar 2013
Thanks: 426 given,
I've been reading along, as well, and I just want to add my Thank You to the growing list of other futures.io (formerly BMT) members that appreciate what you are doing with this journal.
For what it's worth, unless you had an incubation system that was showing vastly better results, I believe you should continue trading the plan that you established at the outset. However, if you do end up substituting a system, perhaps you would be willing to continue this thread with the new substituted system. Heck, I bet if you put the substituted back test trade data up that you could get some of your readers to even take a crack at helping you from Monte Carlo forward--kind of a far flung idea, but perhaps both parties would realize some value from the additional exercise(s).
In any event, thanks.
The following 2 users say Thank You to aventeren for this post:
I think Tharp's book are great (I have them all, I believe), but besides his position sizing ideas he does not provide much value for algorithmic traders. Kevin, which books are on your 'must read' reading list? (Besides, obviously, your own forthcoming book ).
I don't mean this harsh, but I realize that it may sound that way if I type it here, but:
Shouldn't you call it quits with this strategy?
It is true that your maximum drawdown has not been reached yet. However, your strategy severely lags the performance it "should" be able to achieve. That got me thinking about the following. If you go back multiple steps, and forget all things about money and trading, you'll arrive at your alpha model and its alternative hypothesis.
Given the recent lack of performance, isn't the market telling you that, for the time being, your alternative hypothesis is wrong?
Depending on the value of the statistical test you choose to use for that, shouldn't that also mean that you might be better of not trading the strategy for a while, until its performance indicates that its results are better in 2014 than might be expected by chance?
(I also think that one should stop trading if the theoretical assumptions underlying the alpha model are invalidated. Are your theoretical assumptions public, btw? It might be interesting to explore these. Perhaps something changed in 2014 in the market that might explain the lack of performance.)
I know this is a little bit too strict , but let's look at something different than trading and assume that your model does not predict prices but predicts which teaching method is best. Then, looking at the data, we can see that children's grades have been around the low 10% of their predicted grades for the last 72 days. Would that not give strong evidence that, for the time being, the teaching method does not work? Even though the teaching budget (i.e. the max drawdown) has not been evaporated yet?
(I'm really curious what you, and other people in this thread, think about that. Thanks in advance for any comment. ).
The following 3 users say Thank You to Jura for this post:
Thanks for the post. You are not being harsh at all. I welcome civil criticism and discussion/debate of what I am doing here. It is just when people get personal, accusatory and use inappropriate language and bad manners that I get ticked off. Some people post on the Internet stuff they'd never say face to face, and that is bad. But no worries, because that is what the ignore button is for, right?
Anyhow, some algo system authors I like:
Tomasini and Jaekel
Keith Fitschen (I haven't read his book, but I know he is a sharp guy)
As far as my system goes, before I started I had established my "quitting" criteria in advance. Why? Because I found that I make bad decisions in the heat of the moment. Most people do. I turn things off based on emotional responses. That is not good for me.
Ironically, I am friends with a CTA (Commodity Trading Advisor) who creates algo systems, but when he senses the market has changed, he turns the strategy on or off. It works for him, judged by his performance, so I can't say it is a bad thing for him to do.
He'd probably look at my performance, and say that the system is just not acting well. My guess is that because the volatility has gone down, there are less trading opportunities, and the ones that do appear are of possibly lower quality. One indication of this is that I have not had a "big winner" since going live. If you look at statistics (buried somewhere in this thread), you'll see I should have had at least one big winner by now.
What to do? Well, my original criteria was drawdown based. If I had made a quitting point based on lower 10% performance (from the running chart I show), I probably would have quit. But I didn't, so I'm not going to add that criteria now. I try not to second guess myself, as that can have long term bad consequences. Maybe, though, when I rollout a similar system in the future, I will include that criteria.
One other "quit" point I mention in the thread is the "next best alternative." Every few months or so, I look at the portfolio of systems I trade, possibly adjust the capital I allocate, and possibly replace underperformers with something new. When that happens, I might decide to temporarily retire this system, and stick in another comparable system. Again, though, I wait to do it on a regular schedule. That keeps me from making a rash decision.
So, I hope I have explained my rationale. I understand your point of view, and clearly the strategy fits what you are saying (it is not working particularly well right now - nowhere near expectations). If I had more of a quick trigger, I might indeed stop trading it. But, I have a plan, and so I must stick to it.
If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
The following 5 users say Thank You to kevinkdog for this post: