How I Backtest
|December 11th, 2011, 12:27 PM||#1 (permalink)|
User requested to be banned
Futures Experience: None
Favorite Futures: happy
Posts: 6,462 since May 2011
Thanks: 2,090 given, 4,985 received
How I Backtest
Backtesting can be a valuable source of information. It also leads a lot of traders headed down the wrong path. I have read some posts lately regarding automation and backtesting and decided to share some of the process of how I approach it. This will not be a "how to trade" example, or show an exact trade system, but just offer a view into another trader's head of how to use backtesting.
I am a purely discretionary trader. I do not use any sort of automation. I do not have buy/sell signals that I believe should be taken every time they trigger. The way I approach backtesting is it allows me to know more about market movement. I digest the information, take indicators that keep me aware of the information I have learned, and then combine that with other information and experience to decide when to take a trade.
Now that I have lost the interest of possibly the majority of readers...
I trade mostly crude oil, and I like to watch 6 and 9 range charts in addition to time based. I find that when I use range charts, certain patterns re-appear that moving averages can help identify. I like T3 moving averages on range charts because, in addition to having backtested well, they respond very well visually.
When I backtest, my initial run is based on Maximum Net Profit, and I am looking for 50-60 results to go through. I use very wide range settings, like 30 though 300 at intervals of 30, to just try to locate a "sweet spot" in each of the moving averages. I use "Genetic" testing on initial passes to keep the backtest time limited in the initial broad passes. After I have found some tighter criteria, I then limit the range of optimizing and switch to standard backtesting.
As the testing continues, I shift the backtest priority to issues other than maximum profitability. Some personal favorites are Maximum Profit Factor and Minimum Drawdown, but there are several things to consider depending on what the initial runs showed.
The image above shows a test of 7 different T3 moving averages. Some of them work together to provide a trade entry, two work together to show an optimized reversal point, two work to create an alternate trade exit (the exit can also be a setup in the reverse direction or a stop loss). The settings shown above are what I determined were the most beneficial to me after running through the entire backtest process. Again, this started with a very broad sweep.
Below is what this set of averages looks like on a 6-range chart, labeled to show how they work together. I have shown two possible trade triggers, one short and one long, noted by red/green arrows
The way I ultimately decided on those exact settings;
1) I took the information from my initial passes, distilled it down to much tighter ranges per moving average, and exported to MS Excel. I use excel to sort by Win Loss Ratio and Profit Factor and Max Drawdown until I find what I am looking for. The Excel image as I was finishing my work there is shown below.
2) I went back into Ninja Trader and ran the standard backtesting on the selections I had focused on in Excel, and looked at the following 3 priorities;
All looked good to me, based on what I was wanting to know. You may have already noticed that the Percentage Profitable is not what you might agree with for acceptable criteria. I look for profit that is at a minimum 2x loss. I pay attention to Win Percentage, but do not necessarily prioritize it. I want to know what a 2x win or better looks like. I'll explain further down.
3) I then drill down to see if there is anything else to be learned from the backtest. What I saw was, when viewing the Day of Week, it shows Wednesdays showed the worst performance. That seemed logical to me immediately, EIA Report is often a trend changer for crude oil (typically 10:30am EST on Wednesdays).
I also wanted to see if market open, news reports, after hours, 2:30pm EST close, 5:15pm EST close had any affect on performance.
Now, back to the 6-range chart. Should I have taken both the long and the short trades? Should I have chosen one over the other? Having put all of the effort into backtesting, optimizing, color-coding, should I have felt confident about taking those trades based on reward to risk?
The answer is, I took neither.
That is where the discretionary part takes over. I wound up taking a long trade but passing on the earlier two.
1) Crude had taken a massive nosedive the day before, the overall mood was down. There is a lot of potential profit built up in the market on the short side.
2) The chart shown is on a Friday. Futiures stay open 23 hours a day, EXCEPT at the Friday close, they don't re-open until 6pm Sunday night EST. Nearly all traders have some version of fear.
3) To the left and right of the green arrow on the 6-range, we have a minor inverted head and shoulders pattern, suggesting support had taken hold for the day, and shorts might get trapped.
I did eventually enter a long trade, towards the end of the day when I felt crude had told me all it was going to say. The entry location is shown in the chart below. Note that based on the moving averages, the entry still met all of the criteria. However, that entry location seemed like the less obvious point than the ones in the 6-range chart shown previously.
Getting back to the Percentage Profitable of only 33%. That percentage is only if I traded that setup in a vacuum, without any additional information. If you want to increase your real world odds, you have to take the blinders off. No matter what the computer says, be aware. There was far more to the story.
It's Friday, with shorts most likely stuck at the end of the live pit, selling volume had dried up, pit close approaching, support looking ominous to short traders who were still hoping for more, potential double bottom with a prior support area, the prior high on a minor scale (6 range) after the H&S just was broken to the upside...
What were my percentage odds then? I couldn't tell you. But I knew, that was a darn good trade.
I took that trade live on futures.io (formerly BMT) and posted my thoughts about it as it unfolded. Here is a link the approximately start of that trade as it occured.
If you follow that link you may think, "Now wait, where are all the moving averages? That chart looks nothing like the one that was backtested! This guy is full of it!" Well, I don't use the exact T3s from the backtest, but I use T3s that are close. Truthfully, I adjusted them visually based on typical movemtn rather than optimum conditions.
And yes, I did cherry pick that example, as that trade could have just as easliy failed. Trading comes with no guarantees.
But, anyway, that's how I backtest.
The example backtest that was run on moving averages was only a single piece to the puzzle. I use that in combination with analysis of several other timeframes, like what is shown below.
On a 60 minute timeframe, we had a potential major (relative to 60 minute charts) double bottom. That was surely catching some attention at the end of day on Friday as well. If you are trying to trade range charts, tick charts, volume charts, that is a very good thing to do. It gives you a microscopic view, which helps minimize risk and spot opportunity faster. But what really improves your odds is only taking the microscopic signal when the macro view aligns with it.
What I believe; use backtesting to learn your market, use what you learn to complement something else you have learned. Backtest hundreds of things until it all starts to come together in a single understanding.
Optimize yourself and your understanding of the market you choose to trade.
Good Trading and STAY SAFE!!
Last edited by GaryD; December 15th, 2011 at 12:01 AM. Reason: Typos & hoping to improve the intended message