When backtesting, how far back in time do you usually prefer to observe a profitable system?
Let's say for example, a strategy was significantly profitable in the last 2 years, but in the years prior to that..not so much so. Would you continue to move forward with it?
- Jay
Can you help answer these questions from other members on futures io?
For me that would depend on if it's a short term or a long term strategy. For how long does this strategy dictate that you keep your trades / positions open?
Hey Jay; Depends upon your strategy's timeframe. If you're swing trading in the daily chart, and volatility of the instrument has been roughly the same during those two years, I'd say your time frame is two years (or 480 days) of daily candles. Round up to 500. If your trading in the 1 hour chart, then the past 500 hours, the one minute chart, then the past 500 minutes.
That's the basic answer to your question but leaves a lot out when in comes to back testing.
E.g., to differentiate between upswings due to positive economic performance and upswings due to ...oh... prolonged stock buybacks, I back test enough periods to catch a sustained down market, up market, and (if I'm lucky) sideways market. I perform a walk forward and then look at the walk forward intervals to see how the strategy performed in the up, down, and sideways market. If the overall affect of the strategy is positive, I look first for consistency between markets -- i.e. are all the down markets more profitable than the up markets, etc. Then I look at what inputs change that result. Then I decide what the current market is doing and apply the strategy to the past 500 periods and play with the inputs.
And I consider that the tip of the iceberg when it comes to back testing. Back testing is notorious for being inaccurate for a host of reasons. The most obvious is the oft seen statement "Past results does not guarantee future results." or something similar. Another is the temptation to follow the results of a curve fitting back test. I consider back testing to be best used as an indicator that adds a few percentage points of probability to your strategy. E.g. when back testing shows me a 70% win rate and a profit factor of 3.5 or something like that, I feel more comfortable with the possibility of at 51% or better chance of being profitable over a series of trades. I don NOT expect a 70% win rate or a p 3.5 profit factor.
Like said before me it depends on what time frame you are using for your system?
Hourly chart? daily chart?
perhaps volume or range bar charts that are very short term?
Also depends on the market? specific contract?
Does the system works well on more than one market?
PM with any questions about Cannon Trading (800) 454-9572 (310) 859-9572. Trading commodity futures, forex and options involves substantial risk of loss. The recommendations contained in this post are of opinion only and do not guarantee any profits. These are risky markets and only risk capital should be used. Past performance is not necessarily indicative of future results.
Thanks for the replies all, I guess should've been more clear. I was talking about intraday strategies, say on a 5 minute chart. Is 2 years of backtest data enough? What if the strategy proved to be profitable in the past 2 years, but not in the 2 years prior to that. Would you move forward with it in terms of further development and possibly simulation testing?
> What if the strategy proved to be profitable in the past 2 years
How profitable was it?
> Would you move forward with it in terms of further development and possibly simulation testing?
Yes, I'd move forward -- why not go ahead with some live simulated testing? The sim testing will give you more data to evaluate the strategy.
The main cost of going forward with simulated testing is the opportunity cost of time spent getting that extra data, isn't it? The question in my mind would be: what are the odds that this strategy will pan out to be profitable, and is it worth spending the time required to find out?
5 minute bars, intraday strategy. MES and MNQ only. Proved to significantly profitable in 2022. on MES > 20% return on a 50k account, after commission and slippage. Unprofitable in 2019 and 2020 but only small red on the year.
I was trying to get feel of how others decide to move forward with a strategy. Some folks need 5, 10+ years of profitable results so I am wondering if just 2 is enough.
I would probably run it until i see it is running out of steam or behaves "out of character".
Would not hurt to run it back a bit more and evaluate results based on volatility / ATR?
PM with any questions about Cannon Trading (800) 454-9572 (310) 859-9572. Trading commodity futures, forex and options involves substantial risk of loss. The recommendations contained in this post are of opinion only and do not guarantee any profits. These are risky markets and only risk capital should be used. Past performance is not necessarily indicative of future results.
Sorry if my answer's unhelpful, but these aren't really quite the questions I ask myself.
For me, it isn't about "how long" as much as about "over what range of conditions". I want to test robustness, and for that - in principle - I’d much rather have four 6-month (or maybe even 3-month) data-periods when the market was behaving differently than a single 2-year period in which most of the data’s broadly similar.
The following user says Thank You to Tymbeline for this post:
