First time post. Over the years I have had a limited amount of exposure to experienced traders, which I am hoping to change via this forum.
I am trying to define what is a good strategy. Beyond the obvious that it should make money, what else do you look at?
I am currently using Ninja Trader to back test results and use for RT trading. I am currently trading a couple of stratagies with real money.
Here is what I know, or at least what I think I know:
Profit factor of 2 or above is best
Sharpe Ratio of 2 or better is best
Low drawdown is a must.
It must make money
it must be consistent over time.
The part I struggle with is determining what is a good bottom line for a strategy is. I often wonder if I need to tweak my stratagy more to squeeze more out of it or if what I have is good enough. (see attached) For example, if a strategy produces 2K in profits over 6 months trading a single 10K currency lot. Is that considered good? Does it depend on your point of view? or does it matter. Should I be more concerned about making good trades and let the money follow? Am I over analyzing?
Bottom line is, don't curve fit, and if you find something that works out-of-sample, then trade it. If the profit's not enough to make you a gazillionaire, just look for more systems but don't stop trading that one.
You can discover what your enemy fears most by observing the means he uses to frighten you.
The following user says Thank You to Adamus for this post:
Unless you have multiple strategies and logic for regime switching I don't see the point in going back 10 years. Throwing out a system because it doesn't work during the financial crisis is asking for an impossibly robust single system.
I actually think 6 months is about right as far as when things seemed to have changed. It is not a terrible idea to at least have some triggers based on the VIX as a rough estimate of volatility. Trading a system that tested good with the VIX at 10 makes no sense with it at 45.
You can make a philosophical argument that if you knew a trade would even be +EV by 1 cent after transaction cost you should always take it. Obviously, you can't know that with such precision so with those numbers you really need to come up with your own estimates of model error. At some point you do have to gamble, this is not an exact science.
I know I have read Simons say that at the end of all the modeling at Rentech the last question before going live is "does this make sense?".
I would forward test 30 trades on that system, if the numbers look good and you can answer "yes" to the "does this make sense" question then pull the trigger.
Sharpe ratio is highly overrated as a metric on your own capital.
FWIW I've spent the entire day working with the PerformanceAnalytics package in R, my ExportData strat for NT7 with some logic added from my LogReturns indicator, and Excel. It has the makings of something that should add a good alternative view of Strategy performance. I'll post the code and some examples later in the week.
I'm doing that because I don't trust Stratgey Analyzer and because it lacks some core functionality I need on a portfolio level.
As far as how far back to test, perhaps it is good think of it in terms of data points rather than days/month/years depending on your trading frequency. Trading daily bars? Yeah go back 10 years (3650 data points). Trading 5 min bars during cash hours? 44 trading days will give you the same amount of data points.
If you have the data, test further back. No reason not to. For the Euro you can only go back to 1999 or so anyway, right? If you can test during two bull/bear cycles that's probably a good thing.
1) You should really incorporate your slippage and comission into your backtesting. I know it's easy to calculate based off the top of your head, but it'll help you evaluate results more clearly when you see exactly how much less it's making (in comparison to the risk you're undertaking).
2) RINA Index. I like RINA because it takes into account the number of trades (through exposure time) and rewards strategies that profit while taking the least amount of time in the market to achieve it. If anything else, it's just another index that you can use to compare strategies.
3) The amount of time you go back is really up to you and a matter of flavor. There are legit arguments on both sides of the issue. Going back too far, can be just as deterimental as not enough. What if your strategy goes back 2 years and it was highly profitable in the first year and marginal in the second? Now you've just clouded the performance with data that's less relevant. Conversely, not going back far enough can give you a false sense of confidence via fortunate market trends that may have developed recently (and may or may not continue). The best answer is to do all sorts of backtesting at various time lengths rearward, that will give you a more clear picture of what's ocurring and allow you to at least be more aware of the risk you're taking. It also helps you to evaluate/compare strategies that are nice and smooth over all time periods as opposed to strategies that are more "trendy."
4) Drawdown. I personally take the worst drawdown and multiply it by 50% or 100%, depending on how volitile the line chart in variance from the linear regression. Chances are considerable that the worst drawdown is somewhere in the relevant future. Only then can you get a true picture of the return rate based upon your account size, not your trade size.
I have actually done it both ways, tested 2 years and 6 months. What I found is that my entry is solid, in other words it consistently puts me in positive territory, however, what needed to be changed was my profit targets and stops. When testing over a 2 year period my strategy failed miserably. When I broke the data up into 3 to 6 month chunks I was able fine tune it for the period. Generally speaking those settings work for about 2 to 3 months going forward then I have to adjust. It is easy to see when it needs adjustment because I start missing targets by 10 to 20 ticks.
In case anyone is interested my data shows the sweetspot for profit target for a trending EUR/USD is 80 to 90 pips on a 5 minute chart.
What I have learned: if you are testing a strategy you should always look at the trade data. Try to understand why it failed. Use the MAE, MFE and ETD columns on the trade tab to help understand what is happening during your trade. The definitions for what these mean can be found in the NT help section. Its possible, as in my case, you have profitable strategy, but profit targets are to high or stops are too close. A simple adjustment of 5 to 10 ticks can make all the difference in the world.
The following user says Thank You to Silver Dragon for this post: