I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.

Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.

Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.

If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).

I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.

Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.

Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.

If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).

Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.

In terms of changing your ATR, stops and targets, I suggest you have a look at a thread here on futures.io (formerly BMT) (can't remember where it is, but try search for it) where a few of the fellas here tested whether money management gave you an edge (it didn't). It didn't matter what your R:R was, the drop in win rate will just balance out so you end up with no edge in the end. There really isn't a point in optimising the relation between target and stop, just make sure its wide enough to stay out of the noise and that the target isn't unreasonable (i.e. expecting ES to move 40 points a day). The edge has to come from your entry or exit logic, so optimising ATR, stop and target is not going to help.

In terms of ATR, I have found in the past ATR is good in theory, but I prefer to just use a fixed target/stop (are you using an ATR mutliple for target and stops? Not sure if that's what you mean by dynamic). The problem is that markets transition from high vol to low vol and back to high, a lot of the times you get caught with stops and targets that are too tight when it transitions to high, and too loose when it transitions to low. When you use something like a 14 day ATR, the transition can be a bit too slow and catch you out quite a lot.

I did a lot of testing with single stock algos, the problem with applying an algo to stocks is that stocks behave in all sorts of ways. You have dividend payers which tend to range, you have growing companies which trends, you have speculative growth like Healthcare doing R&D that suddenly drop or pop with news, you have companies that can decline slowly instead of quickly. A single algo cannot cater for all of those conditions, so you're better off sticking with Futures.

Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.
.

Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.