how long to forward test to know Drawdown and avg monthly PnL
I didnt know about my strategy until I "forward tested" for 9 months, and it was very tedious
of course (after lots of backtesting), so I knew which months it was bad and gave me a drawdown on 4 months out of 9months and avg monthly of ES was quite low, like $724/month with drawdown was $2577
not sure if someone else can do in 3months, how can you catch best and worst months for "your method"
how long do you forward test to know Drawdown and avg monthly PnL
I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.
Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.
Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.
If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).
very good feedback
yes, my system based on Dynamic SR, ATR beside stops/targets, time to enter/exit does go thru all 3 modes uptrend, downtrend or ranging, but it works better for uptrend, downtrend and I have a 2nd system that works better for ranging.
I have done for 3 years to learn and 9months forward test so far and proved the same
But i dont know much about curve fit. does it mean, it is fooling me in backtest by adjusting parameters ?
I just 3 parameters
ATR (variable), Target (fixed), stop(fixed)
so my only parameter that's optimized is ATR, beside Chart's data series. 15/30/60min charts
i thought i kept it simple.
so Do you think would you curve fit bet ATR and 15/30/60min charts to fool me in backtest
Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.
In terms of changing your ATR, stops and targets, I suggest you have a look at a thread here on futures.io (formerly BMT) (can't remember where it is, but try search for it) where a few of the fellas here tested whether money management gave you an edge (it didn't). It didn't matter what your R:R was, the drop in win rate will just balance out so you end up with no edge in the end. There really isn't a point in optimising the relation between target and stop, just make sure its wide enough to stay out of the noise and that the target isn't unreasonable (i.e. expecting ES to move 40 points a day). The edge has to come from your entry or exit logic, so optimising ATR, stop and target is not going to help.
In terms of ATR, I have found in the past ATR is good in theory, but I prefer to just use a fixed target/stop (are you using an ATR mutliple for target and stops? Not sure if that's what you mean by dynamic). The problem is that markets transition from high vol to low vol and back to high, a lot of the times you get caught with stops and targets that are too tight when it transitions to high, and too loose when it transitions to low. When you use something like a 14 day ATR, the transition can be a bit too slow and catch you out quite a lot.
I did a lot of testing with single stock algos, the problem with applying an algo to stocks is that stocks behave in all sorts of ways. You have dividend payers which tend to range, you have growing companies which trends, you have speculative growth like Healthcare doing R&D that suddenly drop or pop with news, you have companies that can decline slowly instead of quickly. A single algo cannot cater for all of those conditions, so you're better off sticking with Futures.
sorry, couldn't get back in time here
no, when I say I am using ATR multiplier, that's parameter in superTrend (that's what I am testing now).
when does the trend change is identified by "ATR multiplier"
Target/stop - fixed numbers and not depend on ATR
and I said dynamic as the indicator is similar to a dynamic S/R (nothing but where the trend changes)
ok, I read that and I just have 1 parameter called ATR multiplier that decides when trend changes, so I hope thats very less curve fitting. beside that i do optimise on diff charts, like
15min, 30min, 60min
and target/stop are fixed numbers