how long to forward test to know Drawdown and avg monthly PnL - Psychology and Money Management | futures io social day trading
futures io futures trading


how long to forward test to know Drawdown and avg monthly PnL
Updated: Views / Replies:449 / 9
Created: by emini_Holy_Grail Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

how long to forward test to know Drawdown and avg monthly PnL

  #1 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received

how long to forward test to know Drawdown and avg monthly PnL

I didnt know about my strategy until I "forward tested" for 9 months, and it was very tedious
of course (after lots of backtesting), so I knew which months it was bad and gave me a drawdown on 4 months out of 9months and avg monthly of ES was quite low, like $724/month with drawdown was $2577

not sure if someone else can do in 3months, how can you catch best and worst months for "your method"

how long do you forward test to know Drawdown and avg monthly PnL

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Elite Member
Sydney, Australia
 
Futures Experience: Intermediate
Platform: Sierra Chart, IRESS
Broker/Data: IB, IQFeed
Favorite Futures: ES, SPI, ASX stocks, options
 
Posts: 399 since Jun 2015
Thanks: 169 given, 446 received


I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.

Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.

Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.

If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).

Reply With Quote
 
  #4 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received


PeakGrowth View Post
I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.

Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.

Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.

If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).

very good feedback
yes, my system based on Dynamic SR, ATR beside stops/targets, time to enter/exit does go thru all 3 modes uptrend, downtrend or ranging, but it works better for uptrend, downtrend and I have a 2nd system that works better for ranging.
I have done for 3 years to learn and 9months forward test so far and proved the same

But i dont know much about curve fit. does it mean, it is fooling me in backtest by adjusting parameters ?

I just 3 parameters
ATR (variable), Target (fixed), stop(fixed)
so my only parameter that's optimized is ATR, beside Chart's data series. 15/30/60min charts
i thought i kept it simple.

so Do you think would you curve fit bet ATR and 15/30/60min charts to fool me in backtest

Reply With Quote
 
  #5 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received


PeakGrowth View Post
I built a lot of automated systems in the past and I looked at system testing differently. You are placing everything in to a "monthly silo" when in fact, the market does not care when the start or the end of the month is.

Markets are usually in 1 of 3 phases, uptrend, downtrend or ranging. Since most algos are built to take advantage of only trends or ranges, your back test should run through all 3 of those scenarios then determine its performance in the three market states and whether your system can survive an unfavourable market condition.

Your forward test therefore, is no different. The forward test needs to survive all 3 scenarios. Depending on the time frame you're testing on, it could be 3 months or 3 years.

If you curve fit your system to work great across all 3 types of markets without isolating and testing each market condition, it will fail because your curve fit will make it perform poorly during unfavourable conditions and mediocre during conditions which are favourable to your algo (trend or range). A good system should perform strongly for its intended market condition and not at all during unfavourable conditions (in which case you switch it off, or use a different strategy).

1 another thing to backtest stocks, the last 6 years market has been BULLISH, it is some times pain to check in BEARISH, so I pick up some months that are BEARISH in every year

futures swing both way a lot even in one BULLISH market, so testing Futures is quite diff from stocks (there are 1000s and many dont even care of market direction)

Reply With Quote
 
  #6 (permalink)
Elite Member
Sydney, Australia
 
Futures Experience: Intermediate
Platform: Sierra Chart, IRESS
Broker/Data: IB, IQFeed
Favorite Futures: ES, SPI, ASX stocks, options
 
Posts: 399 since Jun 2015
Thanks: 169 given, 446 received

Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.

In terms of changing your ATR, stops and targets, I suggest you have a look at a thread here on futures.io (formerly BMT) (can't remember where it is, but try search for it) where a few of the fellas here tested whether money management gave you an edge (it didn't). It didn't matter what your R:R was, the drop in win rate will just balance out so you end up with no edge in the end. There really isn't a point in optimising the relation between target and stop, just make sure its wide enough to stay out of the noise and that the target isn't unreasonable (i.e. expecting ES to move 40 points a day). The edge has to come from your entry or exit logic, so optimising ATR, stop and target is not going to help.

In terms of ATR, I have found in the past ATR is good in theory, but I prefer to just use a fixed target/stop (are you using an ATR mutliple for target and stops? Not sure if that's what you mean by dynamic). The problem is that markets transition from high vol to low vol and back to high, a lot of the times you get caught with stops and targets that are too tight when it transitions to high, and too loose when it transitions to low. When you use something like a 14 day ATR, the transition can be a bit too slow and catch you out quite a lot.

I did a lot of testing with single stock algos, the problem with applying an algo to stocks is that stocks behave in all sorts of ways. You have dividend payers which tend to range, you have growing companies which trends, you have speculative growth like Healthcare doing R&D that suddenly drop or pop with news, you have companies that can decline slowly instead of quickly. A single algo cannot cater for all of those conditions, so you're better off sticking with Futures.

Reply With Quote
 
  #7 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received


PeakGrowth View Post
Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.

In terms of changing your ATR, stops and targets, I suggest you have a look at a thread here on futures.io (formerly BMT) (can't remember where it is, but try search for it) where a few of the fellas here tested whether money management gave you an edge (it didn't). It didn't matter what your R:R was, the drop in win rate will just balance out so you end up with no edge in the end. There really isn't a point in optimising the relation between target and stop, just make sure its wide enough to stay out of the noise and that the target isn't unreasonable (i.e. expecting ES to move 40 points a day). The edge has to come from your entry or exit logic, so optimising ATR, stop and target is not going to help.

In terms of ATR, I have found in the past ATR is good in theory, but I prefer to just use a fixed target/stop (are you using an ATR mutliple for target and stops? Not sure if that's what you mean by dynamic). The problem is that markets transition from high vol to low vol and back to high, a lot of the times you get caught with stops and targets that are too tight when it transitions to high, and too loose when it transitions to low. When you use something like a 14 day ATR, the transition can be a bit too slow and catch you out quite a lot.

I did a lot of testing with single stock algos, the problem with applying an algo to stocks is that stocks behave in all sorts of ways. You have dividend payers which tend to range, you have growing companies which trends, you have speculative growth like Healthcare doing R&D that suddenly drop or pop with news, you have companies that can decline slowly instead of quickly. A single algo cannot cater for all of those conditions, so you're better off sticking with Futures.

sorry, couldn't get back in time here
no, when I say I am using ATR multiplier, that's parameter in superTrend (that's what I am testing now).
when does the trend change is identified by "ATR multiplier"
Target/stop - fixed numbers and not depend on ATR
and I said dynamic as the indicator is similar to a dynamic S/R (nothing but where the trend changes)

Reply With Quote
 
  #8 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received


PeakGrowth View Post
Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.
.

ok, I read that and I just have 1 parameter called ATR multiplier that decides when trend changes, so I hope thats very less curve fitting. beside that i do optimise on diff charts, like
15min, 30min, 60min
and target/stop are fixed numbers

Reply With Quote
 
  #9 (permalink)
Elite Member
Dallas,TX
 
Futures Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker/Data: Zaner/Zen Fire
Favorite Futures: ES,6E,6B,GC,CL
 
Posts: 590 since Nov 2009
Thanks: 176 given, 116 received


PeakGrowth View Post
Have a google on curve fitting, it's when you change parameters and re-run a backtest to make the performance come out better than before. The purpose of back testing is not to fine tune your algo so that it eeks out an extra 1% per year (that would be curve fitting), it's to ensure that you do have an edge and you don't blow up.

ok, couldnt complete in previous posts.

you say purpose of backtest "it's to ensure that you do have an edge and you don't blow up"
how would you do without changing few parameters atleast

Reply With Quote
 
  #10 (permalink)
Elite Member
Sydney, Australia
 
Futures Experience: Intermediate
Platform: Sierra Chart, IRESS
Broker/Data: IB, IQFeed
Favorite Futures: ES, SPI, ASX stocks, options
 
Posts: 399 since Jun 2015
Thanks: 169 given, 446 received


You can definitely change parameters to make it work, but don't tweak the lengths and what not just to eek out a few more percentage gains.

i.e. if you are changing your ATR length from 14, 15, 16 and picking 16 because it gives you the best results, it will generally be curve fitting

Reply With Quote

Reply



futures io > > > how long to forward test to know Drawdown and avg monthly PnL

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
What's a good batting avg? TemporarySanity Beginners and Introductions 8 August 19th, 2014 03:04 PM
Marketanalyseralert setup for crossing the avg fossil NinjaTrader 0 December 11th, 2012 08:22 AM
nt7 Mov avg behind candlestick bobarian NinjaTrader 6 December 31st, 2011 02:51 PM
Simple Hull Avg... Serger Elite EasyLanguage Automated Trading 3 September 2nd, 2011 10:39 AM
Moving Avg of Indicator ceramictilepro NinjaTrader Programming 6 February 1st, 2011 10:09 PM


All times are GMT -4. The time now is 10:15 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-17 in 0.17 seconds with 19 queries on phoenix via your IP 107.20.115.174