What should be the next step in my backtesting?? - Emini Index Futures Trading | futures io social day trading
futures io futures trading


What should be the next step in my backtesting??
Updated: Views / Replies:6,401 / 76
Created: by budfox Attachments:4

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 4  
 
Thread Tools Search this Thread
 

What should be the next step in my backtesting??

  #61 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

UPDATE> Backtesting complete>> please advise

Hi Guys,


so I have completed my manual backtesting of 101 trades


First sample is from 1 Jan - 23 April 2014.

Second OS is from 1 Aug - Oct 30 2014 .

My first question is should I add the begining equity amount to the end of the first sample?

ie if the ending equity balance at 23 April was $5.00 should I add it to 1 August and start the second sample with $5.00? and build on that?


(My opening balance for my second sample was set to $1 , commissions was set to .0019 (based on the spread charged by AAAfx at the time I started the backtest)



Here are my stats (calculated based all the trades in both samples):
  • Win Ratio 41%
  • Loss Ratio 59%
  • Ave Win = .0013 (EUR/USD)
  • Ave Loss = .0008
  • N = 101
  • Max DD = 89%




Please let me know if I am not following proper backtesting procedure.


Would you trade this model (in conjunction with another pair using the same model (Portfolio Backtesting)?

What other metrics would you require when deciding to implement this model?

Thank You to those that responded, have a great weekend!










@Farmer George



@Hulk @kevinkdog @tellytub @Ming80 @treydog999 @bobwest @sam028


Reply With Quote
 
  #62 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
Hi Guys,


so I have completed my manual backtesting of 101 trades


First sample is from 1 Jan - 23 April 2014.

Second OS is from 1 Aug - Oct 30 2014 .

My first question is should I add the begining equity amount to the end of the first sample?

ie if the ending equity balance at 23 April was $5.00 should I add it to 1 August and start the second sample with $5.00? and build on that?


(My opening balance for my second sample was set to $1 , commissions was set to .0019 (based on the spread charged by AAAfx at the time I started the backtest)



Here are my stats (calculated based all the trades in both samples):
  • Win Ratio 41%
  • Loss Ratio 59%
  • Ave Win = .0013 (EUR/USD)
  • Ave Loss = .0008
  • N = 101
  • Max DD = 89%




Please let me know if I am not following proper backtesting procedure.


Would you trade this model (in conjunction with another pair using the same model (Portfolio Backtesting)?

What other metrics would you require when deciding to implement this model?

Thank You to those that responded, have a great weekend!










@Farmer George



@Hulk @kevinkdog @tellytub @Ming80 @treydog999 @bobwest @sam028


If this is a walk forward testing model yes you should carry your previous P/L and current position inventory with you. Honestly your max DD is too high, 89% is really bad. When you take this to monte carlo you will bust out almost guaranteed.

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #63 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received



treydog999 View Post
If this is a walk forward testing model yes you should carry your previous P/L and current position inventory with you. Honestly your max DD is too high, 89% is really bad. When you take this to monte carlo you will bust out almost guaranteed.

Hi Trey,

No this was not walkforward testing (not quite sure how you define it), but like I said in the previous post two individual samples (of 50 trades each), First sample = Jan 1 - April 23, 2nd sample = 1 Aug - Oct 30 2014 .

The second sample was my "out of sample" sample.

So you still want me to add my closing account balance from April 23 to the opening balance at 1 Aug? (if this is the case I dont think I will have such a huge drawdown)


What do you mean by 'previous P/L and current position inventory with you' sorry it sounds interesting but do not understand the fancy jargon.


2. What is the purpose of monte carlo?


Last edited by budfox; November 11th, 2014 at 01:29 AM.
Reply With Quote
 
  #64 (permalink)
 Vendor: www.probabletrades.com 
OC, California, USA
 
Futures Experience: Advanced
Platform: IB/TWS, NinjaTrader, thinkorswim
Favorite Futures: stocks, options, futures, VIX
 
shodson's Avatar
 
Posts: 1,857 since Jun 2009
Thanks: 480 given, 3,247 received


budfox View Post
[COLOR="RoyalBlue"][SIZE="4"]Hi Guys,
so I have completed my manual backtesting of 101 trades

I would study your equity graph (if you have one) and honestly consider if you can stomach the ups and downs of your account's journey. See this example.

https://futures.io/elite-trading-journals/14717-shodson-s-automated-trading-journal.html#post167077

Along those lines I like to find streaks (both winning and losing) to prepare myself what kind of emotional turmoil I may have to endure during a long losing streak. With a 41% win rate you're likely to have longer losing streaks than winning streaks.

Also, are you considering slippage? And, depending on the strategy, your actual fill prices may not be what an actual market will give you, or you may not get filled at all but your model assumes you always get filled. This can change things dramatically.

Reply With Quote
The following user says Thank You to shodson for this post:
 
  #65 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


shodson View Post
I would study your equity graph (if you have one) and honestly consider if you can stomach the ups and downs of your account's journey. See this example.

https://futures.io/elite-trading-journals/14717-shodson-s-automated-trading-journal.html#post167077

Along those lines I like to find streaks (both winning and losing) to prepare myself what kind of emotional turmoil I may have to endure during a long losing streak. With a 41% win rate you're likely to have longer losing streaks than winning streaks.

Also, are you considering slippage? And, depending on the strategy, your actual fill prices may not be what an actual market will give you, or you may not get filled at all but your model assumes you always get filled. This can change things dramatically.


How would I factor in slippage in my model?

(I am manually backtesting)

Thanks shodson!

Reply With Quote
 
  #66 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
Hi Trey,

No this was not walkforward testing (not quite sure how you define it), but like I said in the previous post two individual samples (of 50 trades each), First sample = Jan 1 - April 23, 2nd sample = 1 Aug - Oct 30 2014 .

The second sample was my "out of sample" sample.

So you still want me to add my closing account balance from April 23 to the opening balance at 1 Aug? (if this is the case I dont think I will have such a huge drawdown)


What do you mean by 'previous P/L and current position inventory with you' sorry it sounds interesting but do not understand the fancy jargon.


2. What is the purpose of monte carlo?

1) Well walk forward testing is defined by Robert Pardo in his book The Evaluation and Optimization of Trading Strategies. But back to my statement. If Sample 1 is your in sample, and Sample 2 is your out of sample then NO you do not add your equity from 1 period to the next. Sample 2 starts again at initial equity.

If it was walk forward, and Sample 1 was the first testing window and parameter set and Sample 2 was the second testing window and parameter set, then yes you would add the equity from the first period to the second.

2) In short, Monte Carlo is a stress testing method and through sampling without replacement allows you to generate confidence intervals for trading statistics like MaxDD, Net Profit, avg trade value etc... It is basically a type of statistical bootstrap. I will not delve any deeper since there are actually many methods that can use this type of testing. But on futures.io (formerly BMT) trade resampling without replacement is the most common.

If you are having trouble with my "jargon" and the definitions here. I suggest you watch some of kevin daveys webinars I am pretty sure he covers this material. As well as pick up a few algorithmic trading / development books. Not to be a ball buster but most of this is extremely common and essential to building and testing good systems.


In regards to including slippage into your model which was posted above. I take the average spread or the max spread (if i am playing more conservative) and subtract that from every transaction/trade. That will help make your tests more realistic. So you have your spread sheet of trades, look at the maximum spread your broker has and subtract that from every trade and recalculate your stats. Thats a worst case scenario when you have to get out in high volatility, this is especially true in spot OTC FX. spreads can be in excess of 10-15 pips and your stops may trigger doing those times. So it should be built in. At worst you underestimate your profit, but its better to do that than under estimate your risk and exposures.

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #67 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


treydog999 View Post
1) Well walk forward testing is defined by Robert Pardo in his book The Evaluation and Optimization of Trading Strategies. But back to my statement. If Sample 1 is your in sample, and Sample 2 is your out of sample then NO you do not add your equity from 1 period to the next. Sample 2 starts again at initial equity.

If it was walk forward, and Sample 1 was the first testing window and parameter set and Sample 2 was the second testing window and parameter set, then yes you would add the equity from the first period to the second.

2) In short, Monte Carlo is a stress testing method and through sampling without replacement allows you to generate confidence intervals for trading statistics like MaxDD, Net Profit, avg trade value etc... It is basically a type of statistical bootstrap. I will not delve any deeper since there are actually many methods that can use this type of testing. But on futures.io (formerly BMT) trade resampling without replacement is the most common.

If you are having trouble with my "jargon" and the definitions here. I suggest you watch some of kevin daveys webinars I am pretty sure he covers this material. As well as pick up a few algorithmic trading / development books. Not to be a ball buster but most of this is extremely common and essential to building and testing good systems.


In regards to including slippage into your model which was posted above. I take the average spread or the max spread (if i am playing more conservative) and subtract that from every transaction/trade. That will help make your tests more realistic. So you have your spread sheet of trades, look at the maximum spread your broker has and subtract that from every trade and recalculate your stats. Thats a worst case scenario when you have to get out in high volatility, this is especially true in spot OTC FX. spreads can be in excess of 10-15 pips and your stops may trigger doing those times. So it should be built in. At worst you underestimate your profit, but its better to do that than under estimate your risk and exposures.

Hi Trey,

@treydog999 @sam028

What if I tested the same model with the same parameters (during the same timeframe) on the ES or GBP/USD......would you estimate or expect similar metrics?

Thanks, I will read Pardo's book. and I nearly done the lynda course, sort of went through all the lectures but not entirely understanding everything.

Have a good day,

BF
@Farmer George


@Hulk @Ming80 @bobwest @sam028

Reply With Quote
 
  #68 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
Hi Trey,

@treydog999 @sam028

What if I tested the same model with the same parameters (during the same timeframe) on the ES or GBP/USD......would you estimate or expect similar metrics?

Thanks, I will read Pardo's book. and I nearly done the lynda course, sort of went through all the lectures but not entirely understanding everything.

Have a good day,

BF
@Farmer George


@Hulk @Ming80 @bobwest @sam028

I would not expect similar metrics. There is a caveat, that is that this system was developed from stage 1 with the intent to trade multiple instruments or a basket of instruments. Then you would want to see similiar metrics but you may not actually have that result.

Reply With Quote
 
  #69 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

What value do you use for the standard deviation

Hello Trey @treydog999


I was reading Pardo's book and he is giving the formula for standard error:



I am wondering how do you determine what the Standard deviation value is to plug in ?



ie. Std Error = ? / SQRT (100)


I know STDDEV is the SQRT of the variance.........but what value to use in the numerator?


Thanks


@Fat Tails

Reply With Quote
 
  #70 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received



budfox View Post
Hello Trey @treydog999


I was reading Pardo's book and he is giving the formula for standard error:



I am wondering how do you determine what the Standard deviation value is to plug in ?



ie. Std Error = ? / SQRT (100)


I know STDDEV is the SQRT of the variance.........but what value to use in the numerator?


Thanks


@Fat Tails

It depends on what statistic your trying to find the std error of. For example you want to find the standard error of your average trade. Take the StdDev of all of your trades / sqrt(nTrades). This can be used as a std error approximation for pretty much anything, assuming a normal distribution.

Reply With Quote

Reply



futures io > > > > What should be the next step in my backtesting??

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

Dec 7

Linda Bradford Raschke: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Step MA crazybears Sierra Chart 2 July 15th, 2014 06:25 AM
Top Step Trader website down? birdoggg Platforms and Indicators 2 November 7th, 2012 10:07 AM
The Next Step Towards The End Of The Euro Quick Summary News and Current Events 0 November 16th, 2011 12:50 AM


All times are GMT -4. The time now is 03:40 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-11-19 in 0.20 seconds with 20 queries on phoenix via your IP 23.20.129.162