NexusFi: Find Your Edge


Home Menu

 





Backtesting: What is a good sample size?


Discussion in Commodities

Updated
      Top Posters
    1. looks_one ellinas with 2 posts (0 thanks)
    2. looks_two tpredictor with 2 posts (1 thanks)
    3. looks_3 ClearTrades with 2 posts (1 thanks)
    4. looks_4 Bermudan Option with 1 posts (3 thanks)
      Best Posters
    1. looks_one rleplae with 5 thanks per post
    2. looks_two Bermudan Option with 3 thanks per post
    3. looks_3 sixtyseven with 1 thanks per post
    4. looks_4 ClearTrades with 0.5 thanks per post
    1. trending_up 8,609 views
    2. thumb_up 11 thanks given
    3. group 9 followers
    1. forum 12 posts
    2. attach_file 0 attachments




 
Search this Thread

Backtesting: What is a good sample size?

  #1 (permalink)
ClearTrades
Yakima WA
 
Posts: 28 since Mar 2019
Thanks Given: 67
Thanks Received: 20

When you are backtesting a strategy, how many trades is a good sample size? How many trades is considered enough to rely on? So far I have backtested 20 trades on a new strategy I am developing. I don't have any fancy backtesting software, I am just manually filling out an excel spreadsheet based on analysis of the chart.
So far, after 20 trades into backtesting, my strategy is profitable, and I am now temped to trade it live with real money, starting small, with just 1 contract in case of failure. My backtesting is complete with stop loss placement so I feel like its ready to go with risk mitigation, but want some input from the community here.

I am planning to keep backtesting my strategy another 10 trades at least but how far back should I go? Is 30 trades enough? Backtesting this way is very time consuming. What do you think?

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Exit Strategy
NinjaTrader
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
Better Renko Gaps
The Elite Circle
NexusFi Journal Challenge - April 2024
Feedback and Announcements
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
GFIs1 1 DAX trade per day journal
18 thanks
The Program
18 thanks
  #3 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863


what period of time did you test ?

market have various regimes/periods
- calm versus volatile
- trending versus ranging

depending of the type of stratergy and the type of market it best
performs in, you need more or less back testing. you want to be
sure that favorable and less favorable periods were tested. it's
ok, not to make money if the type of market is not good, but you
don't want to go bust during that period, that is what you want
to test..

if your approach is very systematic, then a automation is recommended
i did that for several members and quite often it turns out that a
strategy that looks manually (quite good on excel on paper or screen)
does not do well, when doing systematically...

if it's simple, automate it and run it through a year of data..
it well learn you a lot

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #4 (permalink)
 tpredictor 
North Carolina
 
Experience: Beginner
Platform: NinjaTrader, Tradestation
Trading: es
Posts: 644 since Nov 2011

This is not the way to backtest. I recommend you get Tradestation. You can also paper trade your idea in Ninjatrader.

There is no hard and fast answer as to how many trades. It will depend on how big the edge is (bigger edge, less trades), how many variables you have (more variables more trades), and considerations around your beliefs or Bayesian reasons as to why a system works. The number 30 I believe comes from statistical studies around confidence interval and if I recall the Chi square distribution.

I would typically look for at least 30 trades per year and 10 years of performance data. But, this may not be the way that every system would be evaluated. With less trades, you would need to go back further in history and if you have more trades you might need less history or if you have Bayesian beliefs then that might influence how far back in history you go, as well.




ClearTrades View Post
When you are backtesting a strategy, how many trades is a good sample size? How many trades is considered enough to rely on? So far I have backtested 20 trades on a new strategy I am developing. I don't have any fancy backtesting software, I am just manually filling out an excel spreadsheet based on analysis of the chart.
So far, after 20 trades into backtesting, my strategy is profitable, and I am now temped to trade it live with real money, starting small, with just 1 contract in case of failure. My backtesting is complete with stop loss placement so I feel like its ready to go with risk mitigation, but want some input from the community here.

I am planning to keep backtesting my strategy another 10 trades at least but how far back should I go? Is 30 trades enough? Backtesting this way is very time consuming. What do you think?


Reply With Quote
  #5 (permalink)
ClearTrades
Yakima WA
 
Posts: 28 since Mar 2019
Thanks Given: 67
Thanks Received: 20

Thanks for the reply.
I am looking at only the 30 minute chart, on CL crude oil.
1 round trip trade per day based and analizing the last 48 hours, with analysis of the same timeframe each time. Trade execution goal is to complete the round-trip trade within about a 12 hour trading window.
So far I have tested the last 20 trading days.
I don't really know if it is possible to automate because I am using subjective/relative analysis of trend and momentum of the last 48 hours to give me my entry signal. I have my criteria but it is not numerical, it is basically current momentum and trend relative to historical momentum and trend.

I guess since oil has been uptrending lately I should test it back when it was downtrending huh?

Reply With Quote
Thanked by:
  #6 (permalink)
 tpredictor 
North Carolina
 
Experience: Beginner
Platform: NinjaTrader, Tradestation
Trading: es
Posts: 644 since Nov 2011

@ClearTrades I think, in that case, trading it with Ninjatrader market replay makes sense and paper trading it, i.e. real-time sim account, will be most helpful.

Reply With Quote
  #7 (permalink)
 centaurer 
south africa
 
Posts: 169 since Dec 2018

The heuristic of using 30 samples comes from confidence levels between the normal and student t distribution when sampling.
It has absolutely nothing to do with trading or most other real world problems.

IMO you need to think in more of a Bayesian way than in using the tools for sampling from known distributions with all kinds of assumptions on the variable.

If you take 1 sample, then another, then another each new sample is giving you information about the system.

10 samples with 7 winners of course does not tell you this is a "winning" system but it also isn't meaningless.

The main problem though is like rleplae mentioned. If we know anything for certain with trading it is that the distributions will change.

Don't get too hung up on sampling from stale data and last year's distributions. Like right now I am not even sure how much value samples have from a few months ago when fed fund futures had hugely different distributions than today.

Reply With Quote
  #8 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


ClearTrades View Post
Is 30 trades enough? Backtesting this way is very time consuming. What do you think?

l'll say this nicely.... Don't be lazy.

30 trades is not even in the ballpark. You want well over 100. If you can't be bothered to even do that before throwing money at it, then you will have some very quick and hard lessons with this trading game.

Keep testing for some days. You'll be glad you did, I'll bet this strategy will revert to being no better than random. Strategies that work are not easy to come by. They take a lot of time to find. You will spend a lot of time, which will seem like wasted time as you churn through idea after idea that doesn't work out.

Save your money and your mental capital for those strategies that you have throughly tested out.

Reply With Quote
Thanked by:
  #9 (permalink)
 SWisdom 
Newport Beach
 
Experience: Master
Platform: TradingBlox
Broker: Wisdom Trading
Trading: Global Futures
Posts: 3 since Mar 2019
Thanks Given: 0
Thanks Received: 0

I would recommend thousands of trades in your test sample. Not just one market and not just one time frame.

Reply With Quote
  #10 (permalink)
ellinas
Cyprus, Nico
 
Posts: 33 since Mar 2019
Thanks Given: 10
Thanks Received: 9



ClearTrades View Post
When you are backtesting a strategy, how many trades is a good sample size? How many trades is considered enough to rely on? So far I have backtested 20 trades on a new strategy I am developing. I don't have any fancy backtesting software, I am just manually filling out an excel spreadsheet based on analysis of the chart.
So far, after 20 trades into backtesting, my strategy is profitable, and I am now temped to trade it live with real money, starting small, with just 1 contract in case of failure. My backtesting is complete with stop loss placement so I feel like its ready to go with risk mitigation, but want some input from the community here.

I am planning to keep backtesting my strategy another 10 trades at least but how far back should I go? Is 30 trades enough? Backtesting this way is very time consuming. What do you think?

Use only MR(Market replay) method for backtesting for at least 12-months back. This is the only reliable backtest method. Anything else is fake and fails.
Thanks-

Reply With Quote




Last Updated on December 5, 2020


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts