Is back testing a waste of time? - futures io
futures io futures trading



Is back testing a waste of time?


Discussion in Emini and Emicro Index

Updated by SidewalkAerobics
      Top Posters
    1. looks_one Ninj4Man with 4 posts (3 thanks)
    2. looks_two rleplae with 3 posts (1 thanks)
    3. looks_3 JonnyBoy with 2 posts (8 thanks)
    4. looks_4 TWDsje with 2 posts (4 thanks)
      Best Posters
    1. looks_one JonnyBoy with 4.0 thanks per post
    2. looks_two TWDsje with 2.0 thanks per post
    3. looks_3 Gauer with 2.0 thanks per post
    4. looks_4 Ninj4Man with 0.8 thanks per post
    1. trending_up 4,217 views
    2. thumb_up 19 thanks given
    3. group 14 followers
    1. forum 20 replies
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 100,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Is back testing a waste of time?

(login for full post details)
  #1 (permalink)
Auckland
 
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received

Iíve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I donít believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I canít help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I havenít used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

Reply With Quote
The following user says Thank You to Ninj4Man for this post:

Can you help answer these questions
from other members on futures io?
Seeking an MT4 Indicator/EA that scans all Currency Pairs to identify setup
Platforms and Indicators
Anyone with Overcharts license/experience?
Platforms and Indicators
MulitCharts: Limit Order converted to MIT
Platforms and Indicators
US OTCQB STOCK Brooker for UK BASED RESIDENT
Brokers
Traderoomplus
Trading Reviews and Vendors
 
 
(login for full post details)
  #3 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,797 received


Did you forward run it during 6 months ?
How many trades were taken ?

Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

Follow me on Twitter Visit my Facebook Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #4 (permalink)
Auckland
 
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received


rleplae View Post
Did you forward run it during 6 months ?
How many trades were taken ?

I haven't done a forward run as back test results are leading me to believe results are inconsistent, but I can try this to see how it compares to back test.


rleplae View Post
Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

that's what I am getting at, my idea seems to be working and potentially pursuing with the results I get but then this environment isn't real, also how can you go off MFE/MAE etc when it's not live.


rleplae View Post
Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

you right I do try and filter the bad trades out when back testing and find myself constantly changing parameters to rectify loosing trades which I no is a pointless exercise but unsure where to go from here with my strategy, do I just accept the loosing trades when backtesting as long as winning trades outweigh them or am I missing something?


rleplae View Post
What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

I have yet to experience this yet and maybe I won't ever as I feel I can't fully trust my strategies in this environment.

Reply With Quote
 
(login for full post details)
  #5 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,797 received

May i recommend you take the time and read the book from Kevin ?
Building winning algorithmic trading systems..

It will give you a process, if you stick to, you get to results..

Follow me on Twitter Visit my Facebook Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #6 (permalink)
Auckland
 
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received

Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Reply With Quote
The following user says Thank You to Ninj4Man for this post:
 
(login for full post details)
  #7 (permalink)
Gits (Hooglede) Belgium
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 2,991 since Sep 2013
Thanks: 2,437 given, 5,797 received


Ninj4Man View Post
Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Yes, Kevin explains step by step, validating the idea, testing, etc..
he uses tradestation, but NT is equally good, nothing in there that can not be done with NT

Follow me on Twitter Visit my Facebook Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to rleplae for this post:
 
(login for full post details)
  #8 (permalink)
Legendary Market Wizard
Calgary, Alberta
 
Experience: Advanced
Platform: NinjaTrader 8
Broker: Kinetick
Trading: ES
 
JonnyBoy's Avatar
 
Posts: 1,484 since Apr 2012
Thanks: 633 given, 3,450 received


Ninj4Man View Post
Iíve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I donít believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I canít help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I havenít used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

You have already had a great reply from @rleplae, but I thought I would share my thoughts.

In combination with Price Action I have been trading with a particular setup for quite some time. It was only recently that I decided to see if I could slipstream my setup that had become a hybrid of many individual parts, into one system or one setup if you will.

Cutting a long story short, I am now trying to prove with back testing that the setups I have been trading these recent years, is actually a valid system in the first place. What I mean by that is, when trading discretionary it is very easy to filter trades (for many reasons) and I needed to see if I could slipstream that decision making into the code.

What back testing showed on the first few runs was that yes, it is a profitable system so I should continue. But I also need to bear in mind that the initial back test was against a very small parcel of data from Market Replay. Market Replay available files in NT8 is very limited. I decided to use Historical Data instead and I got vastly different results. I am not saying you will too or have, but for me I just didnít trust it.

The resolution of Historical Data is much less than Market Replay and therefore may not be as accurate. Anyway, I digressÖ

One suggestion is you parcel out your back test data. For example, letís say you have 18 months of data you can back test against - split that into 3 parcels of data. You should then look to exclude an entire parcel from ever being back tested against with your system. Then optimise your setup on the 2 parcels of data you have. Human nature says that you will curve fit to the 2 parcels of data.

When you believe the system is the best you will ever get it, proceed to back test it against the parcel of data you have kept in reserve. That is a good way to avoid curve fitting to your entire back test catalogue of data.

I would say back testing is not a waste of time as it can eliminate some very bad systems very quickly.

--------------------------------------------------------
- Trade what you see. Invest in what you believe -
--------------------------------------------------------
Reply With Quote
The following user says Thank You to JonnyBoy for this post:
 
(login for full post details)
  #9 (permalink)
white plains new york
 
 
Posts: 66 since Apr 2016

Back testing is the right tool to identify potential winning patterns.
If you can identify a pattern with high probability of winners, find out what is the average move (in ticks) per incident. Place auto trades with 70% of the ticks in the average move as TP and same as SL. If your pattern produces over 50% winning, you are on the right path to the next step. (reduce the SL fixed point)

Reply With Quote
 
(login for full post details)
  #10 (permalink)
Leeds UK
 
Experience: Intermediate
Platform: TradingView
Broker: LCG, Oanda
Trading: ES, 6E, Cable
 
Tymbeline's Avatar
 
Posts: 453 since Apr 2015
Thanks: 1,147 given, 607 received



Ninj4Man View Post
Is back testing a waste of time?


My own view is that like so many other issues/parameters involved in trading, it is for some people, but not for others.



Ninj4Man View Post
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all.


I think a couple of good reasons (with which I agree) have been mentioned above. Personally, although I certainly don't take favourable backtesting results as proof that a method will be profitable, I still prefer to analyse in more detail and forward-test something that has performed well on backtesting than something that hasn't.



Ninj4Man View Post
I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false.


I don't quite understand how you're distinguishing between "will work" and "will be profitable": I think of these descriptions as more or less the same thing.



Ninj4Man View Post
I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail


I'm sure that's sometimes so. Favourable backtesting results aren't proof that something will work, but in experienced hands unfavourable backtesting results are a pretty good indication not to spend more time on it.

I look at it a little like how I look at going to a wine-tasting: I don't always expect to find something wonderful at a tasting, but it's worthwhile going to one simply if it can show me the things I want to learn avoid in future and investigate no further.



Ninj4Man View Post
why would all these companies have back testing built in if the results were that far removed from live trading.


I suspect "customer expectation and demand" is the answer to that one. And perhaps an element of being self-perpetuating simply because of competition: nobody would want to market the only product that doesn't offer that facility.

Reply With Quote


futures io Trading Community Traders Hideout Emini and Emicro Index > Is back testing a waste of time?


October 3, 2020


Upcoming Webinars and Events
 

Order Flow Tools w/MotiveWave & Zytrade Brokerage

Oct 22
 

Technical Analysis Workshop w/NinjaTrader

Oct 27
 

Bookmap

Oct TBD
 

SharkIndicators

Oct TBD
 

Carley Garner

Oct TBD
 

GruttePier

Oct TBD
     



Copyright © 2020 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts