NexusFi: Find Your Edge


Home Menu

 





Is back testing a waste of time?


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one Ninj4Man with 4 posts (3 thanks)
    2. looks_two rleplae with 3 posts (1 thanks)
    3. looks_3 JonnyBoy with 2 posts (8 thanks)
    4. looks_4 SpeculatorSeth with 2 posts (4 thanks)
      Best Posters
    1. looks_one JonnyBoy with 4 thanks per post
    2. looks_two SpeculatorSeth with 2 thanks per post
    3. looks_3 Gauer with 2 thanks per post
    4. looks_4 Ninj4Man with 0.8 thanks per post
    1. trending_up 9,016 views
    2. thumb_up 19 thanks given
    3. group 15 followers
    1. forum 20 posts
    2. attach_file 0 attachments




 
Search this Thread

Is back testing a waste of time?

  #1 (permalink)
Ninj4Man
Auckland
 
Posts: 10 since Jul 2016
Thanks Given: 4
Thanks Received: 4

I’ve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I don’t believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean it’s good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I can’t help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I haven’t used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
Better Renko Gaps
The Elite Circle
Futures True Range Report
The Elite Circle
ZombieSqueeze
Platforms and Indicators
Are there any eval firms that allow you to sink to your …
Traders Hideout
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
38 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
GFIs1 1 DAX trade per day journal
19 thanks
The Program
18 thanks
  #3 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863


Did you forward run it during 6 months ?
How many trades were taken ?

Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #4 (permalink)
Ninj4Man
Auckland
 
Posts: 10 since Jul 2016
Thanks Given: 4
Thanks Received: 4


rleplae View Post
Did you forward run it during 6 months ?
How many trades were taken ?

I haven't done a forward run as back test results are leading me to believe results are inconsistent, but I can try this to see how it compares to back test.


rleplae View Post
Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

that's what I am getting at, my idea seems to be working and potentially pursuing with the results I get but then this environment isn't real, also how can you go off MFE/MAE etc when it's not live.


rleplae View Post
Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

you right I do try and filter the bad trades out when back testing and find myself constantly changing parameters to rectify loosing trades which I no is a pointless exercise but unsure where to go from here with my strategy, do I just accept the loosing trades when backtesting as long as winning trades outweigh them or am I missing something?


rleplae View Post
What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

I have yet to experience this yet and maybe I won't ever as I feel I can't fully trust my strategies in this environment.

Reply With Quote
  #5 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863

May i recommend you take the time and read the book from Kevin ?
Building winning algorithmic trading systems..

It will give you a process, if you stick to, you get to results..

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #6 (permalink)
Ninj4Man
Auckland
 
Posts: 10 since Jul 2016
Thanks Given: 4
Thanks Received: 4

Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Reply With Quote
Thanked by:
  #7 (permalink)
 
rleplae's Avatar
 rleplae 
Gits (Hooglede) Belgium
Legendary Market Wizard
 
Experience: Master
Platform: NinjaTrader, Proprietary,
Broker: Ninjabrokerage/IQfeed + Synthetic datafeed
Trading: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
Posts: 3,003 since Sep 2013
Thanks Given: 2,442
Thanks Received: 5,863


Ninj4Man View Post
Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Yes, Kevin explains step by step, validating the idea, testing, etc..
he uses tradestation, but NT is equally good, nothing in there that can not be done with NT

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #8 (permalink)
 
JonnyBoy's Avatar
 JonnyBoy 
Montreal, Quebec
 
Experience: Advanced
Platform: NinjaTrader 8
Broker: Kinetick
Trading: ES
Posts: 1,561 since Apr 2012
Thanks Given: 706
Thanks Received: 3,854


Ninj4Man View Post
I’ve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I don’t believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean it’s good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I can’t help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I haven’t used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

You have already had a great reply from @rleplae, but I thought I would share my thoughts.

In combination with Price Action I have been trading with a particular setup for quite some time. It was only recently that I decided to see if I could slipstream my setup that had become a hybrid of many individual parts, into one system or one setup if you will.

Cutting a long story short, I am now trying to prove with back testing that the setups I have been trading these recent years, is actually a valid system in the first place. What I mean by that is, when trading discretionary it is very easy to filter trades (for many reasons) and I needed to see if I could slipstream that decision making into the code.

What back testing showed on the first few runs was that yes, it is a profitable system so I should continue. But I also need to bear in mind that the initial back test was against a very small parcel of data from Market Replay. Market Replay available files in NT8 is very limited. I decided to use Historical Data instead and I got vastly different results. I am not saying you will too or have, but for me I just didn’t trust it.

The resolution of Historical Data is much less than Market Replay and therefore may not be as accurate. Anyway, I digress…

One suggestion is you parcel out your back test data. For example, let’s say you have 18 months of data you can back test against - split that into 3 parcels of data. You should then look to exclude an entire parcel from ever being back tested against with your system. Then optimise your setup on the 2 parcels of data you have. Human nature says that you will curve fit to the 2 parcels of data.

When you believe the system is the best you will ever get it, proceed to back test it against the parcel of data you have kept in reserve. That is a good way to avoid curve fitting to your entire back test catalogue of data.

I would say back testing is not a waste of time as it can eliminate some very bad systems very quickly.

--------------------------------------------------------
- Trade what you see. Invest in what you believe -
--------------------------------------------------------
Reply With Quote
Thanked by:
  #9 (permalink)
 IzhakHaim 
white plains new york
 
Posts: 66 since Apr 2016

Back testing is the right tool to identify potential winning patterns.
If you can identify a pattern with high probability of winners, find out what is the average move (in ticks) per incident. Place auto trades with 70% of the ticks in the average move as TP and same as SL. If your pattern produces over 50% winning, you are on the right path to the next step. (reduce the SL fixed point)

Reply With Quote
  #10 (permalink)
 
Tymbeline's Avatar
 Tymbeline 
Leeds UK
Market Wizard
 
Experience: Intermediate
Platform: Tradovate
Broker: Tradovate
Trading: MES, MNQ
Frequency: Several times daily
Duration: Minutes
Posts: 644 since Apr 2015
Thanks Given: 2,342
Thanks Received: 1,054



Ninj4Man View Post
Is back testing a waste of time?


My own view is that like so many other issues/parameters involved in trading, it is for some people, but not for others.



Ninj4Man View Post
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all.


I think a couple of good reasons (with which I agree) have been mentioned above. Personally, although I certainly don't take favourable backtesting results as proof that a method will be profitable, I still prefer to analyse in more detail and forward-test something that has performed well on backtesting than something that hasn't.



Ninj4Man View Post
I mean it’s good for testing if strategies work but to get an idea if a strategy will be profitable seems false.


I don't quite understand how you're distinguishing between "will work" and "will be profitable": I think of these descriptions as more or less the same thing.



Ninj4Man View Post
I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail


I'm sure that's sometimes so. Favourable backtesting results aren't proof that something will work, but in experienced hands unfavourable backtesting results are a pretty good indication not to spend more time on it.

I look at it a little like how I look at going to a wine-tasting: I don't always expect to find something wonderful at a tasting, but it's worthwhile going to one simply if it can show me the things I want to learn avoid in future and investigate no further.



Ninj4Man View Post
why would all these companies have back testing built in if the results were that far removed from live trading.


I suspect "customer expectation and demand" is the answer to that one. And perhaps an element of being self-perpetuating simply because of competition: nobody would want to market the only product that doesn't offer that facility.

Reply With Quote




Last Updated on October 3, 2020


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts