Is back testing a waste of time? - Index Futures Trading | futures.io
futures.io futures trading


Is back testing a waste of time?
Started: by Ninj4Man Views / Replies:739 / 12
Last Reply: Attachments:0

Welcome to futures.io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.


You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Reply
 
Thread Tools Search this Thread
 

Is back testing a waste of time?

  #1 (permalink)
Trading Apprentice
Auckland
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Emini
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received

Is back testing a waste of time?

Iíve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I donít believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I canít help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I havenít used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

Reply With Quote
The following user says Thank You to Ninj4Man for this post:
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Elite Member
Omer עומר / Israel י
 
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, CL, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 1,656 since Sep 2013
Thanks: 1,219 given, 2,499 received
Forum Reputation: Legendary


Did you forward run it during 6 months ?
How many trades were taken ?

Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

Reply With Quote
 
  #4 (permalink)
Trading Apprentice
Auckland
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Emini
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received


rleplae View Post
Did you forward run it during 6 months ?
How many trades were taken ?

I haven't done a forward run as back test results are leading me to believe results are inconsistent, but I can try this to see how it compares to back test.


rleplae View Post
Back-testing is useful to :
1. quickly test an idea if it is worth pursuing at all
2. to get 'insight' in the strategy, (%win/loss, MFE, MAE, largest draw down)

that's what I am getting at, my idea seems to be working and potentially pursuing with the results I get but then this environment isn't real, also how can you go off MFE/MAE etc when it's not live.


rleplae View Post
Tweaking is good, if you know what you are doing and not just changing a parameter
to filter those bad trades that ruined your plan, chances are big that in forward testing
another disaster will strike before you get the chance to change the parameter to filter
this one out..

you right I do try and filter the bad trades out when back testing and find myself constantly changing parameters to rectify loosing trades which I no is a pointless exercise but unsure where to go from here with my strategy, do I just accept the loosing trades when backtesting as long as winning trades outweigh them or am I missing something?


rleplae View Post
What is different in real trading ?
1. psychology, the trader panicking if the strategy makes a few losses
2. the liquidity of the market (fills, stop runs, ...

I have yet to experience this yet and maybe I won't ever as I feel I can't fully trust my strategies in this environment.

Reply With Quote
 
  #5 (permalink)
Elite Member
Omer עומר / Israel י
 
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, CL, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 1,656 since Sep 2013
Thanks: 1,219 given, 2,499 received
Forum Reputation: Legendary

Futures Edge on FIO
May i recommend you take the time and read the book from Kevin ?
Building winning algorithmic trading systems..

It will give you a process, if you stick to, you get to results..

Reply With Quote
 
  #6 (permalink)
Trading Apprentice
Auckland
 
Futures Experience: Beginner
Platform: NinjaTrader
Favorite Futures: Emini
 
Posts: 10 since Jul 2016
Thanks: 4 given, 4 received

Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Reply With Quote
The following user says Thank You to Ninj4Man for this post:
 
  #7 (permalink)
Elite Member
Omer עומר / Israel י
 
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, CL, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 1,656 since Sep 2013
Thanks: 1,219 given, 2,499 received
Forum Reputation: Legendary


Ninj4Man View Post
Sure i'll look it up I am always keen on learning new approaches, I guess I can apply his theory to NT and my own strategies?

Yes, Kevin explains step by step, validating the idea, testing, etc..
he uses tradestation, but NT is equally good, nothing in there that can not be done with NT

Reply With Quote
The following user says Thank You to rleplae for this post:
 
  #8 (permalink)
Market Wizard
Calgary, Alberta
 
Futures Experience: Intermediate
Platform: NinjaTrader 8
Broker/Data: Kinetick
Favorite Futures: ES
 
JonnyBoy's Avatar
 
Posts: 910 since Apr 2012
Thanks: 373 given, 1,689 received
Forum Reputation: Legendary


Ninj4Man View Post
Iíve been running my strategies in Ninjatrader for 6 months now, tweaking parameters and always fine tuning, I donít believe in a perfect strategy and I am fully aware that I will lose trades but hopefully win (some)
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all. I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false. Even if fill type is liberal and slippage is set to high I canít help but think the results are pointless, I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail, which makes me wonder why even back test at all. I havenít used other trading platforms but why would all these companies have back testing built in if the results were that far removed from live trading. How many of you back test and if so do the results you get in sim even equate to anything remotely similar in live trading?

You have already had a great reply from @rleplae, but I thought I would share my thoughts.

In combination with Price Action I have been trading with a particular setup for quite some time. It was only recently that I decided to see if I could slipstream my setup that had become a hybrid of many individual parts, into one system or one setup if you will.

Cutting a long story short, I am now trying to prove with back testing that the setups I have been trading these recent years, is actually a valid system in the first place. What I mean by that is, when trading discretionary it is very easy to filter trades (for many reasons) and I needed to see if I could slipstream that decision making into the code.

What back testing showed on the first few runs was that yes, it is a profitable system so I should continue. But I also need to bear in mind that the initial back test was against a very small parcel of data from Market Replay. Market Replay available files in NT8 is very limited. I decided to use Historical Data instead and I got vastly different results. I am not saying you will too or have, but for me I just didnít trust it.

The resolution of Historical Data is much less than Market Replay and therefore may not be as accurate. Anyway, I digressÖ

One suggestion is you parcel out your back test data. For example, letís say you have 18 months of data you can back test against - split that into 3 parcels of data. You should then look to exclude an entire parcel from ever being back tested against with your system. Then optimise your setup on the 2 parcels of data you have. Human nature says that you will curve fit to the 2 parcels of data.

When you believe the system is the best you will ever get it, proceed to back test it against the parcel of data you have kept in reserve. That is a good way to avoid curve fitting to your entire back test catalogue of data.

I would say back testing is not a waste of time as it can eliminate some very bad systems very quickly.

--------------------------------------------------------
- Trade what you see. Invest in what you believe -
--------------------------------------------------------
Reply With Quote
The following user says Thank You to JonnyBoy for this post:
 
  #9 (permalink)
Membership Temporarily Revoked
white plains new york
 
Futures Experience: Intermediate
Platform: NJ
Favorite Futures: CL
 
Posts: 66 since Apr 2016
Thanks: 30 given, 66 received

Back testing

Back testing is the right tool to identify potential winning patterns.
If you can identify a pattern with high probability of winners, find out what is the average move (in ticks) per incident. Place auto trades with 70% of the ticks in the average move as TP and same as SL. If your pattern produces over 50% winning, you are on the right path to the next step. (reduce the SL fixed point)

Reply With Quote
 
  #10 (permalink)
Elite Member
Leeds UK
 
Futures Experience: Intermediate
Platform: FXTrade
Broker/Data: Interactive Brokers, CapitalSpreads, Oanda
Favorite Futures: GBP/USD
 
Tymbeline's Avatar
 
Posts: 364 since Apr 2015
Thanks: 787 given, 445 received



Ninj4Man View Post
Is back testing a waste of time?


My own view is that like so many other issues/parameters involved in trading, it is for some people, but not for others.



Ninj4Man View Post
I am skeptical about using back testing as I can always generate more winning trades than loosing and at the time seems like my strategy is working well, but then I have to realize this is not live, not real and only simulated, which makes me wonder why use back testing at all.


I think a couple of good reasons (with which I agree) have been mentioned above. Personally, although I certainly don't take favourable backtesting results as proof that a method will be profitable, I still prefer to analyse in more detail and forward-test something that has performed well on backtesting than something that hasn't.



Ninj4Man View Post
I mean itís good for testing if strategies work but to get an idea if a strategy will be profitable seems false.


I don't quite understand how you're distinguishing between "will work" and "will be profitable": I think of these descriptions as more or less the same thing.



Ninj4Man View Post
I read time and time again in different forums about people developing new strategies running them on back test and getting some great results only to be told by other members that what they are experiencing is fake and in the real world the strategy will fail


I'm sure that's sometimes so. Favourable backtesting results aren't proof that something will work, but in experienced hands unfavourable backtesting results are a pretty good indication not to spend more time on it.

I look at it a little like how I look at going to a wine-tasting: I don't always expect to find something wonderful at a tasting, but it's worthwhile going to one simply if it can show me the things I want to learn avoid in future and investigate no further.



Ninj4Man View Post
why would all these companies have back testing built in if the results were that far removed from live trading.


I suspect "customer expectation and demand" is the answer to that one. And perhaps an element of being self-perpetuating simply because of competition: nobody would want to market the only product that doesn't offer that facility.

Reply With Quote

Reply



futures.io > > > > Is back testing a waste of time?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

2016 True Edge Awards: Voting is open until Jan 21!

Now
 

Bookmap New Indicator Panel, Cumulative Delta, Order Flow demo

Jan 17
 

An Afternoon with FIO member Massive l

Elite only

Leo Murphy: Making sense of Technical Analysis and Random Walk

Elite only
 

FuturesTrader71: Ask Me Anything

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Running 2 s back testing simulating optimization strategies and the same time fractal TradeStation 0 March 8th, 2012 07:13 AM
Platform for back testing andyw Platforms and Indicators 2 December 15th, 2011 01:10 AM
Back to the Future! Back testing multi time frame strategy trades occuring in future mrticks The Elite Circle 10 April 2nd, 2011 10:54 AM
back testing sam028 Terms (Glossary) 0 June 28th, 2010 07:20 PM
Back Testing / Optimization esscalper The Elite Circle 2 November 19th, 2009 03:00 PM


All times are GMT -4. The time now is 06:24 AM.

no new posts
Page generated 2017-01-18 in 0.16 seconds with 19 queries on phoenix via your IP 54.159.158.180