NexusFi: Find Your Edge


Home Menu

 





How can a strategy be profitable when backtested but not when actually used?


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one Big Mike with 5 posts (3 thanks)
    2. looks_two shanemcdonald28 with 5 posts (0 thanks)
    3. looks_3 kevinkdog with 4 posts (9 thanks)
    4. looks_4 wcandrews with 2 posts (4 thanks)
      Best Posters
    1. looks_one artemiso with 4 thanks per post
    2. looks_two kevinkdog with 2.3 thanks per post
    3. looks_3 wcandrews with 2 thanks per post
    4. looks_4 Big Mike with 0.6 thanks per post
    1. trending_up 7,800 views
    2. thumb_up 22 thanks given
    3. group 10 followers
    1. forum 27 posts
    2. attach_file 0 attachments




 
Search this Thread

How can a strategy be profitable when backtested but not when actually used?

  #1 (permalink)
sra18376
kansas city, ks
 
Posts: 17 since Jun 2014
Thanks Given: 0
Thanks Received: 5

I am completely new to using NT software and I was running through some youtube videos to try and get a hang of programming. I came across a video made by Big Mike where he programs a simple strategy. He then backtests the strategy and it proves to be profitable. However he says not to bother with this strategy, as it is not actually profitable in real practice. How can this be? I know some indicators can have lagging signals, but wouldnt running a backtest also take the lagging signals into consideration? Do lagging signals even matter in this case or can successfully-backtested strategies not be profitable for other reasons? Thanks for the explanation in advance.

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
PowerLanguage & EasyLanguage. How to get the platfor …
EasyLanguage Programming
ZombieSqueeze
Platforms and Indicators
Exit Strategy
NinjaTrader
How to apply profiles
Traders Hideout
REcommedations for programming help
Sierra Chart
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Just another trading journal: PA, Wyckoff & Trends
31 thanks
Spoo-nalysis ES e-mini futures S&P 500
28 thanks
Tao te Trade: way of the WLD
24 thanks
Bigger Wins or Fewer Losses?
20 thanks
GFIs1 1 DAX trade per day journal
17 thanks
  #2 (permalink)
 
tturner86's Avatar
 tturner86 
Portland, Oregon
 
Experience: Intermediate
Platform: F-16CM-40
Trading: GBU-39
Posts: 6,191 since Sep 2013
Thanks Given: 10,459
Thanks Received: 12,695


sra18376 View Post
I am completely new to using NT software and I was running through some youtube videos to try and get a hang of programming. I came across a video made by Big Mike where he programs a simple strategy. He then backtests the strategy and it proves to be profitable. However he says not to bother with this strategy, as it is not actually profitable in real practice. How can this be? I know some indicators can have lagging signals, but wouldnt running a backtest also take the lagging signals into consideration? Do lagging signals even matter in this case or can successfully-backtested strategies not be profitable for other reasons? Thanks for the explanation in advance.

Past performance is not indicative of future results.

Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #3 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,663 since Jul 2012
Thanks Given: 1,892
Thanks Received: 7,357



sra18376 View Post
I am completely new to using NT software and I was running through some youtube videos to try and get a hang of programming. I came across a video made by Big Mike where he programs a simple strategy. He then backtests the strategy and it proves to be profitable. However he says not to bother with this strategy, as it is not actually profitable in real practice. How can this be? I know some indicators can have lagging signals, but wouldnt running a backtest also take the lagging signals into consideration? Do lagging signals even matter in this case or can successfully-backtested strategies not be profitable for other reasons? Thanks for the explanation in advance.


There are ways to trick the backtest software. Also, if you heavily optimize, your results will never be the same going forward.

Follow me on Twitter Reply With Quote
Thanked by:
  #4 (permalink)
 
Hulk's Avatar
 Hulk 
Texas, USA
 
Experience: Advanced
Platform: TT, Custom
Trading: Futures, Spreads
Posts: 369 since May 2014
Thanks Given: 731
Thanks Received: 901


sra18376 View Post
I am completely new to using NT software and I was running through some youtube videos to try and get a hang of programming. I came across a video made by Big Mike where he programs a simple strategy. He then backtests the strategy and it proves to be profitable. However he says not to bother with this strategy, as it is not actually profitable in real practice. How can this be? I know some indicators can have lagging signals, but wouldnt running a backtest also take the lagging signals into consideration? Do lagging signals even matter in this case or can successfully-backtested strategies not be profitable for other reasons? Thanks for the explanation in advance.

Other factors are that fills and slippage are not accounted for correctly in most backtests. The strategy may report an order filled at the price determined by the strategy but in reality, the order may not get filled (if limit order) or may slip against the trade by a tick or more thereby invalidating the strategy results. So you are looking at trades that the strategy reports that a trade entry got filled but in reality you may not get the fill or it may report that a trade exit got filled but in reality it may not etc. You could set some backtesting software to report fills only if the entry price is pierced but that leads to the same result - price may not pierce but in reality you could get filled etc.

Visit my NexusFi Trade Journal Reply With Quote
  #5 (permalink)
 
ChrisTinaBruce's Avatar
 ChrisTinaBruce 
Dallas, TX
 
Experience: Intermediate
Platform: TradeStation/Sierra Chart
Broker: Tradestation
Trading: ES
Posts: 66 since Oct 2012
Thanks Given: 56
Thanks Received: 33

Also consider being filled...on Tradestation there is an option to add slippage and a number or percentage of shares before being filled. This would provide a little more real world analysis.


Hulk View Post
Other factors are that fills and slippage are not accounted for correctly in most backtests. The strategy may report an order filled at the price determined by the strategy but in reality, the order may not get filled (if limit order) or may slip against the trade by a tick or more thereby invalidating the strategy results. So you are looking at trades that the strategy reports that a trade entry got filled but in reality you may not get the fill or it may report that a trade exit got filled but in reality it may not etc. You could set some backtesting software to report fills only if the entry price is pierced but that leads to the same result - price may not pierce but in reality you could get filled etc.


Follow me on Twitter Reply With Quote
  #6 (permalink)
sra18376
kansas city, ks
 
Posts: 17 since Jun 2014
Thanks Given: 0
Thanks Received: 5

Thank you for the answers!

Reply With Quote
  #7 (permalink)
 HitTheCity 
Melbourne
 
Experience: Intermediate
Platform: NinjaTrader
Trading: Forex
Posts: 102 since Apr 2012
Thanks Given: 78
Thanks Received: 46

If the backtest was preceded by an optimization process to select strategy parameters, then over fitting is quite likely a reason why backtest results would not be at all reflective of live results.

Reply With Quote
  #8 (permalink)
shanemcdonald28
new york
 
Posts: 355 since Mar 2012
Thanks Given: 665
Thanks Received: 582

Volatility of price action is not recorded in the data used for backtesting. Wild swings up or down can trigger many signals in live trading, but these same signals may not fire at all in backtesting.

Considering this, I do find decent backtesting with Multicharts if you set the option to use tick data. It does not work on specialty bars like renko or line break and other specialty bars. But I do find it to be very good on time and tick based charts.

Reply With Quote
  #9 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,442 since Jun 2009
Thanks Given: 33,215
Thanks Received: 101,603


shanemcdonald28 View Post
Volatility of price action is not recorded in the data used for backtesting. Wild swings up or down can trigger many signals in live trading, but these same signals may not fire at all in backtesting.

Huh?

Sent from my LG Optimus G Pro

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #10 (permalink)
shanemcdonald28
new york
 
Posts: 355 since Mar 2012
Thanks Given: 665
Thanks Received: 582


quote "huh"


Well if the data being used for backtesting only has the open, close, low and high data, then the buy and sell signals will be totally different than live tick by tick data .

In another scenario, a strategy that uses volume rate of change, would not trigger buy and sell signals in backtesting in the same way as live data.

Also, volatility expansion type signals will fire on live data much differently than backtesting.

A macd crossover on a daily chart would not matter much I suppose.

Reply With Quote




Last Updated on July 14, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts