How can a strategy be profitable when backtested but not when actually used? - futures io
futures io futures trading



How can a strategy be profitable when backtested but not when actually used?


Discussion in NinjaTrader

Updated by shanemcdonald28
      Top Posters
    1. looks_one Big Mike with 5 posts (3 thanks)
    2. looks_two shanemcdonald28 with 5 posts (0 thanks)
    3. looks_3 kevinkdog with 4 posts (9 thanks)
    4. looks_4 wcandrews with 2 posts (4 thanks)
      Best Posters
    1. looks_one artemiso with 4.0 thanks per post
    2. looks_two kevinkdog with 2.3 thanks per post
    3. looks_3 wcandrews with 2.0 thanks per post
    4. looks_4 Big Mike with 0.6 thanks per post
    1. trending_up 4,029 views
    2. thumb_up 22 thanks given
    3. group 10 followers
    1. forum 27 replies
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 100,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

How can a strategy be profitable when backtested but not when actually used?

(login for full post details)
  #21 (permalink)
Russellville, AR
 
Experience: Advanced
Platform: NinjaTrader
Broker: AMP/NinjaTrader/CQG/Kinetick/Tickdata, Inc.
Trading: ES. ZB
 
Posts: 7 since Jul 2010
Thanks: 4 given, 8 received

No mechanical system--totally objective, will be profitable in the future.
They may "work" for a while and you'll think you are going to be rich.
(They would be profitable for longer in the past--1980s and 90s.)
Think that's not true? Then name one.
By the way, why are so many systems offered for sale versus actually trading them and getting rich?

Trading system analysis: Then and now

Excalibur Testing Software by Futures Truth was one of the first full-featured backtesting platforms. The software development process started in 1985 on a CROMEMCO 3 computer utilizing FORTRAN IV. The computer had a whopping 64k of RAM and used 5-1/4-inch floppy drives. Trader John Hill spent nearly $20,000 for the computer and hard drive ó an enormous sum for a machine with the fraction of the computing power available today for a few hundred bucks.

Wayne Andrews, a close friend of Hillís and a computer scientist, was instrumental in helping Hill acquire the CROMEMCO, as well as providing his own personal tick level price data for Hill to use.


I do have a little experience.

Reply With Quote
The following 2 users say Thank You to wcandrews for this post:

Can you help answer these questions
from other members on futures io?
is there an easy way to obtain by code the price variation percentual
MultiCharts
Looking freelancer for MT4 Indicator
Platforms and Indicators
Do large speculators buy from commercials?
Traders Hideout
GBP/ARS Pair
Traders Hideout
Horizontal Line Indicator
Platforms and Indicators
 
Best Threads (Most Thanked)
in the last 7 days on futures io
Legal question and need desperate help
187 thanks
Building a high-performance data system
18 thanks
FIO Journal Challenge - June 2020 w/SharkIndicators
15 thanks
VWAP for stock index futures trading?
11 thanks
Is Orderflow An Outdated Concept?
11 thanks
 
(login for full post details)
  #22 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,791 since Jul 2012
Thanks: 1,439 given, 5,380 received


wcandrews View Post
No mechanical system--totally objective, will be profitable in the future.
They may "work" for a while and you'll think you are going to be rich.
(They would be profitable for longer in the past--1980s and 90s.)
Think that's not true? Then name one.
By the way, why are so many systems offered for sale versus actually trading them and getting rich?

Trading system analysis: Then and now

Excalibur Testing Software by Futures Truth was one of the first full-featured backtesting platforms. The software development process started in 1985 on a CROMEMCO 3 computer utilizing FORTRAN IV. The computer had a whopping 64k of RAM and used 5-1/4-inch floppy drives. Trader John Hill spent nearly $20,000 for the computer and hard drive — an enormous sum for a machine with the fraction of the computing power available today for a few hundred bucks.

Wayne Andrews, a close friend of Hill’s and a computer scientist, was instrumental in helping Hill acquire the CROMEMCO, as well as providing his own personal tick level price data for Hill to use.


I do have a little experience.


I'd agree completely with your first sentence, if you add the word "forever" to the end of it. Mechanical systems can work for a good amount of time before they don't.

Your point is a very good one - traders should assume that at some point their strategy will stop working, and build risk management / position sizing etc. with that in mind. That way, they will avoid being wiped out by a strategy that stops working.

Some of the worst blowouts I have heard about were from traders who stubbornly held on to a method, when the market was telling them the method no longer worked.

Follow me on Twitter Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #23 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,791 since Jul 2012
Thanks: 1,439 given, 5,380 received



wcandrews View Post
By the way, why are so many systems offered for sale versus actually trading them and getting rich?

I believe it is because most are developed the wrong way. Too many rules, too much optimizing, improper data sampling and data mining, improper accounting of slippage and commission, bad assumptions about order fills,etc. Garbage In, Garbage Out as they say.


The end result is a great backtest that can sell the system to the unwary public, but will never work in the real world.

Follow me on Twitter Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #24 (permalink)
Russellville, AR
 
Experience: Advanced
Platform: NinjaTrader
Broker: AMP/NinjaTrader/CQG/Kinetick/Tickdata, Inc.
Trading: ES. ZB
 
Posts: 7 since Jul 2010
Thanks: 4 given, 8 received

"The end result is a great backtest that can sell the system to the unwary public, but will never work in the real world. "

Very true. Years ago there used to be a "Club 3000." Systems used to cost $3000. Big Mike probably knows what I'm
talking about. The club critiqued the various scams, oops, I meant systems.

I am 72 and several years ago I finally figured out the "secret"--it's all about money (and also power).

Apparently there is an endless supply of "marks."

Sorry for the cynicism.

Reply With Quote
The following 2 users say Thank You to wcandrews for this post:
 
(login for full post details)
  #25 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,791 since Jul 2012
Thanks: 1,439 given, 5,380 received


wcandrews View Post
"The end result is a great backtest that can sell the system to the unwary public, but will never work in the real world. "

Very true. Years ago there used to be a "Club 3000." Systems used to cost $3000. Big Mike probably knows what I'm
talking about. The club critiqued the various scams, oops, I meant systems.

I am 72 and several years ago I finally figured out the "secret"--it's all about money (and also power).

Apparently there is an endless supply of "marks."

Sorry for the cynicism.

Your cynicism is VERY well founded!

Follow me on Twitter Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #26 (permalink)
Site Administrator
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 49,035 since Jun 2009
Thanks: 31,651 given, 94,882 received


wcandrews View Post
Apparently there is an endless supply of "marks."

Because there is an endless supply of greed. On both the consumer end and seller side.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my Facebook Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #27 (permalink)
Market Wizard
New York, NY
 
Experience: Beginner
Platform: Vanguard 401k
Broker: Yahoo Finance
Trading: Mutual funds
 
Posts: 1,076 since Jul 2012
Thanks: 739 given, 2,269 received


kevinkdog View Post
There are ways to trick the backtest software. Also, if you heavily optimize, your results will never be the same going forward.

For what my view is worth, I think in the context of the poster's question, this is the most accurate answer. The issues experienced lie mostly with the software.

Reply With Quote
The following 4 users say Thank You to artemiso for this post:
 
(login for full post details)
  #28 (permalink)
new york
 
 
Posts: 355 since Mar 2012
Thanks: 665 given, 580 received


Big Mike View Post
I am saying ...


My point being, I don't agree with anything you've said. Now anyone that reads my posts know that I routinely talk about all the pitfalls of backtesting, so don't misconstrue my meaning. But what you've described here is not accurate, not unless you are purposely making it inaccurate by using minute or daily data.

Mike

I am not purposely making anything inaccurate.

These are situations I have observed during live trading.
These are conclusions I have come to after comparing backtesting with live results. I observed more slippage at certain times than others.
The exchanges use sophisticated algorithms to fill all of the orders and this can push the orders around wildly in the lineup.

Do you think tick data being backtested demonstrates accurately how those orders are being filled ?
I dont think it does.

Reply With Quote


futures io Trading Community Platforms and Indicators NinjaTrader > How can a strategy be profitable when backtested but not when actually used?


July 14, 2014


Upcoming Webinars and Events
 

Getting Comfortable with Price Action Patterns w/Earn2Trade

Aug 4
 

Live Video Panel: Ask Me Anything w/Ironbeam Brokerage

Aug 6
     



Copyright © 2020 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts