NexusFi: Find Your Edge


Home Menu

 





How do you decide when an automated strategy is good enough to take live?


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one 303Trader with 7 posts (7 thanks)
    2. looks_two SydAlgoTrader with 4 posts (4 thanks)
    3. looks_3 sixtyseven with 3 posts (6 thanks)
    4. looks_4 OldGerman with 2 posts (2 thanks)
      Best Posters
    1. looks_one TradingOgre with 2 thanks per post
    2. looks_two sixtyseven with 2 thanks per post
    3. looks_3 303Trader with 1 thanks per post
    4. looks_4 SydAlgoTrader with 1 thanks per post
    1. trending_up 5,294 views
    2. thumb_up 42 thanks given
    3. group 12 followers
    1. forum 29 posts
    2. attach_file 4 attachments




 
Search this Thread

How do you decide when an automated strategy is good enough to take live?

  #21 (permalink)
303Trader
Denver CO
 
Posts: 7 since Dec 2018
Thanks Given: 3
Thanks Received: 7


SidewalkAerobics View Post
Centaurer, what is "out of sample data"? I use TradeStation and don't see it there. Any help on this would be appreciated.

Out of sample data is any data which is not used to develop and optimize a strategy. I'm not familiar with TradeStation, but in NT you can select a date range when running optimization. OOS data would be anything outside of the date range being used to optimize.

Reply With Quote
Thanked by:

Can you help answer these questions
from other members on NexusFi?
Futures True Range Report
The Elite Circle
Deepmoney LLM
Elite Quantitative GenAI/LLM
Exit Strategy
NinjaTrader
ZombieSqueeze
Platforms and Indicators
Are there any eval firms that allow you to sink to your …
Traders Hideout
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
GFIs1 1 DAX trade per day journal
17 thanks
  #22 (permalink)
SydAlgoTrader
Sydney new south Wales
 
Posts: 15 since Sep 2018
Thanks Given: 0
Thanks Received: 10

Typically for ES I want a minimum average trade profit of $50 (after 2 ticks slippage and commission if using market orders).

The short side seems pointless, why not just make it a long only strategy?
Nice amount of trades, but usually if you can get more trades without sacrificing average trade profit thats better (more statistically significant).

Personally I wouldn't trade this strategy, and if I did it would be in sim for 6 months. The equity curve's correlation coefficient isn't high enough for me.

Reply With Quote
Thanked by:
  #23 (permalink)
303Trader
Denver CO
 
Posts: 7 since Dec 2018
Thanks Given: 3
Thanks Received: 7



SydAlgoTrader View Post
Typically for ES I want a minimum average trade profit of $50 (after 2 ticks slippage and commission if using market orders).

The short side seems pointless, why not just make it a long only strategy?

Agreed. This backtest was from a while back. I've since killed the shorts.


SydAlgoTrader View Post
Personally I wouldn't trade this strategy, and if I did it would be in sim for 6 months. The equity curve's correlation coefficient isn't high enough for me.

Can you elaborate on this a little? The correlation coefficient with what? The underlying index?

Reply With Quote
Thanked by:
  #24 (permalink)
SydAlgoTrader
Sydney new south Wales
 
Posts: 15 since Sep 2018
Thanks Given: 0
Thanks Received: 10


303Trader View Post

Can you elaborate on this a little? The correlation coefficient with what? The underlying index?


The correlation coefficient would be compared to the "ideal equity curve" (straight line between start and end of the equity curve)

higher correlation coefficient between the two = straighter equity curve.

Reply With Quote
Thanked by:
  #25 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


SydAlgoTrader View Post
The correlation coefficient would be compared to the "ideal equity curve" (straight line between start and end of the equity curve)

higher correlation coefficient between the two = straighter equity curve.

A system that trades all the time - i.e not regime dependant, but outperforms in certain regimes, and underperforms in others (while still being good enough to not try and filter), would have a poor correlation coefficient. Likewise, a system that dribbles along in a near perfect linear line may not be worth trading as it would tie up trading margin for not much return.

I agree that it would be lovely to have a high correlation, but I wouldn't put much weight in this metric.

Reply With Quote
Thanked by:
  #26 (permalink)
 
cutzpr's Avatar
 cutzpr 
United States
 
Experience: None
Platform: TWS,Ninja Trader
Trading: Forex, Futures, Stocks
Posts: 35 since Apr 2012
Thanks Given: 10
Thanks Received: 10

Just for a warm and fuzzy, I would compare a sample set of timeusing high order fill resolution (essentially places the trade on a 1 tick data series. Run the strategy normally for lets say the last 12 months, then rerun using high fill. Running the strategy on High fill from 2011 will take a long time. Compare the two results and if you are not getting similar results, then it might be back to the drawing board.

Reply With Quote
Thanked by:
  #27 (permalink)
SydAlgoTrader
Sydney new south Wales
 
Posts: 15 since Sep 2018
Thanks Given: 0
Thanks Received: 10


sixtyseven View Post
A system that trades all the time - i.e not regime dependant, but outperforms in certain regimes, and underperforms in others (while still being good enough to not try and filter), would have a poor correlation coefficient. Likewise, a system that dribbles along in a near perfect linear line may not be worth trading as it would tie up trading margin for not much return.

I completely disagree. To give an example just 2 days ago I was sorting through some short ES strategies with equity curves back to 2004 and correlation coefficients in excess of 0.95, strategies which are regime dependent (i.e do well in 2018, 2008 etc, but have marginal return in big bull years.

Anything with a CorrCoef of 0.99 say in back test is probably not going to survive live trading through. It'just a number like Sharpe.

Reply With Quote
Thanked by:
  #28 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


SydAlgoTrader View Post
I completely disagree. To give an example just 2 days ago I was sorting through some short ES strategies with equity curves back to 2004 and correlation coefficients in excess of 0.95, strategies which are regime dependent (i.e do well in 2018, 2008 etc, but have marginal return in big bull years.

Anything with a CorrCoef of 0.99 say in back test is probably not going to survive live trading through. It'just a number like Sharpe.

You are quite right. I was poor at visualising equity curves vs correl. It's not something that I use.

I did just plot some out in excel now to get a feel for it, and it wasn't obvious that it's a useful metric. 50%+ drawdowns would still end up with >0.95 for example. I'll stfu now, as I'll likely not add anything useful given my obvious lack of experience with this metric.

Reply With Quote
Thanked by:
  #29 (permalink)
SydAlgoTrader
Sydney new south Wales
 
Posts: 15 since Sep 2018
Thanks Given: 0
Thanks Received: 10


sixtyseven View Post
You are quite right. I was poor at visualising equity curves vs correl. It's not something that I use.

I did just plot some out in excel now to get a feel for it, and it wasn't obvious that it's a useful metric. 50%+ drawdowns would still end up with >0.95 for example. I'll stfu now, as I'll likely not add anything useful given my obvious lack of experience with this metric.

Nah, metrics are metrics. For example strategies with nice sharpe ratios sometimes don't function IRL as they do in backtest. I obviously would use a combination of factors.
Personally I like NetProfit/MaxDD, but everyones got a method.

Reply With Quote
Thanked by:
  #30 (permalink)
 kjhosken 
Seattle, WA/USA
 
Experience: Intermediate
Platform: TOS, TS
Trading: Forex, crude
Posts: 96 since Sep 2016
Thanks Given: 7
Thanks Received: 35



SidewalkAerobics View Post
Centaurer, what is "out of sample data"? I use TradeStation and don't see it there. Any help on this would be appreciated.

In TS you can find OOS when you optimize parameters. It is only available there since everything is OOS for a naive strategy. After selecting your optimization range it will give you options for standard or walk-forward as well as exhaustive or genetic. There is also a button for optimization settings. In that dialogue you can select your OOS %, date range, fiddle with the genetic algo inputs, etc

Follow me on Twitter Reply With Quote




Last Updated on February 18, 2019


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts