How do you decide when an automated strategy is good enough to take live? - futures io
futures io



How do you decide when an automated strategy is good enough to take live?


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one 303Trader with 7 posts (7 thanks)
    2. looks_two SydAlgoTrader with 4 posts (4 thanks)
    3. looks_3 sixtyseven with 3 posts (6 thanks)
    4. looks_4 TradingOgre with 2 posts (4 thanks)
      Best Posters
    1. looks_one TradingOgre with 2 thanks per post
    2. looks_two sixtyseven with 2 thanks per post
    3. looks_3 303Trader with 1 thanks per post
    4. looks_4 SydAlgoTrader with 1 thanks per post
    1. trending_up 3,360 views
    2. thumb_up 42 thanks given
    3. group 13 followers
    1. forum 29 posts
    2. attach_file 4 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

How do you decide when an automated strategy is good enough to take live?

(login for full post details)
  #1 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received

Hi all. I'm new here and this is my first post. I'm super excited to have found this community!

I've been working on building an automated ES trading strategy for the last several months. I've optimized and backtested and I believe this strategy to be profitable. My goal is to take this strategy live starting in January but I'm getting cold feet. I'd like to gain some confidence from more experienced traders that I'm not fooling myself and I'm making a well informed decision to take this strategy live.

I've optimized my strategy using data from 01/2011 - 12/18. Backtesting using the optimized parameters over the same period I get the results shown in the attached pictures. Even with these results I for some reason have a gut feeling of "this is too good to be true".

With all that in mind here are my two "big picture" questions:

-How do you decide when a strategy is good enough to take live, and what data do you use to determine this?
-Is this simple method of backtesting a reliable qualifier for a strategy? (I'm worried I've simply optimized my strategy for this data set and this won't be representative of how it performs live.)

Thanks for taking the time to read. Any advice is greatly appreciated!



Reply With Quote
The following user says Thank You to 303Trader for this post:

Journal Challenge April 2021 results (now extended!):
Competing for $1800 in prizes from Jigsaw
looks_oneMaking a Living with the Microsby sstheo
(288 thanks from 26 posts)
looks_twoSalao's Journalby Salao
(72 thanks from 13 posts)
looks_3Deetee’s DAX Trading Journal (time based)by Deetee
(50 thanks from 20 posts)
looks_4Learning to Profit - A journey in algorithms and optionsby Syntax
(37 thanks from 15 posts)
looks_5Maybe a little bit different journalby Malykubo
(17 thanks from 18 posts)
 
Best Threads (Most Thanked)
in the last 7 days on futures io
The Crude Dude Oil Trading System
114 thanks
Big Mike in Ecuador
63 thanks
Help improve the FIO community
38 thanks
futures io site changelog and issues/problem reporting
19 thanks
www.isystems.com
15 thanks
 
(login for full post details)
  #3 (permalink)
 TradingOgre 
Evans GA/USA
 
Experience: Intermediate
Platform: NinjaTrader
Broker: NinjaTrader Brokerage - Philip Capital
Trading: NQ,ES,6E,CL
 
TradingOgre's Avatar
 
Posts: 550 since Jul 2018
Thanks: 902 given, 1,646 received


Have you traded it with live data in a sim account yet?
How are you handling potential errors (i.e. loss of connection, loss of power etc)?

Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to TradingOgre for this post:
 
(login for full post details)
  #4 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


TradingOgre View Post
Have you traded it with live data in a sim account yet?
How are you handling potential errors (i.e. loss of connection, loss of power etc)?

I haven't traded it live/with a sim account yet. I had planned on doing this for a month or so to make sure I don't have any outstanding bugs and to make sure my trading platform and brokerage play nice together.

If I lose power/connection I'd either close all positions or continue to monitor any open trades manually from a separate connection. I haven't looked into how to account for this in backtesting.

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #5 (permalink)
 TradingOgre 
Evans GA/USA
 
Experience: Intermediate
Platform: NinjaTrader
Broker: NinjaTrader Brokerage - Philip Capital
Trading: NQ,ES,6E,CL
 
TradingOgre's Avatar
 
Posts: 550 since Jul 2018
Thanks: 902 given, 1,646 received

It seems silly at times but you need to do things like while you are running it on sim, disconnect the ethernet cable or whatever form of internet you are using. Then watch what the code does. In the early days of automated systems I saw one case where this happened on a live account, the software kept attempting to place an order but it wouldn't get confirmation so it just kept placing orders. As soon as the internet came back, all of those pending orders fired off. Wasn't pretty.

I don't think that would happen now but you should test for it to be sure.

Best of luck in automating your system.

Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to TradingOgre for this post:
 
(login for full post details)
  #6 (permalink)
Ozquant
Brisbane Queensland Australia
 
 
Posts: 208 since Aug 2017
Thanks: 151 given, 347 received

Not the sort of expectancy that i would be satisfied with in anyway . I would barely call that an edge . I wouldnt trade it
, thats a lot of trades for not a lot . I set bench marks for WR and expectancy , you dont even come close to them . I am a firm believer a 65% WR is the bar . Keep at it , lots of incremental improvements can make huge improvements over time .

Reply With Quote
The following 3 users say Thank You to Ozquant for this post:
 
(login for full post details)
  #7 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


Ozquant View Post
Not the sort of expectancy that i would be satisfied with in anyway . I would barely call that an edge . I wouldnt trade it
, thats a lot of trades for not a lot . I set bench marks for WR and expectancy , you dont even come close to them . I am a firm believer a 65% WR is the bar . Keep at it , lots of incremental improvements can make huge improvements over time .

This is exactly the kind of stuff I'm looking for. I have no frame of reference when trying to decide if a strategy is worth a damn. I see a positive equity line and get excited when I probably shouldn't be.

When backtesting do you put more weight on WR and expectancy than net profit?

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #8 (permalink)
Ozquant
Brisbane Queensland Australia
 
 
Posts: 208 since Aug 2017
Thanks: 151 given, 347 received


303Trader View Post
This is exactly the kind of stuff I'm looking for. I have no frame of reference when trying to decide if a strategy is worth a damn. I see a positive equity line and get excited when I probably shouldn't be.

When backtesting do you put more weight on WR and expectancy than net profit?

WR and expectancy is net profit really , defines your edge . You havent really given enough detail , its fine to have an equity curve starting from zero but what is the initial capital , then Max DD becomes more relevant in a % sense , Your curve seems to have a lot of gaps in it where it doesnt trade and in times where it clearly is a good time to trade , are you curve fitting ? You havent stated the TF you are trading . Your stats are over 8 years total and trading Emini you starting capital need to be decent , you havent allowed for commision , have you allowed for spread ? Its quite possible buy and hold with divs included might be the same or better return as you strategy . Give more detail and better help will be forthcoming im sure . good luck

Reply With Quote
The following user says Thank You to Ozquant for this post:
 
(login for full post details)
  #9 (permalink)
 iantg 
charlotte nc
 
Experience: Master
Platform: ninjatrader
Broker: NinjaTrader
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 399 since Jan 2015
Thanks: 76 given, 1,089 received

1. It looks like you have data going from 2011 to 2018. So does this imply that over the life of 8 years the system made roughly $15K after commissions? This would be evenly divided equal to roughly $2k per year. Is this what your results are showing or am I missing something? I think you can imagine the answer if the question, "Should I trade a system live that produces $2k per year in back testing." But maybe I am missing how your results are displayed.

2. It looks like you are almost always only taking long positions. Is this by design or is something in your code misfiring?

3. What exactly was your testing on? Strategy Analyzer, Market Replay, Live SIM, etc.? Each of these have levels of accuracy that can impact your results.

4. What is the latency sensitivity of your alpha signal? Some strategies test well, but in live trading speed crushes them. Some signals you can catch from home on a laptop over the internet (Market Orders with a long term trade), Some you would need a VPN / VPS with: Limit order / speed sensitive strategies. And some signals you can't reach in time with NinjaTrader no matter what you do.


In order to test a realistic live trading scenario with NT you should do the following.

Run your test with Market Replay, and put your code in the level 1 data event handler. (OnMarketData). The OnBarUpdate front runs the Level 1 and Level 2 feed and can give you a glimpse into the future if your alpha signals are derived from the level 1 feed. No that Market Replay assumes 0 latency. If you want to really simulate live trading types of latency, you can build in some latency in your code yourself before you release your order.

If you can get passed market replay, then move onto Live SIM. The simulate around 200 MS of latency and the live feed is the live feed. Here you can use the OnBarUpdate to get back some of the lag from your interenet + broker to exchange latency.

Also put in your commissions.... You might be okay based on what I saw, but once you have realistic latency to measure against, who knows.

If you can pass these benchmarks and have a positive expectancy you should be fine.

Ian

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to iantg for this post:
 
(login for full post details)
  #10 (permalink)
 OldGerman 
Legendary Market Wizard
Münster Germany
 
Experience: Intermediate
Platform: NinjaTrader, ProRealtime
Broker: NinjaTrader Brokerage
Trading: FDXM, FDAX, ES, YM, 6B and 6E
 
OldGerman's Avatar
 
Posts: 588 since Mar 2015
Thanks: 656 given, 1,289 received


I always look at the following parameters:
avr trade, avr loosing trade, max cons. loosers, max drawdown and trades per year. ....
and then I calculate the time period in which the system eliminates a drawdown again. For my sense, a drawdown should be earned about 3 times a year. That's not what your system does. Sorry.

If I look at your equity, it looks like the trades will be held for a longer period of time. This requires a VPS. See Speedytradingservers.com in this forum or look at Google.

I wish you success.

The biggest risk you can ever take is not betting on yourself!
(Bill Williams +)
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to OldGerman for this post:
 
(login for full post details)
  #11 (permalink)
 kanepa 
philadelphia pa
 
Experience: Intermediate
Platform: ninja
Broker: NinjaTrader Brokerage
Trading: es
 
Posts: 179 since Jul 2017
Thanks: 330 given, 365 received

Hi. You can check out my journal. I started system trading from summer of 2017 and you could see up and downs of my system.

1. Commission. You need to add them into calculation.
2. Add 1 slippage for every trade to see if it survives.
3. Need to go back at least 08 and 09 to see if system survived and profitable in bear market.


-How do you decide when a strategy is good enough to take live, and what data do you use to determine this?

at least 100% return annually on historic max DD then forwarding test for a month.

-Is this simple method of backtesting a reliable qualifier for a strategy? (I'm worried I've simply optimized my strategy for this data set and this won't be representative of how it performs live.)

No. I would not trade those numbers at this time. but please feel free to forward test it with SIM.

Hope you find a good strategy. Good luck!

Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to kanepa for this post:
 
(login for full post details)
  #12 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received

Thanks Ian. See my responses in red.


iantg View Post
1. It looks like you have data going from 2011 to 2018. So does this imply that over the life of 8 years the system made roughly $15K after commissions? This would be evenly divided equal to roughly $2k per year. Is this what your results are showing or am I missing something? I think you can imagine the answer if the question, "Should I trade a system live that produces $2k per year in back testing." But maybe I am missing how your results are displayed.
No, you're not missing anything. That's what the system is showing. What do you consider to be a reasonable and obtainable goal for yearly profit, trading one contract?


2. It looks like you are almost always only taking long positions. Is this by design or is something in your code misfiring?
Nothing misfiring. After running the Strategy Analyzer it looks like my entry criteria are only effective for long entries.

3. What exactly was your testing on? Strategy Analyzer, Market Replay, Live SIM, etc.? Each of these have levels of accuracy that can impact your results.
I have only tested using Strategy Analyzer. Clearly this isn't enough.

4. What is the latency sensitivity of your alpha signal? Some strategies test well, but in live trading speed crushes them. Some signals you can catch from home on a laptop over the internet (Market Orders with a long term trade), Some you would need a VPN / VPS with: Limit order / speed sensitive strategies. And some signals you can't reach in time with NinjaTrader no matter what you do.
I'm hoping this won't be a problem for my strategy. I'm trading based on a five minute chart.

In order to test a realistic live trading scenario with NT you should do the following.

Run your test with Market Replay, and put your code in the level 1 data event handler. (OnMarketData). The OnBarUpdate front runs the Level 1 and Level 2 feed and can give you a glimpse into the future if your alpha signals are derived from the level 1 feed. No that Market Replay assumes 0 latency. If you want to really simulate live trading types of latency, you can build in some latency in your code yourself before you release your order.

If you can get passed market replay, then move onto Live SIM. The simulate around 200 MS of latency and the live feed is the live feed. Here you can use the OnBarUpdate to get back some of the lag from your interenet + broker to exchange latency.

Also put in your commissions.... You might be okay based on what I saw, but once you have realistic latency to measure against, who knows.

If you can pass these benchmarks and have a positive expectancy you should be fine.

This is excellent info and gives me something to shoot for. Thank you!!

Ian



OldGerman View Post
If I look at your equity, it looks like the trades will be held for a longer period of time. This requires a VPS. See Speedytradingservers.com in this forum or look at Google.

Could you elaborate on this? I would think this would be something that's necessary for very short time frame trading (my avg. time in market is 93minutes). I must be missing something.



kanepa View Post
Hi. You can check out my journal. I started system trading from summer of 2017 and you could see up and downs of my system.

1. Commission. You need to add them into calculation.
2. Add 1 slippage for every trade to see if it survives.
3. Need to go back at least 08 and 09 to see if system survived and profitable in bear market.

I'll definitely take a look at your journal. FYI, If I add 1 slippage my net profit decreases by ~15%. Yikes.

Thank you everyone for your input. I was hoping to come to this forum and be humbled. It seems to be working.

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #13 (permalink)
 OldGerman 
Legendary Market Wizard
Münster Germany
 
Experience: Intermediate
Platform: NinjaTrader, ProRealtime
Broker: NinjaTrader Brokerage
Trading: FDXM, FDAX, ES, YM, 6B and 6E
 
OldGerman's Avatar
 
Posts: 588 since Mar 2015
Thanks: 656 given, 1,289 received


303Trader View Post
Could you elaborate on this? I would think this would be something that's necessary for very short time frame trading (my avg. time in market is 93minutes). I must be missing something.

I use my VPS because I can not secure the internet connection for a long time.
VPS also offer a very low latency. But that should not be so important on most systems.

The biggest risk you can ever take is not betting on yourself!
(Bill Williams +)
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to OldGerman for this post:
 
(login for full post details)
  #14 (permalink)
 centaurer 
south africa
 
 
Posts: 169 since Dec 2018

A strategy doesn't have to go from $0 to 2X your networth on margin.

I am interested in getting a strategy to real money ASAP because it is like a bug, it is not going to have that long of a life anyway.

As it performs better it gets more money to trade and as it performs worse it gets less until it dies.

I do have a real problem with optimization in this context though. All that optimization is doing is more finely tuning the model to conditions in the past that we know will not hold tomorrow. You are really optimizing the brittleness of the model most likely.

What I want to see from my models is that the strategy does not completely fall apart when used on another highly correlated time series.

If something blows the doors off ES but gets killed on SPY it is far more likely your model is simply over fitted to a random realization of the time series for ES than capturing and exploiting a structure of ES that is not part of SPY when we know the time series has the same generating process. Then I want to see SPY vs QQQ, NQ vs ES, YM vs SPY, etc..

You also need to look at as a baseline at the profit/loss if you had just gone long SPY and held since 2011 then normalize the leverage in ES. If you are not beating that then what is the point of trading. A system is not good because it is profitable, a trading system is good because it beats by and hold over the same period.

Don't forget too a purely random bad strategy is going to be between 40%-60% win/loss rate. I mean I would love to find a strategy that loses 95% of the time so I can just flip things. I agree with Ozquant 65% rule of thumb although I am ok with 60%. You need to give yourself a model error/noise cushion to stay above pure randomness.

Reply With Quote
 
(login for full post details)
  #15 (permalink)
 bchip 
Africa
 
Experience: Advanced
Platform: TradeStation
Trading: ES,YM,CL,GC
 
Posts: 98 since Sep 2017
Thanks: 136 given, 76 received


303Trader View Post
Hi all. I'm new here and this is my first post. I'm super excited to have found this community!

I've been working on building an automated ES trading strategy for the last several months. I've optimized and backtested and I believe this strategy to be profitable. My goal is to take this strategy live starting in January but I'm getting cold feet. I'd like to gain some confidence from more experienced traders that I'm not fooling myself and I'm making a well informed decision to take this strategy live.

I've optimized my strategy using data from 01/2011 - 12/18. Backtesting using the optimized parameters over the same period I get the results shown in the attached pictures.

Something I noticed is that you used the full historical data set for optimization?
That would just be curve fitting the rules.

My 2c is that when you test a system, you must always test against out-of-sample data.

Maybe read Kevin Davey's book on "Building an algo", definitely worth a read.

Reply With Quote
The following 3 users say Thank You to bchip for this post:
 
(login for full post details)
  #16 (permalink)
 FlyingMonkey 
Market Wizard
Los Angeles CA
 
Experience: Beginner
Platform: NT8
Trading: NQ YM ES ZN CL GC HG ZS 6E 6A 6J 6C
 
FlyingMonkey's Avatar
 
Posts: 654 since Jan 2016
Thanks: 2,968 given, 2,282 received


bchip View Post
My 2c is that when you test a system, you must always test against out-of-sample data.

This. Building a strategy that optimizes well and gives you a great back test is easy (and I agree with previous posters, your result is not yet a great backtest). Building an edge that tests well on out of sample data using a walk-forward test is difficult. And a test that performs well on live data (in sim) is the final step before unleashing on real money.

Give him the stick.

Don't give him the stick!
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to FlyingMonkey for this post:
 
(login for full post details)
  #17 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


bchip View Post
Maybe read Kevin Davey's book on "Building an algo", definitely worth a read.

I just finished reading Kevin's book a few weeks ago. It definitely put things in perspective and answered many of my questions.


FlyingMonkey View Post
This. Building a strategy that optimizes well and gives you a great back test is easy (and I agree with previous posters, your result is not yet a great back-test). Building an edge that tests well on out of sample data using a walk-forward test is difficult. And a test that performs well on live data (in sim) is the final step before unleashing on real money.

I'm finding this to be very, very true. Since my original post I've increased back-test profitability and win rate substantially. However, as soon as I move to out of sample data it falls apart. Furthermore, I've been live sim trading this strategy since the beginning of the year (I figured why not?) and it's doing extremely well! It's infuriating and fascinating all at the same time.

I'm still convinced that my entry signal can be reliable. The two things I'm really struggling with is how to scale my entry parameters to match market volatility and how to let my winning trades run. I'm also still baffled on why this strategy seems to hate short trades. If I can get these things figured out maybe I'll have something.



Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #18 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
 
Posts: 186 since May 2012
Thanks: 851 given, 336 received


303Trader View Post
Since my original post I've increased back-test profitability and win rate substantially. However, as soon as I move to out of sample data it falls apart.

It wasn't clear to me if you are still working on the same basic strategy, or whether you have moved on.

But just in case - if you test the in-sample against the OOS, modify and re-test then the OOS effectively becomes in-sample. You should only peak at the OOS once. If it doesn't work, then start again from scratch.

The advantage of 'algos', is that you can have a lot of them, which keeps the individual bet size low, which lowers max drawdown. Since you need a lot of them, you need to develop a process so you can churn them out. You'll be relatively indifferent if it works or not. It should be expected that most won't work. Keep in mind, you can get decent OOS results, which in reality is still random noise. That's another reason you want multiple systems.

Reply With Quote
The following 4 users say Thank You to sixtyseven for this post:
 
(login for full post details)
  #19 (permalink)
 centaurer 
south africa
 
 
Posts: 169 since Dec 2018

That PNL is obviously over fitted. It is not reasonable that a system 20 bags and has a life span of that long. Of course falling apart out of sample is a classic symptom of over fitting.

IMO testing back to 2011 just makes no sense in general.

At the most I would use something like whatever the data providers back fill on ticks is, this is your DB. View it as a rolling window that the stale data falls out of. To me that kills two birds with one stone as far as data curation and dumping stale data.

If you start going back longer IMO you need some kind of regime switching classifier.

Reply With Quote
 
(login for full post details)
  #20 (permalink)
 SidewalkAerobics 
Los Angels
 
Experience: Intermediate
Platform: MultiChart
Trading: Emini ES
 
Posts: 115 since Aug 2018
Thanks: 173 given, 71 received


centaurer View Post
That PNL is obviously over fitted. It is not reasonable that a system 20 bags and has a life span of that long. Of course falling apart out of sample is a classic symptom of over fitting.

IMO testing back to 2011 just makes no sense in general.

At the most I would use something like whatever the data providers back fill on ticks is, this is your DB. View it as a rolling window that the stale data falls out of. To me that kills two birds with one stone as far as data curation and dumping stale data.

If you start going back longer IMO you need some kind of regime switching classifier.



Centaurer, what is "out of sample data"? I use TradeStation and don't see it there. Any help on this would be appreciated.

Reply With Quote
The following user says Thank You to SidewalkAerobics for this post:
 
(login for full post details)
  #21 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


SidewalkAerobics View Post
Centaurer, what is "out of sample data"? I use TradeStation and don't see it there. Any help on this would be appreciated.

Out of sample data is any data which is not used to develop and optimize a strategy. I'm not familiar with TradeStation, but in NT you can select a date range when running optimization. OOS data would be anything outside of the date range being used to optimize.

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #22 (permalink)
SydAlgoTrader
Sydney new south Wales
 
 
Posts: 15 since Sep 2018
Thanks: 0 given, 10 received

Typically for ES I want a minimum average trade profit of $50 (after 2 ticks slippage and commission if using market orders).

The short side seems pointless, why not just make it a long only strategy?
Nice amount of trades, but usually if you can get more trades without sacrificing average trade profit thats better (more statistically significant).

Personally I wouldn't trade this strategy, and if I did it would be in sim for 6 months. The equity curve's correlation coefficient isn't high enough for me.

Reply With Quote
The following user says Thank You to SydAlgoTrader for this post:
 
(login for full post details)
  #23 (permalink)
303Trader
Denver CO
 
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


SydAlgoTrader View Post
Typically for ES I want a minimum average trade profit of $50 (after 2 ticks slippage and commission if using market orders).

The short side seems pointless, why not just make it a long only strategy?

Agreed. This backtest was from a while back. I've since killed the shorts.


SydAlgoTrader View Post
Personally I wouldn't trade this strategy, and if I did it would be in sim for 6 months. The equity curve's correlation coefficient isn't high enough for me.

Can you elaborate on this a little? The correlation coefficient with what? The underlying index?

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
(login for full post details)
  #24 (permalink)
SydAlgoTrader
Sydney new south Wales
 
 
Posts: 15 since Sep 2018
Thanks: 0 given, 10 received


303Trader View Post

Can you elaborate on this a little? The correlation coefficient with what? The underlying index?


The correlation coefficient would be compared to the "ideal equity curve" (straight line between start and end of the equity curve)

higher correlation coefficient between the two = straighter equity curve.

Reply With Quote
The following user says Thank You to SydAlgoTrader for this post:
 
(login for full post details)
  #25 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
 
Posts: 186 since May 2012
Thanks: 851 given, 336 received


SydAlgoTrader View Post
The correlation coefficient would be compared to the "ideal equity curve" (straight line between start and end of the equity curve)

higher correlation coefficient between the two = straighter equity curve.

A system that trades all the time - i.e not regime dependant, but outperforms in certain regimes, and underperforms in others (while still being good enough to not try and filter), would have a poor correlation coefficient. Likewise, a system that dribbles along in a near perfect linear line may not be worth trading as it would tie up trading margin for not much return.

I agree that it would be lovely to have a high correlation, but I wouldn't put much weight in this metric.

Reply With Quote
The following user says Thank You to sixtyseven for this post:
 
(login for full post details)
  #26 (permalink)
 cutzpr 
United States
 
Experience: None
Platform: MT4,TWS,Ninja Trader
Trading: Forex
 
cutzpr's Avatar
 
Posts: 35 since Apr 2012
Thanks: 10 given, 10 received

Just for a warm and fuzzy, I would compare a sample set of timeusing high order fill resolution (essentially places the trade on a 1 tick data series. Run the strategy normally for lets say the last 12 months, then rerun using high fill. Running the strategy on High fill from 2011 will take a long time. Compare the two results and if you are not getting similar results, then it might be back to the drawing board.

Reply With Quote
The following user says Thank You to cutzpr for this post:
 
(login for full post details)
  #27 (permalink)
SydAlgoTrader
Sydney new south Wales
 
 
Posts: 15 since Sep 2018
Thanks: 0 given, 10 received


sixtyseven View Post
A system that trades all the time - i.e not regime dependant, but outperforms in certain regimes, and underperforms in others (while still being good enough to not try and filter), would have a poor correlation coefficient. Likewise, a system that dribbles along in a near perfect linear line may not be worth trading as it would tie up trading margin for not much return.

I completely disagree. To give an example just 2 days ago I was sorting through some short ES strategies with equity curves back to 2004 and correlation coefficients in excess of 0.95, strategies which are regime dependent (i.e do well in 2018, 2008 etc, but have marginal return in big bull years.

Anything with a CorrCoef of 0.99 say in back test is probably not going to survive live trading through. It'just a number like Sharpe.

Reply With Quote
The following user says Thank You to SydAlgoTrader for this post:
 
(login for full post details)
  #28 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
 
Posts: 186 since May 2012
Thanks: 851 given, 336 received


SydAlgoTrader View Post
I completely disagree. To give an example just 2 days ago I was sorting through some short ES strategies with equity curves back to 2004 and correlation coefficients in excess of 0.95, strategies which are regime dependent (i.e do well in 2018, 2008 etc, but have marginal return in big bull years.

Anything with a CorrCoef of 0.99 say in back test is probably not going to survive live trading through. It'just a number like Sharpe.

You are quite right. I was poor at visualising equity curves vs correl. It's not something that I use.

I did just plot some out in excel now to get a feel for it, and it wasn't obvious that it's a useful metric. 50%+ drawdowns would still end up with >0.95 for example. I'll stfu now, as I'll likely not add anything useful given my obvious lack of experience with this metric.

Reply With Quote
The following user says Thank You to sixtyseven for this post:
 
(login for full post details)
  #29 (permalink)
SydAlgoTrader
Sydney new south Wales
 
 
Posts: 15 since Sep 2018
Thanks: 0 given, 10 received


sixtyseven View Post
You are quite right. I was poor at visualising equity curves vs correl. It's not something that I use.

I did just plot some out in excel now to get a feel for it, and it wasn't obvious that it's a useful metric. 50%+ drawdowns would still end up with >0.95 for example. I'll stfu now, as I'll likely not add anything useful given my obvious lack of experience with this metric.

Nah, metrics are metrics. For example strategies with nice sharpe ratios sometimes don't function IRL as they do in backtest. I obviously would use a combination of factors.
Personally I like NetProfit/MaxDD, but everyones got a method.

Reply With Quote
The following user says Thank You to SydAlgoTrader for this post:
 
(login for full post details)
  #30 (permalink)
 kjhosken 
Seattle, WA/USA
 
Experience: Intermediate
Platform: TOS, TS
Trading: Forex, crude
 
Posts: 90 since Sep 2016
Thanks: 7 given, 32 received


SidewalkAerobics View Post
Centaurer, what is "out of sample data"? I use TradeStation and don't see it there. Any help on this would be appreciated.

In TS you can find OOS when you optimize parameters. It is only available there since everything is OOS for a naive strategy. After selecting your optimization range it will give you options for standard or walk-forward as well as exhaustive or genetic. There is also a button for optimization settings. In that dialogue you can select your OOS %, date range, fiddle with the genetic algo inputs, etc

Follow me on Twitter Reply With Quote


futures io Trading Community Traders Hideout Emini and Emicro Index > How do you decide when an automated strategy is good enough to take live?


Last Updated on February 18, 2019


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Journal Challenge w/$1,800 in prizes!

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts