How do you decide when an automated strategy is good enough to take live? - Emini Index Futures Trading | futures io social day trading
futures io futures trading


How do you decide when an automated strategy is good enough to take live?
Updated: Views / Replies:1,193 / 31
Created: by 303Trader Attachments:4

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 100,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 4  
 
Thread Tools Search this Thread
 

How do you decide when an automated strategy is good enough to take live?

  #11 (permalink)
Elite Member
philadelphia pa
 
Trading Experience: Intermediate
Platform: ninja
Broker/Data: NinjaTrader Brokerage
Favorite Futures: es
 
Posts: 123 since Jul 2017
Thanks: 296 given, 261 received

thought

Hi. You can check out my journal. I started system trading from summer of 2017 and you could see up and downs of my system.

1. Commission. You need to add them into calculation.
2. Add 1 slippage for every trade to see if it survives.
3. Need to go back at least 08 and 09 to see if system survived and profitable in bear market.


-How do you decide when a strategy is good enough to take live, and what data do you use to determine this?

at least 100% return annually on historic max DD then forwarding test for a month.

-Is this simple method of backtesting a reliable qualifier for a strategy? (I'm worried I've simply optimized my strategy for this data set and this won't be representative of how it performs live.)

No. I would not trade those numbers at this time. but please feel free to forward test it with SIM.

Hope you find a good strategy. Good luck!


Last edited by kanepa; December 23rd, 2018 at 02:00 PM. Reason: adding to a comment
Reply With Quote
The following user says Thank You to kanepa for this post:
 
  #12 (permalink)
Trading Apprentice
Denver CO
 
Trading Experience: Intermediate
Platform: NinjaTrader
Favorite Futures: Emini
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received

Thanks Ian. See my responses in red.


iantg View Post
1. It looks like you have data going from 2011 to 2018. So does this imply that over the life of 8 years the system made roughly $15K after commissions? This would be evenly divided equal to roughly $2k per year. Is this what your results are showing or am I missing something? I think you can imagine the answer if the question, "Should I trade a system live that produces $2k per year in back testing." But maybe I am missing how your results are displayed.
No, you're not missing anything. That's what the system is showing. What do you consider to be a reasonable and obtainable goal for yearly profit, trading one contract?


2. It looks like you are almost always only taking long positions. Is this by design or is something in your code misfiring?
Nothing misfiring. After running the Strategy Analyzer it looks like my entry criteria are only effective for long entries.

3. What exactly was your testing on? Strategy Analyzer, Market Replay, Live SIM, etc.? Each of these have levels of accuracy that can impact your results.
I have only tested using Strategy Analyzer. Clearly this isn't enough.

4. What is the latency sensitivity of your alpha signal? Some strategies test well, but in live trading speed crushes them. Some signals you can catch from home on a laptop over the internet (Market Orders with a long term trade), Some you would need a VPN / VPS with: Limit order / speed sensitive strategies. And some signals you can't reach in time with NinjaTrader no matter what you do.
I'm hoping this won't be a problem for my strategy. I'm trading based on a five minute chart.

In order to test a realistic live trading scenario with NT you should do the following.

Run your test with Market Replay, and put your code in the level 1 data event handler. (OnMarketData). The OnBarUpdate front runs the Level 1 and Level 2 feed and can give you a glimpse into the future if your alpha signals are derived from the level 1 feed. No that Market Replay assumes 0 latency. If you want to really simulate live trading types of latency, you can build in some latency in your code yourself before you release your order.

If you can get passed market replay, then move onto Live SIM. The simulate around 200 MS of latency and the live feed is the live feed. Here you can use the OnBarUpdate to get back some of the lag from your interenet + broker to exchange latency.

Also put in your commissions.... You might be okay based on what I saw, but once you have realistic latency to measure against, who knows.

If you can pass these benchmarks and have a positive expectancy you should be fine.

This is excellent info and gives me something to shoot for. Thank you!!

Ian



OldGerman View Post
If I look at your equity, it looks like the trades will be held for a longer period of time. This requires a VPS. See Speedytradingservers.com in this forum or look at Google.

Could you elaborate on this? I would think this would be something that's necessary for very short time frame trading (my avg. time in market is 93minutes). I must be missing something.



kanepa View Post
Hi. You can check out my journal. I started system trading from summer of 2017 and you could see up and downs of my system.

1. Commission. You need to add them into calculation.
2. Add 1 slippage for every trade to see if it survives.
3. Need to go back at least 08 and 09 to see if system survived and profitable in bear market.

I'll definitely take a look at your journal. FYI, If I add 1 slippage my net profit decreases by ~15%. Yikes.

Thank you everyone for your input. I was hoping to come to this forum and be humbled. It seems to be working.

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
  #13 (permalink)
Elite Member
Münster Germany
 
Trading Experience: Intermediate
Platform: NinjaTrader, ProRealtime
Broker/Data: NinjaTrader Brokerage, IG Markets
Favorite Futures: FDXM, FDAX, ES, YM, 6B and 6E
 
OldGerman's Avatar
 
Posts: 361 since Mar 2015
Thanks: 478 given, 733 received



303Trader View Post
Could you elaborate on this? I would think this would be something that's necessary for very short time frame trading (my avg. time in market is 93minutes). I must be missing something.

I use my VPS because I can not secure the internet connection for a long time.
VPS also offer a very low latency. But that should not be so important on most systems.

The biggest risk you can ever take is not betting on yourself!
(Bill Williams)
Reply With Quote
The following user says Thank You to OldGerman for this post:
 
  #14 (permalink)
Trading for Fun
Batavia NY
 
Trading Experience: Advanced
Platform: KDE Neon, TWS, Python, R
Favorite Futures: Common stock, ETFs, YM, options
 
Posts: 112 since Dec 2018
Thanks: 19 given, 139 received

A strategy doesn't have to go from $0 to 2X your networth on margin.

I am interested in getting a strategy to real money ASAP because it is like a bug, it is not going to have that long of a life anyway.

As it performs better it gets more money to trade and as it performs worse it gets less until it dies.

I do have a real problem with optimization in this context though. All that optimization is doing is more finely tuning the model to conditions in the past that we know will not hold tomorrow. You are really optimizing the brittleness of the model most likely.

What I want to see from my models is that the strategy does not completely fall apart when used on another highly correlated time series.

If something blows the doors off ES but gets killed on SPY it is far more likely your model is simply over fitted to a random realization of the time series for ES than capturing and exploiting a structure of ES that is not part of SPY when we know the time series has the same generating process. Then I want to see SPY vs QQQ, NQ vs ES, YM vs SPY, etc..

You also need to look at as a baseline at the profit/loss if you had just gone long SPY and held since 2011 then normalize the leverage in ES. If you are not beating that then what is the point of trading. A system is not good because it is profitable, a trading system is good because it beats by and hold over the same period.

Don't forget too a purely random bad strategy is going to be between 40%-60% win/loss rate. I mean I would love to find a strategy that loses 95% of the time so I can just flip things. I agree with Ozquant 65% rule of thumb although I am ok with 60%. You need to give yourself a model error/noise cushion to stay above pure randomness.


Last edited by centaurer; January 12th, 2019 at 01:24 PM.
Reply With Quote
The following 2 users say Thank You to centaurer for this post:
 
  #15 (permalink)
Elite Member
Africa
 
Trading Experience: Advanced
Platform: TradeStation
Favorite Futures: Futures
 
Posts: 14 since Sep 2017
Thanks: 22 given, 14 received

Backtesting


303Trader View Post
Hi all. I'm new here and this is my first post. I'm super excited to have found this community!

I've been working on building an automated ES trading strategy for the last several months. I've optimized and backtested and I believe this strategy to be profitable. My goal is to take this strategy live starting in January but I'm getting cold feet. I'd like to gain some confidence from more experienced traders that I'm not fooling myself and I'm making a well informed decision to take this strategy live.

I've optimized my strategy using data from 01/2011 - 12/18. Backtesting using the optimized parameters over the same period I get the results shown in the attached pictures.

Something I noticed is that you used the full historical data set for optimization?
That would just be curve fitting the rules.

My 2c is that when you test a system, you must always test against out-of-sample data.

Maybe read Kevin Davey's book on "Building an algo", definitely worth a read.

Reply With Quote
The following 3 users say Thank You to bchip for this post:
 
  #16 (permalink)
Market Wizard
Los Angeles CA
 
Trading Experience: Beginner
Platform: NT8
Favorite Futures: NQ YM ES ZN CL GC HG ZS 6E 6A 6J 6C
 
FlyingMonkey's Avatar
 
Posts: 625 since Jan 2016
Thanks: 2,858 given, 2,133 received


bchip View Post
My 2c is that when you test a system, you must always test against out-of-sample data.

This. Building a strategy that optimizes well and gives you a great back test is easy (and I agree with previous posters, your result is not yet a great backtest). Building an edge that tests well on out of sample data using a walk-forward test is difficult. And a test that performs well on live data (in sim) is the final step before unleashing on real money.

Give him the stick.

Don't give him the stick!
Reply With Quote
The following user says Thank You to FlyingMonkey for this post:
 
  #17 (permalink)
Trading Apprentice
Pecs, Hungary
 
Trading Experience: Beginner
Platform: Tradestation
Favorite Futures: ES, CL, GC
 
Posts: 14 since Jun 2018
Thanks: 7 given, 9 received

First of all you need to collect your own requirements about the strategy:
Does the new strategy fit in the portfolio? Do you get a higher overall profit and a lower overall drawdown? What is the correlation between the different systems and markets?
Strategy: profit, drawdown, average trade, win percentage based on Monte Carlo tests

During the development process I like to validate a strategy on other bar sizes, other session times, sometimes other markets (correlated and non-correlated).

SIM trading to get more than 30 trades.
Perform a t test to compare the results to the test results.

If this is still within the expected limits, than start live trading.
Start small: one contract

Reply With Quote
The following 3 users say Thank You to Sharon9999 for this post:
 
  #18 (permalink)
Trading Apprentice
Denver CO
 
Trading Experience: Intermediate
Platform: NinjaTrader
Favorite Futures: Emini
 
Posts: 7 since Dec 2018
Thanks: 3 given, 7 received


bchip View Post
Maybe read Kevin Davey's book on "Building an algo", definitely worth a read.

I just finished reading Kevin's book a few weeks ago. It definitely put things in perspective and answered many of my questions.


FlyingMonkey View Post
This. Building a strategy that optimizes well and gives you a great back test is easy (and I agree with previous posters, your result is not yet a great back-test). Building an edge that tests well on out of sample data using a walk-forward test is difficult. And a test that performs well on live data (in sim) is the final step before unleashing on real money.

I'm finding this to be very, very true. Since my original post I've increased back-test profitability and win rate substantially. However, as soon as I move to out of sample data it falls apart. Furthermore, I've been live sim trading this strategy since the beginning of the year (I figured why not?) and it's doing extremely well! It's infuriating and fascinating all at the same time.

I'm still convinced that my entry signal can be reliable. The two things I'm really struggling with is how to scale my entry parameters to match market volatility and how to let my winning trades run. I'm also still baffled on why this strategy seems to hate short trades. If I can get these things figured out maybe I'll have something.

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

Please register on futures.io to view futures trading content such as post attachment(s), image(s), and screenshot(s).

Reply With Quote
The following user says Thank You to 303Trader for this post:
 
  #19 (permalink)
Elite Member
Golden Bay, New Zealand
 
Trading Experience: Beginner
Platform: Sierra Chart
Favorite Futures: ES
 
Posts: 148 since May 2012
Thanks: 692 given, 254 received


303Trader View Post
Since my original post I've increased back-test profitability and win rate substantially. However, as soon as I move to out of sample data it falls apart.

It wasn't clear to me if you are still working on the same basic strategy, or whether you have moved on.

But just in case - if you test the in-sample against the OOS, modify and re-test then the OOS effectively becomes in-sample. You should only peak at the OOS once. If it doesn't work, then start again from scratch.

The advantage of 'algos', is that you can have a lot of them, which keeps the individual bet size low, which lowers max drawdown. Since you need a lot of them, you need to develop a process so you can churn them out. You'll be relatively indifferent if it works or not. It should be expected that most won't work. Keep in mind, you can get decent OOS results, which in reality is still random noise. That's another reason you want multiple systems.

Reply With Quote
The following 4 users say Thank You to sixtyseven for this post:
 
  #20 (permalink)
Trading Apprentice
Pecs, Hungary
 
Trading Experience: Beginner
Platform: Tradestation
Favorite Futures: ES, CL, GC
 
Posts: 14 since Jun 2018
Thanks: 7 given, 9 received


Hi 303Trader,

Your new equity looks better. Average trade is still quite low.

And how does this strategy perform in a downtrend?

I saw in your report you subtracted 6,200 USD commissions for 1,520 trades.
This can only work if you enter with a limit order and exit with a profit target.

In case of a market order or stop order the ES slippage can go up to 1 tick per side.

Reply With Quote
The following user says Thank You to Sharon9999 for this post:

Reply



futures io > > > > How do you decide when an automated strategy is good enough to take live?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)
 

Don’t Let One Bad Trade Wipe Out Your Entire Account w/GFF Brokers

Apr 23
 

New CME Micro Indices: New Trading Opportunities w/FuturesTrader71

Elite only
 

Benefits of Trading the Micro E-Minis w/Stage 5 & CME Group

Apr 30
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
During development, when do you decide to quit or keep coding a Strategy? See Equity Dvdkite Traders Hideout 9 October 4th, 2018 06:01 AM
Is OECTrader stable enough to replace MC for live trading EL strategy? murty Open E Cry 0 November 29th, 2014 08:03 PM
How do you decide WHEN you will trade? thomasblake Emini Index Futures Trading 17 August 2nd, 2014 03:48 PM
Good Economy Not Good Enough for Investors: El-Erian Quick Summary News and Current Events 0 January 6th, 2012 02:30 PM
‘Enough’s Enough’ on China Currency: Obama kbit News and Current Events 0 November 14th, 2011 09:33 AM


Tags
ai, ats, automated strategy, automated systems, back testing, backtested, backtesting, bear market, broker, brokerage, brokers, buy, code, commission, data, dom, dow, drawdown, emini, equity curve, error, es, es trading, est, expectancy, futures, how to, index_futures, level 2, limit, limit order, live sim, live trading, long term, loss, market replay, market_replay, ninja, ninjatrader, onmarketdata, options, orders, platform, positive equity, profitable, replay, sample, short, signal, signals, sim, sim_trading, size, slippage, spread, standing, strategies, strategy, systems, tf, traded, trading, trading strategy, vp, vpn, vps

All times are GMT -4. The time now is 06:54 PM. (this page content is cached, log in for real-time version)

Copyright © 2019 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432 WhatsApp Business, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2019-04-24 in 0.30 seconds with 15 queries on phoenix