Taking a Trading System Live - futures io
futures io futures trading



Taking a Trading System Live


Discussion in Trading Journals

Updated by tturner86
      Top Posters
    1. looks_one kevinkdog with 260 posts (714 thanks)
    2. looks_two Big Mike with 61 posts (97 thanks)
    3. looks_3 deaddog with 25 posts (24 thanks)
    4. looks_4 swz168 with 20 posts (40 thanks)
      Best Posters
    1. looks_one kevinkdog with 2.7 thanks per post
    2. looks_two swz168 with 2.0 thanks per post
    3. looks_3 rk142 with 2.0 thanks per post
    4. looks_4 Big Mike with 1.6 thanks per post
    1. trending_up 60,470 views
    2. thumb_up 1,128 thanks given
    3. group 97 followers
    1. forum 529 replies
    2. attach_file 189 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 100,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Thread Tools Search this Thread
 

Taking a Trading System Live

  #221 (permalink)
Phoenix Arizona
 
Experience: Beginner
Platform: MotiveWave
Trading: CL
 
Posts: 125 since May 2013
Thanks: 325 given, 162 received

Kevin, In the test period, did you have such an imbalance between the size of the winners and the losers? I am concerned to see you taking such large losses (almost 10 handles) with so little profit. Maybe I am misunderstanding things here.

Ddawg

Reply With Quote

Can you help answer these questions
from other members on futures io?
Convert NinjaTrader NT7 indicator to Tradestation EasyLanguage and RADARSCREEN
Platforms and Indicators
Issue meeting order entry conditions
Elite Automated NinjaTrader Trading
Last 5 minutes Label
ThinkOrSwim
Moving average with Lag reduction
NinjaTrader
Linux?
ThinkOrSwim
 
Best Threads (Most Thanked)
in the last 7 days on futures io
Spoo-nalysis ES e-mini futures S&P 500
72 thanks
June Charity Fundraiser - Feeding hungry families
48 thanks
Fixed vs trailing stops
31 thanks
FIO Journal Challenge - June 2020 edition w/SharkIndicators
29 thanks
AMP Futures / AMP Global Review
26 thanks
 
  #222 (permalink)
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 324 received

Kevin,

I don't have a strong intuition on the meaning of that lower 10% line. I have a vague sense of what you are aiming to measure, but only a vague sense - and it's unclear how that lower line realizes that vague sense.

Suppose you wanted to know if your system is underperforming over any 18 trade period. Based on what I've learned from your threads so far, I might expect you to run a monte carlo simulation of 2500 18-trade sequences, and take the equity that demarcates the lower 10% of those results from the upper 90% as your cut off point. That would be a well-defined "lower 10%."

But I can see that it would be tedious to do that for each N-trades - you'd basically have to re-run the simulation every day - and so you're modeling the "lower 10%" in a different way. From your combine thread I have the following formula for your %-bands around the average result:

N * AvgTrade + Sqrt(N) * (Sigma) * X

Where I just assumed that the "X" corresponds to the number of standard deviations away from the mean that marks off the lower 10% of your individual trade results. If I understand Z-scores correctly (I probably don't!), I'm guessing X = -1.28 or so.

Even if I've interpreted the "X" in that formula correctly, I still don't have a very good intuition of just how that realizes what you want to be measuring.

Feel free to respond briefly, or not at all. A link to a well chosen wikipedia page might be enough to clear up my confusion.

thanks,
RK

Reply With Quote
The following 2 users say Thank You to rk142 for this post:
 
  #223 (permalink)
Legendary Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,768 since Jul 2012
Thanks: 1,422 given, 5,326 received



Ddawg View Post
Kevin, In the test period, did you have such an imbalance between the size of the winners and the losers? I am concerned to see you taking such large losses (almost 10 handles) with so little profit. Maybe I am misunderstanding things here.

Ddawg

Thanks for the question. It is a good one. For the NGEC strategy#1, which trades overnight, you are indeed right - the winners are small but frequent (75% of time), and the losers are small but infrequent. Overall, they yield a positive expectancy.

What happened last week is that there were 2 consecutive big losers in start #1, which had not happened in a while. As you can see from the equity curve, this has happened before.



Follow me on Twitter Reply With Quote
 
  #224 (permalink)
Legendary Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,768 since Jul 2012
Thanks: 1,422 given, 5,326 received


rk142 View Post
Kevin,

I don't have a strong intuition on the meaning of that lower 10% line. I have a vague sense of what you are aiming to measure, but only a vague sense - and it's unclear how that lower line realizes that vague sense.

Suppose you wanted to know if your system is underperforming over any 18 trade period. Based on what I've learned from your threads so far, I might expect you to run a monte carlo simulation of 2500 18-trade sequences, and take the equity that demarcates the lower 10% of those results from the upper 90% as your cut off point. That would be a well-defined "lower 10%."

But I can see that it would be tedious to do that for each N-trades - you'd basically have to re-run the simulation every day - and so you're modeling the "lower 10%" in a different way. From your combine thread I have the following formula for your %-bands around the average result:

N * AvgTrade + Sqrt(N) * (Sigma) * X

Where I just assumed that the "X" corresponds to the number of standard deviations away from the mean that marks off the lower 10% of your individual trade results. If I understand Z-scores correctly (I probably don't!), I'm guessing X = -1.28 or so.

Even if I've interpreted the "X" in that formula correctly, I still don't have a very good intuition of just how that realizes what you want to be measuring.

Feel free to respond briefly, or not at all. A link to a well chosen wikipedia page might be enough to clear up my confusion.

thanks,
RK


You seem to understand what I am doing almost perfectly. The only difference I can see is that I run the day by day simulation (to get the +/10% curves) at the start of live trading, using all data to that date. So, I don't have to
re-run it every day - I only have to add in the actual results.


I look at the chart this way: after trading for a while, if the historical walkforward test can be believed still, what are the chances that my actual performance is not within the upper and lower 10% curves?

That means, for the lower line, that there is a 90% chance that my performance will be above the curve. If it isn;t, I start to wonder 1) has the system stopped working, and/or 2) has the market changed to something different, rendering my system ineffective?

This curves really becomes an early warning system that something might be wrong.

I hope this explains it - if not just let me know.

Follow me on Twitter Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #225 (permalink)
Dallas,TX
 
Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker: Zaner/Zen Fire
Trading: ES,6E,6B,GC,CL
 
Posts: 598 since Nov 2009
Thanks: 176 given, 120 received

use auto strategy in Topstep

Kevin
your details are really great and like to try Practice sim in Topstep
topstep says we can't use auto strategy and Ninja trader but you seem to be using strategies. is that allowed in Trade station?

Reply With Quote
The following user says Thank You to emini_Holy_Grail for this post:
 
  #226 (permalink)
Legendary Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,768 since Jul 2012
Thanks: 1,422 given, 5,326 received


emini_Holy_Grail View Post
Kevin
your details are really great and like to try Practice sim in Topstep
topstep says we can't use auto strategy and Ninja trader but you seem to be using strategies. is that allowed in Trade station?

Right now I am not doing anything with TopStep - the system I am currently running is automated through Tradestation, and traded through their brokerage.

When I was doing the Combine, I was still using the same strategies, but I was manually entereing orders in the T4 platform, which connected to TopStep.

I hope this explains it - if not, please feel free to ask more questions.

Follow me on Twitter Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #227 (permalink)
Dallas,TX
 
Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker: Zaner/Zen Fire
Trading: ES,6E,6B,GC,CL
 
Posts: 598 since Nov 2009
Thanks: 176 given, 120 received

thks Kevin

I heard Topstep allowed Ninjatrader for a while , but later they stopped.
T4 is ok, but is not as easy as ATM strategy in Ninja's DOM

Reply With Quote
 
  #228 (permalink)
Legendary Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,768 since Jul 2012
Thanks: 1,422 given, 5,326 received


emini_Holy_Grail View Post
thks Kevin

I heard Topstep allowed Ninjatrader for a while , but later they stopped.
T4 is ok, but is not as easy as ATM strategy in Ninja's DOM

Sierra Charts might be an option, if you are looking to automate with TopStep. I have never used it though.

Follow me on Twitter Reply With Quote
 
  #229 (permalink)
Site Administrator
Manta, Ecuador
 
Experience: Advanced
Platform: My own custom solution
Trading: Emini Futures
 
Big Mike's Avatar
 
Posts: 48,775 since Jun 2009
Thanks: 31,485 given, 94,158 received

Kevin,

Is the system really performing as designed? How many trades would it normally have taken over this period? What do the per-trade stats look like in testing vs live?

My fear here is that it is going sideways and you are giving it more time, but what happens should it take a couple of losses in a row? The ultimate question comes down to the first question above -- is it performing as expected, or not.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/
Follow me on Twitter Visit my Facebook Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Big Mike for this post:
 
  #230 (permalink)
Legendary Market Wizard
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,768 since Jul 2012
Thanks: 1,422 given, 5,326 received



Big Mike View Post
Kevin,

Is the system really performing as designed? How many trades would it normally have taken over this period? What do the per-trade stats look like in testing vs live?

My fear here is that it is going sideways and you are giving it more time, but what happens should it take a couple of losses in a row? The ultimate question comes down to the first question above -- is it performing as expected, or not.

Mike

Good question. After 7 weeks of live trading, is the system performing as designed?

First, let's look at the number of trades it has taken. A sharp increase or decrease in the number of trades, when compared to the walkforward history, would suggest that the market action is different than the historical market, causing many more, or far fewer, trades to be taken than normal. After 7 weeks, the system has traded for 18 days, or 53% of possible days. Historically, it trades about 151 days per year, or 60% of days. So, the system is trading less than the average. But some years it has traded as few as 130 days (51%) and as many as 175 days (70%). Based on all this I'd conclude that the number of trades is generally in line with expectations, although at the low end. Anecdotally, I have felt that the volatility has been lower than usual. There have been a handful of days where an entry was wished by only a few ticks (the big Fed move day Sept 18 was a case in point). A little more volatility in the hours before the announcement and the system would have entered a big winner.

Second, let's look at average performance, versus actual performance. The average performance of the system gives $1441 profit after 18 trading days. Actual performance, however, is at -$746. This is a HUGE discrepancy, and the conclusion obviously is that the system is not performing as well as it should.

But here is where is gets tricky. Take a coin, and flip it 100 times. If you get 60 heads, do you conclude that the coin is "broken," i.e. biased? What if you got 70 heads, or 80, or 90? Even if you flipped 100 heads in a row, could you absolutely conclude that the coin is biased? No! There is a chance, albeit very, very small, that a fair coin could be flipped to heads 100 consecutive times.

It is the same way with a trading system. You can ask "is it broken?" but the answer will always have some degree of uncertainty. That uncertainty sometimes makes all the difference to your conclusions.

Right now, the actual system performance is at the 10th percentile of what was expected. The 50th percentile would be right at the average, so the 10% mark is pretty bad. But, it is still within the realm of possible outcomes. If it was below the 0 percentile - let's say the system had lost $8,000 through the first 18 days - then the system is obviously not performing as expected. That would be an easy decision. It is where there is uncertainty that things get tough.

I know there are statistical tests that could be run to show whether the current trades could be part of the historical distribution of trades, but even that analysis is not definitive. There is always a gray layer.

So, how do I navigate the gray area? First, I try to determine the parameters that will cause me to quit trading a system. I could certainly use the percentile number approach, and have a quitting rule that says "if after X days the performance is below the Yth percentile, I will cease trading." X and Y would be at your discretion, based on your personal preferences. As long as you stick to the rule you create, you'll be doing fine.

For me, and this particular system, earlier I decided that I would quit only when I hit a $5000 drawdown. So, I am not using the data in the tracking graph to decide when to quit. Sometimes I do use it, though.


So, to summarize:

Is the system performing as expected? No, not even close. It is performing much worse than expected.

Is the system "broken?" Maybe, maybe not. It depends how you define broken. One cannot say definitively it is broken, or it is not broken.

Am I going to quit? No. My quit point, established earlier, calls for a single contract drawdown of $5,000. This was a well thought out amount, and I can't just toss it out the window. I'm going to stick to the plan. I realize, though, that sticking to the plan might be akin to the captain sinking with the ship. It was a risk I was comfortable with at the start, and still am.

Follow me on Twitter Reply With Quote
The following 6 users say Thank You to kevinkdog for this post:



futures io Trading Community Trading Journals > Taking a Trading System Live




Upcoming Webinars and Events
 

Free BloodHound Licenses to everyone!

June
 

Every journal equals ten meals for the hungry

June
     



Copyright © 2020 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts