Taking a Trading System Live - futures io
futures io



Taking a Trading System Live


Discussion in Trading Journals

Updated
      Top Posters
    1. looks_one kevinkdog with 260 posts (717 thanks)
    2. looks_two Big Mike with 61 posts (97 thanks)
    3. looks_3 deaddog with 25 posts (24 thanks)
    4. looks_4 swz168 with 20 posts (40 thanks)
      Best Posters
    1. looks_one kevinkdog with 2.8 thanks per post
    2. looks_two swz168 with 2 thanks per post
    3. looks_3 rk142 with 2 thanks per post
    4. looks_4 Big Mike with 1.6 thanks per post
    1. trending_up 67,389 views
    2. thumb_up 1,131 thanks given
    3. group 97 followers
    1. forum 529 posts
    2. attach_file 189 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Taking a Trading System Live

(login for full post details)
  #201 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

Week 5 is in the books. Another dull week. The NEGC strategy #2 missed Wednesday's big move by about 5 ticks, but that's the way it goes.


So, I have a question for everyone reading this: After 5 weeks, and 13 days of trading, based on the current performance, would you still be trading this, or would you have abandoned it in favor of something (hopefully) better? It is a hypothetical question worth asking, and there is no right or wrong answer - there is just an answer for your personal circumstances.

Feel free to share your thoughts...






Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #202 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received


kevinkdog View Post


So, I have a question for everyone reading this: After 5 weeks, and 13 days of trading, based on the current performance, would you still be trading this, or would you have abandoned it in favor of something (hopefully) better? It is a hypothetical question worth asking, and there is no right or wrong answer - there is just an answer for your personal circumstances.

based on 13 samples I'd continue until I had more results.

The info you give per day is hard to decipher. How many trades are involved?

As I've mentioned before I would be concerned that the size of the losers is much greater than the size of the winners. Now that may be because a losing day has a large number of trades with small losses that add up to a large losing day. Likewise a day showing a small win might be a number of small losses and one or two large wins.

You haven't shared your methodology, nor do I expect you to but when day trading why stay in a trade that is going against you? Why not cut your losses and look for a new entry?

Reply With Quote
The following user says Thank You to deaddog for this post:
 
(login for full post details)
  #203 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
based on 13 samples I'd continue until I had more results.

The info you give per day is hard to decipher. How many trades are involved?

As I've mentioned before I would be concerned that the size of the losers is much greater than the size of the winners. Now that may be because a losing day has a large number of trades with small losses that add up to a large losing day. Likewise a day showing a small win might be a number of small losses and one or two large wins.

You haven't shared your methodology, nor do I expect you to but when day trading why stay in a trade that is going against you? Why not cut your losses and look for a new entry?


I trade up to 1 time overnight session, and up to 1 time for day session. So, never more than 2 trades in one day. Usually on days when there is a trade, there is only 1 between the 2 strategies. In 13 days, I'd guess there have been 15-20 trades.

The current stop loss for both strategies is 34 ticks, or $425 before slip and commission. This may change over time, due to re-optimizations.

As far as cutting losses and looking for a new entry, that was just something I had not done when designing the system. Too late to do that for this system. Maybe a brand new system would be better off this way.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #204 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


kevinkdog View Post
After 5 weeks, and 13 days of trading, based on the current performance, would you still be trading this, or would you have abandoned it in favor of something (hopefully) better?


With this question, I am trying to get everyone to think emotionally, not analytically. Here you are, all excited to start trading a system live, one that you predict will more than double your money in a year. Yet, after 5 weeks of trading, you are stuck in neutral. You really need this system to perform, or else that trip to Disney next year is out. And you just read Trader Y's journal, where he is killing it with 100% returns just this last month. Maybe you should abandon your approach and follow his...

It is an exercise, but look at it emotionally, and put yourself in that situation. You'll learn some interesting things about yourself...

Follow me on Twitter Started this thread Reply With Quote
The following 4 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #205 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received


kevinkdog View Post

It is an exercise, but look at it emotionally, and put yourself in that situation. You'll learn some interesting things about yourself...

As an exercise it's easy, unfortunately the emotions don't really kick in until you have your own cash at risk. What are you going to do?

Reply With Quote
 
(login for full post details)
  #206 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


deaddog View Post
As an exercise it's easy, unfortunately the emotions don't really kick in until you have your own cash at risk.

The emotions you experience when you role play it can be helpful, if you are capable of being honest with yourself. Introspection is real, and Kevin is right to pursue the question in this way.

But yes, until you experience the pain and suffering of trading, you should avoid overestimating your self knowledge.

My real life story: I learned setups from a guy who traded those setups with discretion. I lost something like 20% of my account in one month trying to trade them myself in his style. I totally panicked and went back to my prior approach, which was not terribly profitable but worked for me. About a year later I paid to have my trading mechanized by someone with experience. The first month I went live with the new rules (early this year) I experienced a drawdown of roughly similar proportions to the previous debacle, but traded through it, and am now profitable for the year.

I knew from the moment I was exposed to trading that I had the discipline to execute. I also learned pretty quickly that I was too smart for my own good, and without mechanizing my trading I would be paralyzed.

If I were Kevin. I'd keep pulling the trigger. When I was in the nasty drawdown earlier this year, I actually enjoyed pulling the trigger. Every time I did so, I felt like a soldier at Thermopylae.

(full disclosure: I've had two lapses in discipline this year, so don't go awarding me a medal of honor just yet)

-RK

Reply With Quote
The following 5 users say Thank You to rk142 for this post:
 
(login for full post details)
  #207 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
As an exercise it's easy, unfortunately the emotions don't really kick in until you have your own cash at risk. What are you going to do?

That is why it is an exercise - you have to pretend a bit. I agree it is difficult to do.

As for me, I know you've been contributing long enough (thank you!) to know what I plan to do...

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #208 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
The emotions you experience when you role play it can be helpful, if you are capable of being honest with yourself. Introspection is real, and Kevin is right to pursue the question in this way.

But yes, until you experience the pain and suffering of trading, you should avoid overestimating your self knowledge.

My real life story: I learned setups from a guy who traded those setups with discretion. I lost something like 20% of my account in one month trying to trade them myself in his style. I totally panicked and went back to my prior approach, which was not terribly profitable but worked for me. About a year later I paid to have my trading mechanized by someone with experience. The first month I went live with the new rules (early this year) I experienced a drawdown of roughly similar proportions to the previous debacle, but traded through it, and am now profitable for the year.

I knew from the moment I was exposed to trading that I had the discipline to execute. I also learned pretty quickly that I was too smart for my own good, and without mechanizing my trading I would be paralyzed.

If I were Kevin. I'd keep pulling the trigger. When I was in the nasty drawdown earlier this year, I actually enjoyed pulling the trigger. Every time I did so, I felt like a soldier at Thermopylae.

(full disclosure: I've had two lapses in discipline this year, so don't go awarding me a medal of honor just yet)

-RK


Thanks for sharing your story. I assume your second drawdown was in line with your expectations? If so, then it was very wise (but probably difficult) to trade through it.


What's Thermopylae?

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #209 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


kevinkdog View Post
Thanks for sharing your story. I assume your second drawdown was in line with your expectations? If so, then it was very wise (but probably difficult) to trade through it.


What's Thermopylae?

Yes - my analysis isn't quite as rigorous as yours (you are always quick to point out that your way isn't the only way, but I do think you are doing an excellent job modeling a process with better controls than mine and I'm quietly learning - credit where credit is due), but in order to trade these setups and get the returns I'm looking for I need to be prepared to endure around a 25% draw down at some point each year. It was probably good that I had to take my medicine early.

Thermopylae was the sight of a famous last stand by the Spartans against the invading Persians. A unit of 7000 held off an army of around 100,000. And when betrayed a unit of less than 1,000 made the final stand. They were overrun, but in the end the underdog Greeks won the war.

Drawdown & Discipline.

Hope you had a great weekend.

-RK

Reply With Quote
The following user says Thank You to rk142 for this post:
 
(login for full post details)
  #210 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

Trading this system is like watching paint dry!

As I close out Week 6, as you can see nothing exciting is happening. I suppose that it both good and bad - bad I am not making money (or anywhere near the average line), good that I am not losing money either.









Follow me on Twitter Started this thread Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #211 (permalink)
 Ddawg 
Phoenix Arizona
 
Experience: Beginner
Platform: MotiveWave
Trading: CL
 
Posts: 125 since May 2013
Thanks: 325 given, 162 received

Kevin, do you think this has something to do with the market being slow and thin?

I have been experiencing much the same....though
I am negative a bit.

Ddawg

Reply With Quote
The following user says Thank You to Ddawg for this post:
 
(login for full post details)
  #212 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Ddawg View Post
Kevin, do you think this has something to do with the market being slow and thin?

I have been experiencing much the same....though
I am negative a bit.

Ddawg

It certainly could be the case. Just not as many opps to trade. Some of the big movement days (like the Fed day a few weeks ago) I only missed by a few ticks. But I do hope volatility increases - that is usually a good thing.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #213 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
Kevin,

Excellent thread. Thank you.

In response to a question about your approach to optimization you wrote the following:



Could you say a bit more about the tools / decision making process you use to make decisions about the IN/OUT periods you use?

I'm afraid of optimization.

thanks,
RK


I have been a bit behind in my work, so I apologize to @rk142 for not addressing his question yet. I have not forgotten it though, and hope to give a good answer soon.

Thanks for your patience!

Follow me on Twitter Started this thread Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #214 (permalink)
 MBAGearhead 
Chicago, IL
 
Experience: Advanced
Platform: OpenECry
Trading: Crude Oil, Soybeans
 
Posts: 22 since Nov 2010
Thanks: 22 given, 12 received

Hey Kevin, great journal you got going here. I just caught up with the whole thing this week! Your posts regarding trading being boring reminded me of my recent TF system that I thought I'd share. Hopefully readers find it interesting to look briefly at a different system and my experience.

I developed this system earlier this year. I did use your Monte Carlo spreadsheet but must not have saved the results. Thanks for sharing that. Keep in mind this data below does not include slippage or commission. It is based on 1 TF contract per trade. I did run some WFO but didn't find enough difference to bother with it. I ended up optimizing over 3 years on max. PF and used the first 6 months of 2013 as out of sample data. The stats/curve looked the same and I saw no obvious difference in the equity curve.



So I started trading live June 1st, expecting the average profit per trade to be ~3 ticks. Yes this is very low but was better than my recent trading so why not?
Things began with a drawdown (don't they always?) but later had a good run:


By the end of July, after 2 months, I decided to stop trading the system.

I had in the back of my head all along that I never really had an incubation period. It was a "live" incubation! LOL Plus, although I had a small gain (~$700 I think), performance was recently flat/down and I wanted to take my trading a different direction and try a discretionary approach for awhile which would limit my focus on systems. Keep in mind systems I do trade are all mechanical and I do not have a Ninja license so can't automate currently.

Over the past 2 months, the system has been really flat so I have no regrets for now. I will keep an eye on it and may trade it again (or automate it) at some point in the future.
June thru Sept:

Reply With Quote
The following 3 users say Thank You to MBAGearhead for this post:
 
(login for full post details)
  #215 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

Thanks for sharing MBA. Any particular reason why your original analysis did not include slippage or commissions? Is the 3 tick profit gross or net? It looks like your bottom charts expect 4 ticks per trade ($40), so I am a bit confused.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #216 (permalink)
 MBAGearhead 
Chicago, IL
 
Experience: Advanced
Platform: OpenECry
Trading: Crude Oil, Soybeans
 
Posts: 22 since Nov 2010
Thanks: 22 given, 12 received


kevinkdog View Post
Thanks for sharing MBA. Any particular reason why your original analysis did not include slippage or commissions? Is the 3 tick profit gross or net? It looks like your bottom charts expect 4 ticks per trade ($40), so I am a bit confused.

Sorry for confusion. First my bottom chart assumes $43.60 net per trade with standard deviation of $440. And I see I attached an earlier version of my system's equity curve on my 1st chart. Attached is the latest version (the one I had been trading) which shows avg. profit of $48/trade through today. For the period 1/1/10 - 6/1/13 it was $51.24 so it has dropped a bit with the "live/incubation" performance since June 1st. Again, all my Ninja reports do not include slippage or commissions. I just find it easier to subtract that myself later when I export to Excel. The "Expected Performance Curves" above DO adjust for slippage and commission. I only included 1/2 tick of slippage per roundtrip which I realize is much smaller than you would use but I found that pretty realistic with this system trading 1 lots.

So I guess I should have said that in theory, I expected the avg. trade to produce 4.3 ticks of net profit, not 3. But in reality I like to set expectations a bit lower due to unforeseen things, human error, incorrect assumptions, etc. Perhaps not low enough in this case as the average profit in period since June 1st is just $2.37/trade or 1/4 of a tick! That's pretty far from 3 or 4 ticks!

Attached Thumbnails
Click image for larger version

Name:	TFSys7 Results.png
Views:	75
Size:	69.5 KB
ID:	124916  
Reply With Quote
The following user says Thank You to MBAGearhead for this post:
 
(login for full post details)
  #217 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


MBAGearhead View Post
Again, all my Ninja reports do not include slippage or commissions. I just find it easier to subtract that myself later when I export to Excel.

One word of caution with this approach, especially if you optimize based on Net Profit or something similar. The optimizer will usually give you a best set of parameters that make you trade too much. Here is an example:

Without Slippage or Commission:

Parameter Setting #1: Gross Profit/trade = $25, 1000 trades, Gross Profit=$25000

Parameter Setting #2: Gross Profit/trade = $50, 300 trades, Gross Profit=$15000

The optimizer will select Setting #1 as superior.


With $25 Slip and Commission:

Parameter Setting #1: Net Profit/trade = $0, 1000 trades, Gross Profit=$0

Parameter Setting #2: Net Profit/trade = $25, 300 trades, Net Profit=$7500

The optimizer will now select Setting #2 as superior.


So, I hope you can see the that adding in costs up front may be really important.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #218 (permalink)
 MBAGearhead 
Chicago, IL
 
Experience: Advanced
Platform: OpenECry
Trading: Crude Oil, Soybeans
 
Posts: 22 since Nov 2010
Thanks: 22 given, 12 received


kevinkdog View Post
One word of caution with this approach, especially if you optimize based on Net Profit or something similar. The optimizer will usually give you a best set of parameters that make you trade too much.

Excellent point Kevin of which I am very aware. I never optimize on Net Profit, partially for that reason. I prefer to optimize on avg. profit per trade and that is the first metric I look at after Ninja generates a report. If this isn't high enough, it's back to the drawing board.

Reply With Quote
The following user says Thank You to MBAGearhead for this post:
 
(login for full post details)
  #219 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

7 weeks into trading this system live, and things went bad this week. The cumulative equity is hovering right around the 10% line, which is a good warning sign that the system is not working the same as its historical test. If circumstances were different I might consider stopping trading this system. The things working against the "quit now" idea are:

1) There are only 18 days of live trading data - too short in my mind to make a decision to quit

2) Most importantly, when I laid out the criteria for stopping live trading, performance relative to this chart was not considered (some times, I do consider this chart in my "when to quit" analysis).


So, I will keep on keepin' on, realizing that things need to improve.







Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #220 (permalink)
Pinot13
New York
 
 
Posts: 5 since Mar 2013
Thanks: 11 given, 3 received


kevinkdog View Post
With this question, I am trying to get everyone to think emotionally, not analytically. Here you are, all excited to start trading a system live, one that you predict will more than double your money in a year. Yet, after 5 weeks of trading, you are stuck in neutral. You really need this system to perform, or else that trip to Disney next year is out. And you just read Trader Y's journal, where he is killing it with 100% returns just this last month. Maybe you should abandon your approach and follow his...

It is an exercise, but look at it emotionally, and put yourself in that situation. You'll learn some interesting things about yourself...

Great question Kevin!

Reply With Quote
The following user says Thank You to Pinot13 for this post:
 
(login for full post details)
  #221 (permalink)
 Ddawg 
Phoenix Arizona
 
Experience: Beginner
Platform: MotiveWave
Trading: CL
 
Posts: 125 since May 2013
Thanks: 325 given, 162 received

Kevin, In the test period, did you have such an imbalance between the size of the winners and the losers? I am concerned to see you taking such large losses (almost 10 handles) with so little profit. Maybe I am misunderstanding things here.

Ddawg

Reply With Quote
 
(login for full post details)
  #222 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received

Kevin,

I don't have a strong intuition on the meaning of that lower 10% line. I have a vague sense of what you are aiming to measure, but only a vague sense - and it's unclear how that lower line realizes that vague sense.

Suppose you wanted to know if your system is underperforming over any 18 trade period. Based on what I've learned from your threads so far, I might expect you to run a monte carlo simulation of 2500 18-trade sequences, and take the equity that demarcates the lower 10% of those results from the upper 90% as your cut off point. That would be a well-defined "lower 10%."

But I can see that it would be tedious to do that for each N-trades - you'd basically have to re-run the simulation every day - and so you're modeling the "lower 10%" in a different way. From your combine thread I have the following formula for your %-bands around the average result:

N * AvgTrade + Sqrt(N) * (Sigma) * X

Where I just assumed that the "X" corresponds to the number of standard deviations away from the mean that marks off the lower 10% of your individual trade results. If I understand Z-scores correctly (I probably don't!), I'm guessing X = -1.28 or so.

Even if I've interpreted the "X" in that formula correctly, I still don't have a very good intuition of just how that realizes what you want to be measuring.

Feel free to respond briefly, or not at all. A link to a well chosen wikipedia page might be enough to clear up my confusion.

thanks,
RK

Reply With Quote
The following 2 users say Thank You to rk142 for this post:
 
(login for full post details)
  #223 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Ddawg View Post
Kevin, In the test period, did you have such an imbalance between the size of the winners and the losers? I am concerned to see you taking such large losses (almost 10 handles) with so little profit. Maybe I am misunderstanding things here.

Ddawg

Thanks for the question. It is a good one. For the NGEC strategy#1, which trades overnight, you are indeed right - the winners are small but frequent (75% of time), and the losers are small but infrequent. Overall, they yield a positive expectancy.

What happened last week is that there were 2 consecutive big losers in start #1, which had not happened in a while. As you can see from the equity curve, this has happened before.



Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #224 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
Kevin,

I don't have a strong intuition on the meaning of that lower 10% line. I have a vague sense of what you are aiming to measure, but only a vague sense - and it's unclear how that lower line realizes that vague sense.

Suppose you wanted to know if your system is underperforming over any 18 trade period. Based on what I've learned from your threads so far, I might expect you to run a monte carlo simulation of 2500 18-trade sequences, and take the equity that demarcates the lower 10% of those results from the upper 90% as your cut off point. That would be a well-defined "lower 10%."

But I can see that it would be tedious to do that for each N-trades - you'd basically have to re-run the simulation every day - and so you're modeling the "lower 10%" in a different way. From your combine thread I have the following formula for your %-bands around the average result:

N * AvgTrade + Sqrt(N) * (Sigma) * X

Where I just assumed that the "X" corresponds to the number of standard deviations away from the mean that marks off the lower 10% of your individual trade results. If I understand Z-scores correctly (I probably don't!), I'm guessing X = -1.28 or so.

Even if I've interpreted the "X" in that formula correctly, I still don't have a very good intuition of just how that realizes what you want to be measuring.

Feel free to respond briefly, or not at all. A link to a well chosen wikipedia page might be enough to clear up my confusion.

thanks,
RK


You seem to understand what I am doing almost perfectly. The only difference I can see is that I run the day by day simulation (to get the +/10% curves) at the start of live trading, using all data to that date. So, I don't have to
re-run it every day - I only have to add in the actual results.


I look at the chart this way: after trading for a while, if the historical walkforward test can be believed still, what are the chances that my actual performance is not within the upper and lower 10% curves?

That means, for the lower line, that there is a 90% chance that my performance will be above the curve. If it isn;t, I start to wonder 1) has the system stopped working, and/or 2) has the market changed to something different, rendering my system ineffective?

This curves really becomes an early warning system that something might be wrong.

I hope this explains it - if not just let me know.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #225 (permalink)
 emini_Holy_Grail 
Dallas,TX
 
Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker: Zaner/Zen Fire
Trading: ES,6E,6B,GC,CL
 
Posts: 598 since Nov 2009
Thanks: 176 given, 120 received

Kevin
your details are really great and like to try Practice sim in Topstep
topstep says we can't use auto strategy and Ninja trader but you seem to be using strategies. is that allowed in Trade station?

Reply With Quote
The following user says Thank You to emini_Holy_Grail for this post:
 
(login for full post details)
  #226 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


emini_Holy_Grail View Post
Kevin
your details are really great and like to try Practice sim in Topstep
topstep says we can't use auto strategy and Ninja trader but you seem to be using strategies. is that allowed in Trade station?

Right now I am not doing anything with TopStep - the system I am currently running is automated through Tradestation, and traded through their brokerage.

When I was doing the Combine, I was still using the same strategies, but I was manually entereing orders in the T4 platform, which connected to TopStep.

I hope this explains it - if not, please feel free to ask more questions.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #227 (permalink)
 emini_Holy_Grail 
Dallas,TX
 
Experience: Intermediate
Platform: NinjaTrader, OpenQuant
Broker: Zaner/Zen Fire
Trading: ES,6E,6B,GC,CL
 
Posts: 598 since Nov 2009
Thanks: 176 given, 120 received

thks Kevin

I heard Topstep allowed Ninjatrader for a while , but later they stopped.
T4 is ok, but is not as easy as ATM strategy in Ninja's DOM

Reply With Quote
 
(login for full post details)
  #228 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


emini_Holy_Grail View Post
thks Kevin

I heard Topstep allowed Ninjatrader for a while , but later they stopped.
T4 is ok, but is not as easy as ATM strategy in Ninja's DOM

Sierra Charts might be an option, if you are looking to automate with TopStep. I have never used it though.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #229 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received

Kevin,

Is the system really performing as designed? How many trades would it normally have taken over this period? What do the per-trade stats look like in testing vs live?

My fear here is that it is going sideways and you are giving it more time, but what happens should it take a couple of losses in a row? The ultimate question comes down to the first question above -- is it performing as expected, or not.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to Big Mike for this post:
 
(login for full post details)
  #230 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
Kevin,

Is the system really performing as designed? How many trades would it normally have taken over this period? What do the per-trade stats look like in testing vs live?

My fear here is that it is going sideways and you are giving it more time, but what happens should it take a couple of losses in a row? The ultimate question comes down to the first question above -- is it performing as expected, or not.

Mike

Good question. After 7 weeks of live trading, is the system performing as designed?

First, let's look at the number of trades it has taken. A sharp increase or decrease in the number of trades, when compared to the walkforward history, would suggest that the market action is different than the historical market, causing many more, or far fewer, trades to be taken than normal. After 7 weeks, the system has traded for 18 days, or 53% of possible days. Historically, it trades about 151 days per year, or 60% of days. So, the system is trading less than the average. But some years it has traded as few as 130 days (51%) and as many as 175 days (70%). Based on all this I'd conclude that the number of trades is generally in line with expectations, although at the low end. Anecdotally, I have felt that the volatility has been lower than usual. There have been a handful of days where an entry was wished by only a few ticks (the big Fed move day Sept 18 was a case in point). A little more volatility in the hours before the announcement and the system would have entered a big winner.

Second, let's look at average performance, versus actual performance. The average performance of the system gives $1441 profit after 18 trading days. Actual performance, however, is at -$746. This is a HUGE discrepancy, and the conclusion obviously is that the system is not performing as well as it should.

But here is where is gets tricky. Take a coin, and flip it 100 times. If you get 60 heads, do you conclude that the coin is "broken," i.e. biased? What if you got 70 heads, or 80, or 90? Even if you flipped 100 heads in a row, could you absolutely conclude that the coin is biased? No! There is a chance, albeit very, very small, that a fair coin could be flipped to heads 100 consecutive times.

It is the same way with a trading system. You can ask "is it broken?" but the answer will always have some degree of uncertainty. That uncertainty sometimes makes all the difference to your conclusions.

Right now, the actual system performance is at the 10th percentile of what was expected. The 50th percentile would be right at the average, so the 10% mark is pretty bad. But, it is still within the realm of possible outcomes. If it was below the 0 percentile - let's say the system had lost $8,000 through the first 18 days - then the system is obviously not performing as expected. That would be an easy decision. It is where there is uncertainty that things get tough.

I know there are statistical tests that could be run to show whether the current trades could be part of the historical distribution of trades, but even that analysis is not definitive. There is always a gray layer.

So, how do I navigate the gray area? First, I try to determine the parameters that will cause me to quit trading a system. I could certainly use the percentile number approach, and have a quitting rule that says "if after X days the performance is below the Yth percentile, I will cease trading." X and Y would be at your discretion, based on your personal preferences. As long as you stick to the rule you create, you'll be doing fine.

For me, and this particular system, earlier I decided that I would quit only when I hit a $5000 drawdown. So, I am not using the data in the tracking graph to decide when to quit. Sometimes I do use it, though.


So, to summarize:

Is the system performing as expected? No, not even close. It is performing much worse than expected.

Is the system "broken?" Maybe, maybe not. It depends how you define broken. One cannot say definitively it is broken, or it is not broken.

Am I going to quit? No. My quit point, established earlier, calls for a single contract drawdown of $5,000. This was a well thought out amount, and I can't just toss it out the window. I'm going to stick to the plan. I realize, though, that sticking to the plan might be akin to the captain sinking with the ship. It was a risk I was comfortable with at the start, and still am.

Follow me on Twitter Started this thread Reply With Quote
The following 6 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #231 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
Am I going to quit? No. My quit point, established earlier, calls for a single contract drawdown of $5,000. This was a well thought out amount, and I can't just toss it out the window. I'm going to stick to the plan. I realize, though, that sticking to the plan might be akin to the captain sinking with the ship. It was a risk I was comfortable with at the start, and still am.

Thanks for the reply. I am going to pick out just the above portion and ask:

What other "quit points" do you have, other than monetary (if any)?

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #232 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
Thanks for the reply. I am going to pick out just the above portion and ask:

What other "quit points" do you have, other than monetary (if any)?

Mike

For this particular system, the only quit point I am considering is the single contract drawdown. This is simple, and pretty robust. If I am trading the system years from now, with many contracts (my hope, of course), I still have that $5,000 maximum drawdown limit.

In the past, I have used the Monte Carlo simulation results to help me decide when to quit. I have also kicked around, but never implemented, a temporary quit point based on market volatility. When the market gets super crazy, it is best to take a break.

I don't think there is a wrong metric or combination of metrics to use to decide when to quit trading. There is probably no "one size fits all" optimum, either. The key, in my mind, is to select some criteria that you are comfortable with, write it down, and then follow it exactly. Then, if your system fails, there should be no tears. You knew the system could break, and you quit at a pre-ordained spot.

I think where people get in trouble is in not having a "quit point," or their quit point is when their money runs out. Speaking from personal experience in the late 1990's, having to quit trading when your money is gone is not a pleasant way to quit.

Follow me on Twitter Started this thread Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #233 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


kevinkdog View Post
You seem to understand what I am doing almost perfectly. The only difference I can see is that I run the day by day simulation (to get the +/10% curves) at the start of live trading, using all data to that date. So, I don't have to re-run it every day - I only have to add in the actual results.

Thanks.

So do you generate the lower 10% curve by:

1) Running monte carlo simulations for N days of trades, with N varying from 1 to aBigNumber, and taking the desired % cutoff (i.e., "after N trade days 90% of my simulations ended up with an equity above $X"")

That's what seemed tedious to me, and I figured the following was your simple way to get at the same idea:

2) Using the formula: N-trades * AveTradeResult - ZScore * StDevOfTrades * Sqrt(N), where the ZScore corresponds to the %-cutoff you are aiming for.

My confusion is this:

If (1) = no confusion (psych!).

But I thought (2), and can't intuit how that formula gives you a result "close enough" to the result of (1). Can you explain that formula, or wikipedia me to the right spot to do my own digging?

-RK

Reply With Quote
The following 2 users say Thank You to rk142 for this post:
 
(login for full post details)
  #234 (permalink)
 record100 
Toronto, CA
 
Experience: Intermediate
Platform: NT
Broker: IB
Trading: YM, ES
 
record100's Avatar
 
Posts: 105 since Jun 2009
Thanks: 34 given, 44 received


rk142 View Post
Thanks.

So do you generate the lower 10% curve by:

1) Running monte carlo simulations for N days of trades, with N varying from 1 to aBigNumber, and taking the desired % cutoff (i.e., "after N trade days 90% of my simulations ended up with an equity above $X"")

That's what seemed tedious to me, and I figured the following was your simple way to get at the same idea:

2) Using the formula: N-trades * AveTradeResult - ZScore * StDevOfTrades * Sqrt(N), where the ZScore corresponds to the %-cutoff you are aiming for.

My confusion is this:

If (1) = no confusion (psych!).

But I thought (2), and can't intuit how that formula gives you a result "close enough" to the result of (1). Can you explain that formula, or wikipedia me to the right spot to do my own digging?

-RK

Could be central limit theorem or law of big numbers that explains such behavior

Reply With Quote
The following 2 users say Thank You to record100 for this post:
 
(login for full post details)
  #235 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


record100 View Post
Could be central limit theorem or law of big numbers that explains such behavior

Thanks!

I'm trying to learn just enough statistics to improve the evaluation and monitoring of my strategies, without geeking out unnecessarily. I'll look into the central limit theorem - it can up recently in my self-study but I didn't look too closely.

-RK

Reply With Quote
The following 2 users say Thank You to rk142 for this post:
 
(login for full post details)
  #236 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
Thanks.

So do you generate the lower 10% curve by:

1) Running monte carlo simulations for N days of trades, with N varying from 1 to aBigNumber, and taking the desired % cutoff (i.e., "after N trade days 90% of my simulations ended up with an equity above $X"")

That's what seemed tedious to me, and I figured the following was your simple way to get at the same idea:

2) Using the formula: N-trades * AveTradeResult - ZScore * StDevOfTrades * Sqrt(N), where the ZScore corresponds to the %-cutoff you are aiming for.

My confusion is this:

If (1) = no confusion (psych!).

But I thought (2), and can't intuit how that formula gives you a result "close enough" to the result of (1). Can you explain that formula, or wikipedia me to the right spot to do my own digging?

-RK


The plot I show is generated by day by day Monte Carlo analysis. It is not as tedious as it sounds, if you set up the spreadsheet and run a bunch of macros.

You should be able to run the formula in #2 to get pretty close to the same results to the simulation in #1. It won't be exact (there will be a difference, for example, if the Monte Carlo loss limit is hit), but it should be close. Here is a link to another system I ran both the Monte Carlo and the equation method:


I honestly don't know where I got that equation, so I can't point you anywhere. As @record100 says, it is probably due to central limit theorem or some other simple statistics tool (Simple is all the statistics I am good at!).

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #237 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


kevinkdog View Post
The plot I show is generated by day by day Monte Carlo analysis. It is not as tedious as it sounds, if you set up the spreadsheet and run a bunch of macros.

I'm actually a little pleased that I've absorbed enough of your logic to anticipate the day by day Monte Carlo approach, while also a little disappointed that I missed it in your other thread - since I read it from start to finish! I must've not had my head wrapped around what you were doing, so it went in one ear and out the other. Now that I've thought it through for myself the post you linked to makes perfect sense.

I'm working on some Monte Carlo macros in Open Office, and will play around with making the under/over performance tracking curves.

Continued gratitude...

-RK

Reply With Quote
 
(login for full post details)
  #238 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
having to quit trading when your money is gone is not a pleasant way to quit.

I am surprised drawdown is a "make or break" measurement for you (the only item on the list).

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #239 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
I'm actually a little pleased that I've absorbed enough of your logic ...

I'm actually a little scared someone understands me that well!!!

Just remember, I am just presenting the way I do things, and that is not the only way, and very possibly not even the best way...

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #240 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
I am surprised drawdown is a "make or break" measurement for you (the only item on the list).

Mike

I'm curious, what were you expecting, and why? It would be interesting to hear your take on things.


I can say that using more than the max drawdown as quit criteria was driven in part by the small size of the max historical drawdown. I definitely do not always use max drawdown as my criteria.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #241 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
I'm curious, what were you expecting, and why? It would be interesting to hear your take on things.


I can say that using more than the max drawdown as quit criteria was driven in part by the small size of the max historical drawdown. I definitely do not always use max drawdown as my criteria.

I am afraid to comment more because I really don't have experience with systems that take this few trades.

But I do have experience with systems that take far, far more trades. In those examples, say the system trades 50 times a month, then I am more comfortable because I can look at the trade frequency, duration, win/loss percentage, drawdown, MAE, MFE, or any other slew of metrics and only need to wait a month for meaningful results (50 samples).

I always use drawdown as a quitting point, but never exclusively. The system may be within normal behavior on drawdown by wildly deviating on other metrics, and that would cause me great concern.

I was just surprised you listed drawdown as the sole qualifier. In essence by saying your only quitting point is based on drawdown, you are measuring the entire system based on this one thing. Am I wrong?

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #242 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
I am afraid to comment more because I really don't have experience with systems that take this few trades.

But I do have experience with systems that take far, far more trades. In those examples, say the system trades 50 times a month, then I am more comfortable because I can look at the trade frequency, duration, win/loss percentage, drawdown, MAE, MFE, or any other slew of metrics and only need to wait a month for meaningful results (50 samples).

I always use drawdown as a quitting point, but never exclusively. The system may be within normal behavior on drawdown by wildly deviating on other metrics, and that would cause me great concern.

I was just surprised you listed drawdown as the sole qualifier. In essence by saying your only quitting point is based on drawdown, you are measuring the entire system based on this one thing. Am I wrong?

Mike


OK, I think I understand what you are getting at, and it is something I really haven't discussed. What happens when the system performance isn't bad enough to hit your quitting point, and it is not great either, but is somewhere in between? Maybe it is making money, or breaking even, and it is within the bounds of what Monte Carlo sim says is possible. When do you quit, or otherwise cease trading the system?

My general philosophy is to watch the downside, and let the upside take care of itself. So, in this case, I watch the max drawdown, and as long as that is not hit, let the system perform. I do this because I never know month to month or year to year what particular systems I am trading will do good, which will do bad, and which will just tread water. So, normally I'll just let system be, and not turn them off or on.

But, a couple times a year I rebalance systems I am trading - add in new ones, and cull the underperformers and possibly adjust the position sizing. If capital becomes an issue, I might very well stop trading and swap out a mildly profitable, but underperforming, system for a system that I feel has more potential. The analysis details are never the same, and I don't have strict rules on this. I might, for example, stop trading a system because I no longer like it for some reason - maybe it just doesn't fit me anymore.

I realize that I am sort of talking out both sides of my mouth here. On one side I say "maximum drawdown is my only quitting criteria." On the other side I say "unless I come up with another legitimate reason to cease trading it." I rationalize this by saying the maximum drawdown is a hard, solid, worst case criteria, and will not be violated. At the same time, though, there may be other circumstances which arise which cause the system to fall out of favor. These circumstances might cause me to quit earlier. That is a big gray area.

I can say my plan right now only includes maximum drawdown as sole criteria for quitting. With the small size and account size I am trading this system with, I can't foresee needing the capital for a better system. But, you never know...


I hope this answers your question. Putting this down in words makes me realize there are always caveats to hard and fast rules.

Follow me on Twitter Started this thread Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #243 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received


kevinkdog View Post
I'm actually a little scared someone understands me that well!!!

Just remember, I am just presenting the way I do things, and that is not the only way, and very possibly not even the best way...

Of course! I'm aware of the impulse to make an idol or a fetish out of someone else's approach, and I'm sure in the end my decisions will be different than yours (they already are in some respects). But what you've shared so far resonates with me, and I am picking up things. I haven't changed my process at all yet - I monitor in the same way - but I am thinking about how to incorporate some of what I learned when bringing the next ruleset online.


Big Mike View Post
I always use drawdown as a quitting point, but never exclusively. The system may be within normal behavior on drawdown by wildly deviating on other metrics, and that would cause me great concern.

I don't have Mike or Kevin's experience, so all of Kevin's disclaimers about not knowing the one "true way" apply to me but to the nth power, but I intuitively gravitated on my own towards a sole focus on drawdown. Drawdown is the first thing I look at with a new ruleset. If I establish what the drawdowns look like for a system and I have enough capital to size so that the max drawdown is within my pain tolerance, then I can trade that system. When the drawdown exceeds my tolerance I'm done. If the system has a long term positive expectancy, then pretty much nothing else matters (to me).

I have a tendency to overthink, so if I brought in other metrics I'd probably enable my risk-averse self-saboteur to stop the pain early. It's the system-trade version of tightening your stop on a discretionary trade too early:

Put your system stop where you know you are wrong, make sure that that stop subjects you to an amount of psychological and financial pain that leaves you in a position to fight another day, and let the "trade" play out.

Other, more experienced people (Mike and Kevin) can surely assess a system's performance and quit early, but my fiddling, both with rule-based and discretionary trades, has caused demonstrable deterioration in my results.

All right, I'm a hobby trader, and I'm violating my all important time-management rules by doing trade related stuff before 10AM, so I've got to get out of here....

-RK

Reply With Quote
The following 2 users say Thank You to rk142 for this post:
 
(login for full post details)
  #244 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


rk142 View Post
Kevin,

Excellent thread. Thank you.

In response to a question about your approach to optimization you wrote the following:



Could you say a bit more about the tools / decision making process you use to make decisions about the IN/OUT periods you use?

I'm afraid of optimization.

thanks,
RK

One common mistake during walkforward analysis is to surreptitiously optimize the IN and OUT periods. Say, for example, that you run the walkforward analysis with 4 year In period, and 1 year Out period. Walkforward results for that case are good, but not great, so you think "maybe I should use 4 years In, with 2 years out." That case is 200% better, and meets all your goals, so you decide "that's the combination to use. Let's go!"

Stop.

Do you realize what just happened? As soon as you selected a second set of In/Out parameters, reran the results, and selected the best case, you just optimized. Sure, it is not a full optimization, since you only compared two cases, but it was optimization nevertheless. Remembering the rule that optimized results can't be trusted, you have a dilemma here: accept the first run (4 year / 1 year), and then discard the strategy because it did not meet your goals, or accept the second run, and pretend you never optimized.

Once again, I'll admit to doing the above on occasion, although I can't recall it ever ending well. The big question in all this is "is there a way to test multiple IN/OUT periods, and select the best one, while still maintaining walkforward integrity?" The answer, thankfully, is yes. The way to do it is to create, in essence, a second walkforward analysis inside of the first. The way to do this is to run the walkforward analysis, as usual, but leave the last few years of data untouched. I typically will leave three years untouched. Then, with the walkforward data I have, I select the best IN/OUT pair I have, and then run it on the last three years of data. If it passes, then I go on to the next step. If it doesn't, I discard the strategy. But, in either case, at least I have made some effort to select the best IN/OUT combination. The downside to this approach is that you have optimized, and the more optimization you do, the worse off you generally are.

This process would look like this:

1. Years 2000-2008 >> run walkfoward analysis for different combinations of In/Out periods, select the best In/Out

2. Years 2009-present >> run walkforward analysis, using best In/Out determined from Step 1

3a. If walkforward results from 2009-present look good, continue with development

3b. If results do not look good, it is probably best to abandon strategy, rather than try again with another In/Out pair

Follow me on Twitter Started this thread Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #245 (permalink)
 Jura 
The Netherlands
 
Experience: None
Platform: MultiCharts, TradingView
Trading: ...
 
Jura's Avatar
 
Posts: 774 since Apr 2010
Thanks: 2,347 given, 688 received


kevinkdog View Post
Right now I am not doing anything with TopStep - the system I am currently running is automated through Tradestation, and traded through their brokerage.

When I was doing the Combine, I was still using the same strategies, but I was manually entereing orders in the T4 platform, which connected to TopStep.

I hope this explains it - if not, please feel free to ask more questions.

What is your take on the use/value of TST for algorithmic automated traders, Kevin?

I know from your earlier posts that your performance was suffering due to needing to manually enter the orders. But let's assume for a moment that one day in the future TST offers support for automated trading. Would you then do another combine or not? And why?

I on the one hand think that doing a successful combine might be a good way to trade a larger capital and make use of the scalability of a strategy. On the other hand, the parameters that TST use are pretty strict and these probably will add to (even) more overfitting during the strategy design process. Plus, designing a profitable strategy is hard enough already without those 'artificial' parameters, I think.

Reply With Quote
 
(login for full post details)
  #246 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Jura View Post
What is your take on the use/value of TST for algorithmic automated traders, Kevin?

I know from your earlier posts that your performance was suffering due to needing to manually enter the orders. But let's assume for a moment that one day in the future TST offers support for automated trading. Would you then do another combine or not? And why?

I on the one hand think that doing a successful combine might be a good way to trade a larger capital and make use of the scalability of a strategy. On the other hand, the parameters that TST use are pretty strict and these probably will add to (even) more overfitting during the strategy design process. Plus, designing a profitable strategy is hard enough already without those 'artificial' parameters, I think.


I think once TST gets Ninja working, automated strategies will be viable. I think they got rid of many of the performance metrics.

Creating successful intraday strategies is tough, though. I find it much easier to create swing strategies, lasting days to weeks.

Follow me on Twitter Started this thread Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #247 (permalink)
 indextrader7 
Birmingham, AL
 
 
Posts: 1,065 since Apr 2012


kevinkdog View Post
Creating successful intraday strategies is tough, though. I find it much easier to create swing strategies, lasting days to weeks.

Hey Kevin,

Why do you think this is? I'd be curious to know your thoughts on it.

Reply With Quote
 
(login for full post details)
  #248 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


indextrader7 View Post
Hey Kevin,

Why do you think this is? I'd be curious to know your thoughts on it.

Great question, Merritt. Here are my thoughts (I encourage people to disagree with my points, and create discussion from which we all can benefit).


Whenever I begin looking at a new strategy, I almost always see if it can be made into a day trading strategy. I define "day trading" as in and out of a trade, or multiple trades, in a single session. There are some nice benefits to such a strategy:

1. No overnight risk from unexpected events, since you are flat

2. Reduced margin requirements, making it easier to trade with large size (although most people should not be doing this)

3. A "working job" type feeling. You fire up the computer in the morning, trade a while, make your daily nut, turn off the computer, go home and play with your kids the rest of the night. Very satisfying way to live.


So, usually when I start development, I select short timeframe bars (1 minute to 5 minute), throw in the "set exit on close" statement to exit at the end of the day, and jump into development.

Nine times out of ten, though, the strategy fails. Regardless of the entry idea (trend, counter trend, whatever) and exit scheme (fixed stops, moving stops, breakeven stops, profit targets, etc.) nothing ever seems to work.

Inevitably, if I like the strategy idea, I'll then open up the timeframe, to 60 minute bars, 240 minute bars, daily bars. I want to see if my idea has any validity at all.

What almost always happens? Performance gets better! Maybe the performance still doesn't meet my goals, but the performance on a daily chart is almost always better than on a 1 minute chart. I've seen it enough times to realize it is more than a coincidence. The question then becomes: why do I see this behavior? Here's what I have come up with:

1. Number of trades and trading costs. Let's say I have daily bar strategy that trades one time per month, or once every 20 bars. That will cost me roughly $25 in trading costs. If I go down to 1 minute bars, the same strategy might trade 10 times per day (once every 120 bars), or $250 in trading costs. That is a huge difference in costs that must be overcome. Add in the fact that 1 minute moves are smaller than daily moves, and it gets even harder.

2. There seems to be more randomness in the data as you go to smaller timeframe bars. Look at a 1 minute chart of ES and most days it is just narrow range noise. It is harder to find the true price path when the random noise level is high. Daily bars, as an alternative, seem to have more trends. Of course, where I see randomness in data could just be due to other biases I have floating around in my brain.

3. Entry and exits become a much more important part of the system, when you have small stops and targets, as most day trading systems are set up to be. So, I must have really great entries and exits, ones with very good edges. But, the better the entry, the harder it is to find during development. Plus, miss the entry by a tick, and you may lose a good percentage of your profit. If you are swing trading with daily bars, a tick or two at entry probably won't mean as much, relative to overall size of an average trade.

4. With tick charts or 1-5 minute charts, think about who you are trading against. Many times it is high frequency trading firms, who probably have better entries than you, and have a speed advantage over you. I feel the impact of the pros is less noticeable at higher timeframes, although I realize many pros trade daily bars, too.

5. With most strategies, as I mentioned before, I find the less trades there are, the better. This could be due to trading costs, but it also could be due to a very bad reason: maybe you think you have an edge, but with few trades, the statistical confidence that you have an edge is a lot lower. Put another way, if I had 2 strategies that averaged $50 profit per trade, and one had 100 trades over the past ten years, and another had 1,000, I'd always pick the 1,000 trade strategy (so would every rationale person). But, the reality is that the 100 trade strategies are a lot easier to find - maybe it is because they aren't really edges at all?

I really wish that all my strategies were day trading type strategies. In actuality, probably 9 out of 10 are the exact opposite. In fact, my best strategy over the past 4 or 5 years holds a position for weeks to months.


I would love to hear the opposite viewpoint to what I have experienced - why someone finds it easier to develop intraday strats.

Follow me on Twitter Started this thread Reply With Quote
The following 9 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #249 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received

Years ago I did exclusively intraday type strategies. I also believe automation should leverage its strengths of speed (more trades, faster reaction times).

However, over the years I've drifted to portfolio strategies that trade a basket of stocks or futures. This gives me the increased frequency I demand and creates some intraday trades as a group.

I find this to literally be the holy grail and cannot encourage it enough over the traditional one strategy one instrument system.

Portfolio trading is the best way to manage risk IMO. After you do that, profits are the easy part.

Sent from my LG Optimus G Pro

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 9 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #250 (permalink)
 Luger 
Nashville, TN
 
Experience: Intermediate
Platform: NinjaTrader
Broker: IB
Trading: NQ ES
 
Posts: 468 since Feb 2011
Thanks: 323 given, 543 received


kevinkdog View Post
Inevitably, if I like the strategy idea, I'll then open up the timeframe, to 60 minute bars, 240 minute bars, daily bars. I want to see if my idea has any validity at all.

What almost always happens? Performance gets better! Maybe the performance still doesn't meet my goals, but the performance on a daily chart is almost always better than on a 1 minute chart. I've seen it enough times to realize it is more than a coincidence.

I would love to hear the opposite viewpoint to what I have experienced - why someone finds it easier to develop intraday strats.

Sorry, I'm not going to disagree. I didn't have success with automated strategies until I took the leap from intraday to swing. Funny thing is that swing is the one time frame that I never thought I would trade, it felt like limbo-land between technicals and fundamentals, but apparently it is a decent place to trade.

As to the reasons, I believe that @Fat Tails could give us a math lesson that would prove that longer time frames should be more profitable. He posted it in another thread or threads because the timeframe topic is usually addressed as a sub-topic to something else. Though I believe it gets down to the points you made about noise and costs.

Though one thing I personally find interesting is that many times I end up building systems on a lower time frame than expected given the trade duration. I don't think most people would build a system with a 3.5 day average trade duration based on a 10-minute chart (about 140 bars per trade). Then again, I have not heard statistics of many profitable systems that people run for any period of time.

Reply With Quote
The following 3 users say Thank You to Luger for this post:
 
(login for full post details)
  #251 (permalink)
 Fat Tails 
Market Wizard
Berlin, Europe
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
 
Fat Tails's Avatar
 
Posts: 9,855 since Mar 2010
Thanks: 4,238 given, 26,742 received


Luger View Post
As to the reasons, I believe that @Fat Tails could give us a math lesson that would prove that longer time frames should be more profitable. He posted it in another thread or threads because the timeframe topic is usually addressed as a sub-topic to something else. Though I believe it gets down to the points you made about noise and costs.


Technically, there is a sweet spot between smaller timeframe trading (scalping) and larger timeframe trading (investing). The exact location of that sweet spot depends on variable costs such as slippage and commissions. We have already discussed this subject in the thread on "Risk of Ruin".

The question was asked by @Big Mike in this post:



and I had tried to give a simplified answer here:




The main idea was to compare risk-adjusted returns. Usually the shorter timeframe system (intraday trading) comes out winner compared to the longer timeframe system (swing trading), so I am not entirely with you.
The reason is that if you trade a longer timeframe the variance of returns is much larger compared to the variance of returns of the system that trades faster. In the example discussed in the thread on "Risk of Ruin", the higher timeframe system has a risk of ruin about 1,350 times higher than the smaller timeframe system.

As the fast trading system has a lower variance of returns and thus a lower risk of ruin, you may increase leverage and trade a larger number of contracts.

However, intraday trading also has its downsides. Commissions and slippage are much higher in relation to the average return per trade. Also intraday trading takes a higher toll on the health of the trader, unless the trades are fully automated.

Reply With Quote
The following 2 users say Thank You to Fat Tails for this post:
 
(login for full post details)
  #252 (permalink)
 xelaar 
prague, czech republic
 
Experience: Intermediate
Platform: NT7, MT4
Broker: LMAX
Trading: DAX, Gold, Euro
 
xelaar's Avatar
 
Posts: 1,517 since Feb 2013
Thanks: 1,740 given, 2,566 received

Excellent conversation, guys!

Trade to live. Not live to trade.
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #253 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
Years ago I did exclusively intraday type strategies. I also believe automation should leverage its strengths of speed (more trades, faster reaction times).

However, over the years I've drifted to portfolio strategies that trade a basket of stocks or futures. This gives me the increased frequency I demand and creates some intraday trades as a group.

I find this to literally be the holy grail and cannot encourage it enough over the traditional one strategy one instrument system.

Portfolio trading is the best way to manage risk IMO. After you do that, profits are the easy part.

Sent from my LG Optimus G Pro

Mike -

When you develop a portfolio strategy for a basket of futures, are you using the exact same strategy for each futures instrument? If so, are you using the same parameters for each instrument, or do you "tune" for each?

What do you think makes it the Holy Grail? The diversification aspect?

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #254 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
Mike -

When you develop a portfolio strategy for a basket of futures, are you using the exact same strategy for each futures instrument? If so, are you using the same parameters for each instrument, or do you "tune" for each?

What do you think makes it the Holy Grail? The diversification aspect?

I have tried all kinds of options.

Presently spending my time on a single strategy with two parameters, a moving average length and an oscillator length. These same parameters have been tested against dozens and dozens of uncorrelated instruments and have produced good results in the portfolio so far.

In this case, I do use NT to optimize the entire basket/group at one time, for the best parameter across say 20 stocks over say 10 years of data.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 4 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #255 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
I have tried all kinds of options.

Presently spending my time on a single strategy with two parameters, a moving average length and an oscillator length. These same parameters have been tested against dozens and dozens of uncorrelated instruments and have produced good results in the portfolio so far.

In this case, I do use NT to optimize the entire basket/group at one time, for the best parameter across say 20 stocks over say 10 years of data.

Mike

Thanks for sharing. I wonder how much of your performance is attributable to entries and exits being of high quality, as opposed to so-so entries and exits, but with superior diversification. I personally have found that diversification plays a major role in overall success.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #256 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
Thanks for sharing. I wonder how much of your performance is attributable to entries and exits being of high quality, as opposed to so-so entries and exits, but with superior diversification. I personally have found that diversification plays a major role in overall success.

It's all about diversification. Any one strategy on one instrument is nothing special, but put a dozen of them in a basket and the equity curve and drawdowns can become magical.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 3 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #257 (permalink)
 indextrader7 
Birmingham, AL
 
 
Posts: 1,065 since Apr 2012


kevinkdog View Post

So, usually when I start development, I select short timeframe bars (1 minute to 5 minute), throw in the "set exit on close" statement to exit at the end of the day, and jump into development.

Nine times out of ten, though, the strategy fails.

I've heard FT71, on multiple occasion, talk about how it's almost totally about the exit, and that one can make money with a random entry method; and that he can prove it.

What would you say about that statement?

I think THAT would be an excellent webinar..... and thusly, shatter the academic world, and have Eugene Fama running for the hills.

Reply With Quote
 
(login for full post details)
  #258 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


indextrader7 View Post
I've heard FT71, on multiple occasion, talk about how it's almost totally about the exit, and that one can make money with a random entry method; and that he can prove it.

What would you say about that statement?

I think THAT would be an excellent webinar..... and thusly, shatter the academic world, and have Eugene Fama running for the hills.

I would agree in principle with that assertion. Van Tharp and hedge fund manager Tom Basso did the same thing a bunch of years ago. I wrote a few articles for Active Trader Magazine a few years back looking into that - random entries, and random exits.

I think it is timeframe dependent though. A scalping strategy will probably never work with random entries. A long term swing strategy very well might.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #259 (permalink)
 indextrader7 
Birmingham, AL
 
 
Posts: 1,065 since Apr 2012

I read an academic study once in college that showed how there was more skew and kurtosis in longer timeframes than shorter ones (in forex). This could also be another reason to add to the thorough ones you listed previously.

Reply With Quote
 
(login for full post details)
  #260 (permalink)
 ehlaban 
Netherlands
 
Experience: Advanced
Platform: Ensign, Multicharts
Trading: SP500
 
Posts: 92 since Nov 2009
Thanks: 66 given, 57 received


kevinkdog View Post
.... looking into that - random entries, and random exits....

Isn't it about that you can have a winning system with random entries but with non random exits.
The exit counts so it shouldn't be random.

Reply With Quote
 
(login for full post details)
  #261 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received

Check John Ehlers webinar on futures.io (formerly BMT) for an engineers take on time frames and efficiency, noise, etc.

Sent from my LG Optimus G Pro

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #262 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


ehlaban View Post
Isn't it about that you can have a winning system with random entries but with non random exits.
The exit counts so it shouldn't be random.

I'd have to dig up my old work, but if I recall correctly, random entries with real exits did work better than real entries with random exits.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #263 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
I'd have to dig up my old work, but if I recall correctly, random entries with real exits did work better than real entries with random exits.

I did some work on this a few years ago, check here:





Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 2 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #264 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

Week 8 of trading the NGEC system with actual money is now complete.





Summary: First, let's look at the big picture. I like to do this every month or so, because at a glance I can tell if things are going as planned or not. From look at this first chart, a couple of things are clear:


1. Over the whole course of the system history (walkforward, incubation, live), the system performance hasn't changed much. I could draw a line from the start of walkforward to the start on incubation, and then another line from the start of incubation until the present time, and the slopes of those two lines would be about the same. This gives me some reassurance that the system is behaving, subject to point 2 below.

2. It is easy to see that the live trading (green line) has not been up to par at all. The performance these past 2 months has been down, and while it has not crashed and burned, it certainly has been a disappointment.


So, after 8 weeks of trading this system live, I am down about 5% from the start.

Am I surprised at this result? Absolutely not. It is well within expectations.

Am I disappointed in the results so far? Yes. After 8 weeks, I had hoped to be making some money. The performance these past 8 weeks is way behind the long term average, so it is very disappointing.

Are results in line with expectations? Yes. The current profit is below the average I expect, and it is right around lower 10% line. So, it is underperforming currently, and I will be concerned if equity drops below that 10% line. Also, I have had 4 winning weeks, and 4 losing weeks. Over time, I expect about 60% of my weeks to be profitable, so the performance is a bit behind in that regard. Plus, one week was really, really bad.

Are fills and trades live comparable to Tradestation strategy report? Yes, in fact in most cases my fills are better than what I had anticipated. Slippage is usually less than I had expected.

Do I see any reason to stop trading this system? No.

Do I see any reason to change my position sizing plan, i.e. reduce or increase my risk? No.


So, after 8 weeks, I will keep on trading per the plan, but storm clouds are forming on the horizon. I need some sunshine instead!








Note that there is some difference in performance numbers, as I did not include commission rebates yet.

Follow me on Twitter Started this thread Reply With Quote
The following 15 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #265 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

You've probably heard the phrase "automated trading does not mean unattended trading." You've also probably read all the disclaimers that the brokerage throws at you before they allow you to turn automation on. With Tradestation, for example, there are a tom of disclaimers you have to sign when you open an account, and then if you want to automate a strategy, there are two more disclaimers you have to click and accept. The first disclaimer is 397 words, and the second is a whopping 593 words. That is a lot of legalese to wade through, just to automate your trading.

But, all those warnings are there for a reason. Ignore them at your own risk. That's what happened to me today. Let me explain...

My NGEC strategies enter on limit orders, which are supposed to be active for the current bar only. After the bar closes, any open orders get cancelled by the software. Depending on the strategy logic, another limit order may be placed for the current bar.

So, Monday night, my night strategy placed a limit order to buy, well below the market. It did not get filled during the bar, so it should have been cancelled. For whatever reason, it was not cancelled. This is the first time I have EVER seen this occur. The success rate of the software auto-cancelling orders, from my experience, has to be well over 99%. That is excellent, but errors can and do occur. Just look at the airline industry, for example. There are roughly 28,000 commercial flights per day, and it 99.99% had successful takeoffs and landings, 2 or 3 planes would crash per day. So, anything less that perfection runs the risk of costing you money.

The order rested at the exchange until 5 AM Tuesday morning, when it was filled. I noticed this rouge position Wednesday morning. Of course, with Murphy's Law in effect here, I noticed it not when the position was profitable, but after it had gone negative. Then, to add insult to injury, while I investigated the issue - before I exited the position - I watched it drop another $125 or so. Once I confirmed the position was wrong, I exited with about a $550 loss.

Who's to blame here? Well, the software did not do its job, since it should have cancelled the order. But, ultimately, I can blame no one, and no thing (software, internet connection, etc.), except myself for the error. I'll repeat that: I am to blame! I am the caretaker, and if things go wrong, as they occasionally will, it is up to me to be aware of it and fix it. I take full responsibility for this screw up, and I am taking some steps to make sure it is not repeated:

1. Check statements every day. If I would have checked it this morning, I'd probably exited with a $300 profit, instead of a $550 loss.

2. Check platform every few hours for uncancelled orders.

3. Improve checking of positions. I normally check my positions every few hours, but somehow this one slipped by me.

4. Make sure filled orders show up on the chart. For some reason, this fill did not - they usually do.



Follow me on Twitter Started this thread Reply With Quote
The following 6 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #266 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,965 since Dec 2010
Thanks: 3,035 given, 2,378 received


kevinkdog View Post
You've probably heard the phrase "automated trading does not mean unattended trading." You've also probably read all the disclaimers that the brokerage throws at you before they allow you to turn automation on. With Tradestation, for example, there are a tom of disclaimers you have to sign when you open an account, and then if you want to automate a strategy, there are two more disclaimers you have to click and accept. The first disclaimer is 397 words, and the second is a whopping 593 words. That is a lot of legalese to wade through, just to automate your trading.

But, all those warnings are there for a reason. Ignore them at your own risk. That's what happened to me today. Let me explain...

My NGEC strategies enter on limit orders, which are supposed to be active for the current bar only. After the bar closes, any open orders get cancelled by the software. Depending on the strategy logic, another limit order may be placed for the current bar.

So, Monday night, my night strategy placed a limit order to buy, well below the market. It did not get filled during the bar, so it should have been cancelled. For whatever reason, it was not cancelled. This is the first time I have EVER seen this occur. The success rate of the software auto-cancelling orders, from my experience, has to be well over 99%. That is excellent, but errors can and do occur. Just look at the airline industry, for example. There are roughly 28,000 commercial flights per day, and it 99.99% had successful takeoffs and landings, 2 or 3 planes would crash per day. So, anything less that perfection runs the risk of costing you money.

The order rested at the exchange until 5 AM Tuesday morning, when it was filled. I noticed this rouge position Wednesday morning. Of course, with Murphy's Law in effect here, I noticed it not when the position was profitable, but after it had gone negative. Then, to add insult to injury, while I investigated the issue - before I exited the position - I watched it drop another $125 or so. Once I confirmed the position was wrong, I exited with about a $550 loss.

Who's to blame here? Well, the software did not do its job, since it should have cancelled the order. But, ultimately, I can blame no one, and no thing (software, internet connection, etc.), except myself for the error. I'll repeat that: I am to blame! I am the caretaker, and if things go wrong, as they occasionally will, it is up to me to be aware of it and fix it. I take full responsibility for this screw up, and I am taking some steps to make sure it is not repeated:

1. Check statements every day. If I would have checked it this morning, I'd probably exited with a $300 profit, instead of a $550 loss.

2. Check platform every few hours for uncancelled orders.

3. Improve checking of positions. I normally check my positions every few hours, but somehow this one slipped by me.

4. Make sure filled orders show up on the chart. For some reason, this fill did not - they usually do.



Ouch, important message, but tougher to live. Fortunately your losses were not that great.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to NJAMC for this post:
 
(login for full post details)
  #267 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


NJAMC View Post
Ouch, important message, but tougher to live. Fortunately your losses were not that great.

Not a killer loss, but enough for me to getting pretty pissed! Percentage wise, for this account, the losses were 7%, so that is pretty bad. Eventually, I want this strategy to get to 10 contracts, and at that point it would be some serious money.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #268 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

It just figures...

I noticed the bad trade about an hour ago, 30+ hours after I entered it. Then, I exited at almost the worst time, with a $550 loss. Now, less than 1 hour later, I could have exited with less than $100 loss.

Is it just me, or does this kind of stuff happen to you too? If I did not know better, I'd swear that someone was controlling prices, and watching my positions, and deliberately doing things to maximize my losses!

I can definitely see why people feel this way - at times, it surely feels like the market is out to get personally get me!


I know why some of this occurs:

I tend to discount - not really notice or dwell on - every good thing that happens to me (mistakes in my favor, excellent news reports right after I enter a position, etc). Money making anomalies become just a blip in the equity curve. They are nice, but I don't really remember them too well.

BUT, for losses due to mistakes, I tend to remember them and keep them filed in my memory bank. Some of these "losing lessons" might be good to remember - today's automated issue is a good example - but most should be forgotten just as easily as the money making mistakes. That doesn't happen for me, at least not usually.

Last year I actually did a study of this. I added up all my winning mistakes with all my losing mistakes. The net impact, from a monetary basis, was just about zero. But, from a psychological standpoint, it definitely was not breakeven.

Follow me on Twitter Started this thread Reply With Quote
The following 4 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #269 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,965 since Dec 2010
Thanks: 3,035 given, 2,378 received


kevinkdog View Post
It just figures...

I noticed the bad trade about an hour ago, 30+ hours after I entered it. Then, I exited at almost the worst time, with a $550 loss. Now, less than 1 hour later, I could have exited with less than $100 loss.

Is it just me, or does this kind of stuff happen to you too? If I did not know better, I'd swear that someone was controlling prices, and watching my positions, and deliberately doing things to maximize my losses!

I can definitely see why people feel this way - at times, it surely feels like the market is out to get personally get me!

@kevinkdog,

Happens to me as well, always out to maximize my losses. Welcome to the club!

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to NJAMC for this post:
 
(login for full post details)
  #270 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


NJAMC View Post
@kevinkdog,

Happens to me as well, always out to maximize my losses. Welcome to the club!


One would think this feeling ("the market is out to get me") would have disappeared after 20+ years of trading. But it hasn't, and I suspect it never will!

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #271 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,965 since Dec 2010
Thanks: 3,035 given, 2,378 received


kevinkdog View Post
One would think this feeling ("the market is out to get me") would have disappeared after 20+ years of trading. But it hasn't, and I suspect it never will!

@kevindog,

I know they are watching me, that is why I need to get my machine learning system running. That is so they can't see anymore! (Paranoia, yes, somewhat accurate, probably)

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #272 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


NJAMC View Post
@kevindog,

I know they are watching me, that is why I need to get my machine learning system running. That is so they can't see anymore! (Paranoia, yes, somewhat accurate, probably)

Won't they still be able to see your machine learning system too?

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #273 (permalink)
 NJAMC 
Atkinson, NH USA
 
Experience: Intermediate
Platform: NinjaTrader 8/TensorFlow
Broker: NinjaTrader Brokerage
Trading: Futures, CL, ES, ZB
 
NJAMC's Avatar
 
Posts: 1,965 since Dec 2010
Thanks: 3,035 given, 2,378 received


kevinkdog View Post
Won't they still be able to see your machine learning system too?

There is just no hope, I am going to drop trading and go back to Basket Weaving. If they watch me do that, maybe they can help me improve my technique.

Nil per os
-NJAMC [Generic Programmer]

LOM WIKI: NT-Local-Order-Manager-LOM-Guide
Artificial Bee Colony Optimization
Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #274 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received

Kevin; Doesn't your system place a stop when you enter a trade?

Reply With Quote
The following user says Thank You to deaddog for this post:
 
(login for full post details)
  #275 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
Kevin; Doesn't your system place a stop when you enter a trade?

Yes it does, but only when the strategy thinks there is an open position. To explain this a bit, there are 2 pieces to this:

Strategy Position - this is the theoretical position, based on the strategy. This will calculate without needing a live account

Live Account Position - this is the position that is actually in your account. It should always match the Strategy Position, but sometimes (like this situation) it doesn't.


So what happened here is that the Strategy Position was flat the whole time. Therefore, the strategy never sent any kind of stop order to the live account.


One possible solution here is to setup a routine to monitor strategy position vs real world position, and notify me when there is a discrepancy. I am pursuing that for a longer term solution.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #276 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received

I take it the breakdown was in the strategy software. Would it be of benefit to set up your live account with an automatic attached order with your max stop.

This way the live account would have a stop order working with each position. I wouldn’t matter if the strategy software malfunctioned.

Reply With Quote
The following user says Thank You to deaddog for this post:
 
(login for full post details)
  #277 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
I take it the breakdown was in the strategy software. Would it be of benefit to set up your live account with an automatic attached order with your max stop.

This way the live account would have a stop order working with each position. I wouldn’t matter if the strategy software malfunctioned.

Unfortunately, the way I have it set up, that is not possible - they do not allow automated OCO orders, unless I do some major rewriting of the code to use order macros.

It is an option that I will consider. But I am not blaming the software for this situation. I should have caught the issue yesterday morning, shortly after it happened.

Follow me on Twitter Started this thread Reply With Quote
 
(login for full post details)
  #278 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received

Might be something to consider re writing your codes for. If your orders go in with an attached stop then whatever disaster happens at your end the stop is with the broker.

Reply With Quote
The following user says Thank You to deaddog for this post:
 
(login for full post details)
  #279 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

What should have been a decent winning week turned out to be a losing week, due to the entry "glitch" described earlier. That cost me $550. I have taken some immediate steps to prevent it from happening again, and I think I am in the clear, at least for the time being. My actual performance now lags the perfect strategy performance, and the issue this past week is to blame for that.

The problem ultimately comes down to this: if I am depending on a computer to place, cancel, replace, etc. orders, unless someone is monitoring it at all times, there is always a possibility of something going haywire. The question is how much of my limited resources (it is just me, after all) do I devote to the effort to make the automation goof proof? Doing the usual amount of effort hasn't worked - since I just lost $550 - but at what point will I be confident those issues will not occur? Plus, it is usually not the issues you know about, but the issues you don;t know about. This case was one of them. After thousands of automated trades over the years, this particular issue had never come up before!

Enough about that issue for now. Performance is still lagging, and the day system has not had a trade in two weeks (I think I can blame lower volatility for that).

Regardless, I will "keep on keepin' on!"








Follow me on Twitter Started this thread Reply With Quote
The following 5 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #280 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received

I do not see the trade where you lost $550. Are you reporting by trade or by the day?

Reply With Quote
 
(login for full post details)
  #281 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
I do not see the trade where you lost $550. Are you reporting by trade or by the day?

It occurred on 10/16. Later in the day I had a profitable trade, so the net for the day was only about -$350. Note the difference that day between "perfect" strategy performance and actual real money performance.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #282 (permalink)
 MWinfrey 
Lubbock TX
 
Experience: Intermediate
Platform: NinjaTrader
Broker: Stage 5 Trading
Trading: CL
 
MWinfrey's Avatar
 
Posts: 1,880 since Jul 2009
Thanks: 1,450 given, 3,325 received

The situation that you describe with the limit order that didn't get cancelled is what I call an orphaned order/position. There are several reasons why this can happen and it really doesn't matter what the reason is. Bottom line is that it would be nice if NT would allow the account orders/positions to be managed the same way that you can manage strategy orders/positions. I mention this here because I have worked on this for a while and haven't found a solution mainly because NT does not support it. Let's say for example that for whatever reason you have an orphaned order/position and want your strategy to adopt that order/position so it can be managed by the strategy. Currently, any attempt to manage that account position will close your strategy and I can't find any way around this. So, if you find a solution, please post it here.

i am asking NT to support this.

Reply With Quote
The following user says Thank You to MWinfrey for this post:
 
(login for full post details)
  #283 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

Week 10 is in the books, and it was a profitable week. Still looking for a big trade from my day strategy, just haven;t gotten it yet. Still hovering around breakeven. Nothing to be proud of, but not losing significantly either.







Follow me on Twitter Started this thread Reply With Quote
The following 5 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #284 (permalink)
 ratfink 
Birmingham UK
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
 
ratfink's Avatar
 
Posts: 3,651 since Dec 2012
Thanks: 17,422 given, 8,403 received


kevinkdog View Post
Week 10 is in the books, and it was a profitable week. Still looking for a big trade from my day strategy, just haven;t gotten it yet. Still hovering around breakeven. Nothing to be proud of, but not losing significantly either.

Interesting. Could you adopt a simple fixed MAE e.g. $-100/-200 or whatever?, as your losses seems less frequent than winners but much larger when they occur. I know sweet f.a. about automation as yet but planning for next year so am taking a greater interest.

Travel Well
Visit my futures io Trade Journal Reply With Quote
The following user says Thank You to ratfink for this post:
 
(login for full post details)
  #285 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


ratfink View Post
Interesting. Could you adopt a simple fixed MAE e.g. $-100/-200 or whatever?, as your losses seems less frequent than winners but much larger when they occur. I know sweet f.a. about automation as yet but planning for next year so am taking a greater interest.


MAE isn't something I normally do walkforward optimization with, but yes - I could have used that as my optimization criteria. It might have made things better, but maybe not. With the night system, losses are bigger than the winners, but a lot less frequent. With the day system, winners are a lot bigger than losers, but win percentage is below 50%

At this point, though, it is too late to go back and try to improve the system. I'd probably easily find something that backtested better, but it would not mean it would work any better going forward. If I really did not like the performance, I'd be better off just starting from scratch with something else.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #286 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received


kevinkdog View Post
Week 10 is in the books, and it was a profitable week.

For the record, and for the "drama" or whatever --- if this were my strategy and what not, I would quit right now -- under the grounds that the strategy is not performing as intended // as it did under tested scenarios.

We can see how that turns out for me later if you decide to continue

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
 
(login for full post details)
  #287 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
For the record, and for the "drama" or whatever --- if this were my strategy and what not, I would quit right now -- under the grounds that the strategy is not performing as intended // as it did under tested scenarios.

We can see how that turns out for me later if you decide to continue

Mike


I can certainly understand that point of view. In my case, I plan on sticking to the original plan.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #288 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received


kevinkdog View Post
I can certainly understand that point of view. In my case, I plan on sticking to the original plan.

At what point do you give up?
Isnt this a bit like staying with a bad trade? You can see it's not doing what you thought it would but some thing keeps you in the trade.

Reply With Quote
 
(login for full post details)
  #289 (permalink)
 rk142 
Atlanta, GA
 
Experience: Intermediate
Platform: Ninjatrader
Trading: N/A
 
Posts: 260 since Dec 2011
Thanks: 117 given, 327 received

Guys,

1) Kevin defined his quit point at the optimal time: before he turned on the system. So his stop is already in place. It is outside of his system's noise, and the amount of his total capital at risk on this system is clearly well within his risk tolerance.

2) Forget about fancy math. Look back at the equity curve from this post:



Are the past 10 weeks that out of character?

3) We also know that the system relies on occasional big winners. It's possible that this is a major problem with the system, but Kevin knew this before he turned it on. He was comfortable with that weakness then - he should be now.

4) None of us know what is going to happen next in the markets, or with our systems. Kevin could have turned his system on just before the system disintegrates. But his process is clear, his quit point is well defined, and he's following his plan.

Based on his posts so far on this forum I'm pretty confident that he will, as he should, soldier on. It's to his credit that he can do so publicly.

I probably couldn't.

-RK

Reply With Quote
The following 5 users say Thank You to rk142 for this post:
 
(login for full post details)
  #290 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
At what point do you give up?
Isnt this a bit like staying with a bad trade? You can see it's not doing what you thought it would but some thing keeps you in the trade.

@rk142 describes the situation well (with my quitting point, etc.), in the post just above this one.

Is this really a "bad trade" at this point? I don't see the performance as good, by any stretch of the imagination, but I don't see it as being bad, either. It is just kind of "blah." But my perception may be different from other people.

Looking at the overall walkforward and live equity curve ( ), I see probably 7 or so similar periods of performance in the past (flat to slight down for an extended period.


I think the real key is that I have a well defined quitting/stopping point in place, and it hasn't hit that yet. If and when it does, I'll quit as planned.

Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #291 (permalink)
 deaddog 
Market Wizard
Prince George BC Canada
 
Experience: Advanced
Platform: IBs TWS
Broker: IB
Trading: Stocks
 
deaddog's Avatar
 
Posts: 856 since May 2013
Thanks: 132 given, 2,327 received

Do you have a time limit? How long are you willing to put your capital at risk without any return?

Reply With Quote
The following 2 users say Thank You to deaddog for this post:
 
(login for full post details)
  #292 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

I am curious what people out there think. If this was your trading system, would you still be trading it?

Please answer a 1 question survey here: https://www.surveymonkey.com/s/XS6X7DN
(If this is against forum rules, I apologize and ask Big mike to delete the post).


Here are some performance curves to help you make a decision. For the sake of the survey, assume you started trading with real money on the green portion of the curve.







Follow me on Twitter Started this thread Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #293 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


deaddog View Post
Do you have a time limit? How long are you willing to put your capital at risk without any return?

Good question. Up to now, I haven't addressed this at all. The answer for me involves the "next best alternative."

Every 6 months or so, I look at every system I am trading, and also the systems waiting in the wings (ready to be traded live, but currently not.) If I find a better system than the one I am trading, and I don't have enough capital to trade both (or perhaps because of correlation issues I do not want to trade both), I will replace it. Thus, even if NGEC is performing decently (making money as opposed to its current situation of modestly losing), I still might replace it with the "next best alternative."

I only do this exercise 2x a year in part because it is involved (correlation studies, etc), but mainly because it is only fair to a new system going live to have some time to prove itself. Most people don't have the patience to do this, and they jump from system to system, never giving any one of them a fair chance. It would almost like pulling a pitcher in baseball as soon as he gives up just one hit.


Of course, even with this analysis, the quitting point is still in effect. If I hit that, I am out, regardless of the next best alternative (which may be cash).

Follow me on Twitter Started this thread Reply With Quote
The following 4 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #294 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

One of the tricks unscrupulous vendors play is to assume all limit orders are filled as soon as the price is touched. You recognize this by looking at a price charts of their trades. If the method shows trades being bought at the exact low of a bar, and/or sold at the exact high of a bar, you can bet this game is being played.

Of course, the reality is that it is hard to buy the low and sell the high. My experience is that, depending on the market and when your order is placed, you can probably do this 20% of the time. The other 80% of the time, price has to trade through your price to get you a fill at the limit price.

This can be an issue with backtesting. If your backtest engine assumes limit orders are filled when touched, the results will be too optimistic. If the backtest engine assumes price must be penetrated to get a fill, then the backtest results will be a bit too pessimistic. I always go with the pessimistic approach. My actual results can then only be better than the backtest.

I bring this up because today my NGEC day strategy bought the exact low of the exact on a limit order. Of course, part of the reason I was filled was because I am trading a one lot - if I was trading a ten lot, I probably would have received only a partial fill.

The interesting thing is that when I refreshed the chart, the trade went away, according to the strategy engine. Since the price did not go one tick below my limit price, the strategy engine assumes there was no trade. But my real account says there was a trade, because I was indeed filled. A nice surprise for once!

So, this is roughly a $400 trade in my real account's favor, when compared to the backtest engine. This makes up for the order problem that cost me $500 a few days ago.





Attached Thumbnails
Click image for larger version

Name:	Capture1.JPG
Views:	41
Size:	28.5 KB
ID:	126986  
Follow me on Twitter Started this thread Reply With Quote
The following 4 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #295 (permalink)
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
 
treydog999's Avatar
 
Posts: 896 since Jul 2012
Thanks: 291 given, 1,028 received

I dont know if its right for me to chime in. But I see this topic on quitting this system all the time on this tread. IMHO the biggest take away here is doing 2 things. Pre-defining your plan, in this case specifically a quitting point before activation of a system. Then having the perseverance to stick with it. Most people here on the thread keep saying the same thing, you should quit, when will you quit, why are you still holding out? For me that was answered at the start of this entire exercise post#10 in fact.

I think what @kevinkdog is doing is absolutely correct. But it also shows the difference from highly seasoned algorithmic traders vs say the "rest of us". Most of the people wouldn't have had the stomach to cope through this 10 weeks of misery, even if they had the pre set quitting point. IMHO everyone who would have quit before the pre defined criteria, changed the criteria (even with some pretty justifiable reasons and maths) or failed to follow the pre set plan in anyway. Made a huge mistake. That's where the failure is, in the trader not following the plan. Not in any single trade, or system being profitable or not profitable.

Reply With Quote
The following 7 users say Thank You to treydog999 for this post:
 
(login for full post details)
  #296 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


treydog999 View Post
I dont know if its right for me to chime in. But I see this topic on quitting this system all the time on this tread. IMHO the biggest take away here is doing 2 things. Pre-defining your plan, in this case specifically a quitting point before activation of a system. Then having the perseverance to stick with it. Most people here on the thread keep saying the same thing, you should quit, when will you quit, why are you still holding out? For me that was answered at the start of this entire exercise post#10 in fact.

I think what @kevinkdog is doing is absolutely correct. But it also shows the difference from highly seasoned algorithmic traders vs say the "rest of us". Most of the people wouldn't have had the stomach to cope through this 10 weeks of misery, even if they had the pre set quitting point. IMHO everyone who would have quit before the pre defined criteria, changed the criteria (even with some pretty justifiable reasons and maths) or failed to follow the pre set plan in anyway. Made a huge mistake. That's where the failure is, in the trader not following the plan. Not in any single trade, or system being profitable or not profitable.

Thanks Trey, I think you hit the nail on the head!

I can tell you there is nothing more deflating than having high hopes for a trading system, only to see it not immediately deliver. I sometimes wonder if this only happens to me - am I the only one who experiences drawdowns, almost without fail, right after I start a new system?

I think a lot of this impatience is because of the age we live in. Nobody wants to wait for anything anymore. I see this a lot with trading systems. I've heard of people abandoning trading systems after 4 or 5 trades. Seriously?

Please weigh in on this topic: Answer this one question survey, and I'll share results with everyone:

https://www.surveymonkey.com/s/XS6X7DN

Follow me on Twitter Started this thread Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #297 (permalink)
 Big Mike 
Site Administrator
Swing Trader
Data Scientist & DevOps
Manta, Ecuador
 
Experience: Advanced
Platform: Custom solution
Trading: Futures & Crypto
 
Big Mike's Avatar
 
Posts: 49,957 since Jun 2009
Thanks: 32,432 given, 98,140 received

I don't believe (in my case) that patience is the issue. It's that the system is not conforming to the original idea. I do understand you set initial stops and quitting points, which is a "must", and you want to see it through. But I struggle with the way the system is performing today vs the way it performed in the tests, to me they are not the same. I would have made that part of my "quitting" points.

However, if the system was performing the same, then you make an excellent point. Most people cannot sit on their hands very well or resist the urge to make changes/tweaks to the system, which will make analysis of it near impossible. You can go back to Richard Dennis's quote about how he could post his trade signals in the WSJ, and most people still wouldn't make money on them because they would try to "improve" the calls or they have different emotional capital than Richard, and therefore would wind up with drastically different trades and results.

Mike

We're here to help -- just ask

For the best trading education, watch our webinars
Searching for trading reviews? Review this list

Follow us on Twitter, YouTube, and Facebook

Support our community as an Elite Member:
https://futures.io/elite/

Visit other sites? Please spread the word about your experience with our community!
Follow me on Twitter Visit my futures io Trade Journal Reply With Quote
The following 3 users say Thank You to Big Mike for this post:
 
(login for full post details)
  #298 (permalink)
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
 
treydog999's Avatar
 
Posts: 896 since Jul 2012
Thanks: 291 given, 1,028 received


kevinkdog View Post
Thanks Trey, I think you hit the nail on the head!

I can tell you there is nothing more deflating than having high hopes for a trading system, only to see it not immediately deliver. I sometimes wonder if this only happens to me - am I the only one who experiences drawdowns, almost without fail, right after I start a new system?

I think a lot of this impatience is because of the age we live in. Nobody wants to wait for anything anymore. I see this a lot with trading systems. I've heard of people abandoning trading systems after 4 or 5 trades. Seriously?

Please weigh in on this topic: Answer this one question survey, and I'll share results with everyone:

https://www.surveymonkey.com/s/XS6X7DN

Nope I have this happen to. My live algo trading journal shows the same thing. Even worse my own errors compounded the problem. To be honest it does feel better to know that even you experience it. Misery loves company

Reply With Quote
The following 2 users say Thank You to treydog999 for this post:
 
(login for full post details)
  #299 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received


Big Mike View Post
I don't believe (in my case) that patience is the issue. It's that the system is not conforming to the original idea. I do understand you set initial stops and quitting points, which is a "must", and you want to see it through. But I struggle with the way the system is performing today vs the way it performed in the tests, to me they are not the same.

That's the $100,000 question: is the system performing the same as it did in historical tests, or did something fundamentally change?

Trying to be objective, here is what I currently see:

* Performance after 26 days of trading is not even close to the average expected performance.

* System is currently performing at about the 15th percentile. If it was average, it would be at the 50% percentile. If it completely fell apart, it would be at the 0 percentile.

* Looking at the equity curve, there are 7 similar flat/down periods in the equity, of roughly the same shape and length.

* There have not been any big winning trades, which is what this system needs to succeed long term. On average, I should see 4-6 large winners per year, or one very 2-3 months.


Based on all this, I am not convinced that the system has stopped working. But, I am open to hearing everyone's thoughts on the matter, especially if you can conclude that from the numbers (I can send you whatever data you need). Maybe there is a measure that says "stop! It is broken!" that I am unaware of-- if so, I'd like to understand it. For example, I might play around with a 2 sample T test, and see what that says.

Follow me on Twitter Started this thread Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #300 (permalink)
 kevinkdog   is a Vendor
 
 
Posts: 2,984 since Jul 2012
Thanks: 1,589 given, 5,924 received

I entered the historical data, and the live data, into the T test calculator here:
Student's t-test: Results


Results are here:





If the result was 0, I believe that indicates that the 2 samples (historical backtest and live testing) are definitely different (ie, the system has changed). If the result was 1, then the 2 samples are definitely the same.

Just for kicks, I took some random groups of 25 historical trades, and put them as sample #2. These are obviously from the historical sample, but the null hypothesis result varied from .09 to 0.82, with a mean of 0.51.

Based on these results, and assuming I did this right and that the analysis is appropriate in the first place, it looks like the two samples are leaning towards being different, but it is not conclusive.

One other interesting tidbit: If I add one $1000 winning trade to my actual results, the null hypothesis test jumps from 0.26 to 0.47. This tells me that the one big winning trade I have been waiting for will move actual results much closer to backtest (which I kind of knew anyhow)...

Follow me on Twitter Started this thread Reply With Quote
The following 3 users say Thank You to kevinkdog for this post:


futures io Trading Community Trading Journals > Taking a Trading System Live


Last Updated on January 6, 2016


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Journal Challenge w/$1,800 in prizes!

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts