NexusFi: Find Your Edge


Home Menu

 





KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)


Discussion in Trading Reviews and Vendors

Updated
      Top Posters
    1. looks_one kevinkdog with 180 posts (416 thanks)
    2. looks_two SMCJB with 23 posts (40 thanks)
    3. looks_3 Mabi with 17 posts (12 thanks)
    4. looks_4 bluefightingcat with 16 posts (9 thanks)
      Best Posters
    1. looks_one grausch with 2.8 thanks per post
    2. looks_two kevinkdog with 2.3 thanks per post
    3. looks_3 SMCJB with 1.7 thanks per post
    4. looks_4 Big Mike with 1.5 thanks per post
    1. trending_up 167,862 views
    2. thumb_up 649 thanks given
    3. group 111 followers
    1. forum 433 posts
    2. attach_file 15 attachments




Closed Thread
 
Search this Thread

KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)

  #11 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039

How do you know when your goals for the system are just too high? I had 3 main criteria: 15000$ a year, +$75 expectancy per average trade and more than 4 trades per month. I was working on the ES on a daily time frame. I am just struggling to find something that worked while keeping it simple. I knew the better I made my backtest, the worse my out of sample would become. I kept getting stuck with more trades vs bigger expected value per trade. The more trades I took, the less my average trade became. So I am wondering if my goals are creating conflict.

What are the most important baseline goals a system should have?
Are there minimum # of trades a system must have?


Can you help answer these questions
from other members on NexusFi?
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
ZombieSqueeze
Platforms and Indicators
Request for MACD with option to use different MAs for fa …
NinjaTrader
NexusFi Journal Challenge - April 2024
Feedback and Announcements
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Retail Trading As An Industry
58 thanks
Battlestations: Show us your trading desks!
48 thanks
NexusFi site changelog and issues/problem reporting
47 thanks
GFIs1 1 DAX trade per day journal
32 thanks
What percentage per day is possible? [Poll]
31 thanks

  #12 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


treydog999 View Post
How do you know when your goals for the system are just too high? I had 3 main criteria: 15000$ a year, +$75 expectancy per average trade and more than 4 trades per month. I was working on the ES on a daily time frame. I am just struggling to find something that worked while keeping it simple. I knew the better I made my backtest, the worse my out of sample would become. I kept getting stuck with more trades vs bigger expected value per trade. The more trades I took, the less my average trade became. So I am wondering if my goals are creating conflict.

What are the most important baseline goals a system should have?
Are there minimum # of trades a system must have?


The biggest clue that you are aiming too high is in this statement: "I knew the better I made my backtest, the worse my out of sample would become." This is almost always a sign you are doing too much (too many rules, or too many optimizations, etc).

My advice is to just start with one criteria, maybe profit per year or average trade profit. See if that helps. You can always then add in more conditions if you find you need to.

Whatever you criteria, don't expect it to be easy to find a strategy that holds up over time. Most ideas will never hold up in testing. You have to sift thru a lot of garbage before you find your gold.

For minimum # trades, a good rule of thumb is to base it on the number of conditions/rules/optimizable variables you have in your code. Let's say you have 5 (2 entry conditions, and 3 exit conditions). Multiply that by 50 or 100, and it says you need roughly 250-500 trades. That is probably a decent amount. Obviously more is better, but at the same time, if I find a nice system with only 200 trades, I might be flexible and still accept it.

Hope this helps.

Follow me on Twitter
The following 6 users say Thank You to kevinkdog for this post:
  #13 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


Hi Kevin,

I'm backtesting in excel and have come across a strategy with a quite nice consistent win% and R:R using a SL/Target adjusted for ATR for each year 2007 through 2011. Apart from 2012 - that was a disaster. Various statistics seem to show a change in 2012 vs the prior 5 years so I flipped the entry from Long to Short (or Short to Long), and hey Presto the results suddenly fall into line.

Would you immediately dump this strategy (curve fitting), or consider the market (somehow) has changed and try and ascertain whether the 'new' entry trigger is viable? This strategy trades once a day - based around the opening swing - so there are quite a few data points. What struck me was how well the results fell into line with prior years. Normally I think, 'hmmmm interesting', and move on.

I suspect you would dump it, but I read (I think in your Combine thread), that you continually update parameters for latest data and adjust triggers accordingly. Probably not to the extreme of flipping entry direction I suspect.

In relation to this - we often hear "the market is constantly changing and strategies effectiveness fade out" - what is your experience in terms of life span of strategies once you actually start trading it with 'big money'? How many months/years do you 'on average' trade a strategy - and how many years of data do you have on that particular strategy? i.e - how long has it worked for?

It seems there is a general contradiction to having years of backtests, walk-forwards, Live trials, Trading with small money, then big money etc, essentially years of data - and then the market changes - and it no longer works. Said another way, as the market is constantly changing it will be damn near impossible to find a successful strategy using many multiple years of data - as it will always have periods (years) where it hasn't worked, or when it stops working. I say near impossible, as I assume that all edges will eventually be discovered and abused due to the amount of people beavering away with high speed computers. In the old days when it was all done with a pad, pencil and abacus the edges would have lasted a lot longer I bet. I'm not saying it's impossible to find an edge, otherwise I wouldn't be writing this!

So, when do you take 'price risk' and get more data, and 'info risk' when you get some good results with less years back test and jump in (once you have traded with small money to check alignment, of course).

What do you do if you see back test results where the first years are horrible, and the latest year/s are glorious. Overall it may have negative expectancy. When do you dig deeper, and when do you toss it into the bin?

I have also developed a monte carlo which adds percentile bands in excel - like you have. A great tool to 'know when to hold em, and know when to fold em'. Given this tool, do you really need to test how a strategy works in all market types - crashes, bull runs etc - i.e years of data. If you don't have the data for a particular market type, and the strategy goes through a period of bad results, as it appears the market is changing (difficult to do this in real time) your model will tell you to stop trading it, incubate etc. Do you really need back test data to tell you it sucks in a crash? I'm still talking of using 3-5 years of data with 250+ signals/data points, but perhaps thinking 6-7+ or the 10+ years I often hear people talk about is not necessary, and perhaps detrimental?

I started out with one question, and only a few lines for you to read. Sorry about that.

The following user says Thank You to sixtyseven for this post:
  #14 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


sixtyseven View Post
Hi Kevin,

I'm backtesting in excel and have come across a strategy with a quite nice consistent win% and R:R using a SL/Target adjusted for ATR for each year 2007 through 2011. Apart from 2012 - that was a disaster. Various statistics seem to show a change in 2012 vs the prior 5 years so I flipped the entry from Long to Short (or Short to Long), and hey Presto the results suddenly fall into line.

Would you immediately dump this strategy (curve fitting), or consider the market (somehow) has changed and try and ascertain whether the 'new' entry trigger is viable?


There are a lot of answer and comments I could make here, so I can't do it all at once. I'll do one at a time over next few days...


Let's just look at 2012. You ran a test. You did not like the results, so you flipped the longs and shorts. Now you like the results. So really you just optimized the entry on 1 year of data, and now you want to trade live based on that in-sample optimization.

That is a really bad idea.


As an alternative, let's say you ran walk forward optimization, and sometimes it told you to "flip" long and short entries. If the test is done correctly, and it is a true walk forward test, then in theory you could do what you are saying.


But based on how you did it, I'd ditch the strategy.

Follow me on Twitter
The following user says Thank You to kevinkdog for this post:
  #15 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


sixtyseven View Post
Hi Kevin,

In relation to this - we often hear "the market is constantly changing and strategies effectiveness fade out" - what is your experience in terms of life span of strategies once you actually start trading it with 'big money'? How many months/years do you 'on average' trade a strategy - and how many years of data do you have on that particular strategy? i.e - how long has it worked for?

I have a couple of strategies right now that I trade live that are, all of a sudden, at or near their maximum drawdown. One I have been trading live for 4 or 5 years successfully.

Are they broken, or just acting as they should? I don't know, but I do know they are having problems.

That is why I always tell people to come up with a plan to quit trading a system, based on some criteria that you develop, that make you think a strategy is broken. Then, if and when that occurs, just cease trading it.


How long will a strategy last? No one really knows. Just look at "buy and hold" for stocks - that worked for many many years, until 2008 and 2009, when it didn't work. Maybe it is working again now, who knows?


I'm afraid I haven't given you a direct answer, and that is because I do not think there is an answer.

I've always thought about developing "switching" algorithms - master algos that could sense when strategies were failing, and switch to a new better performer. Never done it, though...

Follow me on Twitter
The following 2 users say Thank You to kevinkdog for this post:
  #16 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


sixtyseven View Post
Hi Kevin,

What do you do if you see back test results where the first years are horrible, and the latest year/s are glorious. Overall it may have negative expectancy. When do you dig deeper, and when do you toss it into the bin?


I generally put more weight to later year's data, but if the first year's are bad enough, I assume that can happen again. If I could not handle a repeat of those years (or worse), I'll toss it.

Follow me on Twitter
The following 2 users say Thank You to kevinkdog for this post:
  #17 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


sixtyseven View Post

I have also developed a monte carlo which adds percentile bands in excel - like you have. A great tool to 'know when to hold em, and know when to fold em'. Given this tool, do you really need to test how a strategy works in all market types - crashes, bull runs etc - i.e years of data. If you don't have the data for a particular market type, and the strategy goes through a period of bad results, as it appears the market is changing (difficult to do this in real time) your model will tell you to stop trading it, incubate etc. Do you really need back test data to tell you it sucks in a crash? I'm still talking of using 3-5 years of data with 250+ signals/data points, but perhaps thinking 6-7+ or the 10+ years I often hear people talk about is not necessary, and perhaps detrimental?

I started out with one question, and only a few lines for you to read. Sorry about that.


No problem with all the questions, they are all good ones...

I'm in the "more data is better" camp, because I have seen so many strategies look good for a few months, or even a few years, and then fall apart. Mean reversion, back to a "no edge" situation? Very possibly, and that is what I worry about.

That said, it is a ton harder to create systems that hold up over many years. So, it is always a trade off.

This is one reason I try to trade a basket of strategies. In any given year, a few underperform, a few are breakeven, and a few hopefully outperform and give you enough profits to make all the others worthwhile. The problem is you never know which strategies will be best in the next year. But being diversified really helps.


I hope I have answered your questions, but I fear I have not. There are no hard and fast answers to them, at least that I have found. Some of strategy development is an art and a feeling type activity, based on each developer's history of success and failure.

For example, I know a guy who re-optimizes every DAY with last 6 months of data, and claims success. I think this is crazy, but if it works for him (I have to take his word on it), who am I to argue?

Follow me on Twitter
The following 3 users say Thank You to kevinkdog for this post:
  #18 (permalink)
 sixtyseven 
Golden Bay, New Zealand
 
Experience: Beginner
Platform: Sierra Chart
Trading: ES, NQ
Posts: 186 since May 2012
Thanks Given: 851
Thanks Received: 337


kevinkdog View Post

As an alternative, let's say you ran walk forward optimization, and sometimes it told you to "flip" long and short entries. If the test is done correctly, and it is a true walk forward test, then in theory you could do what you are saying.

.

Kevin, thanks for taking the time to respond. You have helped. To clarify the above - is it classed as optimization if I redesign the strategy to sometimes flip long and shorts for all entries for '07-'12 (based on some, as yet undefined criteria) and then run a walk-forward on 2013? So far no walk-forward, monte carlo or bootstrapping has been done.
I.e when does tinkering with a strategy become optimization, and when does it become 'new' development? I'm thinking a lot of strategies are related in some way - so just because I'm redesigning the same strategy now, rather than 'discovering' the redesigned strategy in say 2 years time when I've long since forgotten about this one (and have more and better info - and come up with the 'redesigned' strategy first) actually mean it's no longer optimization? Said another way, if a group of strategies have some form of commonality, after many strategy developments over many years - aren't most of the strategies in that group a subtle form of optimization - without even being aware of it?

This probably falls into the 'Art of Development' as you mention, but thought I'd throw it out there anyway.

Btw, I realise adding filters, and running through numerous permutations of SL/Targets/indicator parameters etc is optimization - but here I'm not trying to filter out trades I'm trying ensure the trade (at a specified point) goes in the right direction. I have no idea how I will add in this criteria or whether this would improve or kill the '07-'11 metrics. I'm also not gung-ho on this particular strategy, and trying to force it to work - I'm trying to ensure I use a sound methodology for all development.

Thanks again for your time.

  #19 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,336


sixtyseven View Post
Kevin, thanks for taking the time to respond. You have helped. To clarify the above - is it classed as optimization if I redesign the strategy to sometimes flip long and shorts for all entries for '07-'12 (based on some, as yet undefined criteria) and then run a walk-forward on 2013? So far no walk-forward, monte carlo or bootstrapping has been done.
I.e when does tinkering with a strategy become optimization, and when does it become 'new' development? I'm thinking a lot of strategies are related in some way - so just because I'm redesigning the same strategy now, rather than 'discovering' the redesigned strategy in say 2 years time when I've long since forgotten about this one (and have more and better info - and come up with the 'redesigned' strategy first) actually mean it's no longer optimization? Said another way, if a group of strategies have some form of commonality, after many strategy developments over many years - aren't most of the strategies in that group a subtle form of optimization - without even being aware of it?

This probably falls into the 'Art of Development' as you mention, but thought I'd throw it out there anyway.

Btw, I realise adding filters, and running through numerous permutations of SL/Targets/indicator parameters etc is optimization - but here I'm not trying to filter out trades I'm trying ensure the trade (at a specified point) goes in the right direction. I have no idea how I will add in this criteria or whether this would improve or kill the '07-'11 metrics. I'm also not gung-ho on this particular strategy, and trying to force it to work - I'm trying to ensure I use a sound methodology for all development.

Thanks again for your time.


Yes, if you run many similar strategies with common elements, that can be viewed as optimization too. Really, once you've tested even 2 strategies, you've optimized, since you'd pick one or the other.

Follow me on Twitter
  #20 (permalink)
 
trendisyourfriend's Avatar
 trendisyourfriend 
Quebec Canada
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: AMP/CQG
Trading: ES, NQ, YM
Frequency: Daily
Duration: Minutes
Posts: 4,526 since Oct 2009
Thanks Given: 4,167
Thanks Received: 6,018


@kevinkdog

What are the core beliefs at the root of your trading ideas? I think this is something you never discussed so far. For a price action trader it may be when prices fail to do something twice for another it may be price valuation as proposed by tools like market or volume profiles for another it can be harmonic patterns etc. but what about you? Do you have any core beliefs or the markets are just a giant puzzle you need to solve and you are in an ever constant search of this elusive idea that will bring you more profit. Thanks.

The following user says Thank You to trendisyourfriend for this post:

Closed Thread




Last Updated on July 5, 2019


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts