KJ Trading Systems Kevin Davey - Ask Me Anything (AMA) - futures io
futures io futures trading



KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)


Discussion in Trading Reviews and Vendors

Updated by justmg
      Top Posters
    1. looks_one kevinkdog with 180 posts (404 thanks)
    2. looks_two SMCJB with 23 posts (39 thanks)
    3. looks_3 Mabi with 17 posts (10 thanks)
    4. looks_4 bluefightingcat with 16 posts (9 thanks)
      Best Posters
    1. looks_one grausch with 2.8 thanks per post
    2. looks_two kevinkdog with 2.2 thanks per post
    3. looks_3 SMCJB with 1.7 thanks per post
    4. looks_4 Big Mike with 1.4 thanks per post
    1. trending_up 94,180 views
    2. thumb_up 632 thanks given
    3. group 111 followers
    1. forum 433 replies
    2. attach_file 15 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 100,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

Closed Thread
 
Search this Thread
 

KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)

(login for full post details)
  #161 (permalink)
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,776 since Jul 2012
Thanks: 1,425 given, 5,344 received

Thanks for checking. I have idea what is going on, but I'll talk to the domain people.

Follow me on Twitter
The following user says Thank You to kevinkdog for this post:

 
 
(login for full post details)
  #162 (permalink)
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,776 since Jul 2012
Thanks: 1,425 given, 5,344 received

Hopefully I fixed the problem, thanks for the help @Big Mike!

Follow me on Twitter
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #163 (permalink)
Barcelona Spain
 
 
Posts: 13 since Aug 2015
Thanks: 15 given, 7 received



kevinkdog View Post
I look at results for that market.

If results in other markets (if I even look) are good, that makes me feel a bit better.

If results in other markets (if I even look) are poor, that probably won't make me throw the system away.

I realize the pitfalls in this approach, but I have never been able to correlate future performance to this kind of metric.

Hi! Thank you very much for your kind and precise response. Very helpful indeed.

The following user says Thank You to Enric66 for this post:
 
(login for full post details)
  #164 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: XTrader and Cloud TT
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,489 since Dec 2013
Thanks: 2,782 given, 6,561 received

Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

The following 5 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #165 (permalink)
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,776 since Jul 2012
Thanks: 1,425 given, 5,344 received


SMCJB View Post
Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

That is a good question. I'm not sure there is a correct answer, but there are some alternatives...

1. What you describe is what many people call a standard "out of sample" test. The typical way these are run do not have any mechanism for re-optimizing. So, whatever model you initially chose, which also worked on out of sample data, would be used indefinitely, and you'd never re-optimize. You could trade this model for 10-20 years and the model would never change. It may or may not hold up.

2. But, you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).

3. Finally, you could re-optimize your model every year, but use all the data available. So, the first optimization is 10 years of data. Next one will use 11 years or data, and so on. The model still gets re-optimized every year. This is a walkforward optimization that is anchored (start point never moves).


Which is best? I guess it depends who you ask. I personally use option 2. I don't like #1 because it never changes, and I don't like #3 because old data keeps impacting the optimization well into the future.


The one thing about your case is that it is not really walkforward optimization yet, since you only had one out of sample period. That one period of out of sample might not be significant - that's why true walkforward testing has 10-20+ out of sample periods.


Question for you: how much does the model change between these 3 alternatives:

1. keep original model
2. re-optimize model with 10 years of data
3. re-optimize model with 11 years of data

Is there a huge difference in the models?

Follow me on Twitter
The following 6 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #166 (permalink)
Market Wizard
OC, California, USA
 
Experience: Advanced
Platform: IB/TWS, NinjaTrader, ToS
Trading: stocks, options, futures, VIX
 
shodson's Avatar
 
Posts: 1,941 since Jun 2009
Thanks: 510 given, 3,588 received

4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Follow me on Twitter Visit my futures io Trade Journal
The following 2 users say Thank You to shodson for this post:
 
(login for full post details)
  #167 (permalink)
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,776 since Jul 2012
Thanks: 1,425 given, 5,344 received


shodson View Post
4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Very good point!

I know some very good traders who do either or both of these things: create models with nothing to optimize, and/or create models that they never reoptimize.

I personally like walkforward, but I know people can and do have success without walkforward.

The key is to have a process/method that proves itself with real money.

Follow me on Twitter
The following 2 users say Thank You to kevinkdog for this post:
 
(login for full post details)
  #168 (permalink)
Legendary Market Wizard
Houston, TX
 
Experience: Advanced
Platform: XTrader and Cloud TT
Broker: Advantage Futures
Trading: Energy
 
Posts: 3,489 since Dec 2013
Thanks: 2,782 given, 6,561 received

Thanks Kevin.

It's interesting what you said in 2) as that relates to the next part of my question.

When I originally started on this model, I arbitrarily picked the last 11 years (04-14), 10 years to look at (04-13), and 1 year (14) as an out of sample test. Once I completed the first model (which is the model I'm actually still trading today) I started delving deeper into the data. I looked at using the last 9, 8, 7, 6 & 5 years rather than 10. The results showed that the model with the best fit used the last 7 years (07-13) rather than 10 years (04-14) of data. I could tell from looking at the models that the higher energy prices of the more recent years (2008, 10-13) where heavily influencing the models. Since the 7 year models results weren't significantly better than the 10 year, I decided to just stick with the original 10 year.

I actually then built a new model, that included a 'year variable'. This model was significantly better than any of the other models. The problem is the addition of the year variable gave values for 2015 significantly higher than anything previously seen, and of course 2016 would probably be even higher. Despite this model having better results I did not use it, and kept with the original 10 year model.

As we all know, energy prices have plummeted in the last year. The model I am using (built based upon 10 years (04-14)) has performed admirably. The model based upon 7 years of data would have been profitable, but not nearly as good. The model that included the 'year variable' would have got destroyed.

I'm comfortable with my decision to use the 10 year vs the 7 year, but by choosing the 10 year over the 'year variable' I feel like I allowed my personal bias to overrule what should be a more systematic and statistical approach. The fact that my personal bias was correct, obviously makes me feel good, but I'm still concerned I've contaminated the process.

So getting back to your point 2
you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).
This is actually what I was planning to do. The problem is, I already know what happened in 2015. If I roll my window forward to 04-14 with an out of sample 2015, I know what the results will be. I know that any model more heavily based upon the recent high energy prices will perform far worse than a model with a longer tenor of data. I strongly suspect that a new model with a "polynomial year variable" (something like -(year-2011.5)^2) would probably perform quite well, but I know that because I know energy prices have had a concave trajectory over the last 15 years. I feel like whatever I do, my personal bias's/knowledge are corrupting the process.

I know there's no actual question in there, but I was curious if you have any additional thoughts on the matter?

The following 3 users say Thank You to SMCJB for this post:
 
(login for full post details)
  #169 (permalink)
Cleveland Ohio/United States
 
Experience: Advanced
Platform: Tradestation
Broker: Tradestation, DeCarley, others
Trading: futures
 
Posts: 2,776 since Jul 2012
Thanks: 1,425 given, 5,344 received

You personal biases may be impacting things, but that is not necessarily a bad thing. Your biases might actually be helping you to make a better decision.

Based on what you've done already, it is hard to recommend a way forward. Like you say, you already know the results, so trying to formalize this into a process probably doesn't make sense.

If I think of a way around this, I'll post it.

Follow me on Twitter
The following user says Thank You to kevinkdog for this post:
 
(login for full post details)
  #170 (permalink)
SE Asia / Canada
 
Experience: Intermediate
Platform: FXCM's Trading Station
Broker: Darwinex, FXCM, Tradingview
Trading: Forex and CFD's
 
Posts: 114 since Jun 2010
Thanks: 26 given, 108 received


Hi Kevin,

Any more youtube videos planned in the close future? those are really good

cheers,

Follow me on Twitter Visit my futures io Trade Journal
The following user says Thank You to DobermanTrading for this post:

Closed Thread

futures io Trading Community Trading Reviews and Vendors > KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)


July 5, 2019


Upcoming Webinars and Events
 

Every journal equals ten meals for the hungry

Now
     



Copyright © 2020 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts