NexusFi: Find Your Edge


Home Menu

 





KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)


Discussion in Trading Reviews and Vendors

Updated
      Top Posters
    1. looks_one kevinkdog with 180 posts (416 thanks)
    2. looks_two SMCJB with 23 posts (40 thanks)
    3. looks_3 Mabi with 17 posts (12 thanks)
    4. looks_4 bluefightingcat with 16 posts (9 thanks)
      Best Posters
    1. looks_one grausch with 2.8 thanks per post
    2. looks_two kevinkdog with 2.3 thanks per post
    3. looks_3 SMCJB with 1.7 thanks per post
    4. looks_4 Big Mike with 1.5 thanks per post
    1. trending_up 170,385 views
    2. thumb_up 649 thanks given
    3. group 111 followers
    1. forum 433 posts
    2. attach_file 15 attachments




Closed Thread
 
Search this Thread

KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)

  #161 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,646 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338

Thanks for checking. I have idea what is going on, but I'll talk to the domain people.

Follow me on Twitter
Thanked by:

Can you help answer these questions
from other members on NexusFi?
Exit Strategy
NinjaTrader
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Better Renko Gaps
The Elite Circle
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
GFIs1 1 DAX trade per day journal
18 thanks
  #162 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,646 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338

Hopefully I fixed the problem, thanks for the help @Big Mike!

Follow me on Twitter
Thanked by:
  #163 (permalink)
Enric66
Barcelona Spain
 
Posts: 13 since Aug 2015
Thanks Given: 15
Thanks Received: 7



kevinkdog View Post
I look at results for that market.

If results in other markets (if I even look) are good, that makes me feel a bit better.

If results in other markets (if I even look) are poor, that probably won't make me throw the system away.

I realize the pitfalls in this approach, but I have never been able to correlate future performance to this kind of metric.

Hi! Thank you very much for your kind and precise response. Very helpful indeed.

Thanked by:
  #164 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

  #165 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,646 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338


SMCJB View Post
Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

That is a good question. I'm not sure there is a correct answer, but there are some alternatives...

1. What you describe is what many people call a standard "out of sample" test. The typical way these are run do not have any mechanism for re-optimizing. So, whatever model you initially chose, which also worked on out of sample data, would be used indefinitely, and you'd never re-optimize. You could trade this model for 10-20 years and the model would never change. It may or may not hold up.

2. But, you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).

3. Finally, you could re-optimize your model every year, but use all the data available. So, the first optimization is 10 years of data. Next one will use 11 years or data, and so on. The model still gets re-optimized every year. This is a walkforward optimization that is anchored (start point never moves).


Which is best? I guess it depends who you ask. I personally use option 2. I don't like #1 because it never changes, and I don't like #3 because old data keeps impacting the optimization well into the future.


The one thing about your case is that it is not really walkforward optimization yet, since you only had one out of sample period. That one period of out of sample might not be significant - that's why true walkforward testing has 10-20+ out of sample periods.


Question for you: how much does the model change between these 3 alternatives:

1. keep original model
2. re-optimize model with 10 years of data
3. re-optimize model with 11 years of data

Is there a huge difference in the models?

Follow me on Twitter
  #166 (permalink)
 
shodson's Avatar
 shodson 
OC, California, USA
Quantoholic
 
Experience: Advanced
Platform: IB/TWS, NinjaTrader, ToS
Broker: IB, ToS, Kinetick
Trading: stocks, options, futures, VIX
Posts: 1,976 since Jun 2009
Thanks Given: 533
Thanks Received: 3,709

4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Follow me on Twitter Visit my NexusFi Trade Journal
Thanked by:
  #167 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,646 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338


shodson View Post
4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Very good point!

I know some very good traders who do either or both of these things: create models with nothing to optimize, and/or create models that they never reoptimize.

I personally like walkforward, but I know people can and do have success without walkforward.

The key is to have a process/method that proves itself with real money.

Follow me on Twitter
Thanked by:
  #168 (permalink)
 
SMCJB's Avatar
 SMCJB 
Houston TX
Legendary Market Wizard
 
Experience: Advanced
Platform: TT and Stellar
Broker: Advantage Futures
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,041 since Dec 2013
Thanks Given: 4,375
Thanks Received: 10,192

Thanks Kevin.

It's interesting what you said in 2) as that relates to the next part of my question.

When I originally started on this model, I arbitrarily picked the last 11 years (04-14), 10 years to look at (04-13), and 1 year (14) as an out of sample test. Once I completed the first model (which is the model I'm actually still trading today) I started delving deeper into the data. I looked at using the last 9, 8, 7, 6 & 5 years rather than 10. The results showed that the model with the best fit used the last 7 years (07-13) rather than 10 years (04-14) of data. I could tell from looking at the models that the higher energy prices of the more recent years (2008, 10-13) where heavily influencing the models. Since the 7 year models results weren't significantly better than the 10 year, I decided to just stick with the original 10 year.

I actually then built a new model, that included a 'year variable'. This model was significantly better than any of the other models. The problem is the addition of the year variable gave values for 2015 significantly higher than anything previously seen, and of course 2016 would probably be even higher. Despite this model having better results I did not use it, and kept with the original 10 year model.

As we all know, energy prices have plummeted in the last year. The model I am using (built based upon 10 years (04-14)) has performed admirably. The model based upon 7 years of data would have been profitable, but not nearly as good. The model that included the 'year variable' would have got destroyed.

I'm comfortable with my decision to use the 10 year vs the 7 year, but by choosing the 10 year over the 'year variable' I feel like I allowed my personal bias to overrule what should be a more systematic and statistical approach. The fact that my personal bias was correct, obviously makes me feel good, but I'm still concerned I've contaminated the process.

So getting back to your point 2
you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).
This is actually what I was planning to do. The problem is, I already know what happened in 2015. If I roll my window forward to 04-14 with an out of sample 2015, I know what the results will be. I know that any model more heavily based upon the recent high energy prices will perform far worse than a model with a longer tenor of data. I strongly suspect that a new model with a "polynomial year variable" (something like -(year-2011.5)^2) would probably perform quite well, but I know that because I know energy prices have had a concave trajectory over the last 15 years. I feel like whatever I do, my personal bias's/knowledge are corrupting the process.

I know there's no actual question in there, but I was curious if you have any additional thoughts on the matter?

  #169 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,646 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338

You personal biases may be impacting things, but that is not necessarily a bad thing. Your biases might actually be helping you to make a better decision.

Based on what you've done already, it is hard to recommend a way forward. Like you say, you already know the results, so trying to formalize this into a process probably doesn't make sense.

If I think of a way around this, I'll post it.

Follow me on Twitter
Thanked by:
  #170 (permalink)
 DobermanTrading 
SE Asia / Canada
 
Experience: Intermediate
Platform: FXCM's Trading Station
Broker: Darwinex, FXCM, Tradingview
Trading: Forex and CFD's
Posts: 114 since Jun 2010
Thanks Given: 26
Thanks Received: 109


Hi Kevin,

Any more youtube videos planned in the close future? those are really good

cheers,

Follow me on Twitter Visit my NexusFi Trade Journal
Thanked by:

Closed Thread



Last Updated on July 5, 2019


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts