KJ Trading Systems Kevin Davey - Ask Me Anything (AMA) - Vendors and Product Reviews | futures io social trading
futures io futures trading


KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)
Updated: Views / Replies:42,689 / 297
Created: by Big Mike Attachments:8

Welcome to futures io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.


You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Reply
 8  
 
Thread Tools Search this Thread
 

KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)

  #161 (permalink)
Elite Member
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,255 since Jul 2012
Thanks: 1,059 given, 4,097 received

Thanks for checking. I have idea what is going on, but I'll talk to the domain people.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #162 (permalink)
Elite Member
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,255 since Jul 2012
Thanks: 1,059 given, 4,097 received

Hopefully I fixed the problem, thanks for the help @Big Mike!

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #163 (permalink)
Trading Apprentice
Barcelona Spain
 
Futures Experience: Intermediate
Platform: MetaTrader
Favorite Futures: Forex
 
Posts: 13 since Aug 2015
Thanks: 15 given, 7 received



kevinkdog View Post
I look at results for that market.

If results in other markets (if I even look) are good, that makes me feel a bit better.

If results in other markets (if I even look) are poor, that probably won't make me throw the system away.

I realize the pitfalls in this approach, but I have never been able to correlate future performance to this kind of metric.

Hi! Thank you very much for your kind and precise response. Very helpful indeed.

Reply With Quote
The following user says Thank You to Enric66 for this post:
 
  #164 (permalink)
Market Wizard
Houston TX
 
Futures Experience: Advanced
Platform: XTrader
Broker/Data: Advantage Futures
Favorite Futures: Energy
 
Posts: 1,862 since Dec 2013
Thanks: 1,585 given, 2,804 received
Forum Reputation: Legendary

Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

Reply With Quote
The following 5 users say Thank You to SMCJB for this post:
 
  #165 (permalink)
Elite Member
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,255 since Jul 2012
Thanks: 1,059 given, 4,097 received


SMCJB View Post
Some advice please Kevin.

Let's say I back tested and tuned a model on 10 years of data 2004-2013. Not that it matters but lets say my instrument isn't something simple like ES, but say an energy spread, that is generally mean reverting.

I then forwarded tested my data on 2014 and it performed well so I went live with my model in 2015 and it has performed well also.

As I go into 2016 should I still be using the model tuned on the original 10 years of data 2004-2013 data? Should I update that to 11 years 2004-2014 or keep it 10 years but move it forward 2005-2014, then re-tune, and walk forward on 2015 data just to make sure its still good? Or should I even consider re-tuning on the 12 years 2004-2015 data or even the most recent 10 years 2006-2015?

That is a good question. I'm not sure there is a correct answer, but there are some alternatives...

1. What you describe is what many people call a standard "out of sample" test. The typical way these are run do not have any mechanism for re-optimizing. So, whatever model you initially chose, which also worked on out of sample data, would be used indefinitely, and you'd never re-optimize. You could trade this model for 10-20 years and the model would never change. It may or may not hold up.

2. But, you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).

3. Finally, you could re-optimize your model every year, but use all the data available. So, the first optimization is 10 years of data. Next one will use 11 years or data, and so on. The model still gets re-optimized every year. This is a walkforward optimization that is anchored (start point never moves).


Which is best? I guess it depends who you ask. I personally use option 2. I don't like #1 because it never changes, and I don't like #3 because old data keeps impacting the optimization well into the future.


The one thing about your case is that it is not really walkforward optimization yet, since you only had one out of sample period. That one period of out of sample might not be significant - that's why true walkforward testing has 10-20+ out of sample periods.


Question for you: how much does the model change between these 3 alternatives:

1. keep original model
2. re-optimize model with 10 years of data
3. re-optimize model with 11 years of data

Is there a huge difference in the models?

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following 6 users say Thank You to kevinkdog for this post:
 
  #166 (permalink)
 Vendor: www.probabletrades.com 
OC, California, USA
 
Futures Experience: Advanced
Platform: IB/TWS, NinjaTrader, thinkorswim
Favorite Futures: stocks, options, futures, VIX
 
shodson's Avatar
 
Posts: 1,848 since Jun 2009
Thanks: 474 given, 3,227 received

or...

4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Reply With Quote
The following 2 users say Thank You to shodson for this post:
 
  #167 (permalink)
Elite Member
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,255 since Jul 2012
Thanks: 1,059 given, 4,097 received


shodson View Post
4. stop re-optimizing, avoid systems that require optimization to be successful (I guess that's the same as #1)

Very good point!

I know some very good traders who do either or both of these things: create models with nothing to optimize, and/or create models that they never reoptimize.

I personally like walkforward, but I know people can and do have success without walkforward.

The key is to have a process/method that proves itself with real money.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following 2 users say Thank You to kevinkdog for this post:
 
  #168 (permalink)
Market Wizard
Houston TX
 
Futures Experience: Advanced
Platform: XTrader
Broker/Data: Advantage Futures
Favorite Futures: Energy
 
Posts: 1,862 since Dec 2013
Thanks: 1,585 given, 2,804 received
Forum Reputation: Legendary

Thanks Kevin.

It's interesting what you said in 2) as that relates to the next part of my question.

When I originally started on this model, I arbitrarily picked the last 11 years (04-14), 10 years to look at (04-13), and 1 year (14) as an out of sample test. Once I completed the first model (which is the model I'm actually still trading today) I started delving deeper into the data. I looked at using the last 9, 8, 7, 6 & 5 years rather than 10. The results showed that the model with the best fit used the last 7 years (07-13) rather than 10 years (04-14) of data. I could tell from looking at the models that the higher energy prices of the more recent years (2008, 10-13) where heavily influencing the models. Since the 7 year models results weren't significantly better than the 10 year, I decided to just stick with the original 10 year.

I actually then built a new model, that included a 'year variable'. This model was significantly better than any of the other models. The problem is the addition of the year variable gave values for 2015 significantly higher than anything previously seen, and of course 2016 would probably be even higher. Despite this model having better results I did not use it, and kept with the original 10 year model.

As we all know, energy prices have plummeted in the last year. The model I am using (built based upon 10 years (04-14)) has performed admirably. The model based upon 7 years of data would have been profitable, but not nearly as good. The model that included the 'year variable' would have got destroyed.

I'm comfortable with my decision to use the 10 year vs the 7 year, but by choosing the 10 year over the 'year variable' I feel like I allowed my personal bias to overrule what should be a more systematic and statistical approach. The fact that my personal bias was correct, obviously makes me feel good, but I'm still concerned I've contaminated the process.

So getting back to your point 2
you could have done this as a walkforward test with a moving window. Then every year, you'd re-tune your model with the most recent 10 years of data. So, your model could change every year. This is walkforward with unanchored testing (the in sample optimization window moves).
This is actually what I was planning to do. The problem is, I already know what happened in 2015. If I roll my window forward to 04-14 with an out of sample 2015, I know what the results will be. I know that any model more heavily based upon the recent high energy prices will perform far worse than a model with a longer tenor of data. I strongly suspect that a new model with a "polynomial year variable" (something like -(year-2011.5)^2) would probably perform quite well, but I know that because I know energy prices have had a concave trajectory over the last 15 years. I feel like whatever I do, my personal bias's/knowledge are corrupting the process.

I know there's no actual question in there, but I was curious if you have any additional thoughts on the matter?

Reply With Quote
The following 3 users say Thank You to SMCJB for this post:
 
  #169 (permalink)
Elite Member
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,255 since Jul 2012
Thanks: 1,059 given, 4,097 received

You personal biases may be impacting things, but that is not necessarily a bad thing. Your biases might actually be helping you to make a better decision.

Based on what you've done already, it is hard to recommend a way forward. Like you say, you already know the results, so trying to formalize this into a process probably doesn't make sense.

If I think of a way around this, I'll post it.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #170 (permalink)
Elite Member
SE Asia / Canada
 
Futures Experience: Intermediate
Platform: FXCM's Trading Station
Broker/Data: Darwinex, FXCM, Tradingview
Favorite Futures: Forex and CFD's
 
Posts: 114 since Jun 2010
Thanks: 26 given, 104 received


Hi Kevin,

Any more youtube videos planned in the close future? those are really good

cheers,

Reply With Quote
The following user says Thank You to DobermanTrading for this post:

Reply



futures io > > > > KJ Trading Systems Kevin Davey - Ask Me Anything (AMA)

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

FIO Journal Challenge featuring NinjaTrader ($2,000+ of prizes)

May
 

Portfolio Diversification w/Brendon Delate @ Diversify Portfolio

Elite only

An overview of volumetric analytical tools w/Sergey Sokolov @ PTMC

Elite only

John Hoagland (TBA)

Elite only

FuturesTrader71 Extended Ask Me Anything (AMA)

Elite only

Pete @ Jigsaw Trading (TBA)

Elite only

John @ No BS Day Trading (TBA)

Elite only

An Afternoon with FIO member Softsoap (being rescheduled)

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Webinar: Designing an Algo Trading System from A-to-Z, code included (Kevin Davey) Big Mike Elite Automated Trading 82 April 21st, 2017 08:51 AM
Webinar: Developing a Rule Based Trading Algorithm - Kevin Davey Big Mike Elite Automated Trading 68 December 1st, 2013 08:05 PM
Market Profile Trading Academy / MissTrade's Matt Davio - Ask Me Anything (AMA) Big Mike Vendors and Product Reviews 81 November 27th, 2013 05:04 PM
DeCarley Trading's Carley Garner (Senior Strategist/Broker) - Ask Me Anything (AMA) Big Mike Options on Futures 1 March 6th, 2013 06:00 PM
Your experience with Kevin Lowry-free range trading ? mjwfinc Vendors and Product Reviews 3 August 16th, 2012 08:12 AM


All times are GMT -4. The time now is 02:58 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-05-25 in 0.14 seconds with 20 queries on phoenix via your IP 54.198.118.102