What should be the next step in my backtesting?? - Emini Index Futures Trading | futures io social day trading
futures io futures trading


What should be the next step in my backtesting??
Updated: Views / Replies:6,585 / 76
Created: by budfox Attachments:4

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 4  
 
Thread Tools Search this Thread
 

What should be the next step in my backtesting??

  #31 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

Going to start backtesting Model2

I think I will put this current model on the shelf, and start manually backtesting my next model

Reasons:
  1. You can not set all the parameters for model1 in NT wizard> too complex
  2. I do not know nearly enough C# > and I figure I want a tradable model before I am a half decent programmer
  3. It challenging to manually backtest model1 since from 14 April to now it has only produced 53 trades. I remember Mike saying in one of his posts that he likes models that trade more frequently
  4. at 57% win rate, it has huge DD and not worth paying someone to code my strategy
  5. Difficulty getting sufficient data for Es on NT.

What you guys think?

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545

Thank You and havea great weekend all

Reply With Quote
 
  #32 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


budfox View Post
I think I will put this current model on the shelf, and start manually backtesting my next model

Reasons:
  1. You can not set all the parameters for model1 in NT wizard> too complex
  2. I do not know nearly enough C# > and I figure I want a tradable model before I am a half decent programmer
  3. It challenging to manually backtest model1 since from 14 April to now it has only produced 53 trades. I remember Mike saying in one of his posts that he likes models that trade more frequently
  4. at 57% win rate, it has huge DD and not worth paying someone to code my strategy
  5. Difficulty getting sufficient data for Es on NT.

What you guys think?

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545

Thank You and havea great weekend all

Hi everyone,

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545@josh

Like I mentioned in this previous post, I put the old model on the shelf and started backtesting a new model.


So the setup on my new model may occur a few times a day (compared to the old one which only setup like a few times a week), so I decided to take the three first trades of the setup of each trading day (sometimes it only setup 2 a day) for a current total of 52 trades.

I am loving this new model, I know its only 52 trades= N, but there is only a losing streak of 2 trades and winning streak of 3.

More stats:
  1. Win ratio = 71%
  2. Ave Winner = 0.98
  3. Ave Loser = 1.13
  4. I havent calcualted the expectancy ratio yet

NEXT STEP: to ensure I am backtesting on a different market period I will backtest another sample from 2 Jan - Feb 28 2014

The current sample is from July 29 - Aug 30

I would like to know what everyone thinks.

Thank You.


Reply With Quote
 
  #33 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received



budfox View Post
Hi everyone,

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545@josh

Like I mentioned in this previous post, I put the old model on the shelf and started backtesting a new model.


So the setup on my new model may occur a few times a day (compared to the old one which only setup like a few times a week), so I decided to take the three first trades of the setup of each trading day (sometimes it only setup 2 a day) for a current total of 52 trades.

I am loving this new model, I know its only 52 trades= N, but there is only a losing streak of 2 trades and winning streak of 3.

More stats:
  1. Win ratio = 71%
  2. Ave Winner = 0.98
  3. Ave Loser = 1.13
  4. I havent calcualted the expectancy ratio yet

NEXT STEP: to ensure I am backtesting on a different market period I will backtest another sample from 2 Jan - Feb 28 2014

The current sample is from July 29 - Aug 30

I would like to know what everyone thinks.

Thank You.



Honestly I don't think putting things on the shelf due to difficulty testing is the way to go. I know its harsh but what your doing is just delaying the inevitable, learning to code in order to test properly. Yes, with this system you have more trades (N) in a shorter period, yes it looks sexier. But the small 30 day window you tested means nothing, its no where near a complete market cycle nor is it representative of a financial year.

My advice stands from what i mentioned above. learn to code, and then you can test properly. This type of testing is going to be prone to finding small blips of activity with may not actually have an edge over the long run. in fact it might just be random phenomenon because of how short the duration is.

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #34 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


treydog999 View Post
Honestly I don't think putting things on the shelf due to difficulty testing is the way to go. I know its harsh but what your doing is just delaying the inevitable, learning to code in order to test properly. Yes, with this system you have more trades (N) in a shorter period, yes it looks sexier. But the small 30 day window you tested means nothing, its no where near a complete market cycle nor is it representative of a financial year.

My advice stands from what i mentioned above. learn to code, and then you can test properly. This type of testing is going to be prone to finding small blips of activity with may not actually have an edge over the long run. in fact it might just be random phenomenon because of how short the duration is.


I understand.

What if January sample (N=50) correlates with the recent August sample? ie. has a similiar win ratio, drawdown etc?


I am considering hiring someone to code this strategy....how much am I looking to spend?

Pretty simple setup, over a two year period.

Reply With Quote
 
  #35 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
I understand.

What if January sample (N=50) correlates with the recent August sample? ie. has a similiar win ratio, drawdown etc?


I am considering hiring someone to code this strategy....how much am I looking to spend?

Pretty simple setup, over a two year period.

Why don't you just test the whole year instead of randomly testing months and "looking for correlation" . I emphasis that because you are putting implicit bias in to your testing and research. Yes it might not be overt but it is there once you begin doing it.

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.

It also limits your flexibility, if you want to do isolated exit, entry or parameter sweeps. Again going to cost you and you are going to have to be very very specific on how you want to do those tests and view the results.

Reply With Quote
 
  #36 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


treydog999 View Post
Why don't you just test the whole year instead of randomly testing months and "looking for correlation" . I emphasis that because you are putting implicit bias in to your testing and research. Yes it might not be overt but it is there once you begin doing it.

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.

It also limits your flexibility, if you want to do isolated exit, entry or parameter sweeps. Again going to cost you and you are going to have to be very very specific on how you want to do those tests and view the results.

Ok.

So I have to learn C# eventually. I am not spending a few grand on anything.

For Model #2 I can B.T. 12 trades in approximately an hour (thats four days of trading), so I could backtest an entire month of trading (20 days) in about 5 hours.

so would there be much error if I backtested the whole of 2014 (so far) assuming there is 55 trades a month? N= 9 x 55 = 495.........N=495 seems enough?

Hey Trey thanks for all the great advice, I owe you big time.


Model #1 is really hard to 'code' or implement in the wizard.

Reply With Quote
 
  #37 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received


budfox View Post
Ok.

So I have to learn C# eventually. I am not spending a few grand on anything.

For Model #2 I can B.T. 12 trades in approximately an hour (thats four days of trading), so I could backtest an entire month of trading (20 days) in about 5 hours.

so would there be much error if I backtested the whole of 2014 (so far) assuming there is 55 trades a month? N= 9 x 55 = 495.........N=495 seems enough?

Hey Trey thanks for all the great advice, I owe you big time.


Model #1 is really hard to 'code' or implement in the wizard.

As with anything learn to walk before you can run. Its better to code some easy strategies that may not be profitable. But I can tell you from personal experience simple strategies can be profitable and are usually more robust. Then work up to your complex ones. Not only will it give you practice in the development cycle, which is as crucial to an algo tester as is trade evaluation cycle for a manual trader. But you will get a sense of what "works" and what "doesn't".

Yes if you did a year and had N=495 I would consider that a much fairer sample than what you have shown before. Honestly 5 hours isn't that long. I have spent many many more than that debugging alone.

No problem for the advice. Hope to see your progress soon.

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #38 (permalink)
Elite Member
seoul, Korea
 
Futures Experience: Intermediate
Platform: Multicharts
Broker/Data: CQG, DTN IQfeed
Favorite Futures: YM 6E
 
treydog999's Avatar
 
Posts: 894 since Jul 2012
Thanks: 291 given, 1,006 received

Personally I learned C++ 15+ years ago in school but i thought this would help you with some learning resources. I have used the Lynda series to refresh myself and also Microsoft virtual academy. Lynda is paid but pretty good (or you can find it some other method wink wink). But the Microsoft virtual academy is really great for a free resource with high quality vids.

C# Fundamentals for Beginners | Microsoft Virtual Academy

Reply With Quote
The following user says Thank You to treydog999 for this post:
 
  #39 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received


treydog999 View Post

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.
.

Hell0 @treydog999,

I would like to inquire who would be doing the backtesting (that I am paying), is it a quant developer? or just someone who knows how to code?

That leads me to my next question, do I have to hire a "professional backtester" , or can I just hire some kid who knows how to code and write a script with no trading experience?


Thank You trey

Reply With Quote
 
  #40 (permalink)
Membership Temporarily Revoked
Toronto
 
Futures Experience: Beginner
Platform: Sierra
Broker/Data: MB
Favorite Futures: ES
 
Posts: 313 since Jun 2013
Thanks: 182 given, 58 received

Update


So I did manually and painfully backtest the month of January for a total of 106 trades (N).


I also did add a another parameter (adding in a trailing stop as opposed to the fixed stop).

The results were completely different:

  • Max DD = 96%
  • win rate = 51%
  • Ave Winner = 0.95 (points)
  • Ave Loser = 0.72 (brought down after implementing new trailer rule)
  • Expectancy ratio = 0.18

This is probably one of my most statistically accurate models that I have backtested, with an accuracy score of 1/SQRT (106)= 0.097.

As you can see the metrics are horrible> probably wont continue backtesting this? Model one with an expectancy score of 0.54 is better (but I need to backtest that more, with N= 57)


Yes I am still learning how to code C# .


@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545 @josh @sam028

Reply With Quote

Reply



futures io > > > > What should be the next step in my backtesting??

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Jigsaw Trading: TBA

Elite only

FuturesTrader71: TBA

Elite only

NinjaTrader: TBA

Jan 18

RandBots: TBA

Jan 23

GFF Brokers & CME Group: Futures & Bitcoin

Elite only

Adam Grimes: TBA

Elite only

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Step MA crazybears Sierra Chart 2 July 15th, 2014 06:25 AM
Top Step Trader website down? birdoggg Platforms and Indicators 2 November 7th, 2012 10:07 AM
The Next Step Towards The End Of The Euro Quick Summary News and Current Events 0 November 16th, 2011 12:50 AM


All times are GMT -4. The time now is 02:52 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-18 in 0.18 seconds with 20 queries on phoenix via your IP 54.92.194.75