NexusFi: Find Your Edge


Home Menu

 





What should be the next step in my backtesting??


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one budfox with 40 posts (2 thanks)
    2. looks_two treydog999 with 21 posts (25 thanks)
    3. looks_3 tellytub with 6 posts (4 thanks)
    4. looks_4 shodson with 2 posts (2 thanks)
      Best Posters
    1. looks_one Ming80 with 2.5 thanks per post
    2. looks_two treydog999 with 1.2 thanks per post
    3. looks_3 shodson with 1 thanks per post
    4. looks_4 tellytub with 0.7 thanks per post
    1. trending_up 16,293 views
    2. thumb_up 41 thanks given
    3. group 9 followers
    1. forum 76 posts
    2. attach_file 4 attachments




 
Search this Thread

What should be the next step in my backtesting??

  #31 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013

I think I will put this current model on the shelf, and start manually backtesting my next model

Reasons:
  1. You can not set all the parameters for model1 in NT wizard> too complex
  2. I do not know nearly enough C# > and I figure I want a tradable model before I am a half decent programmer
  3. It challenging to manually backtest model1 since from 14 April to now it has only produced 53 trades. I remember Mike saying in one of his posts that he likes models that trade more frequently
  4. at 57% win rate, it has huge DD and not worth paying someone to code my strategy
  5. Difficulty getting sufficient data for Es on NT.

What you guys think?

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545

Thank You and havea great weekend all

Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Deepmoney LLM
Elite Quantitative GenAI/LLM
Are there any eval firms that allow you to sink to your …
Traders Hideout
NexusFi Journal Challenge - April 2024
Feedback and Announcements
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Futures True Range Report
The Elite Circle
 

  #32 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


budfox View Post
I think I will put this current model on the shelf, and start manually backtesting my next model

Reasons:
  1. You can not set all the parameters for model1 in NT wizard> too complex
  2. I do not know nearly enough C# > and I figure I want a tradable model before I am a half decent programmer
  3. It challenging to manually backtest model1 since from 14 April to now it has only produced 53 trades. I remember Mike saying in one of his posts that he likes models that trade more frequently
  4. at 57% win rate, it has huge DD and not worth paying someone to code my strategy
  5. Difficulty getting sufficient data for Es on NT.

What you guys think?

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545

Thank You and havea great weekend all

Hi everyone,

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545@josh

Like I mentioned in this previous post, I put the old model on the shelf and started backtesting a new model.


So the setup on my new model may occur a few times a day (compared to the old one which only setup like a few times a week), so I decided to take the three first trades of the setup of each trading day (sometimes it only setup 2 a day) for a current total of 52 trades.

I am loving this new model, I know its only 52 trades= N, but there is only a losing streak of 2 trades and winning streak of 3.

More stats:
  1. Win ratio = 71%
  2. Ave Winner = 0.98
  3. Ave Loser = 1.13
  4. I havent calcualted the expectancy ratio yet

NEXT STEP: to ensure I am backtesting on a different market period I will backtest another sample from 2 Jan - Feb 28 2014

The current sample is from July 29 - Aug 30

I would like to know what everyone thinks.

Thank You.


Started this thread Reply With Quote
  #33 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039



budfox View Post
Hi everyone,

@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545@josh

Like I mentioned in this previous post, I put the old model on the shelf and started backtesting a new model.


So the setup on my new model may occur a few times a day (compared to the old one which only setup like a few times a week), so I decided to take the three first trades of the setup of each trading day (sometimes it only setup 2 a day) for a current total of 52 trades.

I am loving this new model, I know its only 52 trades= N, but there is only a losing streak of 2 trades and winning streak of 3.

More stats:
  1. Win ratio = 71%
  2. Ave Winner = 0.98
  3. Ave Loser = 1.13
  4. I havent calcualted the expectancy ratio yet

NEXT STEP: to ensure I am backtesting on a different market period I will backtest another sample from 2 Jan - Feb 28 2014

The current sample is from July 29 - Aug 30

I would like to know what everyone thinks.

Thank You.



Honestly I don't think putting things on the shelf due to difficulty testing is the way to go. I know its harsh but what your doing is just delaying the inevitable, learning to code in order to test properly. Yes, with this system you have more trades (N) in a shorter period, yes it looks sexier. But the small 30 day window you tested means nothing, its no where near a complete market cycle nor is it representative of a financial year.

My advice stands from what i mentioned above. learn to code, and then you can test properly. This type of testing is going to be prone to finding small blips of activity with may not actually have an edge over the long run. in fact it might just be random phenomenon because of how short the duration is.

Reply With Quote
Thanked by:
  #34 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


treydog999 View Post
Honestly I don't think putting things on the shelf due to difficulty testing is the way to go. I know its harsh but what your doing is just delaying the inevitable, learning to code in order to test properly. Yes, with this system you have more trades (N) in a shorter period, yes it looks sexier. But the small 30 day window you tested means nothing, its no where near a complete market cycle nor is it representative of a financial year.

My advice stands from what i mentioned above. learn to code, and then you can test properly. This type of testing is going to be prone to finding small blips of activity with may not actually have an edge over the long run. in fact it might just be random phenomenon because of how short the duration is.


I understand.

What if January sample (N=50) correlates with the recent August sample? ie. has a similiar win ratio, drawdown etc?


I am considering hiring someone to code this strategy....how much am I looking to spend?

Pretty simple setup, over a two year period.

Started this thread Reply With Quote
  #35 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039


budfox View Post
I understand.

What if January sample (N=50) correlates with the recent August sample? ie. has a similiar win ratio, drawdown etc?


I am considering hiring someone to code this strategy....how much am I looking to spend?

Pretty simple setup, over a two year period.

Why don't you just test the whole year instead of randomly testing months and "looking for correlation" . I emphasis that because you are putting implicit bias in to your testing and research. Yes it might not be overt but it is there once you begin doing it.

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.

It also limits your flexibility, if you want to do isolated exit, entry or parameter sweeps. Again going to cost you and you are going to have to be very very specific on how you want to do those tests and view the results.

Reply With Quote
  #36 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


treydog999 View Post
Why don't you just test the whole year instead of randomly testing months and "looking for correlation" . I emphasis that because you are putting implicit bias in to your testing and research. Yes it might not be overt but it is there once you begin doing it.

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.

It also limits your flexibility, if you want to do isolated exit, entry or parameter sweeps. Again going to cost you and you are going to have to be very very specific on how you want to do those tests and view the results.

Ok.

So I have to learn C# eventually. I am not spending a few grand on anything.

For Model #2 I can B.T. 12 trades in approximately an hour (thats four days of trading), so I could backtest an entire month of trading (20 days) in about 5 hours.

so would there be much error if I backtested the whole of 2014 (so far) assuming there is 55 trades a month? N= 9 x 55 = 495.........N=495 seems enough?

Hey Trey thanks for all the great advice, I owe you big time.


Model #1 is really hard to 'code' or implement in the wizard.

Started this thread Reply With Quote
  #37 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039


budfox View Post
Ok.

So I have to learn C# eventually. I am not spending a few grand on anything.

For Model #2 I can B.T. 12 trades in approximately an hour (thats four days of trading), so I could backtest an entire month of trading (20 days) in about 5 hours.

so would there be much error if I backtested the whole of 2014 (so far) assuming there is 55 trades a month? N= 9 x 55 = 495.........N=495 seems enough?

Hey Trey thanks for all the great advice, I owe you big time.


Model #1 is really hard to 'code' or implement in the wizard.

As with anything learn to walk before you can run. Its better to code some easy strategies that may not be profitable. But I can tell you from personal experience simple strategies can be profitable and are usually more robust. Then work up to your complex ones. Not only will it give you practice in the development cycle, which is as crucial to an algo tester as is trade evaluation cycle for a manual trader. But you will get a sense of what "works" and what "doesn't".

Yes if you did a year and had N=495 I would consider that a much fairer sample than what you have shown before. Honestly 5 hours isn't that long. I have spent many many more than that debugging alone.

No problem for the advice. Hope to see your progress soon.

Reply With Quote
Thanked by:
  #38 (permalink)
 
treydog999's Avatar
 treydog999 
seoul, Korea
 
Experience: Intermediate
Platform: Multicharts
Broker: CQG, DTN IQfeed
Trading: YM 6E
Posts: 897 since Jul 2012
Thanks Given: 291
Thanks Received: 1,039

Personally I learned C++ 15+ years ago in school but i thought this would help you with some learning resources. I have used the Lynda series to refresh myself and also Microsoft virtual academy. Lynda is paid but pretty good (or you can find it some other method wink wink). But the Microsoft virtual academy is really great for a free resource with high quality vids.

C# Fundamentals for Beginners | Microsoft Virtual Academy

Reply With Quote
Thanked by:
  #39 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


treydog999 View Post

Paying for coding at first seems reasonable but usually prices get out of hand. In general look at a few hundred $$ just to get it started, even if the strategy test ends up unprofitable. The reason I say that, is your are going to have to pay for any debugging time, changes in code, or any other modifications. You will explain as best you can to the developer but chances that the signals and code comes out exactly as you want it first time, slim to none. So if your time is worth more then that go ahead and do it. But spending a few grand to test these 2 systems to me seems like a lot, but then again I can do it myself.
.

Hell0 @treydog999,

I would like to inquire who would be doing the backtesting (that I am paying), is it a quant developer? or just someone who knows how to code?

That leads me to my next question, do I have to hire a "professional backtester" , or can I just hire some kid who knows how to code and write a script with no trading experience?


Thank You trey

Started this thread Reply With Quote
  #40 (permalink)
 budfox 
Toronto
 
Experience: Beginner
Platform: Sierra
Broker: MB
Trading: ES
Posts: 313 since Jun 2013


So I did manually and painfully backtest the month of January for a total of 106 trades (N).


I also did add a another parameter (adding in a trailing stop as opposed to the fixed stop).

The results were completely different:

  • Max DD = 96%
  • win rate = 51%
  • Ave Winner = 0.95 (points)
  • Ave Loser = 0.72 (brought down after implementing new trailer rule)
  • Expectancy ratio = 0.18

This is probably one of my most statistically accurate models that I have backtested, with an accuracy score of 1/SQRT (106)= 0.097.

As you can see the metrics are horrible> probably wont continue backtesting this? Model one with an expectancy score of 0.54 is better (but I need to backtest that more, with N= 57)


Yes I am still learning how to code C# .


@treydog999 @Ming80 @bobwest @kevinkdog @NoiseTrader716 @daytrader4545 @josh @sam028

Started this thread Reply With Quote





Last Updated on December 1, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts