NexusFi: Find Your Edge


Home Menu

 





Feedback on my approach to backtesting


Discussion in Currencies

Updated
      Top Posters
    1. looks_one thetamax8 with 2 posts (1 thanks)
    2. looks_two Tymbeline with 2 posts (0 thanks)
    3. looks_3 Quick Summary with 1 posts (0 thanks)
    4. looks_4 PeakGrowth with 1 posts (0 thanks)
    1. trending_up 2,178 views
    2. thumb_up 2 thanks given
    3. group 2 followers
    1. forum 6 posts
    2. attach_file 0 attachments




 
Search this Thread

Feedback on my approach to backtesting

  #1 (permalink)
thetamax8
Vancouver, BC
 
Posts: 25 since Sep 2014
Thanks Given: 0
Thanks Received: 17

Hello,

I've been playing around with backtesting a very basic approach for forex. I ran through 3 years of daily prices for USDJPY using the following types of things:

Close at 5 day high AND
6 SMA above 12 SMA AND
12 SMA is higher than it was 6 days ago

Enter long at the "open" the next day
Stop loss is the LOW of the previous day
Exit when a new 2 day low is reached (exit at the open of the following day).


I ran through transaction by transaction for 3 years. Arrived at a positive result (sort of). I'm not looking for feedback on the rules; as I believe that there are many different ways to do this.

What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Any specific points that I should consider (I also tried to see the range of results if I used slightly diff parameters)

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Futures True Range Report
The Elite Circle
ZombieSqueeze
Platforms and Indicators
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
Deepmoney LLM
Elite Quantitative GenAI/LLM
Better Renko Gaps
The Elite Circle
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
Battlestations: Show us your trading desks!
26 thanks
NexusFi site changelog and issues/problem reporting
25 thanks
The Program
17 thanks
  #3 (permalink)
 
Tymbeline's Avatar
 Tymbeline 
Leeds UK
Market Wizard
 
Experience: Intermediate
Platform: Tradovate
Broker: Tradovate
Trading: MES, MNQ
Frequency: Several times daily
Duration: Minutes
Posts: 644 since Apr 2015
Thanks Given: 2,342
Thanks Received: 1,054



thetamax8 View Post
I ran through transaction by transaction for 3 years.

How many trades were there, over that period?

Manually, or software/automated?


thetamax8 View Post
Arrived at a positive result (sort of).

I think I won't be alone in wondering what "sort of" means?


thetamax8 View Post
What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Drawdown.

Also dealing-costs, spread, slippage, statistical significance, profit factor (are the ones that spring to mind), especially when looking back at theoretical trades: it's easy to imagine that you could have entered and exited trades at prices which you might not achieve in reality, in a live account.

I don't mean to sound disparaging, but this isn't one I'd backtest, myself.

Reply With Quote
  #4 (permalink)
 kevinkdog   is a Vendor
 
Posts: 3,645 since Jul 2012
Thanks Given: 1,890
Thanks Received: 7,338


thetamax8 View Post
Hello,

I've been playing around with backtesting a very basic approach for forex. I ran through 3 years of daily prices for USDJPY using the following types of things:

Close at 5 day high AND
6 SMA above 12 SMA AND
12 SMA is higher than it was 6 days ago

Enter long at the "open" the next day
Stop loss is the LOW of the previous day
Exit when a new 2 day low is reached (exit at the open of the following day).


I ran through transaction by transaction for 3 years. Arrived at a positive result (sort of). I'm not looking for feedback on the rules; as I believe that there are many different ways to do this.

What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Any specific points that I should consider (I also tried to see the range of results if I used slightly diff parameters)


Did you use software to do this test? Be very careful if you did it manually, as manual backtests can be very inaccurate.

How many versions of this system did you test before you arrived at this particular one? If you tried 100 versions (different values for moving avg lengths, etc), I'd certainly trust it less than if you only tested one version of it.

Have you accounted for slippage and commission?

Are you using both the bid and ask data feed to evaluate your strategy, to make sure your stops are triggered properly?

I would definitely test for more than 3 years.

If it holds up over say 10 years, then I'd watch it live for a while and see if it still performs.

Follow me on Twitter Reply With Quote
Thanked by:
  #5 (permalink)
 PeakGrowth 
Sydney, Australia
 
Experience: Intermediate
Platform: Sierra Chart, IRESS
Broker: IB, IQFeed
Trading: ES, SPI, ASX stocks, options
Posts: 399 since Jun 2015
Thanks Given: 169
Thanks Received: 465

Apart from the good replies above, I would also be asking, on what basis did you pick those entry and exit conditions?

Entry and exit conditions should be based on sound statistical reasons (mean reversion standard deviations, momentum trend SDs, anomalies due to institutional trading such as fund rebalancing or investment etc), so if you just pulled those entry conditions out of the air because it looked alright on paper I would be very suspicious (and careful) overall. An edge needs to be grounded in a solid reason as to why it exists, if you simply data mined the "edge" (by coming up with a random theory then optimising it), it is certainly a curve fit.

The steps you should take in a back test, each step should show positive expectancy and each step should increase expectancy.
1. Test the entry with a fixed time exit (end of the day, 1 day, 2 days, whatever).
2. Test the exit with the entry and variable target
3. Test exit with variable time stop (if your losing trades are longer in duration than winning, this is often the case)
4. Test exit with variable stop loss
5. Test scale in / scale out
6. Look at entry times and consider optimal entry times as the market can behave differently at different hours of the day

Test your optimised algo on out of sample data. If it doesn't blow up you are probably on to something.

Your equity curve should be smooth, and winners should not be clustered too much. Additionally, check for DD duration and that it is within your tolerance. Whatever you think your tolerance is, reduce it by 50% - everyone always over estimates how tolerant they are until real money comes into play.

Good luck.

Visit my NexusFi Trade Journal Reply With Quote
  #6 (permalink)
thetamax8
Vancouver, BC
 
Posts: 25 since Sep 2014
Thanks Given: 0
Thanks Received: 17

Thanks for the excellent feedback and advice. I'll look into incorporating those thoughts.

What I did was run a macro in Excel to automatically determine which transactions would occur. I did use the opening price of the next day as the entry; so there will need to be adjustments for the spread, slippage on stops, etc.

I'm not really looking to trade this though. It was more for the purposes of thinking, learning and seeing the outcomes of different approaches. I was trying to understand how a very simple approach would perform. One that really just looks at whether prices are moving up or down, goes in the direction of the trend and has consistent simple rules for entry / exit.

From watching the price for about 1yr, I suspected that there was some benefit of a 5 day vs. 2 day move; and a short period SMA. So just tested my theory.

For those interested, I did run this from January 1, 2013 to December 7th 2015. On the LONG and SHORT side (basically reversing the rules for the SHORTS).

There were a total of 101 transactions during this time.

For the 62 LONG transactions, 26 WINS, 36 LOSSES, Avg Win 0.8699 Avg Loss 0.4268
For the 39 SHORT transactions, 13 WINS, 26 LOSSES, Avg Win 0.94808 Avg Loss 0.37106

Not spectacular. Over the total period, it worked out to about 993 pips total. That's why I said it was "sort of" profitable. After factoring in slippage, commissions, I suspect its not worth the effort.

I also wanted to see if it was a fluke; so tested using different periods for the longer and shorter SMA (e.g. 3v12, 3v6, 3v9, etc.). The length of the SMA wasn't really a factor. The results were roughly similar.

I also tried it without any SMA rule at all, and as one would expect...that had poor results (as it basically had no filter for following the trend).

Anyways, it's a long way to go before this becomes any sort of real strategy. However, I did learn alot from doing it. Thanks everyone!!!

Oh, I will also try it with 2012 data , etc.

Reply With Quote
Thanked by:
  #7 (permalink)
 
Tymbeline's Avatar
 Tymbeline 
Leeds UK
Market Wizard
 
Experience: Intermediate
Platform: Tradovate
Broker: Tradovate
Trading: MES, MNQ
Frequency: Several times daily
Duration: Minutes
Posts: 644 since Apr 2015
Thanks Given: 2,342
Thanks Received: 1,054


thetamax8 View Post
Not spectacular. Over the total period, it worked out to about 993 pips total. That's why I said it was "sort of" profitable. After factoring in slippage, commissions, I suspect its not worth the effort.

I would think this must be right, if the figures above are without any allowance for dealing-costs and slippage.


thetamax8 View Post
The length of the SMA wasn't really a factor. The results were roughly similar.

Yes; any other conclusion would be surprising indeed, even on rather a small sample-size. Still, I can completely understand the "education value" of the exercise.

Reply With Quote




Last Updated on December 11, 2015


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts