Feedback on my approach to backtesting - Currency Futures | futures io social day trading
futures io futures trading


Feedback on my approach to backtesting
Updated: Views / Replies:621 / 6
Created: by thetamax8 Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Feedback on my approach to backtesting

  #1 (permalink)
Trading Apprentice
Vancouver, BC
 
Futures Experience: Beginner
Platform: Other
Favorite Futures: MJY, M6A, M6E, M6G, N225M, MCD
 
Posts: 25 since Sep 2014
Thanks: 0 given, 17 received

Feedback on my approach to backtesting

Hello,

I've been playing around with backtesting a very basic approach for forex. I ran through 3 years of daily prices for USDJPY using the following types of things:

Close at 5 day high AND
6 SMA above 12 SMA AND
12 SMA is higher than it was 6 days ago

Enter long at the "open" the next day
Stop loss is the LOW of the previous day
Exit when a new 2 day low is reached (exit at the open of the following day).


I ran through transaction by transaction for 3 years. Arrived at a positive result (sort of). I'm not looking for feedback on the rules; as I believe that there are many different ways to do this.

What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Any specific points that I should consider (I also tried to see the range of results if I used slightly diff parameters)

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Elite Member
Leeds UK
 
Futures Experience: Intermediate
Platform: FXTrade
Broker/Data: Interactive Brokers, CapitalSpreads, Oanda
Favorite Futures: GBP/USD
 
Tymbeline's Avatar
 
Posts: 401 since Apr 2015
Thanks: 894 given, 486 received



thetamax8 View Post
I ran through transaction by transaction for 3 years.

How many trades were there, over that period?

Manually, or software/automated?


thetamax8 View Post
Arrived at a positive result (sort of).

I think I won't be alone in wondering what "sort of" means?


thetamax8 View Post
What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Drawdown.

Also dealing-costs, spread, slippage, statistical significance, profit factor (are the ones that spring to mind), especially when looking back at theoretical trades: it's easy to imagine that you could have entered and exited trades at prices which you might not achieve in reality, in a live account.

I don't mean to sound disparaging, but this isn't one I'd backtest, myself.

Reply With Quote
 
  #4 (permalink)
Market Wizard
Cleveland Ohio/United States
 
Futures Experience: Advanced
Platform: Tradestation
Broker/Data: various
Favorite Futures: futures
 
Posts: 2,388 since Jul 2012
Thanks: 1,138 given, 4,387 received
Forum Reputation: Legendary


thetamax8 View Post
Hello,

I've been playing around with backtesting a very basic approach for forex. I ran through 3 years of daily prices for USDJPY using the following types of things:

Close at 5 day high AND
6 SMA above 12 SMA AND
12 SMA is higher than it was 6 days ago

Enter long at the "open" the next day
Stop loss is the LOW of the previous day
Exit when a new 2 day low is reached (exit at the open of the following day).


I ran through transaction by transaction for 3 years. Arrived at a positive result (sort of). I'm not looking for feedback on the rules; as I believe that there are many different ways to do this.

What I'm wondering though is, what other steps to backtest this should I take? What other things do I need to test or evaluate?

Any specific points that I should consider (I also tried to see the range of results if I used slightly diff parameters)


Did you use software to do this test? Be very careful if you did it manually, as manual backtests can be very inaccurate.

How many versions of this system did you test before you arrived at this particular one? If you tried 100 versions (different values for moving avg lengths, etc), I'd certainly trust it less than if you only tested one version of it.

Have you accounted for slippage and commission?

Are you using both the bid and ask data feed to evaluate your strategy, to make sure your stops are triggered properly?

I would definitely test for more than 3 years.

If it holds up over say 10 years, then I'd watch it live for a while and see if it still performs.

If you have any questions please send me a Private Message or use the futures.io "Ask Me Anything" thread
Reply With Quote
The following user says Thank You to kevinkdog for this post:
 
  #5 (permalink)
Elite Member
Sydney, Australia
 
Futures Experience: Intermediate
Platform: Sierra Chart, IRESS
Broker/Data: IB, IQFeed
Favorite Futures: ES, SPI, ASX stocks, options
 
Posts: 399 since Jun 2015
Thanks: 169 given, 446 received

Apart from the good replies above, I would also be asking, on what basis did you pick those entry and exit conditions?

Entry and exit conditions should be based on sound statistical reasons (mean reversion standard deviations, momentum trend SDs, anomalies due to institutional trading such as fund rebalancing or investment etc), so if you just pulled those entry conditions out of the air because it looked alright on paper I would be very suspicious (and careful) overall. An edge needs to be grounded in a solid reason as to why it exists, if you simply data mined the "edge" (by coming up with a random theory then optimising it), it is certainly a curve fit.

The steps you should take in a back test, each step should show positive expectancy and each step should increase expectancy.
1. Test the entry with a fixed time exit (end of the day, 1 day, 2 days, whatever).
2. Test the exit with the entry and variable target
3. Test exit with variable time stop (if your losing trades are longer in duration than winning, this is often the case)
4. Test exit with variable stop loss
5. Test scale in / scale out
6. Look at entry times and consider optimal entry times as the market can behave differently at different hours of the day

Test your optimised algo on out of sample data. If it doesn't blow up you are probably on to something.

Your equity curve should be smooth, and winners should not be clustered too much. Additionally, check for DD duration and that it is within your tolerance. Whatever you think your tolerance is, reduce it by 50% - everyone always over estimates how tolerant they are until real money comes into play.

Good luck.

Reply With Quote
 
  #6 (permalink)
Trading Apprentice
Vancouver, BC
 
Futures Experience: Beginner
Platform: Other
Favorite Futures: MJY, M6A, M6E, M6G, N225M, MCD
 
Posts: 25 since Sep 2014
Thanks: 0 given, 17 received

Thanks for the excellent feedback and advice. I'll look into incorporating those thoughts.

What I did was run a macro in Excel to automatically determine which transactions would occur. I did use the opening price of the next day as the entry; so there will need to be adjustments for the spread, slippage on stops, etc.

I'm not really looking to trade this though. It was more for the purposes of thinking, learning and seeing the outcomes of different approaches. I was trying to understand how a very simple approach would perform. One that really just looks at whether prices are moving up or down, goes in the direction of the trend and has consistent simple rules for entry / exit.

From watching the price for about 1yr, I suspected that there was some benefit of a 5 day vs. 2 day move; and a short period SMA. So just tested my theory.

For those interested, I did run this from January 1, 2013 to December 7th 2015. On the LONG and SHORT side (basically reversing the rules for the SHORTS).

There were a total of 101 transactions during this time.

For the 62 LONG transactions, 26 WINS, 36 LOSSES, Avg Win 0.8699 Avg Loss 0.4268
For the 39 SHORT transactions, 13 WINS, 26 LOSSES, Avg Win 0.94808 Avg Loss 0.37106

Not spectacular. Over the total period, it worked out to about 993 pips total. That's why I said it was "sort of" profitable. After factoring in slippage, commissions, I suspect its not worth the effort.

I also wanted to see if it was a fluke; so tested using different periods for the longer and shorter SMA (e.g. 3v12, 3v6, 3v9, etc.). The length of the SMA wasn't really a factor. The results were roughly similar.

I also tried it without any SMA rule at all, and as one would expect...that had poor results (as it basically had no filter for following the trend).

Anyways, it's a long way to go before this becomes any sort of real strategy. However, I did learn alot from doing it. Thanks everyone!!!

Oh, I will also try it with 2012 data , etc.

Reply With Quote
The following user says Thank You to thetamax8 for this post:
 
  #7 (permalink)
Elite Member
Leeds UK
 
Futures Experience: Intermediate
Platform: FXTrade
Broker/Data: Interactive Brokers, CapitalSpreads, Oanda
Favorite Futures: GBP/USD
 
Tymbeline's Avatar
 
Posts: 401 since Apr 2015
Thanks: 894 given, 486 received


thetamax8 View Post
Not spectacular. Over the total period, it worked out to about 993 pips total. That's why I said it was "sort of" profitable. After factoring in slippage, commissions, I suspect its not worth the effort.

I would think this must be right, if the figures above are without any allowance for dealing-costs and slippage.


thetamax8 View Post
The length of the SMA wasn't really a factor. The results were roughly similar.

Yes; any other conclusion would be surprising indeed, even on rather a small sample-size. Still, I can completely understand the "education value" of the exercise.

Reply With Quote

Reply



futures io > > > > Feedback on my approach to backtesting

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
why Backtesting stocks difficult vs backtesting Futures emini_Holy_Grail Traders Hideout 8 July 4th, 2015 03:14 PM
Market Replay Approach Auric NinjaTrader 6 June 7th, 2015 09:08 PM
Where Are We Now? A Comparative Timeline Approach Quick Summary News and Current Events 0 November 8th, 2011 05:10 AM
Managed vs Unmanaged approach Merlin997 NinjaTrader Programming 3 October 21st, 2011 06:32 AM
What is your experience with thetradescalper.com approach? vital38 Trading Reviews and Vendors 28 July 16th, 2010 08:40 AM


All times are GMT -4. The time now is 04:16 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-12 in 0.13 seconds with 19 queries on phoenix via your IP 54.163.210.170