Backtesting & Optimization -- worthless? How to get value from them? - NinjaTrader | futures.io
futures io futures trading


Backtesting & Optimization -- worthless? How to get value from them?
Started: by Jayswiss Views / Replies:355 / 9
Last Reply: Attachments:0

Welcome to futures io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.


You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Reply
 
Thread Tools Search this Thread
 

Backtesting & Optimization -- worthless? How to get value from them?

  #1 (permalink)
Elite Member
San Francisco, CA / USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, TradingView
Favorite Futures: ES, TF, NQ, CL, GC
 
Posts: 29 since Jun 2016
Thanks: 4 given, 3 received

Backtesting & Optimization -- worthless? How to get value from them?

I've seen a lot of replies buried in threads that subtlety suggest that backtesting & optimization do not have as much value as we might be led to believe. People have posted "successful" strategies, and advanced users quickly reply to say the results are unrealistic and can not be replied upon (and often fail, miserably, in live trading).

Simple questions -- Why? And how should we be utilizing these functions to realize the most value from them?


I'm sitting here trying to develop a strategies (as a new NT user) using the Backtest feature of Ninjatrader. I have been giving weight and considerations to the minute differences in results from the Backtesting engine. However, after reading some of the posts here at Futures.io, I wonder if I am throwing out profitable strategies, going in the wrong directions towards improving a strategy due to unreliable(?) data coming from the Backtesting .

I've seen some people suggest to use optimizer on half of your data series, then backtest using the optimized parameters for the other half. If that's something that is sound advice, that is the kind of information & tips that I am seeking with this thread.

Hopefully the experts and advanced users can shed some light here.

Thanks!


Last edited by Jayswiss; July 17th, 2016 at 12:43 PM.
Reply With Quote
The following user says Thank You to Jayswiss for this post:
 
  #2 (permalink)
Elite Member
Omer עומר / Israel י
 
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 1,830 since Sep 2013
Thanks: 1,309 given, 2,753 received
Forum Reputation: Legendary

Hi

It boils down to 'know' what you are doing and what to expect from back-testing and optimization.

It is possible to take a period of data and find 'magic' settings that create the best possible (optimized)
result over that period. Now adding data from another period, or just from tomorrow, will change the
results and they may be horrible.

Is all optimization bad ? no, you need to understand what you are doing

If a series of values for a parameter 16,17,18,19,20 all create positive profitable results and you pick
the best value, the chances are that your system is less vulnerable. If you have 18 being profitable
and all the rest loosers, guaranteed your strategy will fail in forward.

Probably a good read would be the book of Kevin. it walks you step by step through an approach.
Is it the only possible approach ? no.. for sure not
Is it a good approach ? if you apply and the strategy passes the tests, then you are on good track

good luck !

Reply With Quote
 
  #3 (permalink)
Elite Member
San Francisco, CA / USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, TradingView
Favorite Futures: ES, TF, NQ, CL, GC
 
Posts: 29 since Jun 2016
Thanks: 4 given, 3 received


Thanks, rleplae.

Can you link to "The book of Kevin"? I tried some searches and I guess that's not the title?

You bring up a good point with regards to general optimizations. For example, many people favor SMAs (or EMAs) of 50/200 or 9/50. I assume these numbers are the favorites because they were tested (long before NT existed) and seemed to be the most consistently proven winners across the board? Or were they just rabbitoutofhat starting points that stuck after a couple people wrote books about them? Even today on many wise TA educational sites, you will find "standard" sets of numbers to use for SMA crossovers (and RSI, and MACD, ADX, etc. etc.). But of course I agree... 50/200 may be hat magic today, but terrible tomorrow.

Reply With Quote
 
  #4 (permalink)
Elite Member
San Francisco, CA / USA
 
Futures Experience: Intermediate
Platform: NinjaTrader, TradingView
Favorite Futures: ES, TF, NQ, CL, GC
 
Posts: 29 since Jun 2016
Thanks: 4 given, 3 received

For optimization specifically though, I do like the idea of using the same specific periods of difficult historical trading periods + a few "regular" trading periods to test against as a sort of benchmark. A backtesting profile? Something like that. Then when you find something that worked through the bulk of those periods within your standard profile of periods to test, retry against a current or more recent period of time and see how it works out.

Reply With Quote
 
  #5 (permalink)
Elite Member
Omer עומר / Israel י
 
Futures Experience: Master
Platform: NinjaTrader, Proprietary,
Broker/Data: Ninjabrokerage/IQfeed + Synthetic datafeed
Favorite Futures: 6A, 6B, 6C, 6E, 6J, 6S, ES, NQ, YM, AEX, CL, NG, ZB, ZN, ZC, ZS, GC
 
rleplae's Avatar
 
Posts: 1,830 since Sep 2013
Thanks: 1,309 given, 2,753 received
Forum Reputation: Legendary

Futures Edge on FIO

What value do you place on the webinars on FIO?

 
"building algorithmic trading systems"
kevin davey

Reply With Quote
The following user says Thank You to rleplae for this post:
 
  #6 (permalink)
Elite Member
Seattle
 
Futures Experience: Intermediate
Platform: MultiCharts 10
Broker/Data: InteractiveBrokers, IQFeed
Favorite Futures: ES
 
Posts: 41 since Nov 2015
Thanks: 17 given, 2 received

Backtest Data

hello all,

With regard to back testing I'm interested in larger historical time periods and thus have a need for a good data source. I am specifically interested in YM, ES, NQ, ZN,ZB,E6 DAX etc.... I'm not interested in free stuff etc... Just a quality source where I can purchase the needed data and upload it into NT7. Anyone have any experience with this, hopefully going back at least 3 years.

Thanks for the help.

GuppyDRV

Reply With Quote
 
  #7 (permalink)
Elite Member
Indianapolis, Indiana/ USA
 
Futures Experience: Advanced
Platform: MT4, Ninjatrader
Broker/Data: AMP / CQG
Favorite Futures: Futures
 
Posts: 45 since Oct 2012
Thanks: 108 given, 11 received

GuppyDRV,

I am also looking; Did you have any success ?

Richard

Reply With Quote
 
  #8 (permalink)
Elite Member
Seattle
 
Futures Experience: Intermediate
Platform: MultiCharts 10
Broker/Data: InteractiveBrokers, IQFeed
Favorite Futures: ES
 
Posts: 41 since Nov 2015
Thanks: 17 given, 2 received

I made a move from NT8 to MC recently. I just find the Easy Language stuff faster for me to work with. I'm still trading with NT but should be migrated shortly. As a result my data needs changed so I switched providers for historical. I had good luck with these guys for NT and for that matter it migrates to MC if you ever use that. Their not cheap but for me that wasn't my prime deciding factor.

I believe Tckdata is cheaper but DTN has the NXCore product with a wide reach.

https://www.tickdata.com

Commodity market trading data - DTN ProphetX

Hope it helps.

GuppyDRV


Last edited by GuppyDRV; March 12th, 2017 at 05:51 PM. Reason: Add second link
Reply With Quote
 
  #9 (permalink)
Elite Member
Indianapolis, Indiana/ USA
 
Futures Experience: Advanced
Platform: MT4, Ninjatrader
Broker/Data: AMP / CQG
Favorite Futures: Futures
 
Posts: 45 since Oct 2012
Thanks: 108 given, 11 received

GuppyDRV,

I just downloaded the MultiCharts.Net platform for a trial. Thanks for the data links.

I had good luck with these guys for NT and for that matter it migrates to MC if you ever use that.

So, to ensure I follow you, you were able to initially use that data in ninjatrader and eventually port it over to MC ? If correct, that is a nice bonus !

Best,
Richard

Reply With Quote
 
  #10 (permalink)
Elite Member
Seattle
 
Futures Experience: Intermediate
Platform: MultiCharts 10
Broker/Data: InteractiveBrokers, IQFeed
Favorite Futures: ES
 
Posts: 41 since Nov 2015
Thanks: 17 given, 2 received


Yes and yes but I'm using MC 10 not the MC.NET

C# is not my strong suit.....well none really but I can get buy with EasyLanguage better.

GuppyDRV

Reply With Quote

Reply



futures io > > > > Backtesting & Optimization -- worthless? How to get value from them?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

PTMC Protrader Platform First Look w/Sergey Sokolov

Elite only

Spring Grains Outlook w/Sean Lusk @ Walsh Trading

Elite only

FIO Video Journal Challenge featuring NinjaTrader ($2,000+ of prizes)

April

Process above all else w/Anthony Crudele @ Futures Radio Show

Elite only

Machine Learning - Quantitative Trading w/Martin Froehler @ Quantiacs

Elite only

Market Dynamics w/Peter Davies @ Jigsaw Trading

Elite only

Ask Me Anything w/Patrick Rooney @ Trading Technologies

Apr 18

Ask Me Anything w/FuturesTrader71

Apr 19

Machine Learning w/Kris Longmore

Elite only

Market Analysis w/Dave Forss

Apr 25

Introducing iSystems with Stage 5 Trading

Apr 27
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
PC requirements for NT backtesting & optimization? w00dmann NinjaTrader 5 November 10th, 2015 12:12 PM
R Backtesting and Optimization: Quantstrat Big Mike Matlab, R project and Python 16 February 14th, 2015 08:06 PM
Multicharts backtesting and optimization ramblings/questions.. benharrell Elite EasyLanguage Automated Trading 4 June 28th, 2012 06:25 PM
Rethinking Backtesting and Optimization gordo Elite Automated Trading 55 March 19th, 2012 05:53 PM
Do we need to use the bar magnifier when backtesting/optimization in MultiCharts? benharrell MultiCharts 8 May 4th, 2011 12:17 AM


All times are GMT -4. The time now is 11:42 PM.

no new posts
Page generated 2017-03-25 in 0.13 seconds with 19 queries on phoenix via your IP 184.73.47.110