Why using the term "curve-fitting" is wrong - Traders Hideout | futures io social day trading
futures io futures trading


Why using the term "curve-fitting" is wrong
Updated: Views / Replies:847 / 1
Created: by Outlier Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Why using the term "curve-fitting" is wrong

  #1 (permalink)
Elite Member
Germany
 
Futures Experience: Advanced
Platform: TradeStation
Favorite Futures: Futures
 
Posts: 88 since May 2012
Thanks: 53 given, 90 received

Why using the term "curve-fitting" is wrong

After having seen the term being misused and abused hundreds of times, and having been guilty of it myself just until a few years ago, I had to write this.

Curve-fitting is a term from non-linear regression analysis and means constructing a curve, such as a high order polynomial, that best fits a series of data points. It is commonly used as an aid for visualization. Think of a curve that cuts right through your data. Curve-fitting alone is neither good nor bad in the sense that it makes no claims at all about any extrapolation or generalization performance.

Over-fitting, the correct term from statistics and machine learning, means that a model generalizes poorly. On the training set (in-sample), the model has good performance. But on the validation set and test set (out-of-sample), the model has bad performance. Over-fitting, also called high variance, occurs when a model has too many degrees of freedom, or capacity, that during training is fitted to random noise (or sampling error), rather than the underlying structure.

The opposite is under-fitting, also called high bias. An under-fitted model has very similar performance on the training and validation set. But each time, the performance is poor because the capacity of the model is too small to capture enough of the underlying structure.

Desirable is a good fit, a solution to the bias-variance tradeoff.

Over-fitting is countered by the following:
- increasing the number of trades, by raising their frequency and/or using more data, which naturally reduces the sampling error (best approach)
- reduction of the model capacity
- advanced techniques like regularization, early stopping, pruning

Under-fitting is countered by increasing the model capacity.

More at https://en.wikipedia.org/wiki/Curve_fitting
https://en.wikipedia.org/wiki/Overfitting

Successful systematic traders know about this.

"I can talk a little more about over-fitting, if not my personal proprietary techniques. First of all I like the [term] over-fitting rather than curve-fitting because curve-fitting is a term from non-linear regression analysis. It is where you have a lot of data and you are fitting the data points to some curve. Well, you are not doing that with futures. Technically there is no curve-fitting here; the term does not apply. But what you can do is you can over-fit. The reason I like the term over-fit rather than curve-fit is that over-fit shows that you also can under-fit. The people who do not optimize are under-fitting." -- William Eckhardt

William Eckhardt: The man who launched 1,000 systems

Related to these issues is data-mining bias. When a large number of systems is evaluated during training or even validation, the best systems may meet your criteria just by chance. The more systems are tested, the higher the random variation in results.

It can be countered by:
- evaluating a selected group of systems on a 2nd validation set, taking into account the whole distribution of performance
- evaluating the final choice of system(s) on the test set that is only used once, to get an unbiased estimate of performance

None of this, however, can protect against regime changes. The markets could change enough to invalidate any statistical and structural edge. The whole training-validation-test approach of data splitting works under the assumption that all segments are drawn from the same distribution. Unfortunately, that distribution may change substantially in the future. For example, the advent of HFT was a serious regime change for discretionary stock scalpers. Fortunately, regime changes happen gradually over time, giving the trader time to adapt. It may also help to be slightly on the high-bias side.

Reply With Quote
The following 7 users say Thank You to Outlier for this post:
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.


Reply



futures io > > > Why using the term "curve-fitting" is wrong

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Fukushima: "China Syndrome Is Inevitable" ... "Huge Steam Explosions", or "Nuclear Bo Quick Summary News and Current Events 0 November 22nd, 2011 02:50 AM
How to change "Stop Loss" and "Take Profit" in "shElderImpulse" Strategy javed759 NinjaTrader 1 November 1st, 2011 11:44 PM
"mid", "buy", "sell" volumes lokgotkent Traders Hideout 6 September 30th, 2011 02:24 PM
Warren Buffet is "wrong" on taxes RM99 News and Current Events 4 July 29th, 2011 11:10 AM
very short-term "Swing" trading heywally The Elite Circle 2 May 13th, 2011 02:30 PM


All times are GMT -4. The time now is 07:49 AM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.08 seconds with 19 queries on phoenix via your IP 54.226.132.197