Statistical significance - Traders Hideout | futures io social day trading
futures io futures trading


Statistical significance
Updated: Views / Replies:757 / 6
Created: by Zwaen Attachments:0

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors – all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you don’t need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 
Thread Tools Search this Thread
 

Statistical significance

  #1 (permalink)
Elite Member
Netherlands
 
Futures Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: 6E
 
Zwaen's Avatar
 
Posts: 247 since Dec 2010
Thanks: 762 given, 221 received

Statistical significance

Suppose you have a database of trades A.

Suppose you want to use a simple ‘filter ‘ to improve statistics. Lets say this is a simple filter which in your opinion is valid, eg don’t take trades after time y. Reason is that after time y price has not enough time/opportunity to travel to your desired target( you don’t go overnight, and there is no exceptional volatility). Another example would be when you want to analyze if your stoploss and profit target can be made dependent on current volatility. Offcourse, the reason could be anything, as long as the trader can suspect there lies some causality or logical thinking behind the rationale to use the filter.

We don’t want to use too many filters, since more filters means more chance of curve fitting.
What number of trades B in dataset A is sufficiënt to be statistically significant? And if this depends on the number in database A, how would you think/know this relation yields?

At this moment I use a rule of thumb that is around 30 ( law of diminishing returns from statistics) observations. So there need to be a minimum of around 30 (=B) observations to let the filter be of any significant value. I also look at scatterplots/histograms of the data and use common sense ( asuming I have some) to ascertain if the difference is statistically significant. But I wonder if others use a more methodical approach to this?

One of my worst enemies are my own false assumptions
Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
 Vendor: sentimentrader.com 
Tulsa, OK
 
Futures Experience: Advanced
Platform: Tradestation, TOS, Python
Broker/Data: IQFeed, Tradestation, TOS
Favorite Futures: ES, SPY, Options
 
ericbrown's Avatar
 
Posts: 201 since Jan 2011
Thanks: 339 given, 234 received


A rule of thumb of 30 observations helps with the Central Limit Theorem issues, but it doesn't prove statistical significance.

I would suggest start here -> https://www.csulb.edu/~msaintg/ppa696/696stsig.htm

From that page, you need to answer these two questions:


Quoting 
1) what is the probability that the relationship exists;
2) if it does, how strong is the relationship

To answer those questions, you'll need to determine how best to study the relationship between the number of trades "B" and the number of trades in your database "A".

Reply With Quote
The following 3 users say Thank You to ericbrown for this post:
 
  #4 (permalink)
Elite Member
Netherlands
 
Futures Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: 6E
 
Zwaen's Avatar
 
Posts: 247 since Dec 2010
Thanks: 762 given, 221 received


ericbrown View Post

Interesting link, thanks! I will certainly read this.

One of my worst enemies are my own false assumptions
Reply With Quote
 
  #5 (permalink)
 Vendor: sentimentrader.com 
Tulsa, OK
 
Futures Experience: Advanced
Platform: Tradestation, TOS, Python
Broker/Data: IQFeed, Tradestation, TOS
Favorite Futures: ES, SPY, Options
 
ericbrown's Avatar
 
Posts: 201 since Jan 2011
Thanks: 339 given, 234 received


Zwaen View Post
Interesting link, thanks! I will certainly read this.

Welcome.

There's a lot on that page to take in. I use quite a bit of those analysis techniques in my own work. I'm not that great at stats but I know some of the basics so feel free to ask questions.

Reply With Quote
The following user says Thank You to ericbrown for this post:
 
  #6 (permalink)
Elite Member
Netherlands
 
Futures Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: 6E
 
Zwaen's Avatar
 
Posts: 247 since Dec 2010
Thanks: 762 given, 221 received


ericbrown View Post
Welcome.

There's a lot on that page to take in. I use quite a bit of those analysis techniques in my own work. I'm not that great at stats but I know some of the basics so feel free to ask questions.



Hi Eric,

Thanks! I did the following calculation, and wondered if my asumptions and calculations correct, or do I make some mistakes?

I will use simple numbers. Reality is offcourse not so simple, but to illustrate the idea, I either win or lose, given a fixed amount ( target is always t, loss is always l, but are not relevant for calculations)

Suppose you have a set of 200 trades ( set A) which have positive EV. You want to evaluate if filter B is relevant. Filter B contains 25 trades, and has negative EV. We want to know if the 25 trades are significant, so we compare the 2 distributions.

Set A:
200 trades
80 trades are closed at target.
120 trades are stopped out.
Pclose target = 80/200= 0.40
Pstopped out/close trade=120/200=0.60

Set B( filter):
25 trades
5 trades are closed at target
20 trades are stopped out.
Pclose target = 5/25= 0.20
Pstopped out/close trade=20/25=0.80

For set B having the same statistics as set A, distribution B would be:
Number of trades closed at target = 0.40*25=10
Number of trades stopped out/close trade = 0.60*25=15

Then
(5-10)^2/5 = 5.0
(20-15)^2/20 =1.25
Sum= 6.25

Degrees of freedom =(2-1)*(2-1)=1

Then I see in table http://sites.stat.psu.edu/~mga/401/tables/Chi-square-table.pdf
that 6.25 lies between 2.5 and 1%, so I can say with a chance of being wright of 97,5-99% that these distributions are significantly different?!

One of my worst enemies are my own false assumptions
Reply With Quote
The following user says Thank You to Zwaen for this post:
 
  #7 (permalink)
 Vendor: sentimentrader.com 
Tulsa, OK
 
Futures Experience: Advanced
Platform: Tradestation, TOS, Python
Broker/Data: IQFeed, Tradestation, TOS
Favorite Futures: ES, SPY, Options
 
ericbrown's Avatar
 
Posts: 201 since Jan 2011
Thanks: 339 given, 234 received


Zwaen View Post
Hi Eric,

Thanks! I did the following calculation, and wondered if my asumptions and calculations correct, or do I make some mistakes?

I will use simple numbers. Reality is offcourse not so simple, but to illustrate the idea, I either win or lose, given a fixed amount ( target is always t, loss is always l, but are not relevant for calculations)

Suppose you have a set of 200 trades ( set A) which have positive EV. You want to evaluate if filter B is relevant. Filter B contains 25 trades, and has negative EV. We want to know if the 25 trades are significant, so we compare the 2 distributions.

Set A:
200 trades
80 trades are closed at target.
120 trades are stopped out.
Pclose target = 80/200= 0.40
Pstopped out/close trade=120/200=0.60

Set B( filter):
25 trades
5 trades are closed at target
20 trades are stopped out.
Pclose target = 5/25= 0.20
Pstopped out/close trade=20/25=0.80

For set B having the same statistics as set A, distribution B would be:
Number of trades closed at target = 0.40*25=10
Number of trades stopped out/close trade = 0.60*25=15

Then
(5-10)^2/5 = 5.0
(20-15)^2/20 =1.25
Sum= 6.25

Degrees of freedom =(2-1)*(2-1)=1

Then I see in table http://sites.stat.psu.edu/~mga/401/tables/Chi-square-table.pdf
that 6.25 lies between 2.5 and 1%, so I can say with a chance of being wright of 97,5-99% that these distributions are significantly different?!

Not having done the calculations myself with your data, I can't say for certainty that this is correct...but a quick glance I can't see anything wrong.

Regarding interpretation, you are testing that the distributions are different. Your null hypothesis is that they are the same or similar.

With your data, the Chi-Square of 6.25 is greater than the p=.025 for df=1, therefore you can reject the null hypothesis (with a 2.5% probability of error) that the distributions are the same. You can't really say they are significantly different...you can only say that the null hypothesis is rejected, which in your case is what you want to see.

Reply With Quote
The following 2 users say Thank You to ericbrown for this post:

Reply



futures io > > > Statistical significance

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Statistical Arbitrage prisonbreaker82 Elite Trading Journals 143 March 12th, 2013 02:30 PM
Frequency distribution and statistical significance in strategy analyzer markus1000 NinjaTrader Programming 2 June 30th, 2012 08:36 PM
Is this a valid approach from a statistical standpoint? TraderTed Psychology and Money Management 12 May 22nd, 2012 10:00 AM
Software for basic statistical analysis? Xyzzy Platforms and Indicators 15 October 29th, 2011 05:22 AM
Statistical Definitions for Newbies NMTrader Traders Hideout 1 July 28th, 2009 02:39 AM


All times are GMT -4. The time now is 03:03 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.11 seconds with 19 queries on phoenix via your IP 54.226.172.30