Suppose you want to use a simple ‘filter ‘ to improve statistics. Lets say this is a simple filter which in your opinion is valid, eg don’t take trades after time y. Reason is that after time y price has not enough time/opportunity to travel to your desired target( you don’t go overnight, and there is no exceptional volatility). Another example would be when you want to analyze if your stoploss and profit target can be made dependent on current volatility. Offcourse, the reason could be anything, as long as the trader can suspect there lies some causality or logical thinking behind the rationale to use the filter.
We don’t want to use too many filters, since more filters means more chance of curve fitting.
What number of trades B in dataset A is sufficiënt to be statistically significant? And if this depends on the number in database A, how would you think/know this relation yields?
At this moment I use a rule of thumb that is around 30 ( law of diminishing returns from statistics) observations. So there need to be a minimum of around 30 (=B) observations to let the filter be of any significant value. I also look at scatterplots/histograms of the data and use common sense ( asuming I have some) to ascertain if the difference is statistically significant. But I wonder if others use a more methodical approach to this?
One of my worst enemies are my own false assumptions
Thanks! I did the following calculation, and wondered if my asumptions and calculations correct, or do I make some mistakes?
I will use simple numbers. Reality is offcourse not so simple, but to illustrate the idea, I either win or lose, given a fixed amount ( target is always t, loss is always l, but are not relevant for calculations)
Suppose you have a set of 200 trades ( set A) which have positive EV. You want to evaluate if filter B is relevant. Filter B contains 25 trades, and has negative EV. We want to know if the 25 trades are significant, so we compare the 2 distributions.
80 trades are closed at target.
120 trades are stopped out.
Pclose target = 80/200= 0.40
Pstopped out/close trade=120/200=0.60
Set B( filter):
5 trades are closed at target
20 trades are stopped out.
Pclose target = 5/25= 0.20
Pstopped out/close trade=20/25=0.80
For set B having the same statistics as set A, distribution B would be: Number of trades closed at target = 0.40*25=10
Number of trades stopped out/close trade = 0.60*25=15
(5-10)^2/5 = 5.0
Not having done the calculations myself with your data, I can't say for certainty that this is correct...but a quick glance I can't see anything wrong.
Regarding interpretation, you are testing that the distributions are different. Your null hypothesis is that they are the same or similar.
With your data, the Chi-Square of 6.25 is greater than the p=.025 for df=1, therefore you can reject the null hypothesis (with a 2.5% probability of error) that the distributions are the same. You can't really say they are significantly different...you can only say that the null hypothesis is rejected, which in your case is what you want to see.
The following 2 users say Thank You to ericbrown for this post: