So, the first thing i notice is low sample size 47 trades. Yes statisticians say 30+ is "significant" for a sample however more is always better. A quick plug into the old formula you get 14.43% standard error with this sample set alone. So any monte carlo based on that is going to have a built in 14% error either way. Monte carlos are only as good as the data set they can take from.

A quick plug into the old formula you get 14.43% standard error with this sample set alone.

Looking at the trade stats you NET .5(pt) per trade and win 55% of the time. Is this with or without commissions? If not then they are going to take a sizable portion of your edge and make this system very difficult to trade. If it is with commissions then go on right ahead to incubation mode.

Another thing you did not mention is how many rules you have or filters. When you have a sample size this small it really can effect and skew both results and monte carlos. If you have 500 + trades and have 5 filters your ok, but if you 50 trades and 5 filters thats not ok. Statistically speaking, although i am a huge proponent of less filters, and less rules are more robust.

Also mention the date range that your using to test. Is this over 1 day, 1 month, 1 year? Both insample and out of sample to cover atleast 1 full market cycle.

First, I have to thank you for taking the time to help me out. I really appreciate it . (sitting here at home trying to figure this all out on my laptop).

Please realize that this is my first model ever I am attempting to implement using the @kevinkdog style. I have no experience whatsoever in backtesting (but did read his 'Taking it Live' thread).

My challenge is being able to backtest 200 samples without using any software.





Yes I realize N is small, this is due to the fact I am manually backtesting this (ie looking at the trade and manually recording the entry/exit, profit/loss details into excel.

Currently N= 50 Ave win = 1.38point Ave loss = 0.87

I currently havent figured out how to use backtesting software, (most likely will get Ninja Strategy Analyzer)

Yes actually having software, is really important as it speeds up your testing. Every software has its pros and cons its up to you to figure out what kind of quirks each system has. I know Ninja is well known to have errors or give erroneous answers in certain situations but have never used it myself. I personally have used MultiCharts and have reported every bug i detected, and i use R (quantstrat).

Sorry, which formula is this? ( I still have a lot to learn).
standard error of a mean is technically StdDev / sqrt(N). I shortcut it and make it 1/sqrt(Trades). I think i did Trades+1 in the previous example cause I was not sure how many degrees of freedom you have. But its a quick and dirty way of getting some error benchmark.


No I forgot to mention commissions, they are 4.7, but I did factor them into the monte carlo analysis.
Where they included in the original trade results as well? If they were included in the monte carlo thats fine also but its good to include them at every stage of the game.


I have 5 rules. So should I eliminate some of them and rebacktest?
Hard to say, as every system is different. Another general rule is managing degrees of freedom. Your degrees of freedom are your trades (N) - 1. So you have 50-1 = 49 degrees of freedom. You do not want more than 1% degrees of freedom so you want as a MAX .49 rules/filters/ anything using a >, < ,and, or, in code terms. So in your example your going to want to have only 1 rule. Again this is because of a small data set, it forces you to use less rules. Usually i stick around the .5% degrees of freedom. Any more then that and you are using rules to force the data to give you the results you want. Any statistician can beat the data enough to show you what you want.


The sample ranges from 14 April to 2 September.
Not nearly long enough, you need at least 2-3 years. Manually that can take you a very long time.
How do you determine a market cycle?
Well you can do a seasonal analysis and determine what its cycle is. I did some seasonal stuff in my development thread a long time ago. Or you can visually inspect to see if you can identify long term bull/bear markets. They key is you want to encapsulate every type of market, trending, ranging, bull bear into your training and testing sets to make it robust. if you only test on bull markets, then a long strategy is always going to work. and vice versa
Btw how much experience do you have backtesting? and what software you use to do it?
I have been backtesting for several years now. I have taken courses in Financial engineering as well, also done some analysis and backtesting for some small/mid sized trading firms. Software for the "Retailer" I believe multicharts is the best. I have tested and debugged it myself and gotten most of the bugs fixed by the MC team. I trust the results since my live vs backtesting has been within 1-2 ticks of entry and exit when coded properly. Also it supports easylanguage or C# depending on your level of coding ability.

Flip side for more detailed work and for something I have to submit to others. I almost always use R, in a combination of packages including, quantstrat, TTR, quantmod, and redis (for redistributable computing). Along with some custom coded functions and trading reports I developed myself. R simulates an Orderbook rather than trades, so you can see the entire market timeline and how orders became trades and then how trades were worked out. This is better for cross checking, as well as you can add specific delays to see how that affects you. As opposed to ninja trader or multicharts where you just put (next bar). I can set exact latency times for specific situations.
Thanks a lot Trey....I really need your help

Have a great day

Where do you get your data from? how much do you pay for it?



I'm playing around with the Strategy Analyzer in NT now since its free, but at a later stage in my career I will look at some of the quant software.

I am trying to figure out what is a good way to learn all this.

Thanks.