I second that! I have traded for several years, but never spent to much time backtesting. According to the backtests, I am supposed to be well up in 9 figures now. If only someone could convey this to the market, I would be most appreciative...
And why someone would backtest without transaction costs escapes me...
I think we are in the same camp where the coin toss represents a system with no edge, commish means you are paying to toss the coin, so you are doomed to failure no matter what money management system you use if you play the game over time.
You need a system with positive expectancy after commissions and other expenses.
What I've been doing in R (and thanks for making me want to try it in Matlab) is to take the daily lognormal returns of my system under test and see how "randomizing" those results would alter my terminal PnL, my MDD, and the various ratios people look at. I then am trying to work out with what level of confidence will I be making a given level returns, or conversely at what level of confidence will I go bust.
Is that a Monte Carlo sim? I have no idea.
I do know I can't do this natively in Ninja as it is a multi instrument basket, so I don't really use the Monte Carlo sim capabilities of Ninja.
BTW most of the bookmakers (er... equity market liquidity providers) in my town are Dutch.