Your report doesn't appear to include slippage and commissions. I have no idea what (or how much of it) you are trading, but that can quickly turn a strategy negative.
The only way to get and idea is to do in sample and out of sample testing (forward testing) as was talked about above. For example, take 6 months from 2010 to optimize/test you strategy, then see if it worked up until now. That's replicating what you are trying to do. Write something in the past and have it work in the future.
I would then see if it worked in the data you have prior to that period. If not, then you probably need to check your code to make sure there aren't any constants in it, or at least determine if mitigating factor were an issue (the crash, etc).
I've read where people sim test for a year, but I only do so for a week measuring the expected buy and sell points. That's long enough for me to get confortable that the code works exactly like I want. In my opinion, It'll either work or it won't work. If I sim test for a month and its negative, does that invalidate my testing or was it a bad month? I just turn it on, monitor it constantly, and make sure that the percentages and averages I see in the backtest closely mirror the real time testing.
Also, I figure this is your first strategy so here are some things to look out for. I trade in TradeStation, but I know there are issues with limit orders (you may not get filled) and advanced bar types (buy on close instead of next bar). I'm not sure if they exist in Ninja, but before going live, make sure you research and understand any problems that may cause.
The following user says Thank You to shadrock55 for this post:
A) There's nothing that can "tell" if your system will work in the future....it's only relative and a sliding scale, some things give more confidence than others.
B) Forward testing will not indicate if your system will fair well in the future either as there logic challenges with simulation as well (fill orders, slippage, adverse market effects, etc).
C) Obviously simulation will help to iron out bugs with algo guys....but another good reason to sim or forward test is so that you can do a comparison/normalization to your backtest.
I do a forward sim test and then compare that result with the same time period doing a backtest. This will give me an indication of how the back and forward tests compare.
If the backtest is promising and the forward test is promising and the comparison between the 2 is very close, then that's as good as you can do going forward before taking it live. Market conditions can and do change.
Some strategies fair better in trending markets, some in sideways markets, so extended periods of either can make your strategy results seem better or worse.
D) Using an appropriate amount of data to backtest is just as important as the backtest itself. You can actually use too much data/time rearward. You have to carefully breakdown the results to ensure that if the market changed last year (against you) but the backtest was conducted over a 4 year period and seems profitable, then you might be given false confidence taking it live.
E) There are simply some things that become so complicated, you have to employ a "safety factor." I do this for just about everything. Before I put something live in play, I multiply the worst drawdown by 2. I cut the return by 4. I then look at the sheer number of trades and if it's a nibbler (lots of small trades) I try to envison what would happen if it traded half as often (dry spells for entries)....I then apply a very CONSERVATIVE estimate for slippage and I apply the maximum for commission (even though I know my commissions shrink with increased trade volume). IF, and only if it still makes money forward and backward, do I feel confident enough to play it live.
For example....when you backtest CL, you have to use assumption based data like @CL, and anyone trading futures will tell you, there's always a transition volume at the turn of the contract, it's very difficult to replicate how you will approach that....will you split your trades (between May and June), will you transition at the same time each month? Will you employ a rule to switch once the newer contract has more volume than the current? Those are the sort of things that are just very difficult to wrap your arms around and continuous backtest data can't account for the effect without some very sophisticated coding. THAT is why I overengineer my strategies, because I'd rather shoot for the stars, that way, if I fall short, at least I land on the moon.
I used to go thru almost every forum to find strategy and test them. I found that same conclusion that what was claimed to be great, fall short in live market. But after years of looking at many stategy I learned something from years of system evaluation. At a glance I can tell that such system work on certain market condition (trending, ranging, congestion) but not all at the same time.
Its like a racing car that have to be adjusted to the race track ( not the Nascar oval). The problem with trading system, the market is the unknown factor so adjusting to is difficult. Some tried and call them trading system but they are more prediction or what if trading system.
The following user says Thank You to torroray for this post:
Did you disclose the bar type and period? Exotic bars are not reliable for backtesting, and large bars (ie: 60 min) will produce inaccurate results due to NT backtest engine OHLC ordering.
Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.
Need help? 1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first. 2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses. 3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make. 4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance. 5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers. 6) Help using the forum? Watch this video to learn general tips on using the site.
If you want to support our community, become an Elite Member.
Always test your strategy with real money before you can say it works or not!
I use a autoprogram on Silver that gives me a minimum profit of 10k a month,
but it has cost 25k for the developper before it was really working as it should! backtesting or testing in simulated trading did'nt give the real results!
Favorite Futures: YM & Equities & Options & Mutual Funds
Posts: 174 since Jun 2010
Thanks: 56 given,
I think exotic instruments are the problem....
If the bar isn't full of ticks then you're going to have room for error. An instrument that is very liquid and doesn't jump 5 or six ticks in a second, because there is a deep depth of market is going to be more accurate.