Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
How can I tell I my system will work in the Future
Your report doesn't appear to include slippage and commissions. I have no idea what (or how much of it) you are trading, but that can quickly turn a strategy negative.
The only way to get and idea is to do in sample and out of sample testing (forward testing) as was talked about above. For example, take 6 months from 2010 to optimize/test you strategy, then see if it worked up until now. That's replicating what you are trying to do. Write something in the past and have it work in the future.
I would then see if it worked in the data you have prior to that period. If not, then you probably need to check your code to make sure there aren't any constants in it, or at least determine if mitigating factor were an issue (the crash, etc).
I've read where people sim test for a year, but I only do so for a week measuring the expected buy and sell points. That's long enough for me to get confortable that the code works exactly like I want. In my opinion, It'll either work or it won't work. If I sim test for a month and its negative, does that invalidate my testing or was it a bad month? I just turn it on, monitor it constantly, and make sure that the percentages and averages I see in the backtest closely mirror the real time testing.
Also, I figure this is your first strategy so here are some things to look out for. I trade in TradeStation, but I know there are issues with limit orders (you may not get filled) and advanced bar types (buy on close instead of next bar). I'm not sure if they exist in Ninja, but before going live, make sure you research and understand any problems that may cause.
A) There's nothing that can "tell" if your system will work in the future....it's only relative and a sliding scale, some things give more confidence than others.
B) Forward testing will not indicate if your system will fair well in the future either as there logic challenges with simulation as well (fill orders, slippage, adverse market effects, etc).
C) Obviously simulation will help to iron out bugs with algo guys....but another good reason to sim or forward test is so that you can do a comparison/normalization to your backtest.
I do a forward sim test and then compare that result with the same time period doing a backtest. This will give me an indication of how the back and forward tests compare.
If the backtest is promising and the forward test is promising and the comparison between the 2 is very close, then that's as good as you can do going forward before taking it live. Market conditions can and do change.
Some strategies fair better in trending markets, some in sideways markets, so extended periods of either can make your strategy results seem better or worse.
D) Using an appropriate amount of data to backtest is just as important as the backtest itself. You can actually use too much data/time rearward. You have to carefully breakdown the results to ensure that if the market changed last year (against you) but the backtest was conducted over a 4 year period and seems profitable, then you might be given false confidence taking it live.
E) There are simply some things that become so complicated, you have to employ a "safety factor." I do this for just about everything. Before I put something live in play, I multiply the worst drawdown by 2. I cut the return by 4. I then look at the sheer number of trades and if it's a nibbler (lots of small trades) I try to envison what would happen if it traded half as often (dry spells for entries)....I then apply a very CONSERVATIVE estimate for slippage and I apply the maximum for commission (even though I know my commissions shrink with increased trade volume). IF, and only if it still makes money forward and backward, do I feel confident enough to play it live.
For example....when you backtest CL, you have to use assumption based data like @CL, and anyone trading futures will tell you, there's always a transition volume at the turn of the contract, it's very difficult to replicate how you will approach that....will you split your trades (between May and June), will you transition at the same time each month? Will you employ a rule to switch once the newer contract has more volume than the current? Those are the sort of things that are just very difficult to wrap your arms around and continuous backtest data can't account for the effect without some very sophisticated coding. THAT is why I overengineer my strategies, because I'd rather shoot for the stars, that way, if I fall short, at least I land on the moon.
700 trades spread over 4 years? Get a statistician to compare your results with 700 random trades and check the difference. If your system is good there will be a difference.
I used to go thru almost every forum to find strategy and test them. I found that same conclusion that what was claimed to be great, fall short in live market. But after years of looking at many stategy I learned something from years of system evaluation. At a glance I can tell that such system work on certain market condition (trending, ranging, congestion) but not all at the same time.
Its like a racing car that have to be adjusted to the race track ( not the Nascar oval). The problem with trading system, the market is the unknown factor so adjusting to is difficult. Some tried and call them trading system but they are more prediction or what if trading system.
Did you disclose the bar type and period? Exotic bars are not reliable for backtesting, and large bars (ie: 60 min) will produce inaccurate results due to NT backtest engine OHLC ordering.
Always test your strategy with real money before you can say it works or not!
I use a autoprogram on Silver that gives me a minimum profit of 10k a month,
but it has cost 25k for the developper before it was really working as it should! backtesting or testing in simulated trading did'nt give the real results!
If the bar isn't full of ticks then you're going to have room for error. An instrument that is very liquid and doesn't jump 5 or six ticks in a second, because there is a deep depth of market is going to be more accurate.
