Sounds like you where running version 6.5 as it was a long time ago. In that case you were probably wrong not to blame NT as 6.5 was nortorisly bad at backtesting tick data, even if your PC had ample memory, I know as I had maybe half a dozen installs running on different hardware (including Xeon servers with 8Gig or more RAM) and every single one had major problems with backtesting 6.5, infact it was impossible to use it. Now I don't know about you, but it gets me a bit grumpy when I am paying (forget the money, it's the time thats the killer) for a product that claims to do what I want, but cannot. A tool that promises so much, but dosn't work properly.
Using the 1 tick range, how many months/years of tick data are you backtesting on now with 7 ?
Latest NT7 still crashes when backtesting just 1 year of tick data, this just happened on my second test for today see attached. People can post all they like about how much they love NT, but unfortunately that dosn't help solve these issues.
As a balance to this harsh treatment of NT.... what else is out there that does all this backtesting stuff that you want to do?
I backtested for about 3 years with Tradestation and finally gave up. The reason is very simple - you can never really trust the results.Why?
Because some of the trades are flawed logically - even if the bid got hit would you have got filled - there is no way any engine can replicate that ( unless it's random I suppose). The day I gave up and got on with trading it all became a lot less stressful than trying to find a holy grail system.
The only guys I know who run a completely automated system successfully - spent 2 years, a huge amount of money and employed an Astro Physicist with a pHd who ultimately said - we need a box in the exchange otherwise nothing will work - which they did.
Secondly - your data. I trade 3 min bars real time. I would say every day the chart changes from what actually appeared on the screen yesterday. Only subtly but higher highs don't appear, lower lows magically gone, gaps filled , etc. This is because there were data errors, exchange revisions due to fat fingers and other things like they really are out to get me.... my point is the data you test on is NOT what happend in real time. I produced an example chart somewhere else in this forum if you want to check .
I am not being negative nor trying to put you off trading .... just saying that all the work I did in backtesting produced some lovely coding and sometimes a set of great results that VERY RARELY reproduced real time. so Beware.
You are 100% correct, which is why you need to optimize on years of data to represent as many conditions as possible and then take those same settings from the optimization and verify the results on another untested period, which you will normally find will perform much worse than the optimized period. You need to see hundreds of trades in the results and the system needs to be as simple as possible and you need to take into account slippage and commissions. It is a very difficult task, but I enjoy it myself and you can learn alot from backtesting.
Anyway this thread isn't about the pros and cons of backtesting.
Last edited by webart; March 24th, 2011 at 05:40 AM.
NinjaTrader - this is the one area of frustration for me with NT7. I run lots of optimization, not to tweak and curve fit strategies, rather to produce historical probabilities I can use and plug into decision matrix's etc for execution. For this I run backtests/optimizations for 10 year datasets etc, and then process of opening these via a graph or even a trade list takes considerable time (as you also talk about above).
I understand from what you say above that when I open the detail of a given optimization result I am effectively re-running that individual backtest again. Is there any way to say "save all details" etc or any other way/plans to address that??
Yes but what do you consider extremely large ? Even with just 1 year of data there are problems. I think 3-5 years would be the minimun, but more would be better. I know it's a huge amount of data to crunch, but that's what the software has to be able to do it.in order to give people like me, the end user a decent chance. (of course the hardware has to be able to support it also).
My strats are all simple, normally 1 or 2 parrameters and then perhaps some different trading time periods.
I wouldn't mind waiting longer for the backtests to complete if all of the charts and graphs could be pre rendered, so that I can review them quickly when the time comes.