Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Sorry if this is redundant but I can't find an answered thread on this in several years so I figured I'd make a new one. Basically I am trying to research and test my way to finding the most accurate backtester/optimizing platform and there are too many mixed answers and I need some real non biased advice.
I trade futures and have been for a while. I used Ninjatrader and the coding is great but the backtesting results were horribly inaccurate and not useful. I've now downloaded and started messing with TS and MC to see which one is better. I can code in C# but if something is simple that is a plus. Basically can anyone out there give me a straight answer to which platform can give me the most historically accurate backtest results? Thanks.
Can you help answer these questions from other members on NexusFi?
Trading: Primarily Energy but also a little Equities, Fixed Income, Metals and Crypto.
Frequency: Many times daily
Duration: Never
Posts: 5,049 since Dec 2013
Thanks Given: 4,388
Thanks Received: 10,207
I don't think there is a simple straight answer.
I've used Tradestation and find their 'easylanguage' very easy to use and their back testing to be accurate. When I do have problems with Tradestation is inevitably due to their data. If the data isn't perfect then no matter how good the back testing engine is the results will be bad. Not saying Tradestation's data is bad, in fact I think they have a reputation for some of the best data but...
I purchased a MultiCharts lifetime license back in 2011 and haven't regretted the decision having used it to test thousands of strategies over the years. I dabbled with pretty much every retail platform out there, but found it much easier (i.e. more time efficient) to sling strategy code in Easy/PowerLanguage than Java, C#, etc. I would probably prefer TradeStation over MC given it's OO capabilities, but I use Interactive Brokers so that's pretty much not an option for me. Both TS and MC are extensible via DLLs which I've also utilized to implement a Python-based (PyQt5) dashboard for analytics, synchronization and semi-automated trading functionality. Python is also providing me with access to a lot of machine-learning tools which I hope will help me reduce some of the risk I'm currently trying to mitigate...
MC has it's quirks (as does every other platform) but it's been fast & stable enough over the years to keep me using it.
Thanks SMCJB and Fu510n. I have both those platforms and will probably use a combo of both to try to get the best results. As far as data quality, I've looked in to buying data from an institutional quant platform, but a years data for one instrument is around $1300. However that would guarantee me the best data there is. I figure plugging that into one or both of those platforms would probably be the best option for a retail trader.
If you only/truly need a year's worth of tick-level data, there are a few places around (including here ) that might have what you need for backtesting. I initially used NinjaTrader in demo-mode for just that purpose, importing data into MC before I dove "all in" and committed to DTN/IQFeed. I generally always have charts open for ES, NQ, YM, RTY, CL, RB, NG, and 6E and FANG++ to make sure I have whatever data I may need lest I forget and miss something once past the 180 day IQFeed historical limit.
A DTN/IQ Feed, after looking at their site, there are a separate data provider? What purpose would that serve if, for instance I have tradestation that already has data? Their data is more accurate? Also when you say you leave the charts open, you mean you are recording the live market data and storing it yourself?
Also when you say "including here", do you mean from futures.io or from you?. Thanks for all your help and replies.
AFAIK, TradeStation is a "one stop shop"; i.e. they provide both data and brokerage functionality so DTN/IQFeed wouldn't be needed if you went the TradeStation route. I believe that Interactive Brokers data feeds may conflate data hence why I use IQFeed to be able to access/store "raw" tick data (@ $136/mo).
"More accurate"? Well, there's timeliness/lag and quality to consider - there are a few threads here on FIO that speak to that; Kevin Davey might be able to speak to TradeStation's quality but from what I've read over the years, IQFeed or Rithmic are probably the "best" for retail traders.
An example thread for data here on FIO is here. I could upload tick data in Python "pickle" format (.pkl) if anyone would be interested but that would only be helpful for those doing work in Python.
Hello. Can you provide more detailled information about your experience? What do you mean by "horribly inaccurate" and how exactly did you test? IIRC Ninjatrader is capable of executing market replay for backtesting, isn't it ? if so, this should be accurate depending on the data feed you used. Looking forward to more comments.
I take exception to statements like "horribly inaccurate". NinjaTrader 8 historical data comes with microsecond granularity and you can use tick replay to back test. This is a good thing vs. say NT7.
In my experience market replay data isn't really required for most backtests, but this of course depends on what you are trying to do. If you need to have the bid/ask accurately represented for the backtest then I do agree that market replay is more accurate than tick replay and you will see differences.
People start calling NT names when backtests do not go their way, but the trader backtesting is very likely within the ''problematic zone''. Meaning that they ran a backtest on an exotic bar type or data series that just cannot be backtested accurately. Or maybe they even tried to put in some fancy order strategy that NT just didn't represent well in the backtest.
The more you find out about the backtest the more you can establish the problem. They may have a data issue for example. By that I mean a poor data provider leading to garbage in = garbage out. Or they maybe running a back test on non back adjusted merged data, or they curve fit the entire set of data without parcelling the data in to "test'' packages. Or maybe a combination of the above!
Anyhow, with NT8 backtesting most of the "gotchas" can be side lined which wasn't necessarily the case with NT7. I have had lots of success with NT8 back testing once I realised what it could and couldn't do.
EDIT: I just realised this post sounded like I was directing it at you. I wasn't. I thought I replied to the post you replied to. My bad, but point made I guess.
--------------------------------------------------------
- Trade what you see. Invest in what you believe -
--------------------------------------------------------
Thanks for your thoughts and comment. I am also looking for a good backtesting platform and I'd like to dive right into the "right" one. I have red tons of posts here on this forum, on other forums and a lot of articles on the net. My personal feeling is:
SierraChart and MultiChart are the most named platforms with good reviews and pros when comparing to the others. I am not sure about eSignal or NinjaTraders' backtest capabilities but I really like to know what you Pro's can suggest and propose? I have no problem to use a separate platform for trading and one for backtesting. Currently I am trying to find out which backtesting and trading platform to stick with.
) that might have what you need for 