Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Is anyone else besides me occasionally getting different backtesting performance results when using different platforms with the same exact system and same historical data?
Can you help answer these questions from other members on NexusFi?
Of course, each platforms has its own backtest rules and its own capabilities. Some can test intrabar, some cannot. Some assume winners where they should not, some assume losers. Some are more sophisticated order management, some are not.
I have the same problem. I am testing a strategy on two different plattforms (NinjaTrader7 and TradeNavigator).
With two different data providers. But how do I know which results are correct?
I think TradeNavigator is the better plattform, because it works with TickData and it calculates with tickdata for backtesting.
While I am testing the strategy with NT7 I use the data from InteractiveBrokers.
But the strategy is an automated strategy that will run with NT7 and IB.
So which of the backtesting results can I trust? Or what is the best way to backtest a strategy?
For some time I wanted to to conduct a little experiment: to run one and the same trading system in two different simulation environments.
I'm using Sierrachart and DTN IQFeed as data feed. Sierrachart offers a simulation environment for automatic …
.
I don't know how to decide which platform you should trust. The conservative approach would be to select the one with the worse result. Though now you have to define "worse": less profit, smaller profit factor, larger draw down? Whatever.
I guess live results will be different from any platform anyway.
Your only option is to perform a Market Replay over a shorter (and probably more recent) period of time. Such as the last 2-4 months depending on your processors speed. NinjaTrader provides data for free and you'll probably want to use the AutoIt download script for downloading multiple days.
Even market replay won't totally solve the issue if the data are different. All data is not the same, it's filtered and stamped differently.
This issue is more prevalent/impactful on lower timeframe/increment strategies.....
If you're employing a long term, high net trade strategy, the results will approach identical, as the disparities in data and data procesing will be marginal to non-existant.
This is one of the very reasons that I've steered more away from small trades and scalping.
When your system features a net trade of $30 on an instrument like CL, even a small discrepancy/variation in natural slippage or system slippage or data variation can have drastic and usually catastrophic effects.
It's bad enough that you have to overcome fees and slippage....when you throw in execution variations as well, you'd better make absolutely certain you've anticipated/accounted for any execution variation.
A single tick on each side of a $30 net/trade system can totally destroy profitability and at the very least, make drawdowns simply untenable.
Additionally, beyond simple execution challenges, there are processing variations, where slight variances in data can result indicator variations. When you employ a system that has virtually NO room for error, even the signal generation for entries and exits can have a drastic effect on how close your system will execute forward live, vs. backtest.
This is why strategy development and process is so important. Even the most solid backtests need to be thoroughly tested forward, in real time. I've seen indicators that were slightly different on live data than the engine's replication in backtesting.
"A dumb man never learns. A smart man learns from his own failure and success. But a wise man learns from the failure and success of others."
