I read your original post again and remembered that I experienced the same thing at one time. Trades not being entered at the same time in live versus historic or trades entered in live that dont' exist in historic and trades in historic that don't exist in live. Reading your post reminded me of a commercial system I was looking at called Viper Trading Systems which is talked about in a thread on this forum. It was horrible about that kind of thing and they just explained it away. They used tick charts.
I was doing some development around that same time using tick and volume charts and was experiencing the same thing. It was so bad that I gave up on the effort. Eventually I forgot about that and picked up the strategy development again. However, this time I used modified renko bricks and I don't see that same behavior. My entries, aside from expected slippage, are very close to what I see in backtesting. There are times that slippage cause a backtested winner to be a loser just because the move wasn't enough to get my target. However, that is an exception.
As I write this, I'm recalling a possible reason for the problem is the way we get tick data. I think the reason I got was that the timestamp on live tick data isn't the same as historic data for tick based bars. I remember that my broker or maybe NT helped me get to the bottom of that but don't remember for sure. You might ask them about your problem and see if you get anything reasonable. I think I'll do the same and post here if I get anything.
The following user says Thank You to MWinfrey for this post:
I did bring this up on their forums. They can be extremely helpful over there. But, I get the feeling that they think it is my imagination or isn't a big deal. Just part of backtesting that should be accepted. That scares me a little. Mainly because I find backtesting a little better than useless if the backtest results are so far off from live/sim. I accept slippage and all that goes along with it. But what I have been experiencing is another story.
With that in mind. I let my strategies run all day long yesterday. Unlike all the previous days with this problem, I did not edit or compile any code during the trading day. I basically let it trade and didn't tinker with anything. And the backtest vs live/sim results were much closer than they had been on previous days. Far fewer wacked out discrepancies.
From my understanding after asking the question on their forum, editing and compiling code shouldn't affect running strategies. Which I accept, but it is an interesting coincidence. Granted, one day doesn't make a pattern. But, I'll try it again to see what happens.
And before anyone asks the question. My multicore computer is quite beefy. So, the simple task of editing code and compiling shouldn't really affect running strategy calculations that much unless perhaps NT pauses for the 10 or so seconds when it is compiling code. I'd be surprised if that were the case.
I haven't jumped ship from NT yet. But, I have to say, the backtesting capabilities of Multicharts is somewhat intriguing. And if I can't get stable backtest/live results from NT that are in the same ballpark, I won't stick around.
Just for curiosity sake, have you tried running your strategies on timebased or even betterrenko bricks? I'm sure you've written your strategies for tick but just as a test, putting them on betterrenko might be an interesting test as well to see if the same problems occur. I know I don't have those problems when using betterrenko or my own implementation of the renko brick.
Yes, I tried a 1min chart and the results seemed more stable. But, I haven't really given it a fair chance yet. I plan on doing more today.
As far as using BetterRenko's. I've never touched them. I'm a little afraid of anything that has the word Renko in it. Too many issues with those. What advantage does BetterRenko have over time based or tick based? Also, I'm assuming they don't repaint?
I may of stumbled across the problem. If at the very least, I stumbled across another problem that affects running strategies.
It appears that Ninja Trader FREEZES whenever you do certain backtesting operations. Even compiling a code will freeze quotes. And when I've been heavily backtesting and writing/compiling code over the last weeks, it seems to have been severely affecting my backtest vs live/sim results.
I don't know that this is the sole issue with the problem I've been having. But it certainly must be a serious issue.
It is a sad day if we cannot backtest/compile code/etc while running strategies during market hours.