Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Anyone Build A True Backtest Engine for CalculateOnBarClose = false in NT7?
Just curious if anyone had developed a true backtest engine for NT7 for those of us who live in the CalculateOnBarClose = false world? As we know FirstTickOfBar doesn't solve the backtest problem. There's got to be one vendor who wanted to get rich off this idea...
If you are looking for accuracy with NT I can explain how a few things work under the hood.
First off, unless you are using market orders only, and only taking a few trades a day, throw away the strategy analyzer. It will overstate on every single scalping / limit order based strategy by a 100,000x magnitude.
Market replay uses actual tick data to simulate your fills, so you would want to run your tests off of this, however there are differences between 7 and 8. 7 is more optimistic, and 8 is more pessimistic with how the queue placement works. In case you want a look under the hood you can actually see the estimated place in the queue code with NT 8. They only fill from the back of the queue in all cases. This is fairly close to reality for most any retail trader.
But there are two huge issues with market replay you will need to account for and overcome if you want a halfway accurate test for scalping / limit order based strategies.
1. The execution latency is 0! So this assumes you have an FPGA and are sitting at the exchange. So you will have to bake in some latency into your code yourself to get it realistic for your use case. This can be done but it takes extra work.
2. The bigger issue is that if you run your code off of the OnBarUpdate method, this method front runs the OnMarketData event which is what is used to simulate your fills. So you can actually front run your own simulation by accident by using OnBarUpdate. You can also land cancels that occur after price level changes, and get nearly perfect queue placement as a function of this front running. The solution: Only run your code from the OnMarketData feed. Do not ever touch the OnBarUpdate feed for testing.
If you do live SIM, they add a built in latency of 150 MS on top of what you have from your actual latency. This might be overly pessimistic for some. But generlaly speaking the same rules apply, don't ever trade with OnBarUpdate if you want a valid test.
By Contrast if you ever trade live and want fast, the fastest event handler they have is the OnBarUpdate event. Which sucks because it only runs off of transactions and misses cancels and adds. Overall NT is only appropriate for longer term strategies. If you try scalping live with it, you will be too slow.
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Not much to add since @iantg is a lot more familiar with the specifics of NT7 and wrote a good post above. But that's effectively what Deltix, QuantConnect, Quantopian and SmartQuant are after on the lower end.
On the more institutional end, Flextrade entered this space fairly recently and they're decent in the connectivity world. RTS used to have one then they got acquired by Bloomberg and it's probably in some Bloomberg offering now. Itiviti also has one.
Ian, Market replay even at 500X is way too slow to accomplish back-testing for any legit timeframe (like at least a couple of years). I agree with your assessment though, it appears to be the only option. There are literally tens of thousands of NT users that are scalping, so I am surprised NT hasn't developed something themselves. Seems like a huge gap in a need for which many people I am sure would happily pay.
Yeah, retail platforms are awfully slow. Unfortunately this is one of those areas where if you've developed something good, you're more incentivized to keep it for yourself. I have a friend who runs one of the 3 largest trading firms in U.S. equities by volume, and he once mentioned a huge part of their "secret sauce" is the 50 man years of development time across their researchers (who all have PhDs) that were sunk into their simulation engine.
It's not difficult to write something faster and more accurate than NT market replay though if backtesting is all you're gunning for. I think it's a good learning experience that everyone who uses a backtester should go through.
Your absolutely right about NT 7 being crazy slow. I used to setup 4 or 5 laptops and run them 24 hours a day to get a years worth of data over a weekend tested. But then NT 8 came along..... It may be painful to migrate your code, but at max speed on a solid laptop NT 8 can do the job of 10 laptops running market replay on NT 7.
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
My work with NT 7 & 8 totally agrees and confirms what @iantg has stated above.
NT is not that suitable for scalping. NT 8 is getting better, but I still would stay away from using scalping systems on it.
Sorry to say, that Ninja's customer base is to small to support a niche product like that at $99/mo. It would need to start at $250/mo for a company to stay in business. That type of product would need very well educated support team to properly help customers of that nature. And, those type of employees are not cheap.
Like @iantg said NT 8 Replay Connection is much faster. And, I've heard, Xeon processors give an extra boost in speed. I don't plan on spending the extra $ to find out though.
I have quite a few data points, usually the retail processor is faster for a single host machine because you can overclock it easily. What the Xeon processors are usually better for are ECC support, dual/quad processor support, more memory support, larger cache size, and better binning for base clock, most of which become more meaningful if you have a cluster.
