Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I am trying to download market depth events through Ninjatrader custom code that calls OnMarketDepth event handler. My intention is to carryout offline research to understand dynamics order book by analyzing Add and Cancel order events. I have come across various posts on this forum that employs OnMarketDepth event handler to get market depth data but I could not find any example where number of cancelled volume is calculated. I tried using Operation.Remove option to look for cancelled order but it looks like this option is meant for situation where entire order book level needs to be removed from market depth.
I would appreciate if someone can help me pointing in right direction.
Thank you,
Vaibhav
Can you help answer these questions from other members on NexusFi?
Broker: NT Brokerage, Kinetick, IQFeed, Interactive Brokers
Trading: ES
Posts: 159 since Dec 2014
Thanks Given: 40
Thanks Received: 166
Ok, I will preface this with, it's not easy. I've been working on this very thing for the better part of a year and here's what you have to do.
You have to track your own entire order book. Essentially, what you see with a super Dom, you have to create in code. If you want any sort of reasonable performance, you'll need to use structs instead of classes and be ready to do some unsafe pointer stuff.
You're on the right track with that override event handler and depending on whether you have an add, remove, or update, you'll have to update your order book builder accordingly.
I'd recommend starting with a sorted dictionary to track each feature you're interested in. I'd use the time converted to the tick (long data type) as the key then a list of structs as your value. Since the updates arrive in time order, the sorted dictionary will give you inherently/significantly better lookup performance. I believe it used binary search underneath in combination with calculating lookup hashes for you.
I'm not able to share the code I have unfortunately but hopefully that helps and gets you pointed in the right direction. You may want to look on GitHub for an order book builder to see how people are doing this stuff. That's what I would do if I were starting over from scratch.
Broker: NT Brokerage, Kinetick, IQFeed, Interactive Brokers
Trading: ES
Posts: 159 since Dec 2014
Thanks Given: 40
Thanks Received: 166
The other bit of information I would offer would be to look into the multivariate hawkes-gamma point process. The are some good papers on arxiv showing some very promising results. Just search for those terms and that will get you going.
When I mentioned features, I mean order cancellation, order adds, etc and store those in your fast collection. You'll have to decide what you consider an order book update. If you take any change in volume/price, be prepared to deal with 8GB+ per day session of the ES. Storing is not an issue but you'll need an efficient binary storage format for however you're doing your analytics.
There is a ton of misinformation out there about the usefulness and utility of a lot of trading concepts. A recent poll on here is showing that there is more interest on the topics of order flow / and scalping than all the other trading …
It's honestly not too hard to code this. Unless you can get MBO data, most volume data comes by aggregate so you have to un-bundle it yourself with the following logic.
1. Create a variable to hold the "Last update" to a given price level. X price Y volume
2. If and when the OnMarketDepth event handler gives a new update to this price level, you compare this to the variable and determine if this new event added volume or removed volume.
3. If the new event added volume, this is easy part, added volume is just added volume. Create a new variable to hold all your added volume and just keep a running total if you like.
4. For price / volume updates that are less than your variable in step 1, they will only be cancels until the price level gets to the top of the book, but once it hits the level 1 feed then they can be either cancels or transactions, so you have to reference either the OnBarUpdate, or OnMarketData to get the actual transactions. Anything that is subtracting that can't be attributed to the transactions will be your cancels.
Check out the model that I built in the thread I referenced. For every price, For both the bid and ask, I collected the following data.
I did my model by price level, so it's the most granular that you can get. I did it all with ninjatrader too, so it can be done, and it wasn't too hard. You can probably code it over a weekend.
Best of luck!
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Apologies guys for not getting back to you all earlier but I must thank wonderful folks from this equally wonderful forum.
Many thanks @bobwest for creating thread for my question, thanks @iantg and @Jasonnator for your inputs. I am big fan of @iantg's posts and especially his Red Pill post.
I am working for suggestion given by you and if possible I will share by feedback soon.
