Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
In terms of what you need data structure wise.... You need to start by defining your bet, and then work backwards to define the variables you need to solve for. Right or wrong, most people will gravitate to a type of bet that looks like this image. I'll explain....
For every price level there is a best bid and best ask, and at any given point in time these will both have their respective volumes. Now all things even (here we assume the flow of market orders coming in are relatively even on both sides) one would assume the side (best bid or best ask) that has the higher volume will survive the incoming market orders. If this volume ratio is something like 1 to 1 or .8 to 1.2 then meh... No big deal, but if this gets more lopsided then one could start to make a fairly accurate bet that the side with a higher volume will win. Before I go any further let me just state that this particular bet (best ask vs. best bid) will almost always play out the same way on every price level. But unfortunately virtually no one can take advantage of this. Here is the detailed explanation for this: Every price level starts with a weak side and a strong side. The strong side gets to drag in resting volume to start the level, and the weak side has to start the price level with 0 volume. So ultimately the strong side will prevail way more often than not. For example on the ES, the strong side win rate is close to 90%. So what this translates to is that 100% of the queue on the weak side will get filled but only a tiny amount of people on the winning strong side will get filled. After studying this for a considerable amount of time, I will save you all the headache and tell you, there is no bet here you can get in on. The winning strong side only typically fills 10% or less of it's queue, so this isn't a bet for retail traders to attempt.
But there are other bets that you can get in on. The fight is never between the best bid and best ask but between the two adjacent strong sides. In the image above this is noted by the two sections with 1 tick under them. The way the market moves is that if the best bid breaks then the market moves to the left to the 1 tick strong side resting volume as a starting point and the ask will have to spontaneously back fill it's starting volume, it will likely not be able to do this for very long.... So then the Ask side breaks and the market moves to the right and we repeat. This fight typically lasts on the ES for 10 to 15 price levels before one of the two adjacent strong sides actually breaks. So your bet is which strong side will break. On each price level this fight will be between the winning (Best Bid / Best Ask which is usually a strong side) and opposite side resting volume 1 tick out. I hope the crude illustration helps a little bit.
Tuning your bet: So assume that we are in a period where the market order distribution is relatively even... This part is key. If you can extract your strong side resting volumes 1 level out then you can evaluate if one side is definitively stacked more than the other. You may also run this bet in connection with the strong side volumes 2 levels out to 3 levels out. I don't personally thing there is too much value in looking more than a few levels out because in most cases the top 2-3 levels will determine what happens next. Resting volume 7-10 levels out won't influence the next few immediate price level changes at all. So you can typically stop after maybe 3 levels when building your bet criteria.
The image below summarizes the full level 1 micro-structure so you can see what you are solving for.
Here what you are really comparing are the two adjacent strong side volumes. You can quickly spot the pattern and tell that Bid typically starts as the weak side when the price moves up, and starts as the strong side when the price moves down, and vice versa for the ask. The highlighted in bold rows denote when a real strong side price breaks. The other price level changes are just flips between adjacent strong sides breaking the opposite weak side over and over. if you look to the last strong side price level you will see that the bid strong side is starting out with 30% to 40% more than than the ask strong side and ultimately wins the price level. I am not making the claim that this is always the case or even a solid bet, I am just explaining the data and how you could derive a bet.
So in order to get what I have and also have your level 2 resting volume sequenced properly you will need to pull in these additional variables to the ones I already have listed.
Bid Resting 1 level out
Ask Resting 1 level out
And you might want to also pull in the volumes at 2 ticks out and maybe 3 ticks out. After that, I don't think the data will be of much use.
So for every row, you will need to ignore the actual price, and just have a variable called Bid Resting 1 level out (as an example) and capture the level 2 data that is in position 1 on the ladder for example. Once you have this data structure built, then you can look ahead and test your bet like this:
If (Condition on current row) Volume of Ask vs. Volume of Bid 1 level out, or adjacent strong side volumes, etc has a ratio of X then bet Y.
To Test if you are correct in your bet, you just do a look forward formula in excel checking the next strong side price level to break a few rows down the spreadsheet... If you are right give yourself a +1, if you are wrong give yourself a -1.
That's it.
I am enclosing a sample level 2 book in NinjaTrader you can start playing with to get you familiar with the OnMarketDepth feed as well.
Hope this example helps get you started!
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Thanks a lot Ian. I will try to start testing different data structures according to several theories, doing it backwards, like you said.
Your post is really good, it makes you think, it opens your mind to new scenarios.
I think it's a lot of work... some statistics knowledge could help :P
I will see what i can do!
Very nice. It will make a great differences if you are getting this data in every 100msec as a retail trader or colocated.
Or it doesn’t matter?
Otherwise they will be 100msec ahead of you every time let alone the latency.
Regarding this type of data, and the speed of processing, it has several different purposes. On the one hand you are correct that there is a latency threshold.... Beyond a certain point certain bets will not work. But on the other hand, there are some bets that are not latency sensitive at all.
The purpose of this type of data structure modeling is two fold.
1. Source the raw data, synthesize it to build a complete level 3 - level 4 micro-structure fully analyzed. This part has no latency, since you are doing it after the fact for strictly analytical purposes. Here you are exploring the raw structure of the market and learning which edges truly exist by testing every type of bet possible. I urge everyone to do this step, no matter what their trading style is. The better the data set, the better and more accurate their bet - testing will be.
2. Once you have identified several bets with potential edges, you will need to analyze the time stamps of the price level changes and determine which bets you will ever have a chance of pulling off given your commission cost structure, co-location or lack-thereof, etc.
When you build your production code to run in real time, you likely will not be trying to build this type of information in real time because it will draw too much overhead and slow your execution code. So you will need to strip away the extraneous bells and whistles that are more for analytical purposes like tuning your bet, and focus on the alpha signals that you need to place your bet.
Hope this helps a little.
Ian
In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
