Market Microstructures - The Red Pill - futures io
futures io



Market Microstructures - The Red Pill


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one iantg with 20 posts (130 thanks)
    2. looks_two quants00 with 6 posts (11 thanks)
    3. looks_3 options with 4 posts (6 thanks)
    4. looks_4 varaamo with 3 posts (8 thanks)
      Best Posters
    1. looks_one iantg with 6.5 thanks per post
    2. looks_two varaamo with 2.7 thanks per post
    3. looks_3 quants00 with 1.8 thanks per post
    4. looks_4 options with 1.5 thanks per post
    1. trending_up 22,005 views
    2. thumb_up 182 thanks given
    3. group 79 followers
    1. forum 55 posts
    2. attach_file 8 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 150,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Market Microstructures - The Red Pill

(login for full post details)
  #51 (permalink)
 chipwitch 
Nashville, TN
 
Experience: Beginner
Platform: NinjaTrader
Broker: NinjaTrader, Continuum Data
Trading: MES for now... baby steps
 
chipwitch's Avatar
 
Posts: 320 since Feb 2022
Thanks: 229 given, 595 received

... also, "market maker id" is also coming up "". I would appreciate any insight.

Reply With Quote

Can you help answer these questions
from other members on futures io?
NT8 - Footprint Chart adding on code
NinjaTrader
use of Ultimate Oscillator in NT8
NinjaTrader
NT deleting my Strategies on charts
NinjaTrader
TOS Fold Loop to NT8 For Loop
NinjaTrader
Moving Average Price Scan
ThinkOrSwim
 
 
(login for full post details)
  #52 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 405 since Jan 2015
Thanks: 84 given, 1,121 received

There are 3 data feeds in NinjaTrader, and they will go out of sync often. Depending on what you are trying to do you may see some weird stuff.
For modeling purposes, stay away from the OnBarUpdate event. It front runs the other two (OnMarketData, OnMarketDepth) and will be off by a magnitude too far to be able to reconcile. OnMarketData and OnMarketDepth run pretty close together, at least if you are reading a playback file. Trying to process a live data feed 100% in sync would be challenging if not impossible. The way these cycles run, you will see throttling due to internet latency, data feed latency, or .net CG, then you will see a burst of data, then back to throttling, etc.



chipwitch View Post
@iantg and @birdyalex

Did you need to sort the streamed data by time? I just started messing with the NT OnMarketDepth() event. When I streamed a data sample direct to a text file I noticed that the orders are somewhat out of chronological order? Sometimes by as much as 30 seconds?


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Started this thread Reply With Quote
The following 2 users say Thank You to iantg for this post:
 
(login for full post details)
  #53 (permalink)
 chipwitch 
Nashville, TN
 
Experience: Beginner
Platform: NinjaTrader
Broker: NinjaTrader, Continuum Data
Trading: MES for now... baby steps
 
chipwitch's Avatar
 
Posts: 320 since Feb 2022
Thanks: 229 given, 595 received



iantg View Post
There are 3 data feeds in NinjaTrader, and they will go out of sync often. Depending on what you are trying to do you may see some weird stuff.
For modeling purposes, stay away from the OnBarUpdate event. It front runs the other two (OnMarketData, OnMarketDepth) and will be off by a magnitude too far to be able to reconcile. OnMarketData and OnMarketDepth run pretty close together, at least if you are reading a playback file. Trying to process a live data feed 100% in sync would be challenging if not impossible. The way these cycles run, you will see throttling due to internet latency, data feed latency, or .net CG, then you will see a burst of data, then back to throttling, etc.

When you were writing about the feeds losing sync, I didn't expect it within the same data feed. I thought you were speaking about the level 2 and the t&s losing sync with each other! But you're talking about even within the same stream? The level 2 data can be out of sync with other level 2 data on the same feed?

I have no code in the OBU and I have Calculate.OnBarClose set on a one minute chart so OBU shouldn't really be causing a problem, I wouldn't think. Are you aware of any white papers or other tech links anywhere? This seems pretty bizarre.

Reply With Quote
 
(login for full post details)
  #54 (permalink)
 terminator8 
Melbourne, Australia
 
Experience: Beginner
Platform: Sierra Charts, IRESS
Broker: FP, IB, ANZ
Trading: ASX Equities
 
terminator8's Avatar
 
Posts: 10 since Sep 2014
Thanks: 125 given, 5 received


iantg View Post

99% of this stuff won't help the average retail trader a bit, I might post a few things more applicable for the majority of users on this site at some point though.

Thank you for sharing your awesome insights. Are you able to please post a few tips and hints for how it can be applicable for us? Maybe a pseudo-trade with random price levels and order quantities.

Cheers!

Reply With Quote
 
(login for full post details)
  #55 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 405 since Jan 2015
Thanks: 84 given, 1,121 received

Sorry for the delay responding to this. I haven't been keeping up with my old email notifications from Futures.IO. Here is an old NT forum post I started like 7 years ago that gets into this in detail. This will be useful for you to understand how it all works under the hood. https://ninjatrader.com/support/forum/forum/ninjatrader-8/platform-technical-support-aa/107287-onmarketdata-vs-onbarupdate-price-level-question

The TLDR is that OnBarUpdate front runs the other two data feeds often by several seconds. Calling GetCurrentBid() or GetCurrentAsk() from OnBarUpdate is the fastest way to see the current price. Their product manager confirmed this on the thread. I tried to build some algo's with NinjaTrader back in the day. The fastest I ever got it was running Unmanaged code using mostly OnBarUpdate / GetCurrentBid(), GetCurrentAsk() calls from a VPS in Chicago, but it was several orders of magnitude too slow during the US cash sessions to even read the current price level. All of the level2 modeling was so far behind that nothing from the top of the book was even slightly relevant. You may have a shot at signals from the top of the book at night, or if the market is extremely slow in the afternoon, but there is virtually no change you will ever be able to do the type of modeling I described in this blog with NinjaTrader in real time.



chipwitch View Post
When you were writing about the feeds losing sync, I didn't expect it within the same data feed. I thought you were speaking about the level 2 and the t&s losing sync with each other! But you're talking about even within the same stream? The level 2 data can be out of sync with other level 2 data on the same feed?

I have no code in the OBU and I have Calculate.OnBarClose set on a one minute chart so OBU shouldn't really be causing a problem, I wouldn't think. Are you aware of any white papers or other tech links anywhere? This seems pretty bizarre.


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Started this thread Reply With Quote
 
(login for full post details)
  #56 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
 
Posts: 405 since Jan 2015
Thanks: 84 given, 1,121 received


terminator8 View Post
Thank you for sharing your awesome insights. Are you able to please post a few tips and hints for how it can be applicable for us? Maybe a pseudo-trade with random price levels and order quantities.

Cheers!

Your best bet if you are a discretionary trader is this: Use one of the "estimated place in the queue" tools from your favorite software. NinjaTrader has one, I think Sierra and maybe a few others do as well. Get in line early and often, wait until you get close to the front of the line. If you don't make it to the top 10% or at least top 25%, cancel and repeat. Once you get near the top, your chances of landing a non toxic limit order fill increase 10 fold. This will impact your P&L in a very significant way. The take away and relevance from this blog and the analysis relating to this point is: The strong side always wins the initial tick vs. tick battle, but only fills a tiny fraction of the queue. So if you can get into that top 10% / 25% you have a chance at both a fill and a +1 tick starting position.

But aside from this, I don't think any of this type of modeling could be applicable to discretionary traders. It wouldn't be possible to process this type of data in real time with retail tools, let alone using heuristics you could memorize and click on. The best analogy for this would be attempting one of those impossible user designed Mario maker courses.

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my futures io Trade Journal Started this thread Reply With Quote


futures io Trading Community Traders Hideout Emini and Emicro Index > Market Microstructures - The Red Pill


Last Updated on June 20, 2022


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
     



Copyright © 2022 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts