NexusFi: Find Your Edge


Home Menu

 





Market Microstructures - The Red Pill


Discussion in Emini and Emicro Index

Updated
      Top Posters
    1. looks_one iantg with 20 posts (140 thanks)
    2. looks_two quants00 with 6 posts (11 thanks)
    3. looks_3 options with 4 posts (6 thanks)
    4. looks_4 varaamo with 3 posts (8 thanks)
      Best Posters
    1. looks_one iantg with 7 thanks per post
    2. looks_two varaamo with 2.7 thanks per post
    3. looks_3 quants00 with 1.8 thanks per post
    4. looks_4 options with 1.5 thanks per post
    1. trending_up 31,072 views
    2. thumb_up 192 thanks given
    3. group 86 followers
    1. forum 56 posts
    2. attach_file 8 attachments




 
Search this Thread

Market Microstructures - The Red Pill

  #51 (permalink)
 
chipwitch's Avatar
 chipwitch 
Nashville, TN
 
Experience: Beginner
Platform: NinjaTrader
Broker: NinjaTrader, Continuum Data
Trading: MES for now... baby steps
Posts: 322 since Feb 2022
Thanks Given: 230
Thanks Received: 631

... also, "market maker id" is also coming up "". I would appreciate any insight.

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Pivot Indicator like the old SwingTemp by Big Mike
NinjaTrader
MC PL editor upgrade
MultiCharts
ZombieSqueeze
Platforms and Indicators
How to apply profiles
Traders Hideout
Quant vue
Trading Reviews and Vendors
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
What is Markets Chat (markets.chat) real-time trading ro …
70 thanks
Spoo-nalysis ES e-mini futures S&P 500
55 thanks
Just another trading journal: PA, Wyckoff & Trends
31 thanks
Bigger Wins or Fewer Losses?
24 thanks
The Program
20 thanks
  #52 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,148

There are 3 data feeds in NinjaTrader, and they will go out of sync often. Depending on what you are trying to do you may see some weird stuff.
For modeling purposes, stay away from the OnBarUpdate event. It front runs the other two (OnMarketData, OnMarketDepth) and will be off by a magnitude too far to be able to reconcile. OnMarketData and OnMarketDepth run pretty close together, at least if you are reading a playback file. Trying to process a live data feed 100% in sync would be challenging if not impossible. The way these cycles run, you will see throttling due to internet latency, data feed latency, or .net CG, then you will see a burst of data, then back to throttling, etc.



chipwitch View Post
@iantg and @birdyalex

Did you need to sort the streamed data by time? I just started messing with the NT OnMarketDepth() event. When I streamed a data sample direct to a text file I noticed that the orders are somewhat out of chronological order? Sometimes by as much as 30 seconds?


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #53 (permalink)
 
chipwitch's Avatar
 chipwitch 
Nashville, TN
 
Experience: Beginner
Platform: NinjaTrader
Broker: NinjaTrader, Continuum Data
Trading: MES for now... baby steps
Posts: 322 since Feb 2022
Thanks Given: 230
Thanks Received: 631



iantg View Post
There are 3 data feeds in NinjaTrader, and they will go out of sync often. Depending on what you are trying to do you may see some weird stuff.
For modeling purposes, stay away from the OnBarUpdate event. It front runs the other two (OnMarketData, OnMarketDepth) and will be off by a magnitude too far to be able to reconcile. OnMarketData and OnMarketDepth run pretty close together, at least if you are reading a playback file. Trying to process a live data feed 100% in sync would be challenging if not impossible. The way these cycles run, you will see throttling due to internet latency, data feed latency, or .net CG, then you will see a burst of data, then back to throttling, etc.

When you were writing about the feeds losing sync, I didn't expect it within the same data feed. I thought you were speaking about the level 2 and the t&s losing sync with each other! But you're talking about even within the same stream? The level 2 data can be out of sync with other level 2 data on the same feed?

I have no code in the OBU and I have Calculate.OnBarClose set on a one minute chart so OBU shouldn't really be causing a problem, I wouldn't think. Are you aware of any white papers or other tech links anywhere? This seems pretty bizarre.

Reply With Quote
  #54 (permalink)
 
terminator8's Avatar
 terminator8 
Melbourne, Australia
 
Experience: Beginner
Platform: Sierra Charts, IRESS
Broker: FP, IB, ANZ
Trading: ASX Equities
Posts: 11 since Sep 2014
Thanks Given: 145
Thanks Received: 7


iantg View Post

99% of this stuff won't help the average retail trader a bit, I might post a few things more applicable for the majority of users on this site at some point though.

Thank you for sharing your awesome insights. Are you able to please post a few tips and hints for how it can be applicable for us? Maybe a pseudo-trade with random price levels and order quantities.

Cheers!

Reply With Quote
  #55 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,148

Sorry for the delay responding to this. I haven't been keeping up with my old email notifications from Futures.IO. Here is an old NT forum post I started like 7 years ago that gets into this in detail. This will be useful for you to understand how it all works under the hood. https://forum.ninjatrader.com/forum/ninjatrader-8/platform-technical-support-aa/107287-onmarketdata-vs-onbarupdate-price-level-question

The TLDR is that OnBarUpdate front runs the other two data feeds often by several seconds. Calling GetCurrentBid() or GetCurrentAsk() from OnBarUpdate is the fastest way to see the current price. Their product manager confirmed this on the thread. I tried to build some algo's with NinjaTrader back in the day. The fastest I ever got it was running Unmanaged code using mostly OnBarUpdate / GetCurrentBid(), GetCurrentAsk() calls from a VPS in Chicago, but it was several orders of magnitude too slow during the US cash sessions to even read the current price level. All of the level2 modeling was so far behind that nothing from the top of the book was even slightly relevant. You may have a shot at signals from the top of the book at night, or if the market is extremely slow in the afternoon, but there is virtually no change you will ever be able to do the type of modeling I described in this blog with NinjaTrader in real time.



chipwitch View Post
When you were writing about the feeds losing sync, I didn't expect it within the same data feed. I thought you were speaking about the level 2 and the t&s losing sync with each other! But you're talking about even within the same stream? The level 2 data can be out of sync with other level 2 data on the same feed?

I have no code in the OBU and I have Calculate.OnBarClose set on a one minute chart so OBU shouldn't really be causing a problem, I wouldn't think. Are you aware of any white papers or other tech links anywhere? This seems pretty bizarre.


In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
  #56 (permalink)
 iantg 
charlotte nc
 
Experience: Advanced
Platform: My Own System
Broker: Optimus
Trading: Emini (ES, YM, NQ, ect.)
Posts: 408 since Jan 2015
Thanks Given: 90
Thanks Received: 1,148


terminator8 View Post
Thank you for sharing your awesome insights. Are you able to please post a few tips and hints for how it can be applicable for us? Maybe a pseudo-trade with random price levels and order quantities.

Cheers!

Your best bet if you are a discretionary trader is this: Use one of the "estimated place in the queue" tools from your favorite software. NinjaTrader has one, I think Sierra and maybe a few others do as well. Get in line early and often, wait until you get close to the front of the line. If you don't make it to the top 10% or at least top 25%, cancel and repeat. Once you get near the top, your chances of landing a non toxic limit order fill increase 10 fold. This will impact your P&L in a very significant way. The take away and relevance from this blog and the analysis relating to this point is: The strong side always wins the initial tick vs. tick battle, but only fills a tiny fraction of the queue. So if you can get into that top 10% / 25% you have a chance at both a fill and a +1 tick starting position.

But aside from this, I don't think any of this type of modeling could be applicable to discretionary traders. It wouldn't be possible to process this type of data in real time with retail tools, let alone using heuristics you could memorize and click on. The best analogy for this would be attempting one of those impossible user designed Mario maker courses.

In the analytical world there is no such thing as art, there is only the science you know and the science you don't know. Characterizing the science you don't know as "art" is a fools game.
Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #57 (permalink)
 
maxtrade's Avatar
 maxtrade 
Torino, ITALY
 
Experience: Advanced
Platform: Multicharts
Trading: ES
Posts: 44 since Dec 2015
Thanks Given: 18
Thanks Received: 12

Hi @iantg I'm interested about your study....I work with Multicharts, but make fatigue to understand and implemented your search.... can u help me please?

Thanks

Follow me on Twitter Reply With Quote




Last Updated on April 7, 2023


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts