NexusFi: Find Your Edge


Home Menu

 





The Truth: NinjaTrader


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one Fat Tails with 120 posts (165 thanks)
    2. looks_two Big Mike with 54 posts (93 thanks)
    3. looks_3 MetalTrade with 43 posts (10 thanks)
    4. looks_4 ZTR with 41 posts (25 thanks)
      Best Posters
    1. looks_one AynRandFan with 34 thanks per post
    2. looks_two Big Mike with 1.7 thanks per post
    3. looks_3 Fat Tails with 1.4 thanks per post
    4. looks_4 NinjaTrader with 0.9 thanks per post
    1. trending_up 355,732 views
    2. thumb_up 802 thanks given
    3. group 145 followers
    1. forum 1,059 posts
    2. attach_file 65 attachments




 
Search this Thread

The Truth: NinjaTrader

  #231 (permalink)
sandie
California
 
Posts: 8 since Dec 2009
Thanks Given: 2
Thanks Received: 5

I have a question to ask someone familiar with the internals of NT7. I have read where they are storing bid, ask, and last tick in separate databases with a resolution of “One second.” If this is true it seems impossible to build historical data in the proper sequence. Seems like MC and NT can’t seem to get their act together regarding this vital piece of information.

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Deepmoney LLM
Elite Quantitative GenAI/LLM
Futures True Range Report
The Elite Circle
Exit Strategy
NinjaTrader
NexusFi Journal Challenge - April 2024
Feedback and Announcements
ZombieSqueeze
Platforms and Indicators
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
Battlestations: Show us your trading desks!
26 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
  #232 (permalink)
 NetTecture 
Szczecin
 
Experience: Intermediate
Platform: Ninja, writing own now
Posts: 211 since Mar 2010

You son't get an answer here. That said, remember that there are also market replay files that they already creaste from 6.5 upward. I can not imagine hey store things with a 1 second resolution there - this would mean in a 1:1 time replay you get "blocks" of updates every second, which TOTALLY alters the feeling of something like tape reading and would - in the sense of being an epic idiotical programming thing - make market replay totally unusuble as training tool for many things. Heck, you possibly could not even see anything on development of the order book. So, I think they actually store a higher resolution at least there, but do not expose it via API.

Storing historical data in separate files for ticks, bid / ask and with a 1 second resolution would mean any stream reconstruction is simply impossible.

Reply With Quote
  #233 (permalink)
ScoobyStoo
London
 
Posts: 16 since Mar 2010
Thanks Given: 0
Thanks Received: 13


See my previous post with regards to NT timestamp resolution.

I don't use the historical data so can't comment on that but I know from discussions with the NT support team that market replay data is timestamped with 1 sec granularity when it is recorded. This becomes a massive problem because the L1 and L2 data streams are recorded into separate files. In doing this the original sequencing of the L1 and L2 data messages is permanently lost. When the market replay engine comes to play the data back the only thing it has to work with when trying to intersplice the L1 and L2 data is the timestamps. But because these have only 1 sec resolution the engine replays ALL the L1 data for a particular 1 second period first and then replays ALL the L2 data for the particular 1 second period. The data is not correctly interspliced.

You probably won't notice the effect with your eye but try feeding market replay data into an algo strategy and during fast markets the result is a complete mess.

If you want to see the effect yourself just create an NT indicator using the attached code. All this code does is dump out each market data message to the Output window. Attach the indicator to a chart, replay some fast data (e.g. around a major news release) and watch the unsynchronised chaos unfold.

Why is this important? Well, if your algos are basing their decisions on a combination of both L1 and L2 data and those data streams are out of synch then those decisions are going to be heavily flawed.

I know that the NT development team monitor this thread because they have PMed me regarding my previous comments, so if anything I have said here is wrong then please step in and correct me...

Attached Files
Elite Membership required to download: MarketDataEventDumpIndicator.txt
Reply With Quote
  #234 (permalink)
 NetTecture 
Szczecin
 
Experience: Intermediate
Platform: Ninja, writing own now
Posts: 211 since Mar 2010

Ok, I am reinstating my statement Ninja is written by people with no idea waht they do. WHOW. Did they ever get a sanity check on that? You are right, it deal any recorded data totally useless. But I think you underestimate the gravity... it should be visible with the eye when you run a DOM in replay - especially during active times, you should see bid/ask move out of sync with actual trades, T&S should be a total fuckup as the bid/ask coloring will obviously be totally whacky during active trading, too.

The idea to have a 1 second timestamp was stupid to start with (there is nothing lost going lower). But then storing L2 separate from the tick data makes it something I really have to write up in a nice blog post I would fire anyone responsible for such a great architecutral decision.

Let's hope you are wrong

Reply With Quote
  #235 (permalink)
ScoobyStoo
London
 
Posts: 16 since Mar 2010
Thanks Given: 0
Thanks Received: 13


NetTecture View Post
Ok, I am reinstating my statement Ninja is written by people with no idea waht they do. WHOW. Did they ever get a sanity check on that? You are right, it deal any recorded data totally useless. But I think you underestimate the gravity... it should be visible with the eye when you run a DOM in replay - especially during active times, you should see bid/ask move out of sync with actual trades, T&S should be a total fuckup as the bid/ask coloring will obviously be totally whacky during active trading, too.

The idea to have a 1 second timestamp was stupid to start with (there is nothing lost going lower). But then storing L2 separate from the tick data makes it something I really have to write up in a nice blog post I would fire anyone responsible for such a great architecutral decision.

Let's hope you are wrong

I have never used any of NT's visual toolset so can't comment on output via the DOM or T&S windows. I use NT purely as a data replay engine, pumping data to my algos via their API. To be fair to NT I can see why they split out the data into 2 files. The L2 data files are large and a lot of their customers probably aren't interested in anything beyond the top of the book. However, the 1 second granularity thing is dangerous and I personally think that people should be made aware of the potential consequences. It can lead to backtest results during fast markets being wildly different to reality.

caveat emptor...

Reply With Quote
  #236 (permalink)
sandie
California
 
Posts: 8 since Dec 2009
Thanks Given: 2
Thanks Received: 5

NetTecture, ScoobyStoo,

Thank you for your replies. Pretty much confirms my information. Sad both NT and MC have these issues. At least MC may have a handle on theirs soon.

Reply With Quote
  #237 (permalink)
ScoobyStoo
London
 
Posts: 16 since Mar 2010
Thanks Given: 0
Thanks Received: 13


sandie View Post
NetTecture, ScoobyStoo,

Thank you for your replies. Pretty much confirms my information. Sad both NT and MC have these issues. At least MC may have a handle on theirs soon.

No problem.

NetTecture: Throw over a copy of your app if you want some constructive feedback. I've been following your exploits over on ET for a while. What's your blog?

Reply With Quote
  #238 (permalink)
 MXASJ 
Asia
 
Experience: Beginner
Platform: NinjaTrader, TOS
Posts: 796 since Jun 2009
Thanks Given: 109
Thanks Received: 800

I have to ask... what API are you coding to? Or are you going straight to QuickFIX?

Reply With Quote
  #239 (permalink)
ScoobyStoo
London
 
Posts: 16 since Mar 2010
Thanks Given: 0
Thanks Received: 13


MXASJ View Post
I have to ask... what API are you coding to? Or are you going straight to QuickFIX?

Rithmic. Much faster than QuickFIX.

Reply With Quote
Thanked by:
  #240 (permalink)
 
Zondor's Avatar
 Zondor 
Portland Oregon, United States
 
Experience: Beginner
Platform: NinjatraderŽ
Broker: CQG, Kinetick
Trading: Gameplay KlownbineŽ Trading of Globex
Posts: 1,333 since Jul 2009
Thanks Given: 1,246
Thanks Received: 2,731


If, at the time of adding a trade price to the Last file, NT also recorded the bid and ask prices, it would reduce the problems caused by the coarse timestamping.

I suggested that they do this by adding four decimal points to the Volume field. The first two would be the offset in tick size units between the last and the bid, and the second two would be the offset in tick size units from the last to the ask. Then there would be no need for bid and ask files anymore.

The Volume indicator could easily be set to ignore the decimal portion of the stored values.

I posted this in the NT forum and they ignored it. They are truly in their own world.....

Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote




Last Updated on April 22, 2017


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts