NexusFi: Find Your Edge


Home Menu

 





How to measure data feed latencies between continents?


Discussion in Traders Hideout

Updated
      Top Posters
    1. looks_one Koepisch with 22 posts (5 thanks)
    2. looks_two Big Mike with 18 posts (6 thanks)
    3. looks_3 artemiso with 11 posts (1 thanks)
    4. looks_4 gomi with 7 posts (21 thanks)
      Best Posters
    1. looks_one gomi with 3 thanks per post
    2. looks_two Fat Tails with 3 thanks per post
    3. looks_3 Big Mike with 0.3 thanks per post
    4. looks_4 Koepisch with 0.2 thanks per post
    1. trending_up 27,989 views
    2. thumb_up 48 thanks given
    3. group 8 followers
    1. forum 66 posts
    2. attach_file 15 attachments




 
Search this Thread

How to measure data feed latencies between continents?

  #1 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518

Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Are there any eval firms that allow you to sink to your …
Traders Hideout
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
ZombieSqueeze
Platforms and Indicators
Futures True Range Report
The Elite Circle
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
39 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
The Program
18 thanks
GFIs1 1 DAX trade per day journal
18 thanks
  #3 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,396 since Jun 2009
Thanks Given: 33,172
Thanks Received: 101,536


If you use IQFeed it is pretty simple. They provide the Exchange timestamp, then just compare it to local time. I imagine that other API's do this as well. AFAIK however, NinjaTrader does not give you access to it but you should check with them to confirm.

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #4 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518


Big Mike View Post
If you use IQFeed it is pretty simple. They provide the Exchange timestamp, then just compare it to local time. I imagine that other API's do this as well. AFAIK however, NinjaTrader does not give you access to it but you should check with them to confirm.

Mike

Thanks Mike, i will check that issue with ninjatrader directly next week. But i think the API approach is the only way to get thrustworthy comparisions. But that means that i have to pay for the feed + developer license and perhaps twice because i have to run it on 2 different systems. I hope NT can give me a hint doing this within the NT scope.

Koepisch

Started this thread Reply With Quote
  #5 (permalink)
 
Fat Tails's Avatar
 Fat Tails 
Berlin, Europe
Market Wizard
 
Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker: Interactive Brokers
Trading: Keyboard
Posts: 9,888 since Mar 2010
Thanks Given: 4,242
Thanks Received: 27,102


Koepisch View Post
Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

@Koepisch: If you wish to run automated strategies on fast tick based charts for US traded futures or stocks, then you will never be happy with a European based server, but should opt for a US based VPS. Let me explain the reasons.

You said that you enable the strategy a few seconds before the actual signal occurs. This means that the activation of the strategy is not time-critical and can be done from most locations in the world. However, when the actual signal occurs, you want that your automated strategy places an entry order in a timely manner. So let us examine the path from the entry signal until the order is placed at the exchange.


The path of the signal

(1) entry signal occurs at exchange
(2) data feed collects signal at exchange and transmits that data to your trading PC or VPS
(3) NinjaTrader automated strategy triggers entry order
(4) entry order is transmitted from your PC to access point of the broker
(5) broker performs credit control in order to check your margin
(6) broker transmits entry order to exchange
(7) exchange executes order
(8) confirmation of exchange is transmitted to trading server (this step is only required for measuring the lag, see below)

If you have a European based server, there are some obvious disadvantages. In step (2) the data has to travel about 8,000 km (Chicago) through a translantic fiberoptic network. If you select a broker with an access point near the exchange, the order has to travel back another 8,000 km before it reaches the broker server that performs the credit control and relays your order to the exchange.

This means that you need to take into account the transmission time for another 16,000 km. In fiberoptical cables you may assume that signals travel at about 60% of the speed of light, which is 300,000 km/ sec. A quick calculation shows that the signal will at least take 90 msec for a round trip. Add processing times of intermittent routers and you will find a typical delay of about 120 msec to Chicago.

Now this is not all. Most of the internet providers such as the Telekom do not have own transatlantic networks, but rent transmission capacity from one of the larger operators of global networks such as NTT. During the European afternoon, when both Europeans and Americans are up, those networks can be easily jammed by YouTube watchers and online gamers. This may increase the lag to more than 120 msec. Best use a broker who has an access point in Europe, if you trade via a European server. The broker probably has rented a fixed line, which is not affected by YouTube traffic.


Measuring the lag

The best way to measure the lag is to run the same automated strategy in parallel on two different servers

- a European based server
- a US based server located near the broker's access point

You will need two live accounts to do this. Once the same orders have been executed all you need to do is to study the NinjaTrader trace files. The trace files allow you to study the global time for exuting steps (3) to (8). This includes the time from the moment NinjaTrader generates the signal until the confirmation of the order placement has been transmitted to your trading station. The Ninjatrader trace file has everything teimstamped in milliseconds, and the difference between the end of step (8) and beginning of the step (3) can be calculated for both the European and US server. If you compare the two values, you will have an idea of the lag.

I am aware that this method is just an approximation, as you cannot measure the lag of the data feed. The method assumes that the lag introduced by the order confirmation is about the same as introduced by the data feed, as the distance that the signal needs to travel is similar.


Direct measurement of the lag of the data feed is not possible

If you want to measure the lag of the data feed directly, this is impossible. You would need to use the same reference time server. Unfortunately, if that internet time server is located in the US, the time signal also has to travel the same distance, and it arrives at your PC with the typical lag of 50 to 60 msec. If you owned a little tool that would be able to synchronize your PC up to 1 millisecond with the US time server, then your PC would still lag at least 50 msec, if located in Europe. But such a synchronistation of your PC cannot be achieved.


The minimum lag is 100 to 120 milliseconds

Physics tells us that the minimum lag you will be exposed to is about 100 milliseconds. This is quite a lot for tick-based strategies, so you do not need to measure anything. Just go for the US based VPS if you wish to pursue high frequency strategies at CME, and go for a European based server, if you wish to trade on EUREX.

Reply With Quote
  #6 (permalink)
 artemiso 
New York, NY
 
Experience: Beginner
Platform: Vanguard 401k
Broker: Yahoo Finance
Trading: Mutual funds
Posts: 1,152 since Jul 2012
Thanks Given: 784
Thanks Received: 2,685


Koepisch View Post
Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

It's probably better to program an Xbox or Playstation controller instead of designing a custom keyboard from scratch for this purpose.

Reply With Quote
  #7 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518


artemiso View Post
It's probably better to program an Xbox or Playstation controller instead of designing a custom keyboard from scratch for this purpose.

I don't want to play games - i won't make manual trading faster and simpler. MS + Sony designed controller to offer excellent game control, vendors like KeybordTrader designed keyboards for traders. There is no intersection between both requirements. Keyboard hardware design is a no brainer. Use a controller (atmel) with ready to use usb stack, design YOUR PCB + integrate it in YOUR trading solution, thats it.

Started this thread Reply With Quote
  #8 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518

@Fat Tails, thanks for your answer. I've done a lot of research of VPS and trade routing, so your answer is welcome but targeting not my intension. I don't want to measure the latencies with snapshots (single executed trades) - i want to continuous measure the lag over a certain time to capture some events too.

I have understand that's difficult to grab the absolut times of the arrival. I don't want to dig too deep into this but i think that if you use a GPS based clock, that issue would be a non brainer. But i can't install them on the VPS side - so this isn't an option. I don't need the analysis results in realtime, so the delta approach via shifting + correlation analysis should get the approbiate results. But thats much efforts i've wanted to avoid.

Perhaps it is possible to use this method to get the Delta from exchange to europe:

1. Use R|API or CQG to get the EXCHANGE timestamp of the tick tuple.
2. Install a GPS-based time module at your PC
3. Match the incoming tick with the current time of your GPS time

But i need the extra feeds + have to pay developer fees too. But it where only for a 1 or 2 month...

To clearify my setup, i've never mentioned that i'm doing automatic strategies, trading from 1 tick charts nor i'm doing high frequency trading. Usualy i have to place a STOP order a few seconds before the trade is triggered. So i could live with an lag in my data, but i want to know the deviation of data feed lag.

Started this thread Reply With Quote
Thanked by:
  #9 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,396 since Jun 2009
Thanks Given: 33,172
Thanks Received: 101,536

Unless you are wanting microsecond precision, I think using NTP is sufficient instead of GPS.

You need a server in Chicago. Depending on your budget I would say either @sam028 which uses Steadfast backbone at 350 E. Cemak, or go with an Aurora server from your execution broker next to CME. The difference between these will be less than 1ms, but depending on your app that could be an eternity.

Sent from my LG Optimus G Pro

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #10 (permalink)
 
sam028's Avatar
 sam028 
Site Moderator
 
Posts: 3,765 since Jun 2009
Thanks Given: 3,825
Thanks Received: 4,629


I think @MXASJ did what you're trying to do, see his posts here, from Singapore in his case.
He didn't log on nexusfi.com (formerly BMT) for a while, but maybe he'll see that we're looking for him .

Success requires no deodorant! (Sun Tzu)
Follow me on Twitter Reply With Quote




Last Updated on February 17, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts