How to measure data feed latencies between continents? - Traders Hideout | futures io social day trading
futures io futures trading


How to measure data feed latencies between continents?
Updated: Views / Replies:5,879 / 66
Created: by Koepisch Attachments:18

Welcome to futures io.

(If you already have an account, login at the top of the page)

futures io is the largest futures trading community on the planet, with over 90,000 members. At futures io, our goal has always been and always will be to create a friendly, positive, forward-thinking community where members can openly share and discuss everything the world of trading has to offer. The community is one of the friendliest you will find on any subject, with members going out of their way to help others. Some of the primary differences between futures io and other trading sites revolve around the standards of our community. Those standards include a code of conduct for our members, as well as extremely high standards that govern which partners we do business with, and which products or services we recommend to our members.

At futures io, our focus is on quality education. No hype, gimmicks, or secret sauce. The truth is: trading is hard. To succeed, you need to surround yourself with the right support system, educational content, and trading mentors Ė all of which you can find on futures io, utilizing our social trading environment.

With futures io, you can find honest trading reviews on brokers, trading rooms, indicator packages, trading strategies, and much more. Our trading review process is highly moderated to ensure that only genuine users are allowed, so you donít need to worry about fake reviews.

We are fundamentally different than most other trading sites:
  • We are here to help. Just let us know what you need.
  • We work extremely hard to keep things positive in our community.
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts.
  • We firmly believe in and encourage sharing. The holy grail is within you, we can help you find it.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

Reply
 18  
 
Thread Tools Search this Thread
 

How to measure data feed latencies between continents?

  #1 (permalink)
Elite Member
@ Germany
 
Futures Experience: Beginner
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: FDAX
 
Posts: 439 since Nov 2011
Thanks: 254 given, 368 received

How to measure data feed latencies between continents?

Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

Reply With Quote
 
  #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.

 
  #3 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,215 received


If you use IQFeed it is pretty simple. They provide the Exchange timestamp, then just compare it to local time. I imagine that other API's do this as well. AFAIK however, NinjaTrader does not give you access to it but you should check with them to confirm.

Mike

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #4 (permalink)
Elite Member
@ Germany
 
Futures Experience: Beginner
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: FDAX
 
Posts: 439 since Nov 2011
Thanks: 254 given, 368 received


Big Mike View Post
If you use IQFeed it is pretty simple. They provide the Exchange timestamp, then just compare it to local time. I imagine that other API's do this as well. AFAIK however, NinjaTrader does not give you access to it but you should check with them to confirm.

Mike

Thanks Mike, i will check that issue with ninjatrader directly next week. But i think the API approach is the only way to get thrustworthy comparisions. But that means that i have to pay for the feed + developer license and perhaps twice because i have to run it on 2 different systems. I hope NT can give me a hint doing this within the NT scope.

Koepisch

Reply With Quote
 
  #5 (permalink)
Elite Member
Berlin, Europe
 
Futures Experience: Advanced
Platform: NinjaTrader, MultiCharts
Broker/Data: Interactive Brokers
Favorite Futures: Keyboard
 
Fat Tails's Avatar
 
Posts: 9,651 since Mar 2010
Thanks: 4,226 given, 25,599 received
Forum Reputation: Legendary


Koepisch View Post
Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

@Koepisch: If you wish to run automated strategies on fast tick based charts for US traded futures or stocks, then you will never be happy with a European based server, but should opt for a US based VPS. Let me explain the reasons.

You said that you enable the strategy a few seconds before the actual signal occurs. This means that the activation of the strategy is not time-critical and can be done from most locations in the world. However, when the actual signal occurs, you want that your automated strategy places an entry order in a timely manner. So let us examine the path from the entry signal until the order is placed at the exchange.


The path of the signal

(1) entry signal occurs at exchange
(2) data feed collects signal at exchange and transmits that data to your trading PC or VPS
(3) NinjaTrader automated strategy triggers entry order
(4) entry order is transmitted from your PC to access point of the broker
(5) broker performs credit control in order to check your margin
(6) broker transmits entry order to exchange
(7) exchange executes order
(8) confirmation of exchange is transmitted to trading server (this step is only required for measuring the lag, see below)

If you have a European based server, there are some obvious disadvantages. In step (2) the data has to travel about 8,000 km (Chicago) through a translantic fiberoptic network. If you select a broker with an access point near the exchange, the order has to travel back another 8,000 km before it reaches the broker server that performs the credit control and relays your order to the exchange.

This means that you need to take into account the transmission time for another 16,000 km. In fiberoptical cables you may assume that signals travel at about 60% of the speed of light, which is 300,000 km/ sec. A quick calculation shows that the signal will at least take 90 msec for a round trip. Add processing times of intermittent routers and you will find a typical delay of about 120 msec to Chicago.

Now this is not all. Most of the internet providers such as the Telekom do not have own transatlantic networks, but rent transmission capacity from one of the larger operators of global networks such as NTT. During the European afternoon, when both Europeans and Americans are up, those networks can be easily jammed by YouTube watchers and online gamers. This may increase the lag to more than 120 msec. Best use a broker who has an access point in Europe, if you trade via a European server. The broker probably has rented a fixed line, which is not affected by YouTube traffic.


Measuring the lag

The best way to measure the lag is to run the same automated strategy in parallel on two different servers

- a European based server
- a US based server located near the broker's access point

You will need two live accounts to do this. Once the same orders have been executed all you need to do is to study the NinjaTrader trace files. The trace files allow you to study the global time for exuting steps (3) to (8). This includes the time from the moment NinjaTrader generates the signal until the confirmation of the order placement has been transmitted to your trading station. The Ninjatrader trace file has everything teimstamped in milliseconds, and the difference between the end of step (8) and beginning of the step (3) can be calculated for both the European and US server. If you compare the two values, you will have an idea of the lag.

I am aware that this method is just an approximation, as you cannot measure the lag of the data feed. The method assumes that the lag introduced by the order confirmation is about the same as introduced by the data feed, as the distance that the signal needs to travel is similar.


Direct measurement of the lag of the data feed is not possible

If you want to measure the lag of the data feed directly, this is impossible. You would need to use the same reference time server. Unfortunately, if that internet time server is located in the US, the time signal also has to travel the same distance, and it arrives at your PC with the typical lag of 50 to 60 msec. If you owned a little tool that would be able to synchronize your PC up to 1 millisecond with the US time server, then your PC would still lag at least 50 msec, if located in Europe. But such a synchronistation of your PC cannot be achieved.


The minimum lag is 100 to 120 milliseconds

Physics tells us that the minimum lag you will be exposed to is about 100 milliseconds. This is quite a lot for tick-based strategies, so you do not need to measure anything. Just go for the US based VPS if you wish to pursue high frequency strategies at CME, and go for a European based server, if you wish to trade on EUREX.

Reply With Quote
The following 9 users say Thank You to Fat Tails for this post:
 
  #6 (permalink)
Elite Member
Manchester, NH
 
Futures Experience: Beginner
Platform: thinkorswim
Broker/Data: TD Ameritrade
Favorite Futures: Stocks
 
Posts: 902 since Jul 2012
Thanks: 603 given, 1,785 received


Koepisch View Post
Hi,

i want to make a test to compare the data feed latencies between a neuropean hosted system and an us based system (VPS and or Colocation). Both accessing us exchanges and are running Ninjatrader. I don't want to utilize ping -based Systems, because there DON'T targeting the question. So with time based charts or with a little more effort with any other kind of charts i can use correlation analysis to get the relative delta time between both. If i swap to R|API i can get the EXCHANGE time stamp of the incoming tick data and i need precise synchronized clocks at both systems to measure the delta. Has anybody some experience how to achieve that goal? Is the "Internet clock adjustment" acceptable? Should i buy some hardware based clocks to get the precise timings? Any other ideas to get the difference of data arrivals?

If i could get the exchange time from within ninjatrader, then that issue would be a non-brainer.

Practical Background:
I working with fast (for humans) tick-based charts and can enable / disable automatic entry- and exit strategies. I could do this only from europe and have to trust that the feed is relative lag free or i run my strategies at an us-based vps and only CONTROL the strategies from europe. The data feed (and especially the data bursts) have not to flow thru many routers and not over the ocean. The direction from europe->us for the few bytes of enabling the auto entry- and exit strategies are not that critical. Usually i have a few seconds between ENABLE entry strategy and the event of real entry. Thats why i also have plans to build a custom trading keyboard to work with.

Thanks,
Koepisch

It's probably better to program an Xbox or Playstation controller instead of designing a custom keyboard from scratch for this purpose.

Reply With Quote
 
  #7 (permalink)
Elite Member
@ Germany
 
Futures Experience: Beginner
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: FDAX
 
Posts: 439 since Nov 2011
Thanks: 254 given, 368 received


artemiso View Post
It's probably better to program an Xbox or Playstation controller instead of designing a custom keyboard from scratch for this purpose.

I don't want to play games - i won't make manual trading faster and simpler. MS + Sony designed controller to offer excellent game control, vendors like KeybordTrader designed keyboards for traders. There is no intersection between both requirements. Keyboard hardware design is a no brainer. Use a controller (atmel) with ready to use usb stack, design YOUR PCB + integrate it in YOUR trading solution, thats it.


Last edited by Koepisch; February 1st, 2014 at 03:43 AM.
Reply With Quote
 
  #8 (permalink)
Elite Member
@ Germany
 
Futures Experience: Beginner
Platform: NinjaTrader
Broker/Data: Mirus Futures/Zen-Fire
Favorite Futures: FDAX
 
Posts: 439 since Nov 2011
Thanks: 254 given, 368 received

@Fat Tails, thanks for your answer. I've done a lot of research of VPS and trade routing, so your answer is welcome but targeting not my intension. I don't want to measure the latencies with snapshots (single executed trades) - i want to continuous measure the lag over a certain time to capture some events too.

I have understand that's difficult to grab the absolut times of the arrival. I don't want to dig too deep into this but i think that if you use a GPS based clock, that issue would be a non brainer. But i can't install them on the VPS side - so this isn't an option. I don't need the analysis results in realtime, so the delta approach via shifting + correlation analysis should get the approbiate results. But thats much efforts i've wanted to avoid.

Perhaps it is possible to use this method to get the Delta from exchange to europe:

1. Use R|API or CQG to get the EXCHANGE timestamp of the tick tuple.
2. Install a GPS-based time module at your PC
3. Match the incoming tick with the current time of your GPS time

But i need the extra feeds + have to pay developer fees too. But it where only for a 1 or 2 month...

To clearify my setup, i've never mentioned that i'm doing automatic strategies, trading from 1 tick charts nor i'm doing high frequency trading. Usualy i have to place a STOP order a few seconds before the trade is triggered. So i could live with an lag in my data, but i want to know the deviation of data feed lag.

Reply With Quote
The following user says Thank You to Koepisch for this post:
 
  #9 (permalink)
Site Administrator
Manta, Ecuador
 
Futures Experience: Advanced
Platform: My own custom solution
Favorite Futures: E-mini ES S&P 500
 
Big Mike's Avatar
 
Posts: 46,238 since Jun 2009
Thanks: 29,350 given, 83,215 received

Unless you are wanting microsecond precision, I think using NTP is sufficient instead of GPS.

You need a server in Chicago. Depending on your budget I would say either @sam028 which uses Steadfast backbone at 350 E. Cemak, or go with an Aurora server from your execution broker next to CME. The difference between these will be less than 1ms, but depending on your app that could be an eternity.

Sent from my LG Optimus G Pro

Due to time constraints, please do not PM me if your question can be resolved or answered on the forum.

Need help?
1) Stop changing things. No new indicators, charts, or methods. Be consistent with what is in front of you first.
2) Start a journal and post to it daily with the trades you made to show your strengths and weaknesses.
3) Set goals for yourself to reach daily. Make them about how you trade, not how much money you make.
4) Accept responsibility for your actions. Stop looking elsewhere to explain away poor performance.
5) Where to start as a trader? Watch this webinar and read this thread for hundreds of questions and answers.
6)
Help using the forum? Watch this video to learn general tips on using the site.

If you want
to support our community, become an Elite Member.

Reply With Quote
 
  #10 (permalink)
Administrator: Retired Backtester
 Vendor: speedytradingservers.com 
Rennes France
 
Futures Experience: Advanced
Platform: NinjaTrader
Broker/Data: IB/Kinetick
Favorite Futures: Futures
 
sam028's Avatar
 
Posts: 3,366 since Jun 2009
Thanks: 3,571 given, 3,982 received


I think @MXASJ did what you're trying to do, see his posts here, from Singapore in his case.
He didn't log on futures.io (formerly BMT) for a while, but maybe he'll see that we're looking for him .

Success requires no deodorant! (Sun Tzu)
Reply With Quote

Reply



futures io > > > How to measure data feed latencies between continents?

Thread Tools Search this Thread
Search this Thread:

Advanced Search



Upcoming Webinars and Events (4:30PM ET unless noted)

Linda Bradford Raschke: Reading The Tape

Elite only

Adam Grimes: TBA

Elite only

NinjaTrader: TBA

January

Ran Aroussi: TBA

Elite only
     

Similar Threads
Thread Thread Starter Forum Replies Last Post
Best data feed for stocks Day Trading Fool Reviews of Brokers and Data Feeds 10 August 7th, 2016 10:54 AM
Data Feed: How do you confirm Accuracy vs Historical tick data provided. FAUGH45568 Reviews of Brokers and Data Feeds 2 February 14th, 2013 07:04 PM
WHat data feed for ICE US Data? samadams Reviews of Brokers and Data Feeds 7 September 20th, 2012 04:08 PM
A better retail data feed LittleCatJones Elite Automated Trading 6 April 27th, 2012 11:52 AM
Data feed comparison - tick data Big Mike Reviews of Brokers and Data Feeds 6 March 18th, 2010 01:52 PM


All times are GMT -4. The time now is 01:31 PM.

Copyright © 2017 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts
Page generated 2017-12-11 in 0.20 seconds with 38 queries on phoenix via your IP 54.145.16.43