FXCM historical data has DST offset in raw "UTC" data - Brokers and Data Feeds | futures.io
futures.io futures trading

Go Back   futures.io

> Futures Trading, News, Charts and Platforms > Traders Hideout > Brokers and Data Feeds

FXCM historical data has DST offset in raw "UTC" data
Started:October 18th, 2015 (05:54 PM) by noetus Views / Replies:365 / 2
Last Reply:October 19th, 2015 (06:17 AM) Attachments:0

Welcome to futures.io.

Welcome, Guest!

This forum was established to help traders (especially futures traders) by openly sharing indicators, strategies, methods, trading journals and discussing the psychology of trading.

We are fundamentally different than most other trading forums:
  • We work extremely hard to keep things positive on our forums.
  • We do not tolerate rude behavior, trolling, or vendor advertising in posts.
  • We firmly believe in openness and encourage sharing. The holy grail is within you, it is not something tangible you can download.
  • We expect our members to participate and become a part of the community. Help yourself by helping others.

You'll need to register in order to view the content of the threads and start contributing to our community. It's free and simple, and we will never resell your private information.

-- Big Mike

Thread Tools Search this Thread

FXCM historical data has DST offset in raw "UTC" data

Old October 18th, 2015, 05:54 PM   #1 (permalink)
Trading Apprentice
Aix-en-Provence, France
Futures Experience: Intermediate
Platform: Ninjatrader
Favorite Futures: Currency Spot
Posts: 5 since May 2013
Thanks: 0 given, 0 received

FXCM historical data has DST offset in raw "UTC" data

I have been using FXCM's "Historical Data Downloader" software to download historical forex data on 20 different currency pairs (data from late 2001 through to the present). Backtesting on at least 10 years of data is vital to my trading strategy. The retail cost of this software is $500.

I downloaded minute data; the software claims the data has UTC timestamps and you can choose the offset from UTC when you download it. Stating that the data is UTC timestamped means the data should have NO DST offsets in it. Whatever charting software you use would then apply DST offsets depending on what timezone you happen to be in; Ninjatrader for example (what I use) uses the computer clock (i.e. local time) to calculate and apply timestamps to chart candles (and Ninjatrader's data importer asks for the data to be UTC, i.e. with no DST offsets).

I have analysed their data and I have come to the rather shocking conclusion that the data from 2001 through to 9 March 2008 has a DST offset in it, with the offset boundaries corresponding to the dates of DST in the New York (Eastern) timezone. From spring 2008 onwards to the present the data has no DST offset in it. Furthermore, the data from 2001 to 9 March 2008 uses EDT (i.e., summer time) as the reference for calculating the UTC offset when you select that using their Historical Data Downloader. In other words, it is not that during the summer there is an offset of +1 hour applied to all the data. Rather it is that during the winter there is a -1 hour offset applied to all the data.

Given that this is a retail product I am really shocked. It matters a lot to people like me whose trading strategies depend on accurate absolute timestamps. It now seems that all my backtesting is invalid. I have to correct their data (I guess I will have to write a script to change the timestamps in their raw data) and run my backtests again (months of work).

Have I stumbled upon a well-known fact about FXCM's data? Do I have the basis of a complaint against them?

If anyone is interested, I can supply the steps of my analysis that demonstrate that their data contains DST offsets during the time period I mentioned (although my analysis could only be verified by someone with access to the same data I have).

Reply With Quote

Old October 18th, 2015, 05:54 PM   #2 (permalink)
Quick Summary
Quick Summary Post

Quick Summary is created and edited by users like you... Add FAQ's, Links and other Relevant Information by clicking the edit button in the lower right hand corner of this message.


Old October 19th, 2015, 06:17 AM   #3 (permalink)
Elite Member
Los Angeles
Futures Experience: Intermediate
Platform: NinjaTrader/Sierra
Favorite Futures: ES
srgtroy's Avatar
Posts: 1,316 since Jan 2011
Thanks: 952 given, 2,019 received

I would be interested to know what their response is. Please keep us informed.

Reply With Quote


futures.io > Futures Trading, News, Charts and Platforms > Traders Hideout > Brokers and Data Feeds > FXCM historical data has DST offset in raw "UTC" data

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Upcoming Webinars and Events (4:30PM ET unless noted)

An Afternoon with FIO trader bobwest

Elite only

NinjaTrader 8: Programming Profitable Trading Edges w/Scott Hodson

Elite only

Anthony Drager: Executing on Intermarket Correlations & Order Flow, Part 2

Elite only

Adam Grimes: Five critically important keys to professional trading

Elite only

Machine Learning Concepts w/FIO member NJAMC

Elite only

MarketDelta Cloud Platform: Announcing new mobile features

Dec 1

NinjaTrader 8: Features and Enhancements

Dec 6

Similar Threads
Thread Thread Starter Forum Replies Last Post
Does Elite circle "QCollector tick data sharing thread" contain latest 2015 data? ayteck Traders Hideout 5 February 15th, 2016 01:41 PM
Big S&P (SP) "Pit" Historical Data OZTrade Brokers and Data Feeds 5 May 20th, 2015 03:10 PM
Ninja7 "reload Historical data"grayed out traderTX NinjaTrader 4 April 6th, 2014 01:07 PM
Loading historical data via "Restore" jnorris Elite Automated NinjaTrader Trading 5 October 27th, 2013 11:00 PM
Is it possible to get "historical" Time and Sales data? adaseb Brokers and Data Feeds 2 September 8th, 2010 11:18 AM

All times are GMT -4. The time now is 06:28 AM.

Copyright © 2016 by futures.io. All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts

Page generated 2016-10-25 in 0.08 seconds with 19 queries on phoenix via your IP