FXCM historical data has DST offset in raw "UTC" data

October 18th, 2015, 05:54 PM

I have been using FXCM's "Historical Data Downloader" software to download historical forex data on 20 different currency pairs (data from late 2001 through to the present). Backtesting on at least 10 years of data is vital to my trading strategy. The retail cost of this software is $500.

I downloaded minute data; the software claims the data has UTC timestamps and you can choose the offset from UTC when you download it. Stating that the data is UTC timestamped means the data should have NO DST offsets in it. Whatever charting software you use would then apply DST offsets depending on what timezone you happen to be in; Ninjatrader for example (what I use) uses the computer clock (i.e. local time) to calculate and apply timestamps to chart candles (and Ninjatrader's data importer asks for the data to be UTC, i.e. with no DST offsets).

I have analysed their data and I have come to the rather shocking conclusion that the data from 2001 through to 9 March 2008 has a DST offset in it, with the offset boundaries corresponding to the dates of DST in the New York (Eastern) timezone. From spring 2008 onwards to the present the data has no DST offset in it. Furthermore, the data from 2001 to 9 March 2008 uses EDT (i.e., summer time) as the reference for calculating the UTC offset when you select that using their Historical Data Downloader. In other words, it is not that during the summer there is an offset of +1 hour applied to all the data. Rather it is that during the winter there is a -1 hour offset applied to all the data.

Given that this is a retail product I am really shocked. It matters a lot to people like me whose trading strategies depend on accurate absolute timestamps. It now seems that all my backtesting is invalid. I have to correct their data (I guess I will have to write a script to change the timestamps in their raw data) and run my backtests again (months of work).

Have I stumbled upon a well-known fact about FXCM's data? Do I have the basis of a complaint against them?

If anyone is interested, I can supply the steps of my analysis that demonstrate that their data contains DST offsets during the time period I mentioned (although my analysis could only be verified by someone with access to the same data I have).

October 19th, 2015, 06:17 AM

I would be interested to know what their response is. Please keep us informed.

FXCM historical data has DST offset in raw "UTC" data

Discussion in Brokers

FXCM historical data has DST offset in raw "UTC" data