Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Accuracy of SC Denali data, and a data reconciliation request
Thanks, it does indeed appear as though this has been corrected. Without any followup comment to me from SC support, I might add. Meanwhile, the incorrect volume figures from Dec. 5, at least in TY, still have not been corrected. I presume this would be the case in other products as well.
A few questions remain
What caused these errors in the first place?
Would anything have been corrected it if I hadn't brought it to their attention?
What other errors remain unnoticed and therefore uncorrected?
What protocols are in place to prevent future errors from occurring?
This is a data integrity issue. One that concerns me still. These incorrect volumes were obvious, and thus easy to detect, but what about less obvious errors? I don't have the time and resources to check them all, and frankly it isn't my job to do so. I should be able to trust the data that I'm paying for, and right now, unfortunately, I don't.
Conclusions:
Denali data may not be accurate and reliable Sierra Chart Support staff remain rude and unprofessional
I don't think the Historical Daily data is from the same provider as the Denali data.
However, I'm not yet able to confirm this in the documentation.
There is at least one SC Support staff that is professional (John). We know who he is because he's the only one that includes his name. IMO, they should all include their names.
There is also one that should be removed from forum duty.
@tomgilb I'm pretty sure that you're correct about the historical data coming from a different source than the intraday data. In fact yesterday I saw somewhere in the documentation that the historical data comes directly from the exchange. While trying to reconcile the daily to the intraday I also noticed that it appears the CME data for Treasuries uses the start of the 18:00 evening session as the open and 15:00 as the close. But, strangely, sometimes the daily close matches the OPEN of the period beginning at 15:00, and other times it matches the CLOSE of the period ending at 15:00 (they don't always match). This obviously makes reconciliation of daily and intraday a bit squirrely since the sources are different and don't always agree. Moreover, what happens between 15:00 and 17:00 is not captured in the historical daily data and instead is shown as a gap. Just something to be aware of when analyzing the historical data.
When running backtests, I believe the Denali intrada data is used, so hopefully there would be no gaps or errors there. I say hopefully because I haven't been able to independently confirm this.
Here's a great example of the daily / intraday data reconciliation issue noted above
This is the bond contract from yesterday to today. The historical daily chart shows a closing price of 159-00 for April 15. The 30 minute chart shows an OPEN price of 159-00 at 15:00. But then, starting almost precisely at 15:00, the market puked by more than a point on heavy volume, so that by 17:00 it ended way down at 157-29
Meanwhile, the volume reported on the historical daily chart for ZB is 614,615, which EXCEEDS the volumes shown on the intraday chart between 15:00 on April 14 and 18:00 on April 15. Perhaps this is due to block trades conducted off exchange and thus not included in the intraday volumes but then added to the daily total after the close of the session. I really can't say for sure. But the point is, I can't seem to get the volumes to match, and the daily data perhaps includes post 15:00 volumes but NOT post 15:00 price.
Again, just something to be aware of. Personally I think it's weird and also a bit annoying but what can I do
I just remembered something that may only create more confusion, but CME has a procedure to determine the daily "settlement" price that is almost always not going to be the same as the actual intra-day last price. I don't know if the CME data uses this settlement price as the daily close or not.
Here's what they do:
"Equity Futures: For S&P and NASDAQ, the settlement price of the lead* month contract is the midpoint of the
closing range determined based on pit trading activity between 15:14:30-15:15:00 Central Time (“CT”). For all other
equity indices, the Volume Weighted Average Price (VWAP) of trades executed on Globex between 15:14:30-
15:15:00 CT is used to determine the settlement prices for the lead month contracts. Back month contract months are
settled to traded or quoted spread relationships. E-mini S&P and Nasdaq are settled to the value derived from the Big
S&P and Nasdaq. "
Is this likely to be relevant to your issue? I would say, probably not. But here it is, just in case it might.
Bob.
-----------------
Edit: I just noticed that this pdf refers to the "pit trading activity," which no longer exists. So this procedure may be an old one that Google served up, that is no longer in effect.
But I do think that they use the short-term (final x seconds) VWAP method, and probably for ES and NQ now.
When one door closes, another opens.
-- Cervantes, Don Quixote
Thanks, Bob. Yes, you're right about the settlement price, but thankfully that shouldn't affect these data files, as the settlement price is more of a procedural thing used for mark-to-market purposes. With regards to the price data I'm studying - the raw data that underlies the charts we see on our screens - the only thing that should be reported there is actual traded prices and volumes.
