Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
For those of you who maintain your own databases of stock histories, how do you keep track of stock delistings, symbol name changes, new listings etc?
Every day I download EOD data from iqfeed for +-6 000 symbols. I then check their website which has 3 sections, namely: Additions, Changes, Deletions. I go through those manually and update my db accordingly. This is a tedious process to do manually. Also, those pages only list +- 1 week worth of changes.
Does anyone handle this in a more automated way (particularly those who use iqfeed?). I guess I could write a script to scrape those 3 pages each day, but my guess is there must be a more obvious way to do it that im not aware of.
IQFeed does return error codes such as: 'E' and 'NO DATA' but that does not necessarily mean a stock has been delisted, it may have just not traded yesterday (or for whatever period you're requesting data).
Diversification is the only free lunch
Can you help answer these questions from other members on NexusFi?
I pull the entire IQFeed repository daily with my stuff. Daily, minutes, and tick.
Never had a need for a flag for new or delisted. Why not just use the daily data and check for history? Should tell you if it's active, last day it traded, etc.
Maybe im just being overly pedantic, but lets say a stock is delisted. I then send a request to download any new data for it each day. Of course nothing new is available, but every day I continue requesting data because it might just be a stock that doesn't generally trade every day. Thinking about it now like you say,...maybe the best thing would just be to implement a rule whereby if a stock hasn't traded for a week, I will consider it delisted and stop requesting data for it.
However that doesn't address new listings. I have one table that lists all the NYSE+NASDAQ stocks which I request data from each day. How would I know to request data for a new listing if I don't already have that symbol name in my db?
When you say you pull the entire repository, I presume you have a table listing all the symbols you will download? Because iqfeed has +- 18 100 NYSE symbols, I presume you don't poll all of them?
Thanks Mike, I wasn't aware of the daily ftp from NASDAQ. That's excellent.
This is going to make things so much easier!
One last question, how do you handle name changes? I presume when a symbol name changes, the new name all of a sudden starts appearing in the ftp file and the old one just disappears, however it is the same company. Do you just download the full history for the new company name (even though it is now a duplicate of an existing company in your db - seems like a waste having both the old and new).
Thanks Mike, i'll have a look into the Bloomberg option, but I think the Nasdaq daily ftp is going to work perfectly.
Regarding name changes, re-downloading everything for the new name will not work for me because when I run backtests that involve sector/industry comparisons there are going to be more stocks than there should be, thus skewing the strength/weakness and size of that sector. (im surprised that is not also a concern for you, but you've probably got some clever way of handling that which I haven't thought of)
At least all I have to do now is check iqfeed daily for any name changes, then change the relevant symbol name. I don't have to worry about new listings / de-listings thanks to the daily ftp.
My testing doesn't really rely on a single stock, or even a few small stocks, within a sector changing. Not unless AMZN dropped off and appeared a second time under a new ticker or something of similar consequence.
I agree it would be nice to be totally accurate, I've simply not put the time into it.