Stocks Data base - futures io
futures io futures trading



Stocks Data base


Discussion in Platforms and Indicators

Updated
      Top Posters
    1. looks_one Amiuser with 10 posts (10 thanks)
    2. looks_two timmymagic07 with 6 posts (0 thanks)
    3. looks_3 colion with 2 posts (0 thanks)
    4. looks_4 StockJock with 2 posts (0 thanks)
    1. trending_up 15,130 views
    2. thumb_up 10 thanks given
    3. group 9 followers
    1. forum 23 posts
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Stocks Data base

(login for full post details)
  #1 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

Here is an empty data base of US stocks Download US Stocks.zip You just need to download the data (e.g. Amiquote). The symbols are sorted by industries and sectors. It's an empty data base someone made one year ago. So it's not up to date since there are also OTC stocks included so if a firm doesn't exist anymore no (current) data will be available. But of course there are also major firms included that have been existing for years and decades.

What to do with the zip file?

Unzip it and put broker.industries, broker.sectors and us_stocks.tls to ... Program Files\Amibroker
Put the import.types and the tls.format file to ... Program Files\Amibroker\Formats folder

Create a new folder called USStocks or anything similar and put it to i.e. ....Program Files\Amibroker or wherever you want it to be. Then start Amibroker and create a new data base (File>New>DataBase). 'Browse' to the USStocks folder you have created before, open it and press save. (Un-)check 'Load data base at startup'. Then press 'Create'.

Now choose data source (most probably 'local data base' as it gets updated via Amiquote). Base time interval EOD but you can choose any other interval.

Now go to File>Import ASCII, click on it, choose file type US_Stocks (*.tls) and browse to the us_stock.tls file located in ....Program Files\Amibroker. Then click open. The stock symbols will be getting imported. But the data base will be still empty so the last thing to do is to start Amiquote ( Tools>Auto update quotes (Amiquote only) ). Choose download range there and start downloading. Of course you can explore other settings of Amiquote before downloading.

Reply With Quote

Can you help answer these questions
from other members on futures io?
Short Interest Ratio indicator?
ThinkOrSwim
What blockvolume is visible? - order processing
NinjaTrader
Indicators for pair trading MES and M2K
Platforms and Indicators
NT8 indicator to simulate a Funding Evaluation where to …
NinjaTrader
Help Needed: TOS Options Chain IV & IMPL MOVE Calcul …
ThinkOrSwim
 
 
(login for full post details)
  #2 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

If you for example just need stocks of S&P 500 then here there is the .tls file for that Download S&P 500.tls
Put it to the Amiquote folder, then start Amibroker and Amiquote and in Amiquote choose that tls file (File>Open).

Reply With Quote
 
(login for full post details)
  #3 (permalink)
Bay Area, CA - US
 
Experience: Beginner
Platform: TOS, TS & MC
Broker: TDA & TS
Trading: Stock, Options and now Futures (ES)
 
TonyB's Avatar
 
Posts: 516 since Dec 2010
Thanks: 227 given, 140 received


Good stuff Amiuser. Thanks for the helpful how-to. I like how it utilizes industries and sectors. I usually get current NYSE, NASDAQ and AMEX listings from the web, I believe on NASDAQ's website, downloadable...

Reply With Quote
 
(login for full post details)
  #4 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Hello,

TonyB - FYI you can also find some additional useful .TLS files here;

<had to remove url as I do not have 5 posts, but simply google "how to use amiquote" and select the third link, bottom of the page are extra *.TLS files, you have probably seen them already, but just in case.../>

I, like you, am looking into AmiBroker as a solution for my needs. I will be longer term trading/investing, so only need a platform that deals with EOD data (preferably the free stuff!) that is also cost effective. So far this would appear to be the best option, along with ProTA, but I do not have a Mac.

It would be nice if they had up to date *.TLS files for FTSE100, FTSE350, Europe, etc. considering they charge a fee for AmiQuote, but it would appear they do not. It wouldn't take too much effort to keep them up to date, assuming you put in the effort to create them in the first place, but it is still effort I would rather not have to make..

Having said that - I have been in contact with customer support with no word on additional sources of .TLS files, having asked, but I do not suppose there are a selection on their forums perhaps? Amiuser, perhaps you might be able to shed some light on this?

I am a programmer by trade, albeit very early into my career, so AmiBroker certainly appeals in that sense, I just don't want to go spending hard earned cash until I am happy it is the right decision.

Reply With Quote
 
(login for full post details)
  #5 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

They don't charge a monthly fee for Amiquote. It's a one time price for that ticker downloader. By additional providing and keeping all thousands of world wide tickers up-to-date T.J. would have a lot less time being available to improve his software. I wouldn't like that. Even more so since I'm not into stocks that much. The standard data base provides index and stocks examples of the US market so it's up to the user to add additional ones. But here are some worldwide ticker lists that you can use Download tickers worldwide.rar There is a read me file included. So yeah read it.

Reply With Quote
 
(login for full post details)
  #6 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Cheers Amiuser, that's useful.

Are these generally kept up to date by users on the yahoo group at all?

Reply With Quote
 
(login for full post details)
  #7 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

One general note in regards to broker.sectors and broker.industries files of the Amibroker installation folder. Those two files are needed only for creating a data base. After creating of a DB those two files of the .rar archive or any other custom made ones are not needed anymore so the two backed up standard files that are part of the standard installation of Amibroker can be "re-installed".

Reply With Quote
 
(login for full post details)
  #8 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received


timmymagic07 View Post
Cheers Amiuser, that's useful.

Are these generally kept up to date by users on the yahoo group at all?

Ask Market Monk of the yahoo group of AB. He keeps a data base up to date and is providing it to other users. I think he is charging a small donation of around $20. But I don't remember what kind of stocks are included. I think he is an American. So I guess his DB mainly consists of US stocks but ask him.

Reply With Quote
 
(login for full post details)
  #9 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

Another note you can download symbol lists from finviz and simply import them into AB, also can download lists from official US market websites. There was a thread about on the yahoo list recently. I will look at it.

Reply With Quote
 
(login for full post details)
  #10 (permalink)
Bay Area, CA - US
 
Experience: Beginner
Platform: TOS, TS & MC
Broker: TDA & TS
Trading: Stock, Options and now Futures (ES)
 
TonyB's Avatar
 
Posts: 516 since Dec 2010
Thanks: 227 given, 140 received


timmymagic07, I appreciate the follow-up, thank you.

If I had such a programming background, such as yourself, AB would have most likely been my platform of choice. Best to you.

Reply With Quote
 
(login for full post details)
  #11 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received


timmymagic07 View Post
Cheers Amiuser, that's useful.

Are these generally kept up to date by users on the yahoo group at all?

You can keep those lists up to date very easily by yourself as you mentioned that you were a programmer.
Here is an example Amibroker - import Finviz stock lists of how it could be done via AFL and using finviz Free Stock Screener
The new app looks like this with one added option

Urlget can be downloaded from AB website https://www.amibroker.com/bin/URLGet.zip Location of the extracted zip files is Amibroker directory.
The AFL's first red button downloads the raw list(s) from finviz.
This is one raw list from there (Click pic to enlarge)


Then the AFL removes the quotations (button 2 of my AFL).

Before creating a data base containing symbols with sector and industry info you would need to create one custom broker.sectors and one custom broker.industries files according to Finviz' Morningstar Industry Classification (backup the old broker files before).

The content of the finviz broker.sectors file looks like this (Undefined Sectors is an addition by me for symbols that have no sector/industry assignment (that was a tip by Market Monk, thanks btw) ):

Undefined Sector
Basic Materials
Conglomerates
Consumer Goods
Financial
Healthcare
Industrial Goods
Services
Technology
Utilities

the same one as a picture and with additional info

So this is pretty short and simple

The broker.industries file is larger. One click of the third button of my AFL and it gets formated using the finviz raw file seen above.

The formated broker.industries file looks as follows (needs to be in place before new database creation) :


This is a snippet of the formated symbols list (button 4 of my AFL)


The last thing you need is the format file to import the symbols of the symbols list (format file saved as i.e. finviz_NYSE.format or just finviz.format to the Formats folder of AB. Format is the file extension):
 
Code
$FORMAT TICKER,FULLNAME,Skip,INDUSTRY,Skip,Skip,Skip 
$SKIPLINES 1 
$BREAKONERR 1 
$AUTOADD 1 
$NOQUOTES 1 
$MARKET 0 
$OVERWRITE 1 
$SEPARATOR |
the same one as picture


the same AFL (of the video) can be used for any provided symbols list like the ones downloaded from here
NYSE Companies - NASDAQ.com

If you don't need sectors and industries assignments at all then it's even much more simpler to import own/external lists of symbols

Reply With Quote
 
(login for full post details)
  #12 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Thanks Amiuser - that is really very useful.

With a little modification, could this also be used for the FTSE350, for example, and other nations/indices? I assume so and see no reason why not?

Am I right in saying AmiBroker's language is essentially C/C++?

Also - how accurate is the Google finance/Yahoo finance data? I would be relying on it to make decisions, I just wonder whether it would be more worthwhile investing in a paid for provider? I would only be using EOD data mind, not intra-day, would Google/Yahoo be sufficient for my needs? If so, I see no reason not to go for AmiBroker..

Just out of interest, how far back can you get data from Google/Yahoo? Do they put a limit on how much you can download? I would be interested in as much back data as possible for the S&P500, for example...

Thank you for your help Amiuser, it has been most useful!

Reply With Quote
 
(login for full post details)
  #13 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received

You can get a list of symbols from here Symbols lists Unfortunately it's just symbols without names, sectors and industries information. For FTSE350 scroll down on the left side. Copy and paste that list and save it as .tls file and import it via Amiquote.

I'm not into stocks (or EOD) that much as mentioned but Google and Yahoo are free sources. And because they are American sources American stocks are treated better than foreign stocks there. The data for S&P500 goes back to the '50s, AFAIK.

If you want reliable data then you should use the real-time plugins of AB.

Reply With Quote
 
(login for full post details)
  #14 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Thanks Amiuser, I will give that a go.

Would you suggest Yahoo and Google are not sufficient end of day data providers, in terms of quality/accuracy? I am looking to keep costs down as much as possible, but don't want to rely on inaccurate data.

Think I will buy Amibroker at the weekend and quickly become an active member of the yahoo group I expect!

Many thanks

Reply With Quote
 
(login for full post details)
  #15 (permalink)
Asheville, North Carolina
 
 
Posts: 24 since Sep 2010
Thanks: 1 given, 15 received


timmymagic07 View Post
Thanks Amiuser, I will give that a go.

Would you suggest Yahoo and Google are not sufficient end of day data providers, in terms of quality/accuracy? I am looking to keep costs down as much as possible, but don't want to rely on inaccurate data.

Think I will buy Amibroker at the weekend and quickly become an active member of the yahoo group I expect!

Many thanks

Yahoo and Google are not data suppliers. They simply post what is sent by their data sources which include all of the standard names such as CSI. Any errors should be reported to them or the data suppliers.

Reply With Quote
 
(login for full post details)
  #16 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Hi Colion,

So the data Yahoo/Google provide should be of good quality?

Cheers,

Tim

Reply With Quote
 
(login for full post details)
  #17 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received


timmymagic07
Thanks Amiuser, I will give that a go. Would you suggest Yahoo and Google are not sufficient end of day data providers, in terms of quality/accuracy? I am looking to keep costs down as much as possible, but don't want to rely on inaccurate data. Think I will buy Amibroker at the weekend and quickly become an active member of the yahoo group I expect! Many thanks

They provide good quality but Yahoo is quite a bit slow when it comes to stock splits of foreign stocks other than American stocks. It's is not a fact but I've got the feeling that Google seems to be the more reliable one of those two free sources. Long story short you should treat them as what they are ... free sources. But you can choose other sources (free or commercial) with AB too. YHOO and GOOG are just two free options. Pretty much every software offers them. As for AB itself ... you won't regret using it. No frustrations because of slowness and software hanging up, i.e. Ninjatrader or Multicharts. It's a (very) fast, flexible, powerful beast. It's an addicting piece of software and pretty easy to use once you have dug up the rabbits hole, IMO.

Reply With Quote
 
(login for full post details)
  #18 (permalink)
Asheville, North Carolina
 
 
Posts: 24 since Sep 2010
Thanks: 1 given, 15 received


timmymagic07 View Post
Hi Colion,

So the data Yahoo/Google provide should be of good quality?

Cheers,

Tim

As good as their data supplier. Yahoo and Google do not change what they receive from their data suppliers so check them out if you like.

Reply With Quote
 
(login for full post details)
  #19 (permalink)
London, United Kingdom
 
 
Posts: 33 since Apr 2012
Thanks: 3 given, 16 received

Amiuser,

First of all - thank you for all your help on the forum, I have now purchased AmiBroker!

I wanted to ask whether there were some good guides for getting started with the AmiBroker code? Or perhaps some code samples for simple trading system design?

Also - did you create the Finviz stock download/import yourself? I am interested in doing something similar, for EOD data from Google/Yahoo and then ensuring the S&P500 (for example) is kept up to date.

I have been trying to read the Yahoo Group, is there a reader or mail application you can use that you know of? The web interface is really annoying imo.

Reply With Quote
 
(login for full post details)
  #20 (permalink)
New York NY
 
 
Posts: 21 since Mar 2012
Thanks: 3 given, 10 received


timmymagic07 View Post
I wanted to ask whether there were some good guides for getting started with the AmiBroker code? Or perhaps some code samples for simple trading system design?

The AB manual, the AFL library at amibroker.com, wisestocktrader.com, Howard Bandy books, ... You could ask here in the AB section.


timmymagic07 View Post
Also - did you create the Finviz stock download/import yourself? I am interested in doing something similar, for EOD data from Google/Yahoo and then ensuring the S&P500 (for example) is kept up to date.

Yes, I did. It uses some code snippets of different sources that I've changed to fit my needs but 70% of that AFL program has been made by me.


timmymagic07 View Post
I have been trying to read the Yahoo Group, is there a reader or mail application you can use that you know of? The web interface is really annoying imo.

You could use Thunderbird with gmail
Thunderbird 3.0 - Gmail Help
Enabling IMAP - Gmail Help

Reply With Quote
 
(login for full post details)
  #21 (permalink)
Chicago + Illinois/USA
 
 
Posts: 256 since Aug 2010
Thanks: 15 given, 153 received


Amiuser View Post
You can keep those lists up to date very easily by yourself as you mentioned that you were a programmer. Here is an example Amibroker - import Finviz stock lists of how it could be done via AFL and using finviz The new app looks like this with one added option Urlget can be downloaded from AB website Location of the extracted zip files is Amibroker directory.

Is this utility available for Big Mike's Elite members?

Reply With Quote
 
(login for full post details)
  #22 (permalink)
Albuquerque NM
 
 
Posts: 1 since Aug 2012
Thanks: 0 given, 0 received

Hi,
I am very new to Amibroker. After downloading the zip file I tried to istall it used tools/customized/tools entered Finviz as new tool, then added C:\Program Files\AmiBroker\URLGet.zip in command line and also in initial directory. It gave me "you have to define url and local file name in program arguments". also tried with exe file with C:\Program Files\AmiBroker\URLGet.exe as command and C:\Program Files\AmiBroker as initial directory but the same result. Please let me know what I'm doing wrong.

Reply With Quote
 
(login for full post details)
  #23 (permalink)
Chicago + Illinois/USA
 
 
Posts: 256 since Aug 2010
Thanks: 15 given, 153 received

Why doesn't AmiBroker already have this code built into it or does anyone need these categories for trading? Maybe the AmiBroker programmers don't think it's needed and pointless to pursue it.

=========================================


Amiuser View Post
You can keep those lists up to date very easily by yourself as you mentioned that you were a programmer.
Here is an example Amibroker - import Finviz stock lists of how it could be done via AFL and using finviz Free Stock Screener
The new app looks like this with one added option
Then the AFL removes the quotations (button 2 of my AFL).

Before creating a data base containing symbols with sector and industry info you would need to create one custom broker.sectors and one custom broker.industries files according to Finviz' Morningstar Industry Classification (backup the old broker files before).

The content of the finviz broker.sectors file looks like this (Undefined Sectors is an addition by me for symbols that have no sector/industry assignment (that was a tip by Market Monk, thanks btw) ):

Undefined Sector
Basic Materials
Conglomerates
Consumer Goods
Financial
Healthcare
Industrial Goods
Services
Technology
Utilities

the same one as a picture and with additional info

So this is pretty short and simple

The broker.industries file is larger. One click of the third button of my AFL and it gets formated using the finviz raw file seen above.

The formated broker.industries file looks as follows (needs to be in place before new database creation) :


This is a snippet of the formated symbols list (button 4 of my AFL)


The last thing you need is the format file to import the symbols of the symbols list (format file saved as i.e. finviz_NYSE.format or just finviz.format to the Formats folder of AB. Format is the file extension):
 
Code
$FORMAT TICKER,FULLNAME,Skip,INDUSTRY,Skip,Skip,Skip 
$SKIPLINES 1 
$BREAKONERR 1 
$AUTOADD 1 
$NOQUOTES 1 
$MARKET 0 
$OVERWRITE 1 
$SEPARATOR |
the same one as picture


the same AFL (of the video) can be used for any provided symbols list like the ones downloaded from here
NYSE Companies - NASDAQ.com

If you don't need sectors and industries assignments at all then it's even much more simpler to import own/external lists of symbols


Reply With Quote
 
(login for full post details)
  #24 (permalink)
Prague
 
 
Posts: 5 since Sep 2014
Thanks: 0 given, 1 received

Good stuff mate. Where I can download this AFL file mate ?

Amiuser View Post
the same AFL (of the video) can be used for any provided symbols list like the ones downloaded from here


Reply With Quote


futures io Trading Community Platforms and Indicators > Stocks Data base


Last Updated on November 7, 2014


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Battlestations! Show us your trading desk - $1,500 in prizes!

March
 

Importance of Finding Your Own Way w/Adam Grimes

Elite only
 

Journal Challenge w/Jigsaw

April
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, +507 833-9432, info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts