NexusFi: Find Your Edge


Home Menu

 





Yahoo Finance - historical daily data retrieved programmatically


Discussion in Brokers

Updated
    1. trending_up 11,549 views
    2. thumb_up 5 thanks given
    3. group 2 followers
    1. forum 5 posts
    2. attach_file 0 attachments




 
Search this Thread

Yahoo Finance - historical daily data retrieved programmatically

  #1 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

Hi,

In this thread, I will propose several ways to retrieve historical data from Yahoo Finance :
- Web Query (from Java code),
- YQL (from Java code),
- Quantmod (with R).

If you have other or better ideas, do not hesitate to contribute.

EDIT: Yahoo Finance EOD data is available through Ninja Trader: refer to @ratfink 's message below.

Nicolas

Visit my NexusFi Trade Journal Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
Exit Strategy
NinjaTrader
Deepmoney LLM
Elite Quantitative GenAI/LLM
NexusFi Journal Challenge - April 2024
Feedback and Announcements
New Micros: Ultra 10-Year & Ultra T-Bond -- Live Now
Treasury Notes and Bonds
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
59 thanks
Funded Trader platforms
36 thanks
NexusFi site changelog and issues/problem reporting
25 thanks
GFIs1 1 DAX trade per day journal
19 thanks
The Program
18 thanks
  #3 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769


1) Web query and related Java code

Let's consider the following URL :
https://ichart.yahoo.com/table.csv?s=^GSPC&a=3&b=1&c=2014&d=3&e=15&f=2014&g=d

It generates a csv file for ^GSPC, that is to say S&P 500, with daily data (g=d) from April 1st, 2014 (a=3&b=1&c=2014) to April 15th, 2014 (d=3&e=15&f=2014).

More information on the syntax here : https://code.google.com/archive/p/yahoo-finance-managed/wikis/csvHistQuotesDownload.wiki

Other examples of symbols:
MSFT for Microsoft
EN.PA for French 'Bouygues' stock

However, it seems that no historical data is available for future. Refer to python - Download future price series from Yahoo! with Pandas - Stack Overflow

In the CSV file, last data is Adjusted Close, which takes into account split and dividend.
References: https://uk.help.yahoo.com/kb/finance/historical-prices-adjusted-close-sln2311.html?impressions=true and https://help.yahoo.com/kb/finance/historical-prices-sln2311.html?impressions=true

Example of Java code:
 
Code
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

// inspired by http://stackoverflow.com/questions/9093000/read-csv-file-from-internet
public class Yahoo_ReadCSVFromWebQuery {

    public static void main(String[] args) {
        try {
            URL url = new URL("http://ichart.yahoo.com/table.csv?s=^GSPC&a=3&b=1&c=2014&d=3&e=15&f=2014&g=d");
            URLConnection urlConn = url.openConnection();
            InputStreamReader inputStreamReader = new InputStreamReader(urlConn.getInputStream());
            BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                System.out.println(line);
            }
            bufferedReader.close();
            inputStreamReader.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
It produces the following output:

Quoting 
Date,Open,High,Low,Close,Volume,Adj Close
2014-04-15,1831.45,1844.02,1816.29,1842.98,3736440000,1842.98
2014-04-14,1818.18,1834.19,1815.80,1830.61,3111540000,1830.61
2014-04-11,1830.65,1835.07,1814.36,1815.69,3743460000,1815.69
2014-04-10,1872.28,1872.53,1830.87,1833.08,3758780000,1833.08
2014-04-09,1852.64,1872.43,1852.38,1872.18,3308650000,1872.18
2014-04-08,1845.48,1854.95,1837.49,1851.96,3721450000,1851.96
2014-04-07,1863.92,1864.04,1841.48,1845.04,3801540000,1845.04
2014-04-04,1890.25,1897.28,1863.26,1865.09,3583750000,1865.09
2014-04-03,1891.43,1893.80,1882.65,1888.77,3055600000,1888.77
2014-04-02,1886.61,1893.17,1883.79,1890.90,3131660000,1890.90
2014-04-01,1873.96,1885.84,1873.96,1885.52,3336190000,1885.52


Visit my NexusFi Trade Journal Started this thread Reply With Quote
  #4 (permalink)
 
ratfink's Avatar
 ratfink 
Birmingham UK
Market Wizard
 
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425

Don't forget you do get free access to the Yahoo and Kinetic EOD data streams in the free version of NinjaTrader, you could always just use that as your download route if you don't want to run with it.

Travel Well
Visit my NexusFi Trade Journal Reply With Quote
Thanked by:
  #5 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

2) YQL and related Java code

According to Yahoo, "The Yahoo Query Language is an expressive SQL-like language that lets you query, filter, and join data across Web services. With YQL, apps run faster with fewer lines of code and a smaller network footprint."
Documentation: https://developer.yahoo.com/yql/

Let's go to YQL Console: https://developer.yahoo.com/yql/console/

On the left, tick "Show Community Tables".

Then, the following query:

Quoting 
select * from yahoo.finance.historicaldata where symbol = "^GSPC" and startDate = "2014-04-01" and endDate = "2014-04-15"

...returns XML code with S&P 500 daily quotes for the 15 first days of April:

Quoting 
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="11" yahoo:created="2014-04-16T17:48:03Z" yahoo:lang="en-US">
<diagnostics>
<url execution-start-time="0" execution-stop-time="33" execution-time="33"><![CDATA[http://www.datatables.org/yahoo/finance/yahoo.finance.historicaldata.xml]]></url>
<publiclyCallable>true</publiclyCallable>
<cache execution-start-time="36" execution-stop-time="36"
execution-time="0" method="GET" type="MEMCACHED"><![CDATA[1dfb58ee222318d70f556698d247d216]]></cache>
<url execution-start-time="36" execution-stop-time="139" execution-time="103"><![CDATA[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]]></url>
<query execution-start-time="36" execution-stop-time="140"
execution-time="104" params="{url=[http://ichart.finance.yahoo.com/table.csv?g=d&amp;f=2014&amp;e=15&amp;c=2014&amp;b=1&amp;a=3&amp;d=3&amp;s=%5eGSPC]}"><![CDATA[select * from csv(0,1) where url=@url]]></query>
<cache execution-start-time="141" execution-stop-time="141"
execution-time="0" method="GET" type="MEMCACHED"><![CDATA[015e0267fe0040cfc71fc09364faf796]]></cache>
<url execution-start-time="141" execution-stop-time="203" execution-time="62"><![CDATA[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]]></url>
<query execution-start-time="141" execution-stop-time="204"
execution-time="63" params="{columnsNames=[Date,Open,High,Low,Close,Volume,Adj_Close], url=[http://ichart.finance.yahoo.com/table.csv?g=d&amp;f=2014&amp;e=15&amp;c=2014&amp;b=1&amp;a=3&amp;d=3&amp;s=%5eGSPC]}"><![CDATA[select * from csv(2,0) where url=@url and columns=@columnsNames]]></query>
<javascript execution-start-time="34" execution-stop-time="209"
execution-time="174" instructions-used="90942" table-name="yahoo.finance.historicaldata"/>
<user-time>210</user-time>
<service-time>198</service-time>
<build-version>0.2.2430</build-version>
</diagnostics>
<results>
<quote Symbol="%5eGSPC">
<Date>2014-04-15</Date>
<Open>1831.45</Open>
<High>1844.02</High>
<Low>1816.29</Low>
<Close>1842.98</Close>
<Volume>3736440000</Volume>
<Adj_Close>1842.98</Adj_Close>
</quote>
<quote Symbol="%5eGSPC">
<Date>2014-04-14</Date>
<Open>1818.18</Open>
<High>1834.19</High>
<Low>1815.80</Low>
<Close>1830.61</Close>
<Volume>3111540000</Volume>
<Adj_Close>1830.61</Adj_Close>
</quote>
[...]
</results>
</query>

The following Java code sends the same query:
 
Code
// requires JDOM API: http://www.jdom.org/
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.Iterator;
import java.util.List;
import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.input.SAXBuilder;

// inspired by http://cynober.developpez.com/tutoriel/java/xml/jdom/#LIII-A
public class Yahoo_ReadFromYQL {

    public static void main(String[] args) {
        try {
            //
            // 1. Retrieve the XML file
            //
            String prefix = "http://query.yahooapis.com/v1/public/yql?q=";
            String query = "select * from yahoo.finance.historicaldata where symbol = \"^GSPC\" and startDate = \"2014-04-01\" and endDate = \"2014-04-15\"";
            String suffix = "&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys";
            String s = prefix + query.replaceAll(" ", "%20") + suffix;
            URL url = new URL(s);
            URLConnection urlConn = url.openConnection();
            InputStreamReader inputStreamReader = new InputStreamReader(urlConn.getInputStream());
            BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
            final StringBuffer buffer = new StringBuffer();
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                buffer.append(line);
            }
            bufferedReader.close();
            inputStreamReader.close();

            //
            // 2. Parse XML file
            //
            SAXBuilder sxb = new SAXBuilder();
            Document document = sxb.build(new ByteArrayInputStream(buffer.toString().getBytes()));
            Element query0 = document.getRootElement();
            Element results = query0.getChild("results");
            List quotes = results.getChildren();
            Iterator i = quotes.iterator();
            while (i.hasNext()) {
                Element quote = (Element) i.next();
                System.out.print(" "+quote.getChild("Date").getText()); 
                System.out.print(" "+quote.getChild("Open").getText()); 
                System.out.print(" "+quote.getChild("High").getText()); 
                System.out.print(" "+quote.getChild("Low").getText()); 
                System.out.print(" "+quote.getChild("Close").getText()); 
                System.out.print(" "+quote.getChild("Volume").getText()); 
                System.out.println(" "+quote.getChild("Adj_Close").getText()); 
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
It returns:

Quoting 
2014-04-15 1831.45 1844.02 1816.29 1842.98 3736440000 1842.98
2014-04-14 1818.18 1834.19 1815.80 1830.61 3111540000 1830.61
2014-04-11 1830.65 1835.07 1814.36 1815.69 3743460000 1815.69
2014-04-10 1872.28 1872.53 1830.87 1833.08 3758780000 1833.08
2014-04-09 1852.64 1872.43 1852.38 1872.18 3308650000 1872.18
2014-04-08 1845.48 1854.95 1837.49 1851.96 3721450000 1851.96
2014-04-07 1863.92 1864.04 1841.48 1845.04 3801540000 1845.04
2014-04-04 1890.25 1897.28 1863.26 1865.09 3583750000 1865.09
2014-04-03 1891.43 1893.80 1882.65 1888.77 3055600000 1888.77
2014-04-02 1886.61 1893.17 1883.79 1890.90 3131660000 1890.90
2014-04-01 1873.96 1885.84 1873.96 1885.52 3336190000 1885.52


Visit my NexusFi Trade Journal Started this thread Reply With Quote
Thanked by:
  #6 (permalink)
 
Nicolas11's Avatar
 Nicolas11 
near Paris, France
 
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769

3) R's quantmod

The following R code
 
Code
library(quantmod)

SnP500 <- getSymbols("^GSPC", src="yahoo", from=as.Date("2014-4-1") , to=as.Date("2014-4-15"), auto.assign=FALSE)
print(SnP500)
...returns:

Quoting 
GSPC.Open GSPC.High GSPC.Low GSPC.Close GSPC.Volume GSPC.Adjusted
2014-04-01 1873.96 1885.84 1873.96 1885.52 3336190000 1885.52
2014-04-02 1886.61 1893.17 1883.79 1890.90 3131660000 1890.90
2014-04-03 1891.43 1893.80 1882.65 1888.77 3055600000 1888.77
2014-04-04 1890.25 1897.28 1863.26 1865.09 3583750000 1865.09
2014-04-07 1863.92 1864.04 1841.48 1845.04 3801540000 1845.04
2014-04-08 1845.48 1854.95 1837.49 1851.96 3721450000 1851.96
2014-04-09 1852.64 1872.43 1852.38 1872.18 3308650000 1872.18
2014-04-10 1872.28 1872.53 1830.87 1833.08 3758780000 1833.08
2014-04-11 1830.65 1835.07 1814.36 1815.69 3743460000 1815.69
2014-04-14 1818.18 1834.19 1815.80 1830.61 3111540000 1830.61
2014-04-15 1831.45 1844.02 1816.29 1842.98 3736440000 1842.98

quantmod::adjustOHLC ( adjustOHLC {quantmod} | inside-R | A Community Site for R) could be considered to take into account splits and dividends.

This is the end of this series.

Nicolas

Visit my NexusFi Trade Journal Started this thread Reply With Quote




Last Updated on April 16, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts