Download Stock Market Data R
Update #2: Track this Issue Here
Updates to this post are more about which API's are still supported than how to access them with R, Python, or any other language. Follow the hilarious change history of EOD stock data API's at my other post: https://chrisconlan.com/download-daily-data-every-sp-500-stock-r/.
Update: Using Quandl's API
Because everything I write about breaks, the Google Finance API stopped taking requests at this URL. It only returns a year's worth of daily data as of the time of writing.
We will move on to Quandl. They have a stable key-driven API that doesn't seem to be going anywhere. Python code for Quandl coming soon. R code below.
Users will need to visit Quandl's website and sign up for an API key to access the data.
Python Code
Users will need install the Quandl library from pip to use the script with: pip install quandl.
import quandl import datetime quandl.ApiConfig.api_key = 'your_api_key' def quandl_stocks(symbol, start_date=(2000, 1, 1), end_date=None): """ symbol is a string representing a stock symbol, e.g. 'AAPL' start_date and end_date are tuples of integers representing the year, month, and day end_date defaults to the current date when None """ query_list = ['WIKI' + '/' + symbol + '.' + str(k) for k in range(1, 13)] start_date = datetime.date(*start_date) if end_date: end_date = datetime.date(*end_date) else: end_date = datetime.date.today() return quandl.get(query_list, returns='pandas', start_date=start_date, end_date=end_date, collapse='daily', order='asc' ) if __name__ == '__main__': apple_data = quandl_stocks('AAPL') print(apple_data) R Code
Users will need to download the Quandl package from CRAN to run this using: install.packages('Quandl').
Credit to GitHub user johnatasjmo for this solution:
# Quandl package must be installed library(Quandl) # Get your API key from quandl.com quandl_api = "MYAPIKEY" # Add the key to the Quandl keychain Quandl.api_key(quandl_api) quandl_get <- function(sym, start_date = "2017-01-01") { require(devtools) require(Quandl) # create a vector with all lines tryCatch(Quandl(c( paste0("WIKI/", sym, ".8"), # Adj. Open paste0("WIKI/", sym, ".9"), # Adj. High paste0("WIKI/", sym, ".10"), # Adj. Low paste0("WIKI/", sym, ".11"), # Adj. Close paste0("WIKI/", sym, ".12")), # Adj. Volume start_date = start_date, type = "zoo" )) } Original Post
In one of my most popular posts, Download Price History for Every S&P 500 Stock, other traders and I despaired over the death of the Yahoo! Finance API. After batting around a lot of potential replacements, I was still left searching for a good free source of data to use for education and retail trading. I found the answer by searching through the R package quantmod, which was successfully downloading data from Google despite this message on developers.google.com/finance/.
It was Hidden!
As it turns out, quantmod was using a hidden Google Finance API that was quite easy to reverse engineer. In this post, we will build functions for accessing that API in both R and Python. For readers of my book, Automated Trading with R, this will serve as a replacement for the often-referenced yahoo() function, but not as a perfect replacement.
R Code
# Make sure data.table is installed if(!'data.table' %in% installed.packages()[,1]) install.packages('data.table') # Function to fetch google stock data google_stocks <- function(sym, current = TRUE, sy = 2005, sm = 1, sd = 1, ey, em, ed) { # sy, sm, sd, ey, em, ed correspond to # start year, start month, start day, end year, end month, and end day # If TRUE, use the date as the enddate if(current){ system_time <- as.character(Sys.time()) ey <- as.numeric(substr(system_time, start = 1, stop = 4)) em <- as.numeric(substr(system_time, start = 6, stop = 7)) ed <- as.numeric(substr(system_time, start = 9, stop = 10)) } require(data.table) # Fetch data from google google_out = tryCatch( suppressWarnings( fread(paste0("http://www.google.com/finance/historical", "?q=", sym, "&startdate=", paste(sm, sd, sy, sep = "+"), "&enddate=", paste(em, ed, ey, sep = "+"), "&output=csv"), sep = ",")), error = function(e) NULL ) # If successful, rename first column if(!is.null(google_out)){ names(google_out)[1] = "Date" } return(google_out) } # Test it out apple_data = google_stocks('AAPL') Giving this a quick plot, we can see it is working.
Python Code
See the below Python code that accomplishes the same thing using the pandas, io, requests, and time modules. Many people will not have requests or pandas installed by default, so check your package managers if need be.
import pandas as pd import io import requests import time def google_stocks(symbol, startdate = (1, 1, 2005), enddate = None): startdate = str(startdate[0]) + '+' + str(startdate[1]) + '+' + str(startdate[2]) if not enddate: enddate = time.strftime("%m+%d+%Y") else: enddate = str(enddate[0]) + '+' + str(enddate[1]) + '+' + str(enddate[2]) stock_url = "http://www.google.com/finance/historical?q=" + symbol + \ "&startdate=" + startdate + "&enddate=" + enddate + "&output=csv" raw_response = requests.get(stock_url).content stock_data = pd.read_csv(io.StringIO(raw_response.decode('utf-8'))) return stock_data if __name__ == '__main__': apple_data = google_stocks('AAPL') print(apple_data) apple_truncated = google_stocks('AAPL', enddate = (1, 1, 2006)) print(apple_truncated) Running this code, we can see the function works correctly with the dates and fetches the data quickly. Popping it into matplotlib look good, too.
Notes on Data Structure
Unlike the Yahoo! Finance API, this will not return the adjusted close as a separate column. In Automated Trading with R, we go to great lengths to use the adjusted close to obtain adjusted open, adjusted high, and adjusted low. The Google Finance API used here returns all of that information for you. So, there is no column named "adjusted close". All of the data is adjusted in advance.
Advanced users may be disappointed by this, because there is some interesting information regarding splits and dividends to be gained from the disparity between raw and adjusted closing prices. Regardless, most users should be able to use the data returned by this API to accomplish their simulation goals.
Posted by: clearshands.blogspot.com
Source: https://chrisconlan.com/download-historical-stock-data-google-r-python/