curl (version 2.3)

curl: Curl connection interface

Description

Drop-in replacement for base url that supports https, ftps, gzip, deflate, etc. Default behavior is identical to url, but request can be fully configured by passing a custom handle.

Usage

curl(url = "http://httpbin.org/get", open = "", handle = new_handle())

Arguments

url
character string. See examples.
open
character string. How to open the connection if it should be opened initially. Currently only "r" and "rb" are supported.
handle
a curl handle object

Details

As of version 2.3 curl connections support open(con, blocking = FALSE). In this case readBin and readLines will return immediately with data that is available without waiting. For such non-blocking connections the caller needs to call isIncomplete to check if the download has completed yet. Non-blocking connections do raise an error for non-successful HTTP status; the caller can check this via handle_data.

Examples

Run this code
## Not run: 
# con <- curl("https://httpbin.org/get")
# readLines(con)
# 
# # Auto-opened connections can be recycled
# open(con, "rb")
# bin <- readBin(con, raw(), 999)
# close(con)
# rawToChar(bin)
# 
# # HTTP error
# curl("https://httpbin.org/status/418", "r")
# 
# # Follow redirects
# readLines(curl("https://httpbin.org/redirect/3"))
# 
# # Error after redirect
# curl("https://httpbin.org/redirect-to?url=http://httpbin.org/status/418", "r")
# 
# # Auto decompress Accept-Encoding: gzip / deflate (rfc2616 #14.3)
# readLines(curl("http://httpbin.org/gzip"))
# readLines(curl("http://httpbin.org/deflate"))
# 
# # Binary support
# buf <- readBin(curl("http://httpbin.org/bytes/98765", "rb"), raw(), 1e5)
# length(buf)
# 
# # Read file from disk
# test <- paste0("file://", system.file("DESCRIPTION"))
# readLines(curl(test))
# 
# # Other protocols
# read.csv(curl("ftp://cran.r-project.org/pub/R/CRAN_mirrors.csv"))
# readLines(curl("ftps://test.rebex.net:990/readme.txt"))
# readLines(curl("gopher://quux.org/1"))
# 
# # Streaming data
# con <- curl("http://jeroenooms.github.io/data/diamonds.json", "r")
# while(length(x <- readLines(con, n = 5))){
#   print(x)
# }
# 
# # Stream large dataset over https with gzip
# library(jsonlite)
# con <- gzcon(curl("https://jeroenooms.github.io/data/nycflights13.json.gz"))
# nycflights <- stream_in(con)
# ## End(Not run)

Run the code above in your browser using DataCamp Workspace