jsonlite (version 1.2)

rbind.pages: Combine pages into a single data frame

Description

The rbind.pages function is used to combine a list of data frames into a single data frame. This is often needed when working with a JSON API that limits the amount of data per request. If we need more data than what fits in a single request, we need to perform multiple requests that each retrieve a fragment of data, not unlike pages in a book. In practice this is often implemented using a page parameter in the API. The rbind.pages function can be used to combine these pages back into a single dataset.

Usage

"rbind"(pages)

Arguments

pages

a list of data frames, each representing a page of data

Details

The rbind.pages function generalizes base::rbind and plyr::rbind.fill with added support for nested data frames. Not each column has to be present in each of the individual data frames; missing columns will be filled up in NA values.

Examples

# Basic example
x <- data.frame(foo = rnorm(3), bar = c(TRUE, FALSE, TRUE))
y <- data.frame(foo = rnorm(2), col = c("blue", "red"))
rbind.pages(list(x, y))

## Not run: 
# baseurl <- "http://projects.propublica.org/nonprofits/api/v1/search.json"
# pages <- list()
# for(i in 0:20){
#   mydata <- fromJSON(paste0(baseurl, "?order=revenue&sort_order=desc&page=", i))
#   message("Retrieving page ", i)
#   pages[[i+1]] <- mydata$filings
# }
# filings <- rbind.pages(pages)
# nrow(filings)
# colnames(filings)
# ## End(Not run)