Learn R Programming
⚠️There's a newer version (4.3.0) of this package.Take me there.

FedData is an R package implementing functions to automate downloading geospatial data available from several federated data sources (mainly sources maintained by the US Federal government).

FedData version 2.5 will be the final minor CRAN release of FedData 2. FedData 3 will be released in the coming months, but some code built on FedData 2 will not be compatible with FedData 3.

Currently, the package enables extraction from seven datasets:

This package is designed with the large-scale geographic information system (GIS) use-case in mind: cases where the use of dynamic web-services is impractical due to the scale (spatial and/or temporal) of analysis. It functions primarily as a means of downloading tiled or otherwise spatially-defined datasets; additionally, it can preprocess those datasets by extracting data within an area of interest (AoI), defined spatially. It relies heavily on the sp, raster, and rgdal packages.

This package has been built and tested on a source (Homebrew) install of R on macOS 10.14 (High Sierra), and has been successfully run on Ubuntu 14.04.5 LTS (Trusty), Ubuntu 16.04.1 LTS (Xenial) and binary installs of R on Mac OS 10.14 and Windows 10.

Development

Contributors

  • Dylan Beaudette - USDA-NRCS Soil Survey Office, Sonora, CA
  • Scott Chamberlain - ROpenSci and Museum of Paleontology at UC Berkeley

Install FedData

  • From CRAN:

    install.packages('FedData')

  • Development version from GitHub:

    install.packages("devtools")
devtools::install_github("ropensci/FedData")

  • Linux (Ubuntu 14.04.5 or 16.04.1):

    First, in terminal: bash sudo add-apt-repository ppa:ubuntugis/ppa -y sudo apt-get update -q sudo apt-get install libssl-dev libcurl4-openssl-dev netcdf-bin libnetcdf-dev gdal-bin libgdal-dev Then, in R:

    update.packages("survival")
install.packages("devtools")
devtools::install_github("ropensci/FedData")

Demonstration

This demonstration script is available as an R Markdown document in the GitHub repository: https://github.com/ropensci/FedData.

Load FedData and define a study area

# FedData Tester
library(FedData)
library(magrittr)

# Extract data for the Village Ecodynamics Project "VEPIIN" study area:
# http://veparchaeology.org
vepPolygon <- polygon_from_extent(raster::extent(672800, 740000, 4102000, 4170000),
                                  proj4string = "+proj=utm +datum=NAD83 +zone=12")

Get and plot the National Elevation Dataset for the study area

# Get the NED (USA ONLY)
# Returns a raster
NED <- get_ned(template = vepPolygon,
               label = "VEPIIN")
# Plot with raster::plot
raster::plot(NED)

Get and plot the Daymet dataset for the study area

# Get the DAYMET (North America only)
# Returns a raster
DAYMET <- get_daymet(template = vepPolygon,
               label = "VEPIIN",
               elements = c("prcp","tmax"),
               years = 1980:1985)
# Plot with raster::plot
raster::plot(DAYMET$tmax$X1985.10.23)

Get and plot the daily GHCN precipitation data for the study area

# Get the daily GHCN data (GLOBAL)
# Returns a list: the first element is the spatial locations of stations,
# and the second is a list of the stations and their daily data
GHCN.prcp <- get_ghcn_daily(template = vepPolygon, 
                            label = "VEPIIN", 
                            elements = c('prcp'))
# Plot the NED again
raster::plot(NED)
# Plot the spatial locations
sp::plot(GHCN.prcp$spatial,
         pch = 1,
         add = TRUE)
legend('bottomleft',
       pch = 1,
       legend="GHCN Precipitation Records")

Get and plot the daily GHCN temperature data for the study area

# Elements for which you require the same data
# (i.e., minimum and maximum temperature for the same days)
# can be standardized using standardize==T
GHCN.temp <- get_ghcn_daily(template = vepPolygon, 
                            label = "VEPIIN", 
                            elements = c('tmin','tmax'), 
                            years = 1980:1985,
                            standardize = TRUE)
# Plot the NED again
raster::plot(NED)
# Plot the spatial locations
sp::plot(GHCN.temp$spatial,
         add = TRUE,
         pch = 1)
legend('bottomleft',
       pch = 1,
       legend = "GHCN Temperature Records")

Get and plot the National Hydrography Dataset for the study area

# Get the NHD (USA ONLY)
NHD <- get_nhd(template = vepPolygon, 
               label = "VEPIIN")
# Plot the NED again
raster::plot(NED)
# Plot the NHD data
NHD %>%
  lapply(sp::plot,
         col = 'black',
         add = TRUE)

Get and plot the NRCS SSURGO data for the study area

# Get the NRCS SSURGO data (USA ONLY)
SSURGO.VEPIIN <- get_ssurgo(template = vepPolygon, 
                     label = "VEPIIN")
# Plot the NED again
raster::plot(NED)
# Plot the SSURGO mapunit polygons
plot(SSURGO.VEPIIN$spatial,
     lwd = 0.1,
     add = TRUE)

Get and plot the NRCS SSURGO data for particular soil survey areas

# Or, download by Soil Survey Area names
SSURGO.areas <- get_ssurgo(template = c("CO670","CO075"), 
                           label = "CO_TEST")
#> Warning: 36 parsing failures.
#>  row       col           expected                     actual                                                                                file
#> 4561 ponddurcl 1/0/T/F/TRUE/FALSE Very brief (4 to 48 hours) '/Users/bocinsky/git/FedData/EXTRACTIONS/CO_TEST/SSURGO/CO_TEST_SSURGO_comonth.csv'
#> 4561 ponddep.r 1/0/T/F/TRUE/FALSE 10                         '/Users/bocinsky/git/FedData/EXTRACTIONS/CO_TEST/SSURGO/CO_TEST_SSURGO_comonth.csv'
#> 4561 ponddep.h 1/0/T/F/TRUE/FALSE 15                         '/Users/bocinsky/git/FedData/EXTRACTIONS/CO_TEST/SSURGO/CO_TEST_SSURGO_comonth.csv'
#> 4562 ponddurcl 1/0/T/F/TRUE/FALSE Very brief (4 to 48 hours) '/Users/bocinsky/git/FedData/EXTRACTIONS/CO_TEST/SSURGO/CO_TEST_SSURGO_comonth.csv'
#> 4562 ponddep.r 1/0/T/F/TRUE/FALSE 10                         '/Users/bocinsky/git/FedData/EXTRACTIONS/CO_TEST/SSURGO/CO_TEST_SSURGO_comonth.csv'
#> .... ......... .................. .......................... ...................................................................................
#> See problems(...) for more details.

# Let's just look at spatial data for CO675
SSURGO.areas.CO675 <- SSURGO.areas$spatial[SSURGO.areas$spatial$AREASYMBOL=="CO075",]

# And get the NED data under them for pretty plotting
NED.CO675 <- get_ned(template = SSURGO.areas.CO675,
                            label = "SSURGO_CO675")
               
# Plot the SSURGO mapunit polygons, but only for CO675
plot(NED.CO675)
plot(SSURGO.areas.CO675,
     lwd = 0.1,
     add = TRUE)

Get and plot the ITRDB chronology locations in the study area

# Get the ITRDB records
ITRDB <- get_itrdb(template = vepPolygon,
                        label = "VEPIIN",
                        makeSpatial = TRUE)
# Plot the NED again
raster::plot(NED)
# Map the locations of the tree ring chronologies
plot(ITRDB$metadata,
     pch = 1,
     add = TRUE)
legend('bottomleft',
       pch = 1,
       legend = "ITRDB chronologies")

Get and plot the National Land Cover Dataset for the study area

# Get the NLCD (USA ONLY)
# Returns a raster
NLCD <- get_nlcd(template = vepPolygon,
                 year = 2011,
                 dataset = "landcover",
                 label = "VEPIIN")
# Plot with raster::plot
raster::plot(NLCD)


# You can also download the Canopy or impervious datasets:
NLCD_canopy <- get_nlcd(template = vepPolygon,
                 year = 2011,
                 dataset = "canopy",
                 label = "VEPIIN")
# Plot with raster::plot
raster::plot(NLCD_canopy)


NLCD_impervious <- get_nlcd(template = vepPolygon,
                 year = 2011,
                 dataset = "impervious",
                 label = "VEPIIN")
# Plot with raster::plot
raster::plot(NLCD_impervious)

Acknowledgements

This package is a product of SKOPE (Synthesizing Knowledge of Past Environments) and the Village Ecodynamics Project. This software is licensed under the MIT license.

FedData was reviewed for rOpenSci by @jooolia, and was greatly improved as a result. rOpenSci onboarding was coordinated by @sckott.

Copy Link

Version

Install

install.packages('FedData')

Monthly Downloads

1,068

Version

2.5.7

License

MIT + file LICENSE

Issues

2

Pull Requests

0

Stars

101

Forks

22

Maintainer

R. Bocinsky

Last Published

April 22nd, 2019

Functions in FedData (2.5.7)

download_huc4

Download a zipped directory containing a shapefile of the HUC4 subregions of the NHD.
download_ssurgo_inventory

Download a zipped directory containing a shapefile of the SSURGO study areas.
download_ssurgo_study_area

Download a zipped directory containing the spatial and tabular data for a SSURGO study area.
download_nhd_subregion

Download a zipped NHD HUC4 subregion.
download_nlcd_tile

Download a zipped tile from the National Land Cover Database.
get_nhd

Download and crop the National Hydrography Dataset.
get_ghcn_daily

Download and crop the Global Historical Climate Network-Daily data.
get_daymet_tile

Download and crop a netcdf tile from the 1-km DAYMET daily weather dataset.
get_nhd_subregion

Download and crop data from a zipped HUC4 subregion of the National Hydrography Dataset.
get_nlcd

Download and crop the National Land Cover Database.
get_nlcd_tile

Download and crop a tile from the National Land Cover Database.
pkg_test

Install and load a package.
polygon_from_extent

Turn an extent object into a polygon
spdf_from_polygon

Turn a SpatialPolygons object into a SpatialPolygonsDataFrame.
station_to_data_frame

Convert a list of station data to a single data frame.
download_data

Use curl to download a file.
read_crn

Read a Tucson-format chronology file.
download_daymet_tile

Download a netcdf tile from the 1-km DAYMET daily weather dataset.
read_crn_data

Read chronology data from a Tucson-format chronology file.
download_itrdb

Download the latest version of the ITRDB.
download_ned_tile

Download a zipped tile from the 1 (~30 meter) or 1/3 (~10 meter) arc-second National Elevation Dataset.
extract_ssurgo_data

Extract data from a SSURGO database pertaining to a set of mapunits.
get_ghcn_daily_station

Download and extract the daily data for a GHCN weather station.
get_daymet

Download and crop the 1-km DAYMET daily weather dataset.
get_ghcn_inventory

Download and crop the inventory of GHCN stations.
get_ned

Download and crop the 1 (~30 meter) or 1/3 (~10 meter) arc-second National Elevation Dataset.
get_ssurgo_study_area

Download and crop the spatial and tabular data for a SSURGO study area.
nlcd_canopy_pam

The NLCD canopy PAM attributes.
get_huc4

Download and crop a shapefile of the HUC4 regions of the National Hydrography Dataset.
get_ned_tile

Download and crop tile from the 1 (~30 meter) or 1/3 (~10 meter) arc-second National Elevation Dataset.
nlcd_tiles

The NLCD tiles SpatialPolygonsDataFrame.
pal_nlcd

NLCD colour map palettes
get_itrdb

Download the latest version of the ITRDB, and extract given parameters.
get_ssurgo

Download and crop data from the NRCS SSURGO soils database.
get_ssurgo_inventory

Download and crop a shapefile of the SSURGO study areas.
SDA_query

Submit a Soil Data Access (SDA) Query
daymet_tiles

The DAYMET tiles SpatialPolygonsDataFrame.
nlcd_impervious_pam

The NLCD impervious PAM attributes.
read_crn_metadata

Read metadata from a Tucson-format chronology file.
nlcd_landcover_pam

The NLCD landcover PAM attributes.
sequential_duplicated

Get a logical vector of which elements in a vector are sequentially duplicated.
substr_right

Get the rightmost 'n' characters of a character string.
unwrap_rows

Unwraps a matrix and only keep the first n elements.
download_ghcn_daily_station

Download the daily data for a GHCN weather station.