Learn R
rnoaa (version 0.8.4)

ncdc: Search for and get NOAA NCDC data

Description

Search for and get NOAA NCDC data

Usage

ncdc(datasetid = NULL, datatypeid = NULL, stationid = NULL,
  locationid = NULL, startdate = NULL, enddate = NULL,
  sortfield = NULL, sortorder = NULL, limit = 25, offset = NULL,
  token = NULL, dataset = NULL, datatype = NULL, station = NULL,
  location = NULL, locationtype = NULL, page = NULL, year = NULL,
  month = NULL, day = NULL, includemetadata = TRUE, results = NULL,
  add_units = FALSE, ...)

Arguments

datasetid

(required) Accepts a single valid dataset id. Data returned will be from the dataset specified, see ncdc_datasets

datatypeid

Accepts a valid data type id or a vector or list of data type ids. (optional)

stationid

Accepts a valid station id or a vector or list of station ids

locationid

Accepts a valid location id or a vector or list of location ids (optional)

startdate

(required) Accepts valid ISO formated date (yyyy-mm-dd) or date time (YYYY-MM-DDThh:mm:ss). Data returned will have data after the specified date. The date range must be less than 1 year.

enddate

(required) Accepts valid ISO formated date (yyyy-mm-dd) or date time (YYYY-MM-DDThh:mm:ss). Data returned will have data before the specified date. The date range must be less than 1 year.

sortfield

The field to sort results by. Supports id, name, mindate, maxdate, and datacoverage fields (optional)

sortorder

Which order to sort by, asc or desc. Defaults to asc (optional)

limit

Defaults to 25, limits the number of results in the response. Maximum is 1000 (optional)

offset

Defaults to 0, used to offset the resultlist (optional)

token

This must be a valid token token supplied to you by NCDC's Climate Data Online access token generator. (required) See Authentication section below for more details.

dataset

THIS IS A DEPRECATED ARGUMENT. See datasetid.

datatype

THIS IS A DEPRECATED ARGUMENT. See datatypeid.

station

THIS IS A DEPRECATED ARGUMENT. See stationid.

location

THIS IS A DEPRECATED ARGUMENT. See locationid.

locationtype

THIS IS A DEPRECATED ARGUMENT. There is no equivalent argument in v2 of the NOAA API.

page

THIS IS A DEPRECATED ARGUMENT. There is no equivalent argument in v2 of the NOAA API.

year

THIS IS A DEPRECATED ARGUMENT. Use combination of startdate and enddate arguments.

month

THIS IS A DEPRECATED ARGUMENT. Use combination of startdate and enddate arguments.

day

THIS IS A DEPRECATED ARGUMENT. Use combination of startdate and enddate arguments.

includemetadata

Used to improve response time by preventing the calculation of result metadata. Default: TRUE. This does not affect the return object, in that the named part of the output list called "meta" is still returned, but is NULL. In practice, I haven't seen response time's improve, but perhaps they will for you.

results

THIS IS A DEPRECATED ARGUMENT. See limit.

add_units

(logical) whether to add units information or not. default: FALSE. If TRUE, after getting data from NOAA we add a new column units. See "Adding units" in Details for more

...

Curl options passed on to HttpClient (optional)

Value

An S3 list of length two, a slot of metadata (meta), and a slot for data (data). The meta slot is a list of metadata elements, and the data slot is a data.frame, possibly of length zero if no data is found.

Authentication

Get an API key (aka, token) at http://www.ncdc.noaa.gov/cdo-web/token. You can pass your token in as an argument or store it one of two places:

  • your .Rprofile file with an entry like options(noaakey = "your-noaa-token")

  • your .Renviron file with an entry like NOAA_KEY=your-noaa-token

See Startup for information on how to create/find your .Rrofile and .Renviron files

Flags

The attributes, or "flags", for each row of the output for data may have a flag with it. Each datasetid has it's own set of flags. The following are flag columns, and what they stand for. fl_ is the beginning of each flag column name, then one or more characters to describe the flag, keeping it short to maintain a compact data frame. Some of these fields are the same across datasetids. See the vignette vignette("rnoaa_attributes", "rnoaa") for description of possible values for each flag.

  • fl_c completeness

  • fl_d day

  • fl_m measurement

  • fl_q quality

  • fl_s source

  • fl_t time

  • fl_cmiss consecutive missing

  • fl_miss missing

  • fl_u units

GSOM/GSOY Flags

Note that flags are different for GSOM and GSOY datasets. They have their own set of flags per data class. See system.file("extdata/gsom.json", package = "rnoaa") for GSOM and system.file("extdata/gsom.json", package = "rnoaa") for GSOY. Those are JSON files. The system.file call gives you then path, then read in with jsonlite::fromJSON which will give a data.frame of the metadata. For more detailed info but plain text, open system.file("extdata/gsom_readme.txt", package = "rnoaa") and system.file("extdata/gsoy_readme.txt", package = "rnoaa") in a text editor.

Adding units

The add_units parameter is experimental - USE WITH CAUTION! If add_units=TRUE we pull data from curated lists of data used by matching by datasetid and data type.

We've attempted to gather as much information as possible on the many, many data types across the many different NOAA data sets. However, we may have got some things wrong, so make sure to double check data you get if you do add units.

Get in touch if you find some units that are wrong or missing, and if you are able to help correct information.

Details

Note that NOAA NCDC API calls can take a long time depending on the call. The NOAA API doesn't perform well with very long timespans, and will time out and make you angry - beware.

Keep in mind that three parameters, datasetid, startdate, and enddate are required.

Note that the default limit (no. records returned) is 25. Look at the metadata in $meta to see how many records were found. If more were found than 25, you could set the parameter limit to something higher than 25.

See Also

Other ncdc: ncdc_combine, ncdc_datacats, ncdc_datasets, ncdc_datatypes, ncdc_locs_cats, ncdc_locs, ncdc_plot, ncdc_stations

Examples

Run this code
# NOT RUN {
# GHCN-Daily (or GHCND) data, for a specific station
ncdc(datasetid='GHCND', stationid='GHCND:USW00014895',
   startdate = '2013-10-01', enddate = '2013-12-01')

# GHCND data, for a location by FIPS code
ncdc(datasetid='GHCND', locationid = 'FIPS:02', startdate = '2010-05-01',
   enddate = '2010-05-10')

# GHCND data from October 1 2013 to December 1 2013
ncdc(datasetid='GHCND', startdate = '2013-10-01', enddate = '2013-10-05')

# GHCN-Monthly (or GSOM) data from October 1 2013 to December 1 2013
ncdc(datasetid='GSOM', startdate = '2013-10-01', enddate = '2013-12-01')
ncdc(datasetid='GSOM', startdate = '2013-10-01', enddate = '2013-12-01',
   stationid = "GHCND:AE000041196")

# Normals Daily (or NORMAL_DLY) GHCND:USW00014895 dly-tmax-normal data
ncdc(datasetid='NORMAL_DLY', stationid='GHCND:USW00014895',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Dataset, and location in Australia
ncdc(datasetid='GHCND', locationid='FIPS:AS', startdate = '2010-05-01',
    enddate = '2010-05-31')

# Dataset, location and datatype for PRECIP_HLY data
ncdc(datasetid='PRECIP_HLY', locationid='ZIP:28801', datatypeid='HPCP',
   startdate = '2010-05-01', enddate = '2010-05-10')

# multiple datatypeid's
ncdc(datasetid='PRECIP_HLY', datatypeid = 'HPCP',
   startdate = '2010-05-01', enddate = '2010-05-10')

# multiple locationid's
ncdc(datasetid='PRECIP_HLY', locationid=c("FIPS:30103", "FIPS:30091"),
   startdate = '2010-05-01', enddate = '2010-05-10')

# Dataset, location, station and datatype
ncdc(datasetid='PRECIP_HLY', locationid='ZIP:28801',
   stationid='COOP:310301', datatypeid='HPCP',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Dataset, location, and datatype for GHCND
ncdc(datasetid='GHCND', locationid='FIPS:BR', datatypeid='PRCP',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Normals Daily GHCND dly-tmax-normal data
ncdc(datasetid='NORMAL_DLY', datatypeid='dly-tmax-normal',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Normals Daily GHCND:USW00014895 dly-tmax-normal
ncdc(datasetid='NORMAL_DLY', stationid='GHCND:USW00014895',
   datatypeid='dly-tmax-normal',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Hourly Precipitation data for ZIP code 28801
ncdc(datasetid='PRECIP_HLY', locationid='ZIP:28801', datatypeid='HPCP',
   startdate = '2010-05-01', enddate = '2010-05-10')

# 15 min Precipitation data for ZIP code 28801
ncdc(datasetid='PRECIP_15', datatypeid='QPCP',
   startdate = '2010-05-01', enddate = '2010-05-02')

# Search the NORMAL_HLY dataset
ncdc(datasetid='NORMAL_HLY', stationid = 'GHCND:USW00003812',
   startdate = '2010-05-01', enddate = '2010-05-10')

# Search the GSOY dataset
ncdc(datasetid='ANNUAL', locationid='ZIP:28801', startdate = '2010-05-01',
   enddate = '2010-05-10')

# Search the NORMAL_ANN dataset
ncdc(datasetid='NORMAL_ANN', datatypeid='ANN-DUTR-NORMAL',
   startdate = '2010-01-01', enddate = '2010-01-01')

# Include metadata or not
ncdc(datasetid='GHCND', stationid='GHCND:USW00014895',
   startdate = '2013-10-01', enddate = '2013-12-01')
ncdc(datasetid='GHCND', stationid='GHCND:USW00014895',
   startdate = '2013-10-01', enddate = '2013-12-01', includemetadata=FALSE)

# Many stationid's
stat <- ncdc_stations(startdate = "2000-01-01", enddate = "2016-01-01")
## find out what datasets might be available for these stations
ncdc_datasets(stationid = stat$data$id[10])
## get some data
ncdc(datasetid = "GSOY", stationid = stat$data$id[1:10],
   startdate = "2010-01-01", enddate = "2011-01-01")
# }
# NOT RUN {
# }
# NOT RUN {
# NEXRAD2 data
## doesn't work yet
ncdc(datasetid='NEXRAD2', startdate = '2013-10-01', enddate = '2013-12-01')
# }

Run the code above in your browser using DataLab