Learn R Programming

dataone: R interface to the DataONE network of data repositories

Provides read and write access to data and metadata from the global DataONE network of data repositories, including the KNB Data Repository, Dryad, and the NSF Arctic Data Center and dozens of other data repositories from federal and state agencies and non-profit organizations. Each DataONE repository implements a consistent repository application programming interface. Users call methods in R to access these remote repository functions, such as methods to query the metadata catalog, get access to metadata for particular data packages, and read the data objects from the data repository using the global identifier for each data object. Users can also insert and update data objects on repositories that support these methods. For more details, see the vignettes.

Installation Notes

Version 2.0 of the dataone R package removes the dependency on rJava and significantly changes the base API to correspond to the published DataONE API. Previous methods for accessing DataONE will be maintained, but new methods have been added.

The dataone R package requires the R package redland. If you are installing on Ubuntu then the Redland C libraries must be installed first. If you are installing on Mac OS X or Windows then installing these libraries is not required.

Installing on Mac OS X

On Mac OS X dataone can be installed with the following commands:

install.packages("dataone")
library(dataone)

The dataone R package should be available for use at this point.

Installing on Ubuntu

For ubuntu, install the required Redland C libraries by entering the following commands in a terminal window:

sudo apt-get update
sudo apt-get install librdf0 librdf0-dev

Then install the R packages from the R console:

install.packages("dataone")
library(dataone)

The dataone R package should be available for use at this point

Installing on Windows

For windows, the required redland R package is distributed as a binary release, so it is not necessary to install any additional system libraries.

To install the dataone R packages from the R console:

install.packages("dataone")
library(dataone)

The dataone R package should be available for use at this point.

Windows issues with TLS 1.3

Some users report problems with curl on Windows recently failing to connect when using TLS 1.3 connections, with the following error:

> library(dataone)
> cn <- CNode("PROD")
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Failure when receiving data from the peer [cn.dataone.org]:
schannel: failed to read data from server: SEC_E_CONTEXT_EXPIRED (0x80090317) - The context has expired and can no longer be used.

This seems to be associated with changes to the Schannel SSL backend on Windows in how it handles requests for client-side x509 certificates under TLS1.3, which no longer functions properly under Windows. Switching to using the OpenSSL backend (as described in issue #308) seems to fix the problem but is probably not a great long-term solution. Here's a workaround by setting an environment variable to tell curl to use the OpenSSL backend:

write('CURL_SSL_BACKEND=openssl', file = "~/.Renviron", append = TRUE)

Restart the R session and verify that OpenSSL is now active (no longer in parenthesis):

> curl::curl_version()$ssl_version
Initiating curl with CURL_SSL_BACKEND: openssl
[1] "OpenSSL/3.5.0 (Schannel)"

Now, try the code again and it should work:

> library(dataone)
> cn <- CNode("PROD")

Quick Start

See the full manual (help(dataone)) for documentation.

To search the DataONE Federation Member Node Knowledge Network for Biocomplexity (KNB) for a dataset:

library(dataone)
cn <- CNode("PROD")
mn <- getMNode(cn, "urn:node:KNB")
mySearchTerms <- list(q="abstract:salmon+AND+keywords:spawn+AND+keywords:chinook",
                      fl="id,title,dateUploaded,abstract,size",
                      fq="dateUploaded:[2017-06-01T00:00:00.000Z TO 2017-07-01T00:00:00.000Z]",
                      sort="dateUploaded+desc")
result <- query(mn, solrQuery=mySearchTerms, as="data.frame")
result[1,c("id", "title")]
id <- result[1,'id']

The metadata file that describes the located research can be downloaded and viewed in an XML viewer, text editor after being written to disk, or in R via the commands below:

library(XML)
metadata <- rawToChar(getObject(mn, id))
doc <- xmlRoot(xmlTreeParse(metadata, asText=TRUE, trim = TRUE, ignoreBlanks = TRUE))
tf <- tempfile()
saveXML(doc, tf)
file.show(tf)

This metadata file describes a data file (CSV) in this data collection (package) that can be obtained using the listed identifier, using the commands:

dataRaw <- getObject(mn, "urn:uuid:49d7a4bc-e4c9-4609-b9a7-9033faf575e0")
dataChar <- rawToChar(dataRaw)
theData <- textConnection(dataChar)
df <- read.csv(theData, stringsAsFactors=FALSE)
df[1,]

Uploading a CSV file to a DataONE Member Node requires user authentication. DataONE user authentication is described in the vignette dataone-federation.

Once the authentication steps have been followed, uploading is done with:

library(datapack)
library(uuid)
d1c <- D1Client("STAGING", "urn:node:mnStageUCSB2")
id <- paste("urn:uuid:", UUIDgenerate(), sep="")
testdf <- data.frame(x=1:10,y=11:20)
csvfile <- paste(tempfile(), ".csv", sep="")
write.csv(testdf, csvfile, row.names=FALSE)
# Build a DataObject containing the csv, and upload it to the Member Node
d1Object <- new("DataObject", id, format="text/csv", filename=csvfile)
uploadDataObject(d1c, d1Object, public=TRUE)

In addition, a collection of science metadata and data can be downloaded with one command, for example:

d1c <- D1Client("PROD", "urn:node:KNB")
pkg <- getDataPackage(d1c, id="urn:uuid:04cd34fd-25d4-447f-ab6e-73a572c5d383", quiet=FALSE)

See the R vignette dataone R Package for more information.

Acknowledgments

Work on this package was supported by:

  • NSF-ABI grant #1262458 to C. Gries, M. B. Jones, and S. Collins.
  • NSF-DATANET grants #0830944 and #1430508 to W. Michener, M. B. Jones, D. Vieglais, S. Allard and P. Cruse
  • NSF DIBBS grant #1443062 to T. Habermann and M. B. Jones
  • NSF-PLR grant #1546024 to M. B. Jones, S. Baker-Yeboah, J. Dozier, M. Schildhauer, and A. Budden
  • NSF-PLR grant #2042102 to M. B. Jones, A. Budden, J. Dozier, and M. Schildhauer

Additional support was provided for working group collaboration by the National Center for Ecological Analysis and Synthesis, a Center funded by the University of California, Santa Barbara, and the State of California.

Copy Link

Version

Install

install.packages('dataone')

Monthly Downloads

1,295

Version

2.3.0

License

Apache License 2.0

Issues

67

Pull Requests

0

Stars

35

Forks

19

Maintainer

Matthew B Jones

Last Published

December 10th, 2025

Functions in dataone (2.3.0)

MNode-class

Provides R API to DataONE Member Node services.
MNode

Create a MNode object representing a DataONE Member Node repository.
D1Node-class

A base class for CNode and MNode.
initialize,D1Node-method

Initialize a D1Node
addData,DataPackage,D1Object-method

Add a D1Object containing a data object to a DataPackage
archive

Archive an object on a Member Node or Coordinating Node, which hides it from casual searches.
D1Node

Create a D1Node object.
D1Object-class

D1Object (Defunct) is a representation of a DataObject.
auth_put

PUT a resource with authenticated credentials.
data.formatFamily

Data Format
auth_post

POST a resource with authenticated credentials.
data.tableAttributeNames

returns the attribute names
createD1Object

Create the Object in the DataONE System
canRead,D1Object-method

Test whether the provided subject can read an object.
convert.csv

Convert a DataFrame to Standard CSV.
auth_put_post_delete

POST, PUT, or DELETE a resource with authenticated credentials.
auth_get

GET a resource with authenticated credentials if available.
auth_head

Send a http HEAD request for a resource with authenticated credentials if available.
data.tableAttributeOrientation

The Attribute (Header) Orientation
data.tableQuoteCharacter

Quote Character
data.tableSkipLinesHeader

Number of lines to skip before reading data
data.tableMissingValueCodes

returns missing value codes
data.tableAttributeStorageTypes

returns the attributes' data storage types
d1IdentifierSearch

Query the DataONE Solr endpoint of the Coordinating Node.
documented.entityNames

Get the entity names associated with each table
data.characterEncoding

CharacterEncoding
d1_errors

This function parses a DataONE service response message for errors, and extracts and prints error information.
getAuthExpires

Get the expiration date of the current authentication method.
downloadCert

Open the CILogon Certificate download page in the default browser.
downloadObject

Download an object from the DataONE Federation to Disk.
createDataPackage

Create a DataPackage on a DataONE Member Node
createObject

Create an object on a Member Node.
asDataFrame

return the D1Object data as a data.frame.
auth_delete

DELETE a resource with authenticated credentials.
echoCredentials

Echo the credentials used to make the call.
dataone-deprecated

Deprecated
d1SolrQuery

A method to query the DataONE solr endpoint of the Coordinating Node.
documented.sizes

Get the sizes of the described data tables.
getCapabilities

Get the node capabilities description, and store the information in the MNode.
getCert

Get the DataONE X.509 Certificate location.
dataone-defunct

Defunct
describeObject

Efficiently get systemmetadata for an object.
getD1Object

Download a data object from the DataONE Federation.
getData,D1Object-method

Get the data content of a D1Object.
encodeUrlQuery

Encode the Input for a URL Query Segment.
evaluateAuth

Evaluate DataONE authentication.
getCertLocation

Get the file path on disk of the client certificate file.
getChecksum

Get the checksum for the data object associated with the specified pid.
getCertInfo

Get X.509 Certificate information
getCertExpires

Show the date and time when an X.509 certificate expires.
documented.d1Identifiers

Get DataONE identifiers
getPackage

Download a data package from a member node.
getQueryEngineDescription

Query a node for the list of query engines available on the node
dataone

Search, download and upload data to the DataONE network.
encodeUrlPath

Encode the Input for a URL Path Segment.
data.tableAttributeTypes

returns the attributes' data types
listFormats

List all object formats registered in DataONE.
getDataObject

Download a file (and it's associated system metadata) from the DataONE Federation as a DataObject.
getAuthMethod

Get the current valid authentication mechanism.
getDataPackage

Download data from the DataONE Federation as a DataPackage.
getMNode

Get a reference to a node based on its identifier
getMNodeId

Get the member node identifier associated with this D1Client object..
data.tableFieldDelimiter

Field Delimiter
getCN

Get the coordinating node associated with this D1Client object.
encodeSolr

Encode the input for Solr Queries
getAuthSubject

Get the authentication subject.
listMemberNodes

List DataONE Member Nodes.
obscureAuth

Temporarily disable DataONE authentication.
isAuthExpired

Check if the currently valid authentication method has reached the expiration time.
hasReservation

Checks to determine if the supplied subject is the owner of the reservation of id.
listQueryEngines

Query a node for the list of query engines available on the node
getObject

Get the bytes associated with an object on this Node.
getEndpoint

Return the URL endpoint for the DataONE Coordinating Node.
getMetadataMember

Get the DataObject containing package metadata
get_user_agent

User agent string
getErrorDescription

Extract an error message from an http response.
isAuthValid

Verify authentication for a member node.
getTokenInfo

Get authentication token information
reserveIdentifier

Reserve a identifier that is unique in the DataONE network.
query

Search DataONE for data and metadata objects
isAuthorized

Check if an action is authorized for the specified identifier
updateObject

Update an object on a Member Node, by creating a new object that replaces an original.
getSystemMetadata

Get the metadata describing system properties associated with an object on this Node.
resolve

Get a list of coordinating nodes holding a given pid.
getToken

Get the value of the DataONE Authentication Token, if one exists.
updateSystemMetadata

Update the system metadata associated with an object.
parseSolrResult

Parse Solr output into an R list
ping

Test if a node is online and accepting DataONE requests
obscureCert

Obscure the CILogon Client Certificate
parseCapabilities

Construct a Node, using a passed in capabilities XML
setMNodeId

Set the member node identifier to be associated with the D1Client object.
is_windows

Is the OS Windows?
restoreCert

Restore the CILogon client certificate by renaming it to its original location
isCertExpired

Determine if an X.509 certificate has expired.
getFormatId,D1Object-method

Get the FormatId of the D1Object
generateIdentifier

Get a unique identifier that is generated by the Member Node repository and guaranteed to be unique.
getFormat

Get information for a single DataONE object format
showAuth

Display all authentication information
listObjects

Retrieve the list of objects that match the search parameters
getMN

Get a member node client based on its node identifier.
setObsoletedBy

Set a pid as being obsoleted by another pid
showClientSubject

Get DataONE Identity as Stored in the CILogon Certificate.
getIdentifier,D1Object-method

Get the Identifier of the D1Object
listNodes

Get the list of nodes associated with a CN
restoreAuth

Restore authentication (after being disabled with obscureAuth).
setPublicAccess,D1Object-method

Make the object publicly readable.
uploadDataPackage

Upload a DataPackage to a DataONE member node.
uploadDataObject

Upload a DataObject to a DataONE member node.
AuthenticationManager-class

Manage DataONE authentication.
initialize,D1Client-method

Initialize a D1Client object
CNode-class

Provides R API to DataONE Coordinating Node services.
CNode

Create a CNode object.
AbstractTableDescriber-class

Base Class for Specific Metadata Parsers
CertificateManager

Create a CertificateManager object
D1Client

The DataONE client class used to download, update and search for data in the DataONE network.
AuthenticationManager

Create an AuthenticationManager object
CertificateManager-class

CertficateManager provides mechanisms to obtain, load, verify, and display X509 certificates.
D1Client-class

The D1Client class contains methods that perform high level DataONE tasks
D1Object

Create a D1Object instance.
initialize,D1Object-method

Initialize a D1Object
EMLParser

Construct an EML parser object.
EMLParser-class

Handler for Parsing Table Format Details from Metadata