BOLDconnectR is a package designed for retrieval, transformation and analysis of the data available in the Barcode Of Life Data Systems (BOLD) database. This package provides the functionality to obtain public and private user data available in the database in the Barcode Core Data Model (BCDM) format. Data include information on the taxonomy,geography,collection,identification and DNA barcode sequence of every submission. The manual is currently hosted here (https://github.com/boldsystems-central/BOLDconnectR_examples/blob/main/BOLDconnectR_1.0.0.pdf)

BOLDconnectR requires R version 4.0 or above to function properly. The versions of dependent packages have also been set such that they would work with R >= 4.0. In addition, there are a few suggested packages that are not mandatory for the package to download and install properly, but, are essential for a couple of functions to work. The names and exact versions of the dependencies/suggestions are given here (https://github.com/boldsystems-central/BOLDconnectR/blob/main/DESCRIPTION). More details on Suggested packages provided below.

Installation

The package can be installed using devtools::install_github function from the devtools package in R (which needs to be installed before installing BOLDConnectR).

devtools::install_github("https://github.com/boldsystems-central/BOLDconnectR")

library(BOLDconnectR)

BOLDconnectR has 11 functions currently:

1. bold.fields.info
2. bold.apikey
3. bold.fetch
4. bold.public.search
5. bold.full.search
6. bold.data.summarize
7. bold.analyze.align
8. bold.analyze.tree
9. bold.analyze.diversity
10. bold.analyze.map
11. bold.export

Note on Suggested packages Function 6: bold.data.summarize requires the packages Biostrings to be installed and imported in R session beforehand for generating the barcode_summary. Function 7: bold.analyze.align requires the packages msa and Biostrings to be installed and imported in the R session beforehand. Function 8 also uses the output generated from function 7. msa and Biostrings can be installed using the BiocManager package.

if (!requireNamespace("BiocManager", quietly=TRUE))
install.packages("BiocManager")

BiocManager::install("msa")
BiocManager::install("Biostrings")

library(msa)
library(Biostrings)

Note on API key

The function bold.fetch requires an api key internally to access and download all public + private user data. The API key needed to retrieve BOLD records is found in the BOLD ‘Workbench’ https://bench.boldsystems.org/index.php/Login/page?destination=MAS_Management_UserConsole. After logging in, navigate to ‘Your Name’ (located at the top left-hand side of the window) and click ‘Edit User Preferences’. You can find the API key in the ‘User Data’ section. Please note that to have an API key available in the workbench, a user must have uploaded at least 10,000 records to BOLD. API key can be saved in the R session using bold.apikey() function. Please note that the API keys are regenerated periodically and will be updated in the user's workbench account. Using old keys will result in a HTTP 401 error.

# Substitute ‘00000000-0000-0000-0000-000000000000’ with your key
# bold.apikey(‘00000000-0000-0000-0000-000000000000’)

Basic usage of BOLDConnectR functionality

API key function must be run prior to using the fetch function (Please see above).

Fetch data

BCDM_data<-bold.fetch(get_by = "processid",
                      identifiers = test.data$processid)
#> Initiating download
#>  Downloading data in a single batch  
#> Download complete & BCDM dataframe generated

knitr::kable(head(BCDM_data,4))

Similarly, sampleids or dataset_codes or project_codes can also be used to fetch data. The data can also be filtered on different parameters such as Geography, Attributions and DNA Sequence information using the _filt arguments available in the function

Summarize downloaded data

Downloaded data can then be summarized in different ways. Options currently include a concise summary of all the data, detailed taxonomic counts, data completeness and a barcode-based summary

BCDM_data_summary<-bold.data.summarize(bold_df = BCDM_data,
                               summary_type = "concise_summary")

BCDM_data_summary$concise_summary
#>                        Category   Value
#> 1                 Total_records    1336
#> 2     Total_records_w_sequences    1336
#> 3                Unique_species      80
#> 4                   Unique_BINs     117
#> 5              Unique_countries       1
#> 6             Unique_institutes       6
#> 7          Unique_identified_by       7
#> 8  Unique_specimen_depositories       3
#> 9                Unique_markers  COI-5P
#> 10        Amplicon_length_range 508-658

A concise summary providing a high level overview of the data

Export the downloaded data

Downloaded data can also be exported to the local machine either as a flat file or as a FASTA file for any third party sequence analysis tools.The flat file contents can be modified as per user requirements (entire data or specific presets or individual fields).

# Preset dataframe
# bold.export(bold_df = BCDM_data,
#             export_type = "preset_df",
#             presets = 'taxonomy',
#             cols_for_fas_names = NULL,
#             export = "file_path_with_intended_name.csv")

# Unaligned fasta file
# bold.export(bold_df = BCDM_data,
#             export_type = "fas",
#             cols_for_fas_names = c("bin_uri","genus","species"),
#             export = "file_path_with_intended_name.fas")

Other functions

The package also has functions that provide sequence alignment, NJ clustering, biodiversity analysis and occurrence mapping using the downloaded BCDM data. Additionally, some of these functions also output objects that are commonly used by other R packages (Ex. ‘sf’ dataframe, occurrence matrix for ‘vegan’ and ‘betapart’). Please go through the help manual (Link provided above) for detailed usage of all the functions of BOLDConnectR with examples.

BOLDconnectR can retrieve data very fast (~100k records in a minute on a fast wired connection).

Citation: Padhye SM, Ballesteros-Mejia CL, Agda TJA, Agda JRA, Ratnasingham S. BOLDconnectR: An R package for streamlined retrieval, transformation and analysis of BOLD DNA barcode data (MS in prep).