cansim v0.2.3

0

Monthly downloads

0th

Percentile

Accessing Statistics Canada Data Table and Vectors

Searches for, accesses, and retrieves new-format and old-format Statistics Canada data tables, as well as individual vectors, as tidy data frames. This package deals with encoding issues, allows for bilingual English or French language data retrieval, and bundles convenience functions to make it easier to work with retrieved table data. Optional caching features are provided.

Readme

Build Status CRAN_Status_Badge

cansim

An R package to retrieve and work with public Statistics Canada data tables.

This package:

  • Searches for and retrieves data tables and series from Statistics Canada's socioeconomic data repository (previously known as CANSIM)
  • Prepares retrieved data tables as analysis-ready tidy data frames
  • Accepts legacy CANSIM table catalogue numbers
  • Allows for bilingual data retrieval
  • Offers caching of downloaded data for faster loading and less waiting
  • Includes convenience functions for relabelling and rescaling as well as tools for working with data hierarchies in downloaded table objects

Mentioned in Awesome Official Statistics

Documentation

Cansim R package home page and reference guide

Installation

The cansim package is available on CRAN and can be installed directly.

install.packages("cansim")

Alternatively, the latest development version can be downloaded from Github using either the remotes or devtools packages.

# install.packages("remotes")
remotes::install_github("mountainmath/cansim")

Basic Usage

The package acccepts use of both old-format ("051-0013") or new-format ("17-10-0016-01") table catalogue numbers to download entire data tables as tidy data frames. Calling either the legacy CANSIM table number or the new NDM number will load the same data. Since the transition to the new data repository, existing tables will have retained their old-format numbers, but any newly created tables will have only new-format names.

# Retrieve data for births table: 17-10-0016-01 (formerly: CANSIM 051-0013)
births <- get_cansim("051-0013")
births <- get_cansim("17-10-0016-01")

# Retrieve data for balance of payment table 36-10-0042-01 (formerly CANSIM  376-8105)
bop <- get_cansim("3768105")
bop <- get_cansim("36-10-0042")

See more example usage and worfklow in the Getting started with the cansim package vignette.

Caching

Many of the data tables available in Statistics Canada's data repository are quite large in size. After downloading tables, the cansim package will cache data in a temporary directory for the duration of the current R session. This reduces unnecessary waiting when recompiling code. To force a refresh of the data, pass the refresh=TRUE option in the function call.

Bilingual

Statistics Canada data tables are provided in either English or French formats, including labels and formats. The cansim package allows for download of tables in either English or French. There is an optional language argument to retrieve tables in French:

Le paquet cansim fonctionne en anglais ou en français. Il existe un argument de langue optionnel pour récupérer les tables en français:

naissances <- get_cansim("051-0013",language="fr")

Normalizing values

The package also contains a convenience function that will re-scale and re-label variables that are reported in thousands or millions. Statistics Canada data table values may be scaled by powers of 10.

For example, values in the VALUE field may be reported in "millions", so a VALUE of 10 means 10,000,000. The normalize_cansim_values function automatically scales the VALUE field to be a number, so the VALUE will be converted from 10 to 10,000,000 in the example given.

data <- get_cansim("17-10-0079-01") %>% normalize_cansim_values

To retain the original VALUE field pass the replacement_value = <your field name> option to create a field to contain the normalized values.

data <- get_cansim("17-10-0079-01") %>% normalize_cansim_values(replacement_value="normalized value")

By default percentages will be converted to rates, so instead of being 0-100 it will be normalized to 0-1. To change that behaviour set normalize_percent=FALSE.

Vectors

Many of the time-series data available from Statistics Canada have individual vector codes and some users of Canadian statistical data, who are often concerned with specific time series such as the CPI or international arrivals, will typically know the exact series they need. If, for example, you are tracking the Canadian Consumer Price Index (CPI) over time, you might already know the Statistics Canada vector code the seasonally-unadjusted all-items CPI value: v41690973. To retrieve just this data series on its own without all of the additional data available in related tables, we can use the get_cansim_vector() function with the vector code and the date onwards from which we want to get vector results for.

get_cansim_vector("v41690973","2015-01-01")

To access metadata for the vectors, use

get_cansim_vector_info("v41690973")

More detailed usage examples are available in the Retrieving individual Statistics Canada vectors vignette.

Table overview metadata

The get_cansim_table_overview function displays an overview of table information. If the table is not yet downloaded and cached it will first download the table itself. Let's take a look what's in the table we are interested in.

get_cansim_table_overview(36-10-040)

Listing available tables

Calling list_cansim_tables returns a data frame with useful metadata for available tables. There are 21 fields of metadata for each table including title, in English and French, keyword sets, notes, and table numbers.

list_cansim_tables()

The appropriate table can be found by subsetting or filtering on the properties we want to use to find the appropriate tables. This work well with standard dplyr verbs.

list_cansim_tables() %>% 
  filter(grepl("Labour force characteristics",title), grepl("economic region",title)) %>% 
  select("cansim_table_number","title")

As table search functions require a full scrape of Statistics Canada's data repository webpages, generating this list can be quite slow so a saved list of tables is included with the package. As Statistics Canada adds additional tables and data products, the list that comes with the package will become out of date and will require refreshing. Tables can be refreshed by specifying refresh=TRUE when calling list_cansim_tables. The full list of tables can be cached locally to avoid delays and prevent unnecessary web scraping. This can (and should) be enabled by setting options(cansim.cache_path="your cache path") option so that table information is cached across R sessions.

The Listing Statistics Canada data tables vignette has additional detail and examples.

License

The code in this package is licensed under the MIT license. The bundled table metadata in Sysdata.R, as well as all Statistics Canada data retrieved using this package is made available under the Statistics Canada Open Licence Agreement, a copy of which is included in the R folder. The Statistics Canada Open Licence Agreement requires that:

Subject to this agreement, Statistics Canada grants you a worldwide, royalty-free, non-exclusive licence to:

  - use, reproduce, publish, freely distribute, or sell the Information;
  - use, reproduce, publish, freely distribute, or sell Value-added Products; and,
  - sublicence any or all such rights, under terms consistent with this agreement.

In doing any of the above, you shall:

  - reproduce the Information accurately;
  - not use the Information in a way that suggests that Statistics Canada endorses you or your use of the Information;
  - not misrepresent the Information or its source;
  - use the Information in a manner that does not breach or infringe any applicable laws;
  - not merge or link the Information with any other databases for the purpose of attempting to identify an individual person, business or organization; and
  - not present the Information in such a manner that gives the appearance that you may have received, or had access to, information held by Statistics Canada about any identifiable individual person, business or organization.

Attribution

Subject to the Statistics Canada Open Licence Agreement, licensed products using Statistics Canada data should employ the following acknowledgement of source:

Acknowledgment of Source

(a) You shall include and maintain the following notice on all licensed rights of the Information:

  - Source: Statistics Canada, name of product, reference date. Reproduced and distributed on an "as is" basis with the permission of Statistics Canada.

(b) Where any Information is contained within a Value-added Product, you shall include on such Value-added Product the following notice:

  - Adapted from Statistics Canada, name of product, reference date. This does not constitute an endorsement by Statistics Canada of this product.

Why cansim?

CANSIM was the name of Statistics Canada's legacy socio-economic data repository that was widely used by practitioners, academics, and students, with many still calling the new repository by that name. Statistics Canada refers to the current repository simply as "Statistics Canada data" or "StatCan data". We use the CANSIM name for this package as a nostalgic reference.

Proxy issues

Some users have reported issues accessing and downloading Statistics Canada tables while behind a proxy as is sometimes the case in office environments. A quick fix for this requires specifying a proxy configuration for the httr package.

httr::set_config(use_proxy(url=http_proxy, port=selected_port, username=your_username,password=your_pass))

Contributing

Issues and pull requests are highly appreciated.

If you want to get in touch, we are pretty good at responding via email or via twitter at @dshkol or @vb_jens.

  • There exists a CANSIM2R package on CRAN that shares some functionality.

  • CANSIM-dataviewer is another tool that depends on the existing CANSIM2R package with a focus on uses for the Province of British Columbia.

  • cancensus is a package designed to access, retrieve, and work with Canadian Census data and geography. A stable version is available on CRAN and a development version is on Github.

Cite cansim

If you wish to cite the cansim package in your work:

von Bergmann, J., Dmitry Shkolnik (2018). cansim: functions and convenience tools for accessing Statistics Canada data tables. v0.2.0.

A BibTeX entry for LaTeX users is

  @Manual{,
    author = {Jens {von Bergmann} and Dmitry Shkolnik},
    title = {cansim: functions and convenience tools for accessing Statistics Canada data tables},
    year = {2018},
    note = {R package version 0.2.0},
    url = {https://mountainmath.github.io/cansim/},
  }

Functions in cansim

Name Description
get_cansim_table_subject Retrieve Statistics Canada data table subject detail
get_cansim_column_list Retrieve Statistics Canada data table column list
get_cansim_table_survey Retrieve Statistics Canada data table survey detail
view_cansim_webpage View CANSIM table information in browser
get_cansim_table_list Retrieve list of all Statistics Canada data tables
get_cansim_table_list_page An internal method to get page of Statistics Canada data tables
get_cansim Retrieves a Statistics Canada data table using CANSIM code or NDM table number
cansim_old_to_new Translate deprecated CANSIM table number into new NDM-format table catalogue number
adjust_cansim_values_by_variable (Deprecated) Adjust values in a retrieved Statistics Canada data table using a scaling variable
get_cansim_ndm Retrieve a Statistics Canada data table using NDM catalogue number
get_cansim_changed_tables Retrieve a list of modified tables since a given date
get_cansim_table_url Retrieve a Statistics Canada data table URL given a table number
get_cansim_table_info Retrieve Statistics Canada data table information
get_cansim_data_for_table_coord_periods Retrieve data for specified Statistics Canada data product for last N periods for specific coordinates
get_cansim_cube_metadata Retrieve table metadata from Statistics Canada API
parse_and_fold_in_metadata Parse metadata and fold into data table
get_cansim_vector_for_latest_periods Retrieve data for specified Statistics Canada data vector(s) for last N periods
get_cansim_table_notes Retrieve Statistics Canada data table notes
search_cansim_tables Search through Statistics Canada data tables
get_cansim_vector Retrieve data for a Statistics Canada data vector released within a given time frame
get_cansim_vector_info Retrieve metadata for specified Statistics Canada data vectors
list_cansim_tables Get overview list for all Statistics Canada data tables
get_cansim_table_overview Retrieve Statistics Canada data table overview text
normalize_cansim_values Normalize retrieved data table values to appropriate scales
categories_for_level Use metadata to extract categories for column of specific level
get_cansim_column_categories Retrieve Statistics Canada data table categories for a specific column
correspondence The correspondence file for old to new StatCan table numbers is included in the package
No Results!

Vignettes of cansim

Name
cansim.Rmd
No Results!

Last month downloads

Details

Type Package
License MIT + file LICENSE
Encoding UTF-8
LazyData true
RoxygenNote 6.1.1
URL https://github.com/mountainMath/cansim, https://mountainmath.github.io/cansim/
BugReports https://github.com/mountainMath/cansim/issues
VignetteBuilder knitr
Language en-CA
NeedsCompilation no
Packaged 2019-01-06 13:27:20 UTC; jens
Repository CRAN
Date/Publication 2019-01-07 15:00:03 UTC

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/cansim)](http://www.rdocumentation.org/packages/cansim)