getMissingScrape

From CHCN v1.5 by Steven Mosher

A function to find which scrapes need to be done

During the course of scraping the server sometimes loses the connection or returns an empty file. When that happens the scrapeToCsv should be run again to complete the scrape. That function takes a sequence of files that have not been downloaded. To figure out the missing files, call getMissingScrape

Keywords: files

Usage

getMissingScrape(monthlyList = MONTHLY.STATION.LIST, directory = "EnvCanada")

Arguments

monthlyList: The file name of the list of stations that report monthly. This is created by calling createMonthlyStations
directory: the default directory where scrapes are stored. EnvCanada.

Details

The monthly list of stations contains all the stations that report monthly, numbered from 99111111 upwards. As files are scraped then are downloaded to EnvCanada. If the scrape should fail, or if you want to do it in bits and peices you can find out which files are missing by comparing the master station list with the directory of EnvCanada. This function makes that easy and uses the file names to determin which elements of the monthly station list are missing.

Value

into the monthly station list. That list has 7676 stations. If elements 52,78,954, and 3215 do not have their associated files in envCanada, then those files can be scraped by calling scrapeToCsv and passing the sequence of elements to that function

Examples

 ## Not run: 
#    missing <- getMissingScrape()
#    scrapeToCsv(Stations,get=missing)
#  ## End(Not run)

Documentation reproduced from package CHCN, version 1.5, License: GPL (>= 2)

Community examples

Looks like there are no examples yet.