rdwd (version 0.10.2)

indexFTP: Create a recursive index of the DWD CDC FTP Server

Description

Create a list of all the files (in subfolders) at the Climate Data Center (CDC) FTP-Server from the German Weather Service (DWD, Deutscher WetterDienst) at ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate. The R package RCurl must be available to do this. If RCurl::getURL fails, usually because bot access is detected and denied, there will still be an output which you can pass in a second run via folder to extract the remaining dirs. You might want to wait a bit and set sleep to a higher value in that case. Here's an example: gridindex <- indexFTP("grids_germany","ftp://ftp-cdc.dwd.de/pub/CDC") gridindex <- indexFTP(gridindex,"ftp://ftp-cdc.dwd.de/pub/CDC", sleep=1)

Usage

indexFTP(folder = "currentfindex",
  base = "ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate",
  sleep = 0, dir = "DWDdata", filename = folder[1], overwrite = FALSE,
  quiet = FALSE, progbar = !quiet, verbose = FALSE)

Arguments

folder

Folder(s) to be indexed recursively, e.g. "/hourly/wind/". Use folder="" to search at the location of base itself. If codefolder is "currentfindex" (the default) and base is the default, codefolder is changed to all folders in current fileIndex: unique(dirname(fileIndex$path)). DEFAULT: "currentfindex"

base

Main directory of DWD ftp server, defaulting to observed climatic records. DEFAULT: ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/climate

sleep

If not 0, a random number of seconds between 0 and sleep is passed to Sys.sleep after each read folder to avoid getting kicked off the FTP-Server. DEFAULT: 0

dir

Writeable directory name where to save the downloaded file. Created if not existent. DEFAULT: "DWDdata" at current getwd()

filename

Character: Part of output filename. "INDEX_of_DWD_" is prepended, "/" replaced with "_", ".txt" appended. DEFAULT: folder[1]

overwrite

Logical: Overwrite existing file? If not, "_n" is added to the filename, see berryFunctions::newFilename. DEFAULT: FALSE

quiet

Suppress progbars and message about directory/files? DEFAULT: FALSE

progbar

Logical: present a progress bar in each level? Only works if the R package pbapply is available. DEFAULT: TRUE

verbose

Logical: write a lot of messages from RCurl::getURL? DEFAULT: FALSE (usually, you dont need all the curl information)

Value

currently a vector with file paths (output may change in the future)

Details

It's not suggested to run this for all folders, as it can take quite some time and you may get kicked off the FTP-Server. This package contains an index of the climatic observations at weather stations: View(rdwd:::fileIndex) If it is out of date, please let me know!

Examples

# NOT RUN {
 ## Needs internet connection
sol <- indexFTP(folder="/daily/solar")
head(sol)

mon <- indexFTP(folder="/monthly/kl", verbose=TRUE)
# }
# NOT RUN {
# }