Rnightlights (version 0.2.4)

processNlData: Downloads nightlight tiles and country polygons and calls the function to process them

Description

Downloads nightlight tiles and country polygons in preparation for the appropriate functions to process them. Given a list of countries and nlPeriods and an nlType, processNlData will first determine which countries and periods do not actually have data i.e. have not been previously processed. From the list of countries and prediods that need processing, it determines which nightlight tiles require to be downloaded. At the same time, it downloads any country polygons which have not already been downloaded.

Usage

processNlData(ctryCodes, admLevels, nlTypes, configNames,
  multiTileStrategy = pkgOptions("multiTileStrategy"),
  multiTileMergeFun = pkgOptions("multiTileMergeFun"),
  removeGasFlares = pkgOptions("removeGasFlares"), nlPeriods,
  nlStats = pkgOptions("nlStats"), custPolyPath = NULL,
  gadmVersion = pkgOptions("gadmVersion"),
  gadmPolyType = pkgOptions("gadmPolyType"),
  downloadMethod = pkgOptions("downloadMethod"),
  cropMaskMethod = pkgOptions("cropMaskMethod"),
  extractMethod = pkgOptions("extractMethod"))

Arguments

ctryCodes

the list of countries to be processed

admLevels

the list of admin levels in the given countries at which to calculate stats

nlTypes

the types of nightlights to process

configNames

character the types of raster being processed

multiTileStrategy

character How to handle multiple tiles per nlPeriod

multiTileMergeFun

character The function to use to merge tiles

removeGasFlares

logical Whether to perform gas flare removal pre-processing

nlPeriods

the nlPeriods of interest

nlStats

the statistics to calculate. If not provided will calculate the stats specified in pkgOptions("nlStats")

custPolyPath

Alternative to GADM. A path to a custom shapefile zip

gadmVersion

The GADM version to use

gadmPolyType

The format of polygons to download from GADM

downloadMethod

The method used to download rasters and polygons

cropMaskMethod

The method used to crop and mask satellite rasters

extractMethod

The method used to extract and perform functions on raster data

Value

None

Details

processNlData then calls processNlCountry with the nlType supplied as a parameter. The processing is essentially the same for all nlTypes.

This is the main entry-point to the package and the main user-facing function. However, it works in batch mode and caches all data without returning any data to the user. However, it will return TRUE/FALSE depending on whether it completed successfully. Since it saves state regularly it can be run multiply in case of failure until it finally returns TRUE. This is where the only constraints are downloading and processing errors due to bandwidth or other resource constraints. A good example is running a long-running batch on an AWS EC2 spot-priced machine where the server may be decommissioned at any time. See more in the examples.

Examples

# NOT RUN {
#long running examples which may require large downloads
# }
# NOT RUN {
#Example 1: process monthly VIIRS nightlights for all countries at the
    lowest admin level and for all nlPeriods available e.g. to create a 
    local cache or repo
    
    processNlData() #process VIIRS nightlights for all countries and all periods

#Example 2: process monthly VIIRS nightlights for all countries in 2014 only

    nlPeriods <- getAllNlPeriods("VIIRS.M") #get a list of all nightlight periods to present-day

    nlPeriods <- nlPeriods[grep("^2014", nlPeriods)] #filter only periods in 2014

    processNlData(nlTypes="VIIRS.M", nlPeriods=nlPeriods)

#Example 3: process OLS nightlights for countries KEN & RWA from 1992
#     to 2000

    cCodes <- c("KEN", "RWA")

    nlPeriods <- getAllNlPeriods("VIIRS.M")

    nlPeriods <- nlRange("1992", "2000", "OLS.Y")

    processNlData(ctryCodes=cCodes, nlPeriods=nlPeriods)

#Example 4: process VIIRS nightlights for countries KEN & RWA in 2014 Oct to 2014 Dec only

    processNlData(ctryCodes=c("KEN", "RWA"), nlTypes="VIIRS.M", 
        nlPeriods=c("201410", "201411", "201412"))
    
#Example 5: process all nightlights, all countries, all stats in one thread

   processNlData() 
   
#Example 6: process all VIIRS monthly nightlights, all countries, all stats with each
#   year in a separate thread. Create a separate R script for each year as follows:

    library(Rnightlights)

    nlPeriods <- getAllNlPeriods("VIIRS.M")

    nlPeriods_2014 <- nlPeriods[grep("^2014", nlPeriods)]

    processNlData(nlPeriods=nlPeriods_2014)

    #Run the script from the command line as:
    
    #R CMD BATCH script_name_2014.R
    
# }