Rnightlights (version 0.1.2)

processNlData: Downloads nightlight tiles and country polygons and calls the function to process them

Description

Downloads nightlight tiles and country polygons in preparation for the appropriate functions to process them. Given a list of countries and nlPeriods and an nlType, processNlData will first determine which countries and periods do not actually have data i.e. have not been previously processed. From the list of countries and prediods that need processing, it determines which nightlight tiles require to be downloaded. At the same time, it downloads any country polygons which have not already been downloaded.

Usage

processNlData(ctryCodes = getAllNlCtryCodes("all"),
  nlPeriods = getAllNlPeriods(nlType), nlType = "VIIRS",
  stats = pkgOptions("stats"))

Arguments

ctryCodes

the list of countries to be processed

nlPeriods

the nlPeriods of interest

nlType

the type of nightlights to process i.e. "OLS" or "VIIRS". Default "VIIRS"

stats

the statistics to calculate. If not provided will calculate the stats specified in pkgOptions("stats")

Value

None

Details

processNlData then calls processNlCountry with the nlType supplied as a parameter. The processing is essentially the same for all nlTypes.

This is the main entry-point to the package and the main user-facing function. However, it works in batch mode and caches all data without returning any data to the user. However, it will return TRUE/FALSE depending on whether it completed successfully. Since it saves state regularly it can be run multiply in case of failure until it finally returns TRUE. This is where the only constraints are downloading and processing errors due to bandwidth or other resource constraints. A good example is running a long-running batch on an AWS EC2 spot-priced machine where the server may be decommissioned at any time. See more in the examples.

Examples

# NOT RUN {
#Example 1: process VIIRS nightlights for all countries and all periods available e.g. to create 
    #a local cache or repo
    
    #Recommend running nlInit() to improve performance. It stores some global variables 
    #so that they do not have to be re-evaluated multiply
    nlInit() 
    
    processNlData() #process VIIRS nightlights for all countries and all periods

#Example 2: process nightlights for all countries in 2012 only
    
    nlInit() #for performance. See Example 1

    nlPeriods <- getAllNlYears("VIIRS") #get a list of all nightlight periods to present-day

    nlPeriods <- nlPeriods[grep("^2012", nlPeriods)] #filter only periods in 2012

    processNlData(nlPeriods=nlPeriods)

#Example 3: process VIIRS nightlights for countries KEN & RWA in 2014 Jan to 2014 May only
    
    nlInit()

    cCodes <- c("KEN", "RWA")

    nlPeriods <- getAllNlPeriods("VIIRS")

    nlPeriods <- nlPeriods[grep("^20120[1-5]", nlPeriods)]

    processNlData(ctryCodes=cCodes, nlPeriods=nlPeriods)

#Example 4: process VIIRS nightlights for countries KEN & RWA in 2014 Oct to 2014 Dec only

    processNlData(ctryCodes=c("KEN", "RWA"), nlPeriods=c("201410", "201411", "201412"))
    
#Example 5: process all nightlights, all countries, all stats in one thread

   processNlData() 
   
#Example 6: process all nightlights, all countries, all stats with each
#   year in a separate thread. Create a separate R script for each year as follows:

    library(Rnightlights)

    nlInit()

    nlPeriods <- getAllNlYears("VIIRS")

    nlPeriods_2012 <- nlPeriods[grep("^2012", nlPeriods)]

    processNlData(nlPeriods=nlPeriods_2012)

    #Run the script from the command line as:
    
    #R CMD BATCH script_name_2012.R
    
# }