hipread (version 0.2.1)

hipread_long_yield: Read a hierarchical fixed width data file, in yields

Description

Enhances hipread_long() or hipread_list() to allow you to read hierarchical data in pieces (called 'yields') and allow your code to have full control between reading pieces, allowing for more freedom than the 'callback' method introduced in the chunk functions (like hipread_long_chunked()).

Usage

hipread_long_yield(file, var_info, rt_info = hip_rt(1, 0),
  compression = NULL, skip = 0, encoding = "UTF-8")
hipread_list_yield(file, var_info, rt_info = hip_rt(1, 0),
  compression = NULL, skip = 0, encoding = "UTF-8")
HipLongYield
HipListYield

Arguments

file

A filename

var_info

Variable information, specified by either hip_fwf_positions() or hip_fwf_widths(). For hierarchical data files, there should be a named list, where the name is the value indicated by the record type variable and there is one variable information per record type.

rt_info

A record type information object, created by hip_rt(), which contains information about the location of the record type variable that defines the record type for each observation. The default contains width 0, which indicates that there the data is rectangular and does not have a record type variable.

compression

If NULL, guesses the compression from the file extension (if extension is "gz" uses gzip, otherwise treats as plain text), can specify it with a string ("txt" indicates plain text and "gz" for gzip).

skip

Number of lines to skip at the start of the data (defaults to 0).

encoding

(Defaults to UTF-8) A string indicating what encoding to use when reading the data, but like readr, the data will always be converted to UTF-8 once it is imported. Note that UTF-16 and UTF-32 are not supported for non-character columns.

Value

A HipYield R6 object (See 'Details' for more information)

Format

An object of class R6ClassGenerator of length 24.

Details

These functions return a HipYield R6 object which have the following methods:

• yield(n = 10000) A function to read the next 'yield' from the data, returns a tbl_df (or list of tbl_df for hipread_list_yield()) with up to n rows (it will return NULL if no rows are left, or all available ones if less than n are available).

• reset() A function to reset the data so that the next yield will read data from the start.

• is_done() A function that returns whether the file has been completely read yet or not.

• cur_pos A property that contains the next row number that will be read (1-indexed).

Examples

# NOT RUN {
library(hipread)
data <- hipread_long_yield(
  hipread_example("test-basic.dat"),
  list(
    H = hip_fwf_positions(
      c(1, 2, 5, 8),
      c(1, 4, 7, 10),
      c("rt", "hhnum", "hh_char", "hh_dbl"),
      c("c", "i", "c", "d")
    ),
    P = hip_fwf_widths(
      c(1, 3, 1, 3, 1),
      c("rt", "hhnum",  "pernum", "per_dbl", "per_mix"),
      c("c", "i", "i", "d", "c")
    )
  ),
  hip_rt(1, 1)
)
# Read the first 4 rows
data$yield(4)

# Read the next 2 rows
data$yield(2)

# Reset and then read the first 4 rows again
data$reset()
data$yield(4)
# }