tfdatasets (version 1.9)

dataset_interleave: Maps map_func across this dataset, and interleaves the results

Description

Maps map_func across this dataset, and interleaves the results

Usage

dataset_interleave(dataset, map_func, cycle_length, block_length = 1)

Arguments

dataset

A dataset

map_func

A function mapping a nested structure of tensors (having shapes and types defined by output_shapes() and output_types() to a dataset.

cycle_length

The number of elements from this dataset that will be processed concurrently.

block_length

The number of consecutive elements to produce from each input element before cycling to another input element.

Details

The cycle_length and block_length arguments control the order in which elements are produced. cycle_length controls the number of input elements that are processed concurrently. In general, this transformation will apply map_func to cycle_length input elements, open iterators on the returned dataset objects, and cycle through them producing block_length consecutive elements from each iterator, and consuming the next input element each time it reaches the end of an iterator.

See Also

Other dataset methods: dataset_batch, dataset_cache, dataset_concatenate, dataset_decode_delim, dataset_filter, dataset_map_and_batch, dataset_map, dataset_padded_batch, dataset_prefetch_to_device, dataset_prefetch, dataset_repeat, dataset_shuffle_and_repeat, dataset_shuffle, dataset_skip, dataset_take

Examples

# NOT RUN {
dataset <- tensor_slices_dataset(c(1,2,3,4,5)) %>%
 dataset_interleave(cycle_length = 2, block_length = 4, function(x) {
   tensors_dataset(x) %>%
     dataset_repeat(6)
 })

# resulting dataset (newlines indicate "block" boundaries):
c(1, 1, 1, 1,
  2, 2, 2, 2,
  1, 1,
  2, 2,
  3, 3, 3, 3,
  4, 4, 4, 4,
  3, 3,
  4, 4,
  5, 5, 5, 5,
  5, 5,
)

# }
# NOT RUN {
# }

Run the code above in your browser using DataCamp Workspace