tfdatasets (version 1.13.1)

dataset_map_and_batch: Fused implementation of dataset_map() and dataset_batch()

Description

Maps `map_func`` across batch_size consecutive elements of this dataset and then combines them into a batch. Functionally, it is equivalent to map followed by batch. However, by fusing the two transformations together, the implementation can be more efficient.

Usage

dataset_map_and_batch(dataset, map_func, batch_size,
  num_parallel_batches = NULL, drop_remainder = FALSE,
  num_parallel_calls = NULL)

Arguments

dataset

A dataset

map_func

A function mapping a nested structure of tensors (having shapes and types defined by output_shapes() and output_types() to another nested structure of tensors.

batch_size

An integer, representing the number of consecutive elements of this dataset to combine in a single batch.

num_parallel_batches

(Optional) An integer, representing the number of batches to create in parallel. On one hand, higher values can help mitigate the effect of stragglers. On the other hand, higher values can increase contention if CPU is scarce.

drop_remainder

Ensure that batches have a fixed size by omitting any final smaller batch if it's present. Note that this is required for use with the Keras tensor inputs to fit/evaluate/etc.

num_parallel_calls

(Optional) An integer, representing the number of elements to process in parallel If not specified, elements will be processed sequentially.

See Also

Other dataset methods: dataset_batch, dataset_cache, dataset_concatenate, dataset_decode_delim, dataset_filter, dataset_interleave, dataset_map, dataset_padded_batch, dataset_prefetch_to_device, dataset_prefetch, dataset_repeat, dataset_shuffle_and_repeat, dataset_shuffle, dataset_skip, dataset_take