tfio (version 0.4.1)

kafka_dataset: Creates a KafkaDataset.

Description

Creates a KafkaDataset.

Usage

kafka_dataset(
  topics,
  servers = "localhost",
  group = "",
  eof = FALSE,
  timeout = 1000
)

Arguments

topics

A tf.string tensor containing one or more subscriptions, in the format of [topic:partition:offset:length], by default length is -1 for unlimited.

servers

A list of bootstrap servers.

group

The consumer group id.

eof

If True, the kafka reader will stop on EOF.

timeout

The timeout value for the Kafka Consumer to wait (in millisecond).

Examples

# NOT RUN {
dataset <- kafka_dataset(
    topics = list("test:0:0:4"), group = "test", eof = TRUE) %>%
  dataset_repeat(1)

sess <- tf$Session()
iterator <- make_iterator_one_shot(dataset)
next_batch <- iterator_get_next(iterator)

until_out_of_range({
  batch <- sess$run(next_batch)
  print(batch)
})
# }
# NOT RUN {
# }