RBERT (version 0.1.11)

create_attention_mask_from_input_mask: Create 3D attention mask from a 2D tensor mask

Description

An attention mask is used to zero out specific elements of an attention matrix. (For example, to prevent the model from "paying attention to the answer" in certain training tasks.)

Usage

create_attention_mask_from_input_mask(from_tensor, to_mask)

Arguments

from_tensor

2D or 3D Tensor of shape [batch_size, from_seq_length, ...].

to_mask

int32 Tensor of shape [batch_size, to_seq_length].

Value

float Tensor of shape [batch_size, from_seq_length, to_seq_length].

Examples

# NOT RUN {
with(tensorflow::tf$variable_scope("examples",
  reuse = tensorflow::tf$AUTO_REUSE
), {
  from_tensor <- ids <- tensorflow::tf$get_variable("ften",
    dtype = "float", shape = c(10, 20)
  )
  to_mask <- ids <- tensorflow::tf$get_variable("mask",
    dtype = "int32", shape = c(10, 30)
  )
})
create_attention_mask_from_input_mask(from_tensor, to_mask)
# }