I've been faced with an error in the PositionalEmbedding step on the original notebook

[aXlireza]

class PositionalEmbedding(tf.keras.layers.Layer):
  def __init__(self, vocab_size, d_model):
    super().__init__()
    self.d_model = d_model
    self.embedding = tf.keras.layers.Embedding(vocab_size, d_model, mask_zero=True) 
    self.pos_encoding = positional_encoding(length=2048, depth=d_model)

  def compute_mask(self, *args, **kwargs):
    return self.embedding.compute_mask(*args, **kwargs)

  def call(self, x):
    length = tf.shape(x)[1]
    x = self.embedding(x)
    # This factor sets the relative scale of the embedding and positonal_encoding.
    x *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))
    x = x + self.pos_encoding[tf.newaxis, :length, :]
    return x

embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size(), d_model=512)

pt_emb = embed_pt(pt)

Error Log:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-79-19249302cd7f>](https://localhost:8080/#) in <cell line: 1>()
----> 1 pt_emb = embed_pt(pt)
      2 en_emb = embed_en(en)

1 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py](https://localhost:8080/#) in error_handler(*args, **kwargs)
    120             # To get the full stack trace, call:
    121             # `keras.config.disable_traceback_filtering()`
--> 122             raise e.with_traceback(filtered_tb) from None
    123         finally:
    124             del filtered_tb

[<ipython-input-77-e9ab4e283481>](https://localhost:8080/#) in call(self, x)
     11   def call(self, x):
     12     length = tf.shape(x)[1]
---> 13     x = self.embedding(x)
     14     # This factor sets the relative scale of the embedding and positonal_encoding.
     15     x *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))

ValueError: Exception encountered when calling PositionalEmbedding.call().

Invalid dtype: <property object at 0x7e5961d38810>

Arguments received by PositionalEmbedding.call():
  • x=tf.Tensor(shape=(64, 92), dtype=int64)

[JeremyJian]

I face the exact the same issue!

Replace embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size(), d_model=512) embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size(), d_model=512)

with embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size().numpy(), d_model=512) embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size().numpy(), d_model=512)

might solve this issue?

[StefanWehrenberg]

I face the exact the same issue!

Replace embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size(), d_model=512) embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size(), d_model=512)

with embed_pt = PositionalEmbedding(vocab_size=tokenizers.pt.get_vocab_size().numpy(), d_model=512) embed_en = PositionalEmbedding(vocab_size=tokenizers.en.get_vocab_size().numpy(), d_model=512)

might solve this issue?

Can confirm this. Had the same issue, and adding the conversion to a NumPy array fixed it. This was seemingly corrected in the notebooks in Colab and the repository, but not in the downloadable version of the tutorial.