practical-pytorch icon indicating copy to clipboard operation
practical-pytorch copied to clipboard

seq2seq-translation-batched: Bahdanau attention does not work

Open juditacs opened this issue 8 years ago • 2 comments

__init__.py fails with AttributeError, max_length does not exist. Fixing this results in a concat error in the Attn class:

    45         elif self.method == 'concat':
---> 46             energy = self.attn(torch.cat((hidden, encoder_output), 1))
     47             energy = self.v.dot(energy)
     48             return energy

RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)

replacing the dimension in line 46 to 0 results in this error:

    44 
     45         elif self.method == 'concat':
---> 46             energy = self.attn(torch.cat((hidden, encoder_output), 0))
     47             energy = self.v.dot(energy)
     48             return energy

RuntimeError: inconsistent tensor sizes at /opt/conda/conda-bld/pytorch_1512386481460/work/torch/lib/THC/generic/THCTensorMath.cu:157

juditacs avatar Dec 28 '17 15:12 juditacs

I think there could be a solution to attention calculation in matrix form in #107 .

aevilorz avatar Jul 05 '18 06:07 aevilorz

#119 addresses some of the mentioned issues.

anantzoid avatar Oct 30 '18 18:10 anantzoid