flow icon indicating copy to clipboard operation
flow copied to clipboard

AddCouple中正反向是否写反的问题

Open BlueWinters opened this issue 7 years ago • 1 comments

根据https://kexue.fm/archives/5776中的公式[7]和[9],x->...->h_t的过程表示前向; 那么nice.py中AddCouple的call方法中,

    def call(self, inputs):
        part1, part2, mpart1 = inputs
        if self.isinverse:
            return [part1, part2 + mpart1] # 逆为加
        else:
            return [part1, part2 - mpart1] # 正为减
`
此两种情况是否写反了?

BlueWinters avatar Dec 25 '18 06:12 BlueWinters

I also have this question - I inverted the order in to what the paper looks like

         if self.isinverse:
            return [part1, part2 - mpart1] # 逆为加
        else:
            return [part1, part2 + mpart1] # 正为减

and the code doesn't seem to train anymore. Correct me if I'm wrong but both transformations are valid in both directions, the jacobian would still be the same? I'm confused why this would not work.

k--chow avatar Oct 02 '20 21:10 k--chow