BERT-flow
BERT-flow copied to clipboard
bohanli
TensorFlow implementation of On the Sentence Embeddings from Pre-trained Language Models (EMNLP 2020)
感觉作者训练有造假的嫌疑，首先 ops = [glow_ops.get_variable_ddi, glow_ops.actnorm, glow_ops.get_dropout] encoder = glow_ops.encoder_decoder self.z, encoder_objective, self.eps, _, _ = encoder( "flow", x, self.hparams, eps=None, reverse=False,init=init) objective += encoder_objective 这时的objective 损失为正，原因是其实就是在变换过程中antocrm的参数以及分割向量后经过正态分布密度函数后的log_prob的概率负数，但是作者确实使用+=，有待商榷，参数总和可能为正也可能为负，但是log_prob一定为负，由于概率再0-1之间， self.z_top_shape =self.z.shape prior_dist =...
My dataset only has one sentence a line,How Can I get its embedding? Thanks in advance.
The current script to download the GLUE data is not working. Changed by the one of https://github.com/nyu-mll/jiant-v1-legacy/tree/master/scripts that works well.
我在蚂蚁金融数据集上进行了测试，一个中文的句子对相似任务[0，1] 分类，进行了如下实验： 1.bash scripts/train_siamese.sh train \ "--exp_name=exp_${BERT_NAME}_${RANDOM_SEED} \ --num_train_epochs=1.0 \ --learning_rate=2e-5 \ --train_batch_size=16 \ --cached_dir=${CACHED_DIR}" 在eval 中的acc 为85左右 2.bash scripts/train_siamese.sh train \ "--exp_name_prefix=exp \ --cached_dir=${CACHED_DIR} \ --flow=1 --flow_loss=1 \ --num_examples=0...
Dear BERT-Flow authors, I notice there's a mistake in your code where you attempt to average the last 2 layers of BERT but instead you average the first and last...
loss为负
你好，我在QQP数据集上进行训练时，发现loss为负数，请问是什么原因呢  
It seems that the provided Google drive link only contains BERT-large models, could you release the base version of BERT-flow for purpose of better reproduction?
Will it be better to replace BERT with ELECTRA? Have done relevant experiments？
