Add world_language_model transformer and lstm

Open xuzhao9 opened this issue 3 years ago • 1 comments

Add world_language_model transformer and lstm. They are also used in release testing.

Jul 01 '22 01:07 xuzhao9

The PR is blocked by memory leak issue in wlm_lstm_train_cuda: https://app.circleci.com/pipelines/github/pytorch/benchmark/4705/workflows/b9b7e022-0a50-41eb-8088-4e9ca5d4f169/jobs/4836 @robieta I suspect this is because of cyclic references in the LSTM model (since wlm_transformer_train_cuda) doesn't have this problem. But still there might be a bug somewhere?

Jul 28 '22 20:07 xuzhao9