fused_layer_norm_cuda

[song201216]

Hi, I am very interested in your research. When I reproduced your code, the following problems appeared, and I felt very confused. Traceback (most recent call last): File "generate_samples.py", line 326, in
main()
File "generate_samples.py", line 321, in main
model = setup_model(args)
File "generate_samples.py", line 52, in setup_model
model = get_model(args)
File "E:\CogView-main\pretrain_gpt2.py", line 65, in get_model model = GPT2Model(num_layers=args.num_layers,
File "E:\CogView-main\model\gpt2_modeling.py", line 91, in init self.transformer = mpu.GPT2ParallelTransformer(num_layers,
File "E:\CogView-main\mpu\sparse_transformer.py", line 460, in init
[get_layer(layer_id) for layer_id in range(num_layers)]) File "E:\CogView-main\mpu\sparse_transformer.py", line 460, in
[get_layer(layer_id) for layer_id in range(num_layers)]) File "E:\CogView-main\mpu\sparse_transformer.py", line 441, in get_layer
return GPT2ParallelTransformerLayer( File "E:\CogView-main\mpu\sparse_transformer.py", line 283, in init
self.input_layernorm = LayerNorm(hidden_size, eps=layernorm_epsilon)
File "E:\CogView-main\mpu\sparse_transformer.py", line 42, in init
super().init(*args, **kwargs) File "D:\software\Anaconda3\envs\syq\lib\site-packages\apex\normalization\fuse d_layer_norm.py", line 133, in init fused_layer_norm_cuda = importlib.import_module("fused_layer_norm_cuda")
File "D:\software\Anaconda3\envs\syq\lib\importlib_init_.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'fused_layer_norm_cuda'

[poorfriend]

重新安装apex https://blog.csdn.net/li_jiaoyang/article/details/117431876