oneDNN
oneDNN copied to clipboard
oneapi-src
graph: backend: dnnl: fix decompose kernel select index check
Fix MFDNN-13765.
Add check logic to fall back unsupported sdpa case to large_partition kernel.
ONEDNN_VERBOSE=1,filter=graph KMP_AFFINITY=compact,1,0,granularity=fine OMP_NUM_THREADS=4 OMP_PLACES=threads OMP_PROC_BIND=close numactl --membind=
0 --physcpubind=0-3 ./tests/benchdnn/benchdnn --graph --dt=7:f32+8:f32+9:f32+10:f32 --case=/home/sdp/xiangguo/onednn/oneDNN/mha-pt-starcoder-inf-bf16-bs1.json.json
onednn_verbose,v1,info,oneDNN v3.9.0 (commit 8b485fb2210a8f92b7ea2175c0e4e5692b6fc79e)
onednn_verbose,v1,info,cpu,runtime:OpenMP,nthr:4
onednn_verbose,v1,info,cpu,isa:Intel AVX-512 with float16, Intel DL Boost and bfloat16 support and Intel AMX with bfloat16 and 8-bit integer support
onednn_verbose,v1,info,gpu,runtime:none
onednn_verbose,v1,info,graph,backend,0:dnnl_backend
onednn_verbose,v1,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,v1,graph,info,template:operation,engine,partition_id,partition_kind,op_names,data_formats,logical_tensors,fpmath_mode,implementation,backend,exec_time
onednn_verbose,v1,common,info,filter format is enabled, hit components: graph
onednn_verbose,v1,graph,exec,cpu,100002,sdp,aten::matmul;aten::mul;aten::where;aten::softmax;aten::matmul,,in0_bf16:5:strided:variable:1x117x48x128:718848s6144s128s1 in1_bf16:6:strided:variable:1x1x128x117:14976s14976s117s1 in2_bf16:2:strided:undef:0:2 in3_boolean:1:strided:variable:1x117x1x117:13689s117s117s1 in4_f32:9:strided:undef:0:2 in5_bf16:19:strided:variable:1x1x117x128:14976s14976s128s1 out0_bf16:20:strided:variable:1x117x48x128:718848s6144s128s1,fpm:strict,larger_partition_kernel_t,dnnl_backend,47.416
[8743][0:1:20:39] exp_f32: 0.00468446 exp: 0.00469971 got: 0.00466919 diff:3.05176e-05 rdiff:0.00649351
[9309][0:1:24:93] exp_f32: 2.80984e-05 exp: 2.81334e-05 got: 2.7895e-05 diff:2.38419e-07 rdiff:0.00847458
[9328][0:1:24:112] exp_f32: 1.99666e-05 exp: 1.9908e-05 got: 2.00272e-05 diff:1.19209e-07 rdiff:0.00598802
[41347][0:6:35:3] exp_f32: -0.908203 exp: -0.910156 got: -0.90625 diff:0.00390625 rdiff:0.00429185
[154787][0:25:9:35] exp_f32: 1.11755e-05 exp: 1.11461e-05 got: 1.12057e-05 diff:5.96046e-08 rdiff:0.00534759
[156925][0:25:25:125] exp_f32: 1.78026e-05 exp: 1.77622e-05 got: 1.78814e-05 diff:1.19209e-07 rdiff:0.00671141
[175248][0:28:25:16] exp_f32: 1.11148e-05 exp: 1.10865e-05 got: 1.11461e-05 diff:5.96046e-08 rdiff:0.00537634
[186574][0:30:17:78] exp_f32: -0.873047 exp: -0.871094 got: -0.875 diff:0.00390625 rdiff:0.00448431
[187039][0:30:21:31] exp_f32: -0.118896 exp: -0.119141 got: -0.118652 diff:0.000488281 rdiff:0.00409836
[220921][0:35:45:121] exp_f32: -1.77734 exp: -1.78125 got: -1.77344 diff:0.0078125 rdiff:0.00438596
[COMPARE_STATS]: trh=0 err_max_diff:0.0078125 err_max_rdiff:0.00847458 all_max_diff:0.0078125 all_max_rdiff:0.294304
[COMPARE_STATS] Norm check is allowed; error_to_total_ratio: 24/718848; allowed_ratio: 702/718848;
0:PASSED (316 ms) __REPRO: --graph --dt=7:f32+8:f32+9:f32+10:f32 --case=/home/sdp/xiangguo/onednn/oneDNN/mha-pt-starcoder-inf-bf16-bs1.json.json
tests:1 passed:1 skipped:0 mistrusted:0 unimplemented:0 invalid_arguments:0 failed:0 listed:0
total: 0.32s; create_pd: 0.00s (0%); create_prim: 0.00s (0%); fill: 0.00s (0%); execute: 0.00s (0%); compute_ref: 0.00s (0%); compare: 0.00s (0%);
make test set test_scope=NIGHTLY disable benchdnn_all enable benchdnn_graph
make test set test_scope=NIGHTLY disable benchdnn_all enable benchdnn_graph
make test set test_scope=NIGHTLY disable benchdnn_all enable benchdnn_graph