Jason Chen

How to set batch size with Bert

1

Hi, I am trying to do inference with Bert. However, I don't find any API to set batch size in **run.py**. Could you pls. help me with that? Appreciate a...

Hi, **When I ran resnet50 with cmd "python3.6 python/main.py --profile resnet50-tf --mlperf_conf ../../mlperf.conf --model ../../../resnet50.pb --dataset-path fake_imagenet/ --output /home/jason/resnet50/inference/vision/classification_and_detection/output/test1 --scenario=Offline" There are error logs blow:** ERROR:main:thread: failed on contentid=['fake_imagenet/val/800px-Porsche_991_silver_IAA.jpg', 'fake_imagenet/val/800px-20180630_Tesla_Model_S_70D_2015_midnight_blue_left_front.jpg',...

Inference DLRMv2 on CPU using dlrm_main.py Issue: ``` model, optimizer, lr_scheduler = construct_model(args) TypeError: cannot unpack non-iterable NoneType object ``` This PR is for this issue fixing.

ChatGLM3 output token size is not generated as expected

5

Benchmark cmd (-ic is 128): ``` numactl -C 0-55 -m 0 python benchmark.py -m /root/.cache/huggingface/hub/chatglm3-6b-ov/pytorch/dldt/FP16 -p "It is done, and submitted..." -n 2 -bs 1 -d CPU --torch_compile_backend openvino -ic...

[Improvement] Wrong output size of google/flan-t5-xl

4

Benchmark cmd: ``` numactl -C 0-55 -m 0 python benchmark.py -m /root/.cache/huggingface/hub/flan-t5-xl-ov/pytorch/dldt/FP16 -p "It is done..." -n 3 -bs 1 -d CPU --torch_compile_backend openvino -ic 128 --num_beams 1 -lc bfloat16_config.json...