ONNX to TensorRT Error: Internal Error (kv_slice: optimization profile is missing values for shape input)

[zengrh3]

When I tried to convert my onnx model to TensorRT model, I encountered the following error:

[TRT] [E] 4: [network.cpp::validate::3584] Error Code 4: Internal Error (kv_slice: optimization profile is missing values for shape input)

However, I have explicitly set the shape like below:

shapes = {
    'position' : [(1, 1), (1, 1000), (1, 2000)],
    'inputs_embeds': [(1, 1, 1792), (1, 1000, 1792), (1, 2000, 1792)],
    'past_key_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in2' : [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in2': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'kv_slice': [(1,), (1,), (1,)],
}

for order, (name, shape_list) in enumerate(shapes.items()):
    print(order, name, shape_list)
    min_shape, opt_shape, max_shape = shape_list
    profile.set_shape(name, min_shape, opt_shape, max_shape)
    set_min_shape, set_opt_shape, set_max_shape = profile.get_shape(name)
    

config.add_optimization_profile(profile)
engine = builder.build_serialized_network(network, config)

[yuanyao-nv]

Can you please share your model and the complete repro steps? Thanks