Keep track of inference time

[MaramHasanain]

Keep track of inference time per input sample, maybe just for the successful cases (we can add it to the cache and update it for every new input sample). After experiment is done, the pipeline to report the average inf. time over successful samples.