Add infra to push performance metric to remote performance tracker

[mfuntowicz]

This PR introduces a new subpackage optimum.tools.records which aims at providing the bare-minimum infrastructure required to push performance metrics to our internal tracking system

• [x] Pythonic API RFC
• [ ] CLI interface WIP

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[mfuntowicz]

Example usage:

from optimum.tools.records import AutoPerformanceTracker, PerformanceRecord

if __name__ == '__main__':

    tracker = AutoPerformanceTracker.from_uri(
        "es+aws://benchmarks-kb3me[..]q7deny.us-east-1.es.amazonaws.com"
    )

    record = PerformanceRecord.latency(
        metric="TIME_TO_FIRST_TOKEN",
        value=123.4,
        meta={
            "commit_id": "saflsfkja3115",
            "model_id": "meta-llama/Meta-Llama-3.1-8B-Instruct",
            "dtype": "float16",
            "tgi_version": "v2.1.1",
            "cuda_version": "12.4.1"
        }
    )

    tracker.push("ci_tgi_performances_tracker", record)

[mfuntowicz]

Tentative CLI interface:

optimum benchmark push -m TIME_TO_FIRST_TOKEN -k latency -v 123.4 -t <float(timestamp)> --meta commit_id=saflsfkja3115 --meta model_id=meta-llama/Meta-Llama-3.1-8B-Instruct --meta dtype=float16 [...] es+aws://benchmarks-k[...]deny.us-east-1.es.amazonaws.com

[github-actions[bot]]

This PR has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.