model-diagnostics icon indicating copy to clipboard operation
model-diagnostics copied to clipboard

Enh/compute score uncertainty

Open m-maggi opened this issue 2 years ago • 0 comments

@lorentzenchr it's just a draft PR to receive some feedback on what I did and discuss how to go forward.


>>> from model_diagnostics.scoring import compute_score, SquaredError
>>> 
>>> y_obs = [0, 0, 1, 1]
>>> y_pred = [-1, 1, 1, 2]
>>> 
>>> print(
...     compute_score(
...         y_obs=y_obs,
...         y_pred=y_pred,
...         scoring_function=SquaredError(),
...         confidence_level=0.5,
...     )
... )
shape: (1, 5)
┌────────────────┬────────────────┬─────────────┬───────┬───────────────────────────┐
│ miscalibration ┆ discrimination ┆ uncertainty ┆ score ┆ score_interval            │
│ ---            ┆ ---            ┆ ---         ┆ ---   ┆ ---                       │
│ f64            ┆ f64            ┆ f64         ┆ f64   ┆ list[extension]           │
╞════════════════╪════════════════╪═════════════╪═══════╪═══════════════════════════╡
│ 0.625          ┆ 0.125          ┆ 0.25        ┆ 0.75  ┆ [[0.55877692 0.94122308]] │
└────────────────┴────────────────┴─────────────┴───────┴───────────────────────────┘

m-maggi avatar Mar 05 '24 17:03 m-maggi