model-diagnostics
model-diagnostics copied to clipboard
Enh/compute score uncertainty
@lorentzenchr it's just a draft PR to receive some feedback on what I did and discuss how to go forward.
>>> from model_diagnostics.scoring import compute_score, SquaredError
>>>
>>> y_obs = [0, 0, 1, 1]
>>> y_pred = [-1, 1, 1, 2]
>>>
>>> print(
... compute_score(
... y_obs=y_obs,
... y_pred=y_pred,
... scoring_function=SquaredError(),
... confidence_level=0.5,
... )
... )
shape: (1, 5)
┌────────────────┬────────────────┬─────────────┬───────┬───────────────────────────┐
│ miscalibration ┆ discrimination ┆ uncertainty ┆ score ┆ score_interval │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ f64 ┆ f64 ┆ f64 ┆ f64 ┆ list[extension] │
╞════════════════╪════════════════╪═════════════╪═══════╪═══════════════════════════╡
│ 0.625 ┆ 0.125 ┆ 0.25 ┆ 0.75 ┆ [[0.55877692 0.94122308]] │
└────────────────┴────────────────┴─────────────┴───────┴───────────────────────────┘