scoringutils
scoringutils copied to clipboard
There should be a warning when there is a column called "sample" with a quantile format
I constructed some synthetic data and run into an error that was tricky to spot, caused by the fact that the sample column was automatically ignored:
library(dplyr)
library(tidyr)
library(scoringutils)
n_sim <- 1000
epsilon <- rnorm(n_sim)
Y <- exp(epsilon)
forecasts <- expand.grid(
sigma = 1:20/10,
quantile = c(0.01, 0.025, 1:19/20, 0.975, 0.99)
)
forecasts <- forecasts |>
as_tibble() |>
mutate(model = 10 * sigma,
prediction = exp(qnorm(quantile, sd = sigma)),
true_value = list(Y),
sample = list(1:length(Y))) |>
unnest(c(true_value, sample))
check_forecasts(forecasts)
score(forecasts)