scoringutils icon indicating copy to clipboard operation
scoringutils copied to clipboard

There should be a warning when there is a column called "sample" with a quantile format

Open nikosbosse opened this issue 3 years ago • 0 comments

I constructed some synthetic data and run into an error that was tricky to spot, caused by the fact that the sample column was automatically ignored:

library(dplyr)
library(tidyr)
library(scoringutils)


n_sim <- 1000
epsilon <- rnorm(n_sim)
Y <- exp(epsilon)

forecasts <- expand.grid(
  sigma = 1:20/10, 
  quantile = c(0.01, 0.025, 1:19/20, 0.975, 0.99)
)

forecasts <- forecasts |>
  as_tibble() |>
  mutate(model = 10 * sigma, 
         prediction = exp(qnorm(quantile, sd = sigma)), 
         true_value = list(Y), 
         sample = list(1:length(Y))) |>
  unnest(c(true_value, sample))

check_forecasts(forecasts)
score(forecasts)

nikosbosse avatar Jul 05 '22 19:07 nikosbosse