gamboostLSS icon indicating copy to clipboard operation
gamboostLSS copied to clipboard

Error when predicting with large N

Open bryorsnef opened this issue 6 years ago • 1 comments

I'm getting an error when trying to predict using gamboost, but the error only appears for me when the N becomes large.

E.g. running this code with 10000 rows of data is fine.

try with 10000 obs

p <- 10 n <- 10000

x <- matrix(runif(np), nrow = n, ncol = p) coefs <- ifelse(runif(p) < 0.5, rnorm(p), 0) y <- x %% coefs + rnorm(n) x[,10] <- ifelse(runif(n) < 0.5, x[,10], NA)

xy <- data.frame(x,y)

forms <- list(mu = as.formula(X10 ~ .), phi = as.formula(X10 ~ .))

mod <- gamboostLSS(forms, data = xy[!is.na(xy[,10]),], families = BetaLSS())

preds <- predict(mod, newdata = xy[is.na(xy$X10),], type = "response")

no error

try with 50000 obs, error

p <- 10 n <- 50000

x <- matrix(runif(np), nrow = n, ncol = p) coefs <- ifelse(runif(p) < 0.5, rnorm(p), 0) y <- x %% coefs + rnorm(n) x[,10] <- ifelse(runif(n) < 0.5, x[,10], NA)

xy <- data.frame(x,y)

forms <- list(mu = as.formula(X10 ~ .), phi = as.formula(X10 ~ .))

mod <- gamboostLSS(forms, data = xy[!is.na(xy[,10]),], families = BetaLSS())

preds <- predict(mod, newdata = xy[is.na(xy$X10),], type = "response")

bryorsnef avatar Dec 10 '19 22:12 bryorsnef

as a quick fix you can change the following option to something higher than your n.

options("mboost_indexmin") # check current setting 
options(mboost_indexmin = 100000) # set to new value

sbrockhaus avatar Dec 11 '19 21:12 sbrockhaus