[ENH] Add Lambert W x F distributions

[gmgeorg]

Is your feature request related to a problem? Please describe.

For modeling skewed and/or heavy-tailed distributions i'd like to have support for Lambert W x F distributions. On top of modeling, Lambert W x F distribution allow to "Gaussianize" the observed data.

This is especially useful / prevalent for financial time series data, which is often skewed and/or heavy-tailed.

Describe the solution you'd like

This exists in the LambertW R package and the pylambertw Python module, which is an sklearn transformer/estimator wrapper around torchlambertw.

Describe alternatives you've considered

Other heavy-tailed distributions; but none of the typical ones allow the ease of itnerpretation of the heavy-tail parameter, the input/output system view of transformation, and a bijective back-transformation.

Additional context

• see here for a detailed discussion with references / screenshots etc. https://github.com/StatMixedML/XGBoostLSS/discussions/55

• https://github.com/StatMixedML/XGBoostLSS/pull/65#issuecomment-1911933973

I'd be happy to open a PR to implement a first version of Lambert W x Gaussian distributions, but would like some guidance/pointers on best practices for skpro.

[fkiraly]

Very interesting. For anyone looking for a mathematical reference, the annals article is available on the arxiv: https://arxiv.org/abs/0912.4554.

I am intrigued since, please confirm if I understad this correctly:

• Lambert transformed distributions are actually dependent objects - i.e., Lambert tf of distr D, which makes thes compositional
• the practical intention of introducing them is making distributions more normal for modelling, which is a common assumption in machine learning

If I understand correctly, there are also multiple related "objects":

• the transformation itself, operating on empirical samples (matrices)
• the distribution depending on another distribution that gets transformed
• a regressor that applies the sample transformation on data, and inverts the transformation on a distribution that is predictd

The last one especially is related to the "transformed distribution" proposed in https://github.com/sktime/skpro/issues/30.

[fkiraly]

I'd be happy to open a PR to implement a first version of Lambert W x Gaussian distributions, but would like some guidance/pointers on best practices for skpro. Thanks, that would be nice!

skpro generally follows sklearn extension patterns. The distribution extension contract is not that well-documented at the moment, it is maturing - you could however look at the classes in distributions, all methods have proper docstrings. Perhaps the Normal is the best template for now.

The one thing to note, perhaps, is that distributions are of matrix/table shape, i.e., a matrix/table with distributions (possibly dependnent but usually independent) as entries. This is because in tabular probabilistic regression, this object is the output.

Questions:

• would it not be nicer to have Lambert W x any distribution? Or, are the transforms of Gaussians more explicit than the arbitrary case?
  • this would be representable in the interface, it would be a distribution that takes another distribution as argument.

[gmgeorg]

@fkiraly yes to all your points in first reply.

re 2nd: yes, implementing Lambert W x Gaussian shouldn't be much different from just implementing a Lambert W x F abstract class and then inheriting/setting base_distribution=Gaussian . This is what I ended up doing for torchlambertw as well as the xgboostlss implementations.

I need to get more familiar with skpro first to see how this would actually work in this framework. Will take a look at this and see if I run into any issues trying to implement the generic LambertWDistribution class first, with LambertWGaussian, LambertWExponential, etc as special cases.

[fkiraly]

shouldn't be much different from just implementing a Lambert W x F abstract class and then inheriting/setting base_distribution=Gaussian

I see!

I need to get more familiar with skpro first to see how this would actually work in this framework.

I would recommend to look at distributions.normal for an example. We have not gotten round to write an extension template, but I hope the stucture is self-explanatory.

Will take a look at this and see if I run into any issues trying to implement the generic LambertWDistribution class first, with LambertWGaussian, LambertWExponential

The way I imagined it would be sth around the lines:

any_inner_dist = InnerDist(a=a_arr, b=b_arr)
lambert_trafo_dist = LambertW(any_inner_dist, gamma=0.5)

That is, any distribution can be taken as an argument of LambertW - what is passed is the actual distribution, not a string.

In the example, InnerDist could be Gaussian or Laplace or anything else, and it provides th methods that all distributions have. Do you think it can be implemented in this high degree of generality, or do we need to make case distinctions for inner distributions, e.g., due to limitations in our knowledge of the explicit form of distribution generating functions?