SeGLiR
SeGLiR copied to clipboard
auduno
Javascript library for rapid A/B-testing with Sequential Generalized Likelihood Ratio tests
I'm interested in using this library so I tested to see if the False Positive Rate is controlled within alpha that I set. However, the results are showing that the...
While the Whitehead bias-adjusted estimator seems to work decently, it would be nice to be able to estimate bias conditional on number of samples the test needed to finish. Furthermore,...
It's possible to extend most sequential GLR tests to multi-hypothesis settings, i.e. multivariate testing. It might need ability to calculate several-dimensional optimization, in order to calculate thresholds.
It is fairly trivial to add the possibility to set a minimum samplesize (to ensure that post-analysis estimates are better) since this can be simulated as usual. Adding the possibility...
Similarly to bernoulli comparison of proportions we would like to know the worst-case samplesize at a given level. This can not be simulated, but needs to be calculated analytically.
The library should include more precalculated thresholds than the one currently included.
The current implementation of optimization to find the thresholds at given levels, is a mocked up gradient descent algorithm where simulation is used to find the gradient descent steps. While...