split

split copied to clipboard

Q: Variant consistently strong in the first week, but drops of to no change in the 2nd. Normal?

1

Hi there, we've been using the split gem for a while, but we've seen this behavior a couple of times and wonder if it's normal: We run an A/B test...

christian314159

Option to automatically end test upon reaching N sample size?

2

Just wondering if this can be configured as an option whereby the test can be paused when N sample size of participants is reached?

imweijia

Concurrency issues when entering trials

1

The "choose trial" logic is not atomic, so it is easy to run it multiple times for the same user. Specifically, there is a read here: https://github.com/splitrb/split/blob/d3ef9d79d2fe15d95292072420e62afea151dffd/lib/split/trial.rb#L71 With a matching...

wheeyls

A single ab_test call triggers multiple Redis commands for all experiments

6

**Describe the bug** `ab_test` triggers multiple Redis commands for all experiments for a single experiment `ab_test(:experiment_1)` call. If in a single request flow I call `ab_test(:experiment_1)`, `ab_test(:experiment_2)` and `ab_test(:experiment_3)` it...

phstc

Testing Split leads to variant failing by nearly 20%

2

Hi, we've been running a few experiments using Split and noticed nearly all of our variants were losing over the control version by pretty hefty margins. So we set up...

getabetterpic

Q: Is there a way to remove a user from an experiment?

1

In certain cases, I want to be able to take a user out of an experiment and its statistics. Is it possible?

moriaam

Be able to call "finish" multiple times without resetting the test group

10

I would like to bucket users into two groups, and then measure the number of times each group performs a certain action. Let's say group A has two users and...

henrik

Number of participants exceeds page views

3

Hi there! We recently added Split to our project and have noticed that the number of participants far exceeds the number of page views that we've recorded by as much...

dinkengraven

One of my experiments have one alternative winning by 7% but it has lower probability of winning (41%). Does it make any sense? I can't figure out the math behind...

caepenna

Assets for dashboard are not compatible with Asset Pipeline

2

When deploying to a production environment, the `/split` assets are not made available on the expected path. For example, `/split/reset.css` is a 404. This seems to be because the engine...

iwz