Shipping the winning variation

Overview

This topic explains how to choose and ship a winning variation for a completed experiment.

LaunchDarkly offers both Bayesian and frequentist statistics as analysis methods for experiments.

Bayesian experiments

For experiments using Bayesian statistics, the winning variation is typically the treatment with the highest probability of being the best among those that show a significant high probability of beating the control. If all treatment variations have a significantly low probability of beating the control, then the control is considered the winning variation. To learn more, read Decision making with Bayesian statistics.

If enough contexts have encountered your experiment to determine a winning variation, you can stop the experiment and ship the winning variation to all of your contexts.

To ship a winning variation for a Bayesian experiment:

1. Navigate to the Experiments list.
2. Click on the name of the experiment you want to ship a variation for.
3. If enough contexts have encountered the experiment to determine a winning variation, a winning variation banner is visible.
4. Click Ship it. A "Ship the leading variation" dialog appears.
5. Click Ship it.

The experiment iteration stops. All contexts are now receiving the winning variation of your experiment. LaunchDarkly retains all of the data collected from stopped iterations.

Frequentist experiments

For experiments that use frequentist statistics, the winning variation for an experiment is the variation that has the highlighted p-value.

To ship a winning variation for a Frequentist experiment:

1. Navigate to the Experiments list.
2. Click on the name of the experiment you want to ship a variation for.
3. Click on the Latest results tab.
4. Find the variation with the p-value highlighted in green.
5. Update the flag to serve that variation to all of your contexts. To learn how, read Target with flags.

Funnel experiments

In addition to identifying the winning variation for the entire funnel optimization experiment, each step in the funnel highlights its winning variation in bold. The overall winning variation, determined by performance in the final step, displays at the top of the results section.