• Home
  • Integrations
  • SDKs
  • Guides
  • API docs
No results for ""
EXPAND ALL

EDIT ON GITHUB

Analyzing experiments

Read time: 3 minutes
Last edited: Sep 15, 2023

Overview

This topic explains how to interpret an experiment's results and apply its findings to your product.

Reading experiment data

The data an experiment has collected is represented in a Results tab. The Results tab provides information about each variation's performance in the experiment, how it compares to the other variations, and which variations are likely to be best out of all the tested options. Understanding how to read this tab can help you make informed decisions about when to edit, stop, or choose a winning variation for your experiments.

Here is an example Results tab for a standard experiment:

An experiment's results tab.
An experiment's results tab.

This table explains the different sections of the Results tab and what you can use them for:

SectionExample (click to enlarge)
Attribute filters and the traffic count table help you examine experiment results for certain cross-sections of your experiment audience. To learn more, read Slicing experiment results.
Experiment results sliced by attribute.
Experiment results sliced by attribute.
The sample ratio mismatch section helps you understand when there may be an issue with your JavaScript-based SDKs. To learn more, read Understanding sample ratios.
The sample ratio mismatch section.
The sample ratio mismatch section.
The sample size estimator helps you decide how long to run the experiment for. To learn more, read Experiment size and run time.
An experiment's sample size estimator results.
An experiment's sample size estimator results.
An experiment's probability chart and results table.
An experiment's probability chart and results table.

Choosing a winning variation

The winning variation for a completed standard experiment is the variation that is most likely to be the best option out of all of the variations tested.

In this example, the "true" variation has the highest probability to be best:

An experiment's results tab with the winning variation called out.
An experiment's results tab with the winning variation called out.

To learn more, read Winning variations.

Consider stopping an experiment after you choose a winning variation

If you're done with an experiment and have rolled out the winning variation to your user base, it is a good time to stop your experiment. Experiments running on a user base that only receives one flag variation do not return useful results. Stopping an experiment retains all the data collected so far.

Further analyzing results

If you're using Data Export, you can find experiment data in your Data Export destinations to further analyze it using third-party tools of your own.

To learn more, read Data Export.