Setting up LaunchDarkly to run A/B tests involves a few additional setup steps beyond what's needed to control feature flags. We'll walk through the steps one by one. We'll assume that you've already gone through the basic Setup steps and can successfully toggle features on and off.
To use LaunchDarkly for A/B testing, you need to include an additional client-side snippet in your HTML. Your Quickstart guide contains the exact snippet to include (see "Tracking client-side events").
You'll also need to set the user context on every page by calling
Attributes and identify
You don't need to pass all of the user attributes for targeting and segmentation to
identify unless you're also using Front-end flags. If not, the
anonymous attributes are sufficient.
Goals are the metrics used to measure the effectiveness of a feature. To run an A/B test, you need to define the goals you care about. LaunchDarkly supports three kinds of goals:
- Click goals - track whether a user clicks on a specific page element.
- Page view goals - track whether a user lands on a specific page (for example, a confirmation page).
- Custom goals - track other user interactions that don't correspond to page views or clicks.
You can create goals directly in LaunchDarkly, or import them from Optimizely.
Once you've created your goals, you'll need to decide which goals to track for each feature.
Goals are defined globally in LaunchDarkly and can be re-used for multiple feature flags. In order to indicate which goals are relevant to each flag, you need to associate them with the feature flag. You can do this on the feature flag control page by clicking on the pencil in the header.
When the client-side snippet is live and goals have been created and tracked, you'll see results in the Analytics tab.
Event processing time
We process events on a five minute delay. If you're not seeing the numbers you expect on the analytics tab, ensure that you've waited at least five minutes for your events to be processed.
Once we've determined a winner, we'll show a green checkbox next to the winning variation.
Determining a winner
We'll select a winner when each variation has at least 1000 distinct users (impressions), and the confidence interval is above 95%.
Custom goals are useful for recording interactions that don't directly correspond to clicks or page views. LaunchDarkly allows you to record custom events client-side (via our JS snippet) or on the server (via our SDKs).