Randomization units

Overview

This topic explains what randomization units are and how to use them in LaunchDarkly Experimentation.

An experiment's randomization unit is the context kind the experiment uses to assign traffic to each of its variations. For example, if you choose the user context kind as an experiment's randomization unit, then the experiment will divide contexts into the experiment's variations by individual users.

Randomization units in experiments

Before you begin an experiment, you must:

• choose an industry-standard randomization unit for the experiment,
• map the randomization unit to an appropriate context kind,
• map the context kind to the appropriate metrics, then
• add the randomization unit to the experiment.

These steps are explained below.

Industry standard randomization units

There are six industry-standard randomization units that you can use in Experimentation:

• user
• user-time
• guest
• guest-time
• organization
• request

LaunchDarkly limits experiments to randomizing by these six industry-standard units because other units might result in invalid results. If you have a context kind that doesn't logically map to one of these industry-standard units, it may not be appropriate to use as a randomization unit. For examples of common context kind mapping, read the table in Map randomization units to context kinds.

For example, if you tried to use "country" as a randomization unit, your experiment would serve the same variation to everyone in a particular country. If your experiment had end users in the United States, Mexico, and Canada, the experiment might serve variation A to all end users in the United States and Canada, and variation B to all end users in Mexico. This could result in significantly uneven numbers of end users in each variation, and each variation would not include a random sampling of end users. At the end of the experiment, there would be no way to tell if variation A or B performed better in any given country.

Instead, if you wanted to view results by country, you could filter your results by context attribute. To learn more, read Filtering experiment results.

Map randomization units to context kinds

The built-in user context kind is automatically mapped to the user standard randomization unit. If you have created additional context kinds you want to use in experiments, then you must first map them to one of the standard randomization units. Multiple context kinds can be mapped to the same standard randomization unit. To learn how, read Creating context kinds.

Here is an example of mapping a new context kind of "customer" to the randomization unit "user":

This table includes the industry-standard randomization units and example context kinds that might be associated with them:

Map randomization units to metrics

When you create a metric, you must map the new metric to a context kind you have marked as available for experiments:

This context kind determines which randomization units the metric is compatible with. To learn more, read Metrics.

Choose randomization units for experiments

Finally, when you create an experiment, you will choose a randomization unit that determines both:

• which context kind the experiment will allocate traffic to different variations by, and
• which metrics you can use in the experiment.

In this example, the experiment has a randomization unit of user. It is compatible with the chosen metric, because the metric can measure events from user context kinds.

Using randomization units with multi-contexts

Expand the example below to see how you can use different randomization units within the same experiment.

{
  "kind": "multi",
  "user": {
    "key": "user-key-anna", // Anna has a unique user key
    "name": "Anna",
  },
  "organization": {
    "key": "org-key-global-health", // The org key is the same in both multi-contexts
    "name": "Global Health Services",
  }
}
{
  "kind": "multi",
  "user": {
    "key": "user-key-jesse", // Jesse has a unique user key
    "name": "Jesse",
  },
  "organization": {
    "key": "org-key-global-health", // The org key is the same in both multi-contexts
    "name": "Global Health Services",
  }
}
COPY