Creating metrics

Read time: 7 minutes

Last edited: Mar 25, 2024

Metrics is available for Pro and Enterprise plans

Metrics is available to customers on a Pro or Enterprise plan. To learn more, read about our pricing. To add Experimentation to your plan, contact Sales.

Overview

This topic explains metric types and how to create metrics to use with LaunchDarkly features, such as experiments and guarded releases.

Choosing a metric type

You should choose a metric type that correctly measures the effect of a change on your customers or codebase. If you are unsure of what metric type to use, it may be helpful to begin by determining what kind of data you are trying to measure. For examples of common metrics and how to configure them, read Example metrics.

This table includes examples of different kinds of data you can measure with different metric types, and common randomization unit context kind mappings:

Metric type	Example uses	Example randomization unit
Click conversion	How often do customers click a "Save" button? How many times do customers click on a link? When is the best point during a process to display a sign-up invitation?	`user`
Custom conversion/binary	Do customer searches call a particular service? Do customers contact customer service within a set period of time? Do customers renew their contract within 30 days? Do customer payments succeed?	`user` `organization`
Custom numeric	How much do customers spend per transaction in my store? How much do customers spend in total? How many items do customers purchase per transaction? How many items do customers purchase total? How much time do customers spend on a page? How long does it take for a server to respond to a request? How long until the time to first byte (TTFB)?	`user` `guest` `request`
Page view conversion	How many times do end users view a blog post?	`user`

Metric analysis

When you create a metric, you must decide how you want to handle its metric and unit analysis. Units are the context instances that encounter a metric.

The following sections describe LaunchDarkly's metric analysis options.

Unit aggregation method

Expand Unit aggregation method

When you create a metric, you must select whether to aggregate event values per unit by average or by sum, depending on your use case.

Funnel experiments can only use metrics aggregated by average

For metrics you plan to use in funnel experiments, you must select "Average individual unit values."

For example, if your metric is tracking visitor purchases, you can use average and sum to understand the following information as part of an experiment:

average: the proportion of customers who make at least one purchase
sum: the number of purchases totaled to find the average number of purchases per customer

Imagine you have three visitors in your online store who make the following number of purchases:

Visitor A: 3 purchases
Visitor B: 0 purchases
Visitor C: 2 purchases

In the example above, three customers make a total of five purchases. Because you have configured the number of purchases as your metric, this number is what LaunchDarkly uses to calculate average and sum.

This table shows an example of how LaunchDarkly would calculate the average versus the sum of your metric's results:

Values as an average	Values as a sum
`(1+0+1)/3 = 0.67` Two out of three, or 0.67, of your visitors make at least one purchase	`(3+0+2)/3 = 1.67` Your visitors make an average of 1.67 purchases each

Analysis method

Expand Analysis method

Percentile analysis methods are in beta

The default metric analysis method is "mean." Percentile analysis methods, such as p50 and p99, are in beta. You cannot use a metric with a percentile analysis method in funnel experiments.

You can analyze a metric by the following methods:

mean: the mean is the default analysis method
p50: the 50th percentile, which is also the median
p75: the 75th percentile
p90: the 90th percentile
p95: the 95th percentile
p99: the 99th percentile

Funnel experiments can only use metrics that use the mean analysis method.

The closer the percentile is to 50, the closer it is to the median. Percentiles closer to the median, such as p75, are useful for analyzing general trends. For example, you might want to use the p50 or p75 methods for analyzing things like API latency.

Higher percentiles are better for detecting outliers. For example, if you have an endpoint that generally works well, but handles a single customer with large amounts of data poorly, you might want to use the p95 or p99 methods for the related metric.

Units without events

Expand Units without events

Sometimes a context instance will encounter a metric without completing the action the metric is measuring. When this happens the SDK does not send a metric event to LaunchDarkly. For example, if a conversion metric is measuring completed sign-ups through an email submission form, the SDK will not send an event if the user context opens the form but never clicks the "submit" button. LaunchDarkly automatically sets the value for these units without events to 0 for custom conversion/binary, click, and page view metrics. However, for custom numeric metrics, you can configure LaunchDarkly to exclude these units from the analysis entirely.

For custom numeric metrics that use a percentile analysis method:

LaunchDarkly automatically excludes units without events from the analysis

For custom numeric metrics that use the mean analysis method:

Setting the value for units without events to zero is best for metrics where an incomplete process can be treated the same as 0, such as tracking cart totals for an online store. In this example, customers who put items in their cart but never completed the checkout process are treated as if they purchased $0.
Excluding units without events from the analysis is best for latency metrics. If LaunchDarkly never receives an event for a context instance, you do not want to default to 0 because LaunchDarkly would interpret this as an extremely fast latency time, which would skew or invalidate the results.

Randomization units

Expand Randomization units

When you create a metric, you must select a randomization unit. The randomization unit is the context kind, such as user, device, or request, that the metric can measure events from. To learn more, read Randomization units.