Tracking AI metrics
Read time: 8 minutes
Last edited: Dec 19, 2024
The AI configs product is only available in early access for customers on select plans. To request early access, navigate to AI configs and join the waitlist.
The AI SDKs are designed for use with the AI configs product. The AI SDKs are currently in an alpha version.
Overview
This topic explains how to record metrics from your AI model generation, including duration, satisfaction, and tokens. This feature is available for AI SDKs only.
About AI metrics
To help you track how your AI model generation is performing, the AI SDKs provide options to record metrics from your model generation. LaunchDarkly displays these metrics on the AI config Monitoring tab in the user interface.
The SDK provides separate functions to record metrics for several of the models that you can select when you set up your AI config variation in the LaunchDarkly UI. If your model is not directly supported in the SDK, you can use the SDK's other track* functions to record these metrics manually.
AI SDKs
This feature is available for all of the AI SDKs:
.NET (AI)
Expand .NET (AI) code sample
Use the TrackRequest function to record metrics from your AI model generation.
Here's how:
If you would like to do any additional tracking, besides what LaunchDarkly provides, it is your responsibility to fill in the Response object with the data you want to track.
You can also use the SDK's other Track* functions to record these metrics manually. The TrackRequest function is expecting a response, so you may need to do this if your application requires streaming.
Each of the Track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the Track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call Config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.
Go (AI)
Expand Go (AI) code sample
Use the TrackRequest function to record metrics from your AI model generation.
Here's how:
Alternatively, you can use the SDK's other Track* functions to record these metrics manually. The TrackRequest function is expecting a response, so you may need to do this if your application requires streaming.
Each of the Track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the Track* functions from across all variations of the AI config.
Here's how to record metrics manually:
To learn more, read Tracker.
Node.js (AI)
Expand Node.js (AI) code sample
Use one of the track[Model]Metrics functions to record metrics from your AI model generation. The SDK provides separate track[Model]Metrics functions for several of the models that you can select when you set up your AI config variation in the LaunchDarkly user interface.
Here's how:
You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track[Model]Metrics function. The track[Model]Metrics functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.
Python (AI)
Expand Python (AI) code sample
Use one of the track_[model]_metrics functions to record metrics from your AI model generation. The SDK provides separate track_[model]_metrics functions for several of the models that you can select when you set up your AI config variations in the LaunchDarkly user interface.
Here's how:
You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track_[model]_metrics function. The track_[model]_metrics functions are expecting a response, so you may also need to do this if your application requires streaming.
Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all variations of the AI config.
Here's how to record metrics manually:
Make sure to call config again each time you use the tracker and generate content from your AI model.
To learn more, read LDAIConfigTracker.