Tracking AI metrics

Use one of the track[Model] functions to record metrics from your AI model generation. The SDK provides separate track[Model] functions for several of the models that you can select when you set up your AI config version in the LaunchDarkly user interface.

Here's how:

const { tracker } = aiConfig;
const completion = await tracker.trackOpenAI(
  // Pass in the result of the OpenAI operation.
  // When you call the OpenAI operation, use details from aiConfigValue.
  // For instance, you can pass aiConfig.config.prompt
  // and aiConfig.config.model to your specific OpenAI operation.
  //
  // For a complete example, visit https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/examples/openai.
);
COPY

You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track[Model] function. The track[Model] functions are expecting a response, so you may also need to do this if your application requires streaming.

Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all versions of the AI config.

Here's how to record metrics manually:

// Track your own start and stop time.
// Set duration to the time (in ms) that your AI model generation takes.
// The duration may include network latency, depending on how you calculate it.
aiConfig.tracker.trackDuration(duration);
COPY

To learn more, read LDAIConfigTracker.

Use one of the track_[model] functions to record metrics from your AI model generation. The SDK provides separate track_[model] functions for several of the models that you can select when you set up your AI config version in the LaunchDarkly user interface.

Here's how:

tracker = config_value.tracker
completion = tracker.track_openai(
  # Pass in the result of the OpenAI operation.
  # When calling the OpenAI operation, use details from config_value.
  # For instance, you can pass config_value.config['model']['modelId']
  # and config_value.config['prompt'] to your specific OpenAI operation.
)
COPY

You can also use the SDK's other track* functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track_[model] function. The track_[model] functions are expecting a response, so you may also need to do this if your application requires streaming.

Each of the track* functions sends data back to LaunchDarkly. The Monitoring tab of the AI config in the LaunchDarkly UI aggregates data from the track* functions from across all versions of the AI config.

Here's how to record metrics manually:

# Track your own start and stop time.
# Set duration to the time (in ms) that your AI model generation takes.
# The duration may include network latency, depending on how you calculate it.
config_value.tracker.track_duration(duration)
COPY

To learn more, read LDAIConfigTracker.