AI configs
Read time: 2 minutes
Last edited: Dec 17, 2024
The AI configs product is only available in early access for customers on select plans. To request early access, navigate to AI configs and join the waitlist.
Overview
This category explains how to use LaunchDarkly to manage your AI configs. An AI config is a resource that you create in LaunchDarkly. You can use AI configs to customize, test, and roll out new large language models (LLMs) within your generative AI applications.
With AI configs, you can:
- manage your model configuration outside of your application code. This means you can update your model details and messages at runtime. Additionally, teammates with access to LaunchDarkly, but without familiarity with your codebase, can collaborate and iterate on messages.
- upgrade your app to the newest model version as soon as it's released, then roll out the changes to your customers gradually and safely.
- add a config for a new model provider, and progressively move your production traffic to the new provider.
- compare variations and determine which one performs better, based on satisfaction, cost, or other metrics. You can compare different messages with models from the same provider, or the same messages with different model providers.
About AI configs
Within each AI config, you define one or more variations, each of which includes a model configuration, and one or more messages. You can also define targeting rules, just like you do with feature flags, to make sure that particular messages and model configurations are served to particular end users of your application.
Then, within your application, you use one of LaunchDarkly's AI SDKs. The SDK determines which messages and model your application should serve to which contexts. The SDK can also customize the messages based on context attributes and other variables that you provide. This means both the messages and the model evaluation are specific to each end user, at runtime. You can update your messages, specific to each end user, without redeploying your application.
After you use this customized config in your AI model generation, you can use the SDK to record various metrics, including generation count, tokens, and satisfaction rate. These appear in the LaunchDarkly user interface for each AI config variation.
The topics in this category explain how to create AI configs and variations, update targeting rules, monitor related metrics, and incorporate AI configs in your application.