AI configs
Read time: 2 minutes
Last edited: Nov 08, 2024
The AI configs product is only available in early access for customers on Foundation and Enterprise plans. To request early access, navigate to AI configs and join the waitlist.
Overview
This category explains how to use LaunchDarkly to manage your AI configs. An AI config is a resource that you create in LaunchDarkly. You can use AI configs to customize, test, and roll out new large language models (LLMs) and prompts within your generative AI applications.
Within each AI config, you define one or more versions, each of which includes your AI prompts and model configurations. You can also define targeting rules, just like you do with feature flags, to make sure that particular prompts and model configurations are served to particular end users of your application.
Then, within your application, you use one of LaunchDarkly's AI SDKs. The SDK determines which prompt and model your application should serve to which contexts. The SDK can also customize the prompt based on context attributes and other variables that you provide. This means both the prompt and the model evaluation are specific to each end user, at runtime. You can update your prompts, specific to each end user, without redeploying your application.
After you use this customized config in your AI model generation, you can use the SDK to record various metrics, including duration and tokens. These appear in the LaunchDarkly user interface for each AI config.
The topics in this category explain how to create AI configs and versions, update targeting rules, monitor related metrics, and incorporate AI configs in your application.