Customizing AI configs
Read time: 4 minutes
Last edited: Nov 08, 2024
The AI configs product is only available in early access for customers on Foundation and Enterprise plans. To request early access, navigate to AI configs and join the waitlist.
The AI SDKs are designed for use with the AI configs product. The AI SDKs are currently in an alpha version.
Overview
This topic explains how to customize an AI config. This feature is available for AI SDKs only.
About AI configs
The AI config feature customizes an AI config based on the context that you provide. This means both the prompt and the model evaluation in your generative AI application are specific to each end user, at runtime.
This feature requires an AI config key, the context that encounters the AI config in your application, a fallback value, and, optionally, additional variables used in the prompt template.
This feature returns the value of the customized AI config version to use for this context, based on the AI config's targeting rules. This version includes the model and a customized prompt for the context. Then, you can use the value of the AI config version in your AI model generation.
Customizing the prompt means that LaunchDarkly substitutes context attributes and optional variables into the prompt template associated with the AI config version. If the prompt cannot be retrieved, for example because the AI config key does not exist, or because the SDK cannot connect to LaunchDarkly, then it returns the fallback value.
The AI config feature adds a context to the Contexts list, if a context with the same key does not already exist. However, each SDK customizes the AI config based only on the object you provide in the evaluation call. In other words, the SDK does not automatically use the attributes shown on the Contexts list, and attributes are not synchronized across SDK instances. You must provide all relevant attributes for each customization in order for variables in your prompt to be substituted correctly. To learn more, read Context configuration.
AI SDKs
This feature is available for all of the AI SDKs:
Node.js (AI)
Expand Node.js (AI) code sample
The modelConfig function customizes the AI config. This function returns the customized prompt and model, which you can pass directly to your AI. modelConfig calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config version that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application's error-handling. Alternatively, you can use a JSON object with values from one of the AI config versions you have created in the LaunchDarkly UI.
Here is an example of calling the modelConfig method:
const fallbackModel = {model: { modelId: 'my-default-model', },enabled: true,};const aiConfig: LDAIConfig = aiClient.modelConfig('ai-config-key-123abc',context,fallbackModel,{ 'exampleCustomVariable': 'exampleCustomValue' },);
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read modelConfig.
Python (AI)
Expand Python (AI) code sample
The model_config function customizes the AI config. This function returns the customized prompt and model, which you can pass directly to your AI. model_config calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config version that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application's error-handling. Alternatively, you can use a JSON object with values from one of the AI config versions you have created in the LaunchDarkly UI.
Here is an example of calling the model_config method:
key = 'ai-config-key-123abc'context = Context.builder('context-key-123abc').kind('user').set('name', 'Sandy').build()fallback_value = { 'model': { 'modelId': 'my-default-model' }, 'enabled': True, }variables = { 'example_custom_variable': 'example_custom_value' }config_value = aiclient.model_config(key, context, fallback_value, variables)
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read model_config.