Customizing AI configs
Read time: 7 minutes
Last edited: Dec 21, 2024
The AI configs product is only available in early access for customers on select plans. To request early access, navigate to AI configs and join the waitlist.
The AI SDKs are designed for use with the AI configs product. The AI SDKs are currently in an alpha version.
Overview
This topic explains how to customize an AI config. This feature is available for AI SDKs only.
About AI configs
The AI config feature customizes an AI config based on the context that you provide. This means both the prompt and the model evaluation in your generative AI application are specific to each end user, at runtime.
This feature requires an AI config key, the context that encounters the AI config in your application, a fallback value, and, optionally, additional variables used in the prompt template.
This feature returns the value of the customized AI config variation to use for this context, based on the AI config's targeting rules. This variation includes the model and customized messages for the context. Then, you can use the value of the AI config variation in your AI model generation. You need to use the AI config feature each time you generate content from your AI model.
Customizing messages
Customizing the AI config means that LaunchDarkly substitutes context attributes and optional variables into the messages associated with the AI config variation. If the message has multiple messages, they are all customized and returned. If the variation cannot be retrieved, for example because the AI config key does not exist, or because the SDK cannot connect to LaunchDarkly, then the SDK returns the fallback value.
Syntax for customization
When you create each message in an AI config variation, use the following syntax if you want to substitute context attributes or other variables in the message:
-
Use two curly braces to indicate variables in your application code. For example:
- Enter
This is an {{ example }} messagein the message in your AI config variation in the LaunchDarkly UI. - In the SDK, pass a dictionary or key-value pair with
exampleand yourexample_valueto the config call.
- Enter
-
Use two curly braces, the
ldctxprefix, and dot (.) notation to indicate context attributes. For example:- Enter
Describe the typical weather in {{ ldctx.city }}if you'd like to replace{{ ldctx.city }}with thecityattribute from each context that encounters this AI config. - In the SDK, the context is a required parameter to the config call. You do not need to pass any additional information to the config call.
- Enter
Customization is based on context
The AI config feature adds a context to the Contexts list, if a context with the same key does not already exist. However, each SDK customizes the AI config based only on the object you provide in the config call. In other words, the SDK does not automatically use the attributes shown on the Contexts list, and attributes are not synchronized across SDK instances. You must provide all relevant attributes for each customization in order for variables in your prompt to be substituted correctly. To learn more, read Context configuration.
AI SDKs
This feature is available for all of the AI SDKs:
.NET (AI)
Expand .NET (AI) code sample
The Config function customizes the AI config. You need to call Config each time you generate content from your AI model.
This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI config variation have their values set to the context attributes and variables you pass to Config. Then, you can pass the customized prompt directly to your AI.
Config calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config variation that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use LdAiConfig.Disabled as a fallback value, and then check for this during your application's error-handling. Alternatively, you can create a custom LdAiConfig object using LdAiConfig.New().
Here is an example of calling the Config method:
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read Config.
Go (AI)
Expand Go (AI) code sample
The Config function customizes the AI config. This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI config variation have their values set to the context attributes and variables you pass to Config. Then, you can pass the customized prompt directly to your AI.
Config calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config variation that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty, disabled Config as a fallback value, by calling ldai.Disabled(), and then check for this during your application's error-handling. Alternatively, you can create a custom Config object using NewConfig.
Here is an example of calling the Config method:
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read Config.
Node.js (AI)
Expand Node.js (AI) code sample
The config function customizes the AI config. You need to call config each time you generate content from your AI model.
This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI config variation have their values set to the context attributes and variables you pass to config. Then, you can pass the customized prompt directly to your AI.
config calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config variation that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application's error-handling. Alternatively, you can construct a JSON object, for example with values from one of the AI config variations you have created in the LaunchDarkly UI.
Here is an example of calling the config method:
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read config.
Python (AI)
Expand Python (AI) code sample
The config function customizes the AI config. You need to call config each time you generate content from your AI model.
This function returns the customized prompt and model along with a tracker instance for recording prompt metrics. Customization means that any variables you include in the prompt messages when you define the AI config variation have their values set to the context attributes and variables you pass to config. Then, you can pass the customized prompt directly to your AI.
config calls take the AI config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.
The fallback value is the value of the AI config variation that your application should use in the event of an error, for example, if the AI config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application's error handling. Alternatively, you can construct a JSON object, for example with values from one of the AI config variations you have created in the LaunchDarkly UI.
Here is an example of calling the config method:
After the function call, you can view the context that you provided to it on the Contexts list.
To learn more, read config.