Create AI model configurations

Overview

This topic explains how to create and manage custom models to use in AI configs.

An AI config is a resource that you create in LaunchDarkly. You can use AI configs to customize, test, and roll out new large language models (LLMs) within your generative AI applications.

Within each AI config, you define one or more variations, each of which includes a model configuration, and one or more messages.

When you create a variation, you have the option to use a standard model from the list provided by LaunchDarkly, or define your own custom model. This is called an AI model configuration. After you create an AI model configuration, you can use it in any variation for any AI config in your project.

Create an AI model configuration

You can create an AI model configuration either while you are creating an AI config variation, or from the AI model configs tab under Project settings.

Create an AI model configuration from a variation

To create an AI model configuration from an AI config variation:

1. Navigate to the Variations tab of the AI config and select a variation.
2. Open the Select a model menu.
3. Choose Add a model.
4. Follow the steps for completing the "Add custom model" dialog, below.

Create an AI model configuration from project settings

To create an AI model configuration from your Project settings page:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.
4. Click Add AI model config.
5. Follow the steps for completing the "Add custom model" dialog, below.

Complete the "Add custom model" dialog

To complete the "Add custom model" dialog:

1. Enter a Name for your AI model configuration.

• The name appears in the Select a model dropdown when you create a variation for an AI config.
• You also need the name if you want to create a custom role that allows or denies access to this AI model configuration.

2. Select a Provider.
3. Enter a Model ID.

• The model ID is the identifier for the model that the LLM provider uses. This must match the identifier from the provider exactly.
• You also need the model ID if you want to create a custom role that allows or denies access to this AI model configuration.

4. (Optional) Click + Add suggested parameters. The menu displays model parameters, their types, and their default values.

• Select a model parameter from the list.
• Set its value in the dialog. This will be the default value when you choose this model in an AI config variation. You can set a different value in the AI config variation if needed.
• When you customize an AI config in your SDK, you can access these parameters in the model section of the LDAIConfig.

5. (Optional) Click + Add custom parameters.

• Enter the additional parameters you want to include. Your entry must be a valid JSON object.
• These parameters and the values you specify here will always be used when you choose this model in an AI config variation.
• When you customize an AI config in your SDK, you can access these parameters in the model.custom section of the LDAIConfig.

6. Click Save.

Here is an example:

Delete an AI model configuration

To delete an AI model configuration:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.
4. Click the name of the AI model config you want to delete.
5. Click Delete AI model config.

You cannot an AI model configuration if it is currently used in a variation for an AI config.