Python (AI) SDK reference
Read time: 4 minutes
Last edited: Nov 08, 2024
The AI configs product is only available in early access for customers on Foundation and Enterprise plans. To request early access, navigate to AI configs and join the waitlist.
The AI SDKs are designed for use with the AI configs product. The Python (AI) SDK is currently in an alpha version.
Overview
This topic documents how to get started with the Python (AI) SDK, and links to reference information on all of the supported features.
LaunchDarkly's SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications:
Resource | Location |
---|---|
SDK API documentation | SDK API docs |
GitHub repository | python-server-sdk-ai |
Sample application | |
Published module | PyPI |
Get started
LaunchDarkly AI SDKs interact with AI configs. AI configs are the LaunchDarkly resources that manage your AI prompts and model configurations.
You can use the Python (AI) SDK to customize your AI config based on the context that you provide. This means both the prompt and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDKs to record metrics from your AI model generation, including duration and tokens.
Follow these instructions to start using the Python (AI) SDK in your Python application.
Understand version compatibility
The LaunchDarkly Python (AI) SDK is compatible with Python 3.8.0 and higher.
Install the SDK
First, install the AI SDK as a dependency in your application using your application's dependency manager. If you want to depend on a specific version, refer to the SDK releases page to identify the latest version.
Here's how:
pip install launchdarkly-server-sdk-ai
Next, import the LaunchDarkly LDAIClient
into your application code:
import ldclientfrom ldclient import Contextfrom ldclient.config import Configfrom ldai.client import LDAIClient
Initialize the client
After you install and import the AI SDK, create a single, shared instance of LDAIClient
. Specify your SDK key here to authorize your application to connect to a particular environment within LaunchDarkly.
The Python (AI) SDK uses an SDK key. Keys are specific to each project and environment. They are available from the Environments list for each project. To learn more about key types, read Keys.
Here's how:
ldclient.set_config(Config("sdk-key-123abc"))aiclient = LDAIClient(ldclient.get())
Configure the context
Next, configure the context that will use the AI config, that is, the context that will encounter generated AI in your application. The context attributes determine which version of the AI config LaunchDarkly serves to the end user, based on the targeting rules in your AI config. If you are using template variables in the prompt in your AI config's versions, the context attributes also fill in values for the template variables.
Here's how:
context = Context.builder("context-key-123abc") \.set("firstName", "Sandy") \.set("lastName", "Smith") \.set("email", "sandy@example.com") \.set("groups", ["Google", "Microsoft"]) \.build()
Customize an AI config
Then, use model_config
to customize the AI config. This function returns the customized prompt and model, which you can pass directly to your AI. You can customize the prompt using both context attributes and additional variables that you pass to model_config
.
Here's how:
fallback_value = {'model': { 'modelId': 'my-default-model', },'enabled': True,}config_value = aiclient.model_config('ai-config-key-123abc', context, fallback_value, { 'example_custom_variable': 'example_custom_value'})
To learn more, read Customizing AI configs.
Record metrics from AI model generation
Finally, use one of the track_[model]
functions to record metrics from your AI model generation.
Here's how:
tracker = config_value.trackercompletion = tracker.track_openai(# Pass in the result of the OpenAI operation.# When calling the OpenAI operation, use details from config_value.# For instance, you can pass config_value.config['model']['modelId']# and config_value.config['prompt'] to your specific OpenAI operation.)
Alternatively, you can use the SDK's other track*
functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track_[model]
function. The track_[model]
functions are expecting a response, so you may also need to do this if your application requires streaming.
To learn more, read Tracking AI metrics.
Supported features
This SDK supports the following features: