Amazon Kinesis

Overview

This topic explains how to create and test an Amazon Kinesis destination for Data Export.

You can export all of the flag evaluation and event data generated by LaunchDarkly SDKs to a Kinesis data stream.

Prerequisites

To set up a Kinesis destination, you must configure your AWS account with the following information:

• A Kinesis stream you want to export the data to. You can use an existing stream if you already have one.
• An IAM policy that can put records into your Kinesis stream
• An IAM role that LaunchDarkly can assume that provides access to the policy

If you don't already have a Kinesis stream, create one by reading Amazon's documentation.

Creating the IAM Policy

This section explains how to create an IAM policy to support the Kinesis Data Export.

To create an IAM policy:

1. Sign in to the Identity and Access Management (IAM) console.
2. Follow Amazon's documentation to create an IAM policy that allows LaunchDarkly to write to your Kinesis Stream.
3. Select the Create Policy from JSON option.
4. Use the following template policy to configure the Policy Document field:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
          "kinesis:PutRecord",
          "kinesis:PutRecords"
      ],
      "Resource": [
          "arn:aws:kinesis:{region}:{account-id}:stream/{stream-name}"
      ]
    }
  ]
}
COPY

5. If you have a stream ARN already configured, you can paste the entire ARN in. At least, change the following fields to include the values you specified during IAM policy configuration:

• {region}
• {account-id}
• {stream-name}.

6. Click Save.
7. After you create the policy, write down or save the policy's name.

Creating an IAM Role

This section explains how to create a cross-account IAM role for LaunchDarkly to assume so it can access the policy.

To create a cross-account IAM role:

1. Log into AWS and access the IAM section.
2. Copy the account ID:

554582317989
COPY

3. Enter the account ID into the Account ID field. This allows LaunchDarkly to assume the role:

4. Add the policy you created earlier to this role. Write down or save the role ARN.

Creating the Kinesis Destination in LaunchDarkly

Using the region, stream name, and role ARN from earlier in the procedure, you can now create the Kinesis destination within LaunchDarkly.

To create the Kinesis destination:

1. Navigate to the Integrations page and find "Amazon Kinesis."
2. Click Add integration. The "Create a destination" panel appears.
3. (Optional) Enter a Name for the destination.
4. Choose an environment from the Environment menu.
5. Enter the AWS data you saved earlier:

• AWS Kinesis stream region
• ROle address
• AWS Kinesis stream name

6. Click Save destination.

Testing a Kinesis Destination

After you save the destination, send a test event to confirm that the destination is configured properly.

To send a test event:

1. Navigate to the Integrations page and find "Amazon Kinesis."
2. Click into the "Amazon Kinesis" section to display a list of destinations.
3. Click the edit icon next to the destination you want to test. The "Edit destination" panel appears.
4. In the "Send a test event" section, click Send event.
5. If you have configured the destination correctly, an event is logged in the Kinesis destination.
6. Identify a successful test event by searching for the context key or flag key LaunchDarklyDataExportTestEvent. When this key appears in the destination, you know the test event arrived.

Consuming the Kinesis Data Stream

Kinesis Data Streams are very flexible. You can consume them in a lambda function, write an application to consume the streams, and use a number of pre-built AWS integrations.

A common use case is to use the Data Stream to populate a data store, such as S3 or Redshift, by setting the Kinesis Data Stream as a source to a Kinesis Data Firehose. To learn how to do this, read Amazon's documentation.

Enabling Data Export for flags and environments

After you create a Data Export destination, you must start sending flag or environment event data to it. You can enable Data Export for individual flags, or for all the flags in an environment.

To learn more about exporting event data from flags and environments, read Exporting event data for flags and environments.

Deleting a Kinesis destination

You can delete a Data Export destination from the Integrations screen.

To delete a destination:

1. Navigate to the Integrations page and find "Amazon Kinesis."
2. Click into the "Amazon Kinesis" section to display a list of destinations.
3. Click the edit icon next to the destination you wish to delete. The "Edit destination" panel appears.
4. Click Delete destination.