• HOME
  • INTEGRATIONS
  • SDKS
  • GUIDES
  • API DOCS
No results for ""
EXPAND ALL
CLOSE
launchdarkly.com

EDIT ON GITHUB

Using a persistent feature store

Read time: 6 minutes
Last edited: Sep 19, 2020

Overview

This topic explains how to use a persistent feature store to keep flag data.

In their default configuration, LaunchDarkly's server-side SDKs:

  • connect to LaunchDarkly and receive feature flag data
  • store the flags in memory
  • update the in-memory state if LaunchDarkly sends updates

Flag evaluations always refer to the last known state in memory. This collection of last known flag data is the "feature store."

Alternatively, you can use a database for the feature store. Most of the server-side SDKs support Redis, DynamoDB, and Consul for this purpose.

Whichever database you use, there are two ways to use it:

  • Exactly like the default configuration, except substituting a database for the in-memory store, or
  • Using only the database as a source of flag data, without connecting to LaunchDarkly.

Using a persistent feature store while still connecting to LaunchDarkly

In this configuration, the SDK receives feature flag data from LaunchDarkly and puts it in the feature store. The only difference is that the store is in a database.

When flags are evaluated, the SDK checks the database to get the latest flag state, usually with some form of in-memory caching to improve performance.

The main reason to do this is to accelerate flag updates when your application has to restart, and after restarting, it takes longer to establish a connection to LaunchDarkly than you want. If you have a persistent feature store that has already been populated, the SDK can still evaluate flags using the last known flag state from the store until newer data is available from LaunchDarkly.

To set up this configuration, most people create some kind of object for the specific type of database and put it in the client configuration's feature store property. In PHP, this property is called the "feature requester".

An example configuration is below:

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 examplepackage "github.com/launchdarkly/go-server-sdk-some-example-database"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 examplepackage.DataStore().SomeStoreOptions(),
10)
11client := ld.MakeCustomClient(sdkKey, config, waitTime)
12
13
14// Before version 5.0, the syntax was different:
15import (
16 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
17 examplepackage "gopkg.in/launchdarkly/go-server-sdk.v4/some-example-database"
18)
19
20factory, err := examplepackage.NewExampleFeatureStoreFactory(someStoreOptions)
21config := ld.DefaultConfig
22config.FeatureStoreFactory = factory
23client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read PersistentDataStore.

If there are multiple instances of your application configured in this way with the same database, the same data gets written to the database multiple times, because each instance receives feature flags from LaunchDarkly.

This is harmless, but it is inefficient, so you may want to use a persistent feature store without connecting to LaunchDarkly, as described below.

Using a persistent feature store without connecting to LaunchDarkly

This is similar to the previous configuration, except that the SDK does not connect to LaunchDarkly at all. Instead, it relies on some other process which does have a LaunchDarkly connection to write the latest flag data to the database, where the SDK will then read it.

The other process could be the LaunchDarkly Relay Proxy, or any other application that creates a SDK client with the same persistent store. To learn more about the Relay Proxy, read The LaunchDarkly Relay Proxy.

Since the Relay Proxy is also known as the LaunchDarkly Daemon, some versions of the SDKs refer to this mode as "LDD mode." Creating the client is the same as above in terms of specifying the persistent store, but you must also add an option to make the SDK not connect to LaunchDarkly.

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 examplepackage "github.com/launchdarkly/go-server-sdk-some-example-database"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 examplepackage.DataStore().SomeStoreOptions(),
10)
11config.DataSource = ldcomponents.ExternalUpdatesOnly() // <-- the extra option
12client := ld.MakeCustomClient(sdkKey, config, waitTime)
13
14
15// Before version 5.0, the syntax was different:
16import (
17 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
18 examplepackage "gopkg.in/launchdarkly/go-server-sdk.v4/some-example-database"
19)
20
21factory, err := examplepackage.NewExampleFeatureStoreFactory(someStoreOptions)
22config := ld.DefaultConfig
23config.FeatureStoreFactory = factory
24config.UseLdd = true // <-- the extra option
25client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read ExternalUpdatesOnly.

Using Redis

All of the LaunchDarkly server-side SDKs support Redis (see "Platform-specific notes" below).

The available options are slightly different in each language, but you can always specify the following:

  • The Redis host address (defaults to localhost:6379)
  • A prefix string to add to all keys used by the store, to avoid collisions in case the database is also being used for some other purpose
  • The length of time that recently read or updated data should be cached in memory

In the following examples, the Redis feature store is set to use a host address of my-redis:6379, a prefix string of "my-key-prefix", and a cache TTL of 30 seconds.

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 ldredis "github.com/launchdarkly/go-server-sdk-redis"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 ldredis.DataStore().
10 HostAndPort("my-redis", 6379).
11 Prefix("my-key-prefix"),
12).CacheSeconds(30)
13client := ld.MakeCustomClient(sdkKey, config, waitTime)
14
15
16// Before version 5.0, the syntax was different:
17import (
18 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
19 ldredis "gopkg.in/launchdarkly/go-server-sdk.v4/redis"
20)
21
22factory, err := ldredis.NewRedisFeatureStoreWithDefaults(
23 ldredis.HostAndPort("my-redis", 6379),
24 ldredis.Prefix("my-key-prefix"),
25 ldredis.CacheTTL(30 * time.Second))
26config := ld.DefaultConfig
27config.FeatureStoreFactory = factory
28client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read go-server-sdk-redis.

Platform-specific notes

  • C: This integration is part of the main SDK distribution.
  • Haskell You must install an additional package named launchdarkly-server-sdk-redis.
  • Go: This integration is in github.com/launchdarkly/go-server-sdk-redis for 5.0.0 or higher of the SDK. In earlier SDK versions, it is in the main SDK distribution as the subpackage redis.
  • Java: In Java SDK 5.0 and later, you must install the additional package com.launchdarkly.launchdarkly-java-server-sdk-redis-store. In Java SDK 4.x and earlier, this integration is part of the main SDK distribution.
  • .NET: You must install an additional package named LaunchDarkly.ServerSdk.Redis.
  • Node.js: This integration is part of the main SDK distribution.
  • PHP: This integration is part of the main SDK distribution, but you must also must install the package predis/predis.
  • Python: This integration is part of the main SDK distribution, but you must also install the package redis.
  • Ruby: This integration is part of the main SDK distribution, but you must also install the gems redis and connection_pool.

Using DynamoDB

Data size limit in DynamoDB

DynamoDB does not support storing values greater than 400KB (or slightly less, since that number includes the size of the column metadata). Therefore, using DynamoDB as a persistent feature store will not work if the JSON representation of any feature flag or user segment exceeds that size.

To see the JSON representations of all flags and segments, query https://app.launchdarkly.com/sdk/latest-all with your SDK key in the Authorization header.

All of the current versions of the server-side SDKs except C support DynamoDB (see "Platform-specific notes" below). DynamoDB is a particularly useful solution if you are running code in AWS Lambda, since it can be accessed from Lambda without needing access to any VPC resource.

In your application code, the only required parameter is the table name, although you can also specify any other options supported by AWS; by default, the DynamoDB driver will expect to get your AWS credentials and region from environment variables or local configuration files, as described in the AWS SDK documentation.

The table must already exist before your application starts. It must have a partition key called "namespace", and a sort key called "key". (The SDK does not create the table automatically because it would not know what values to use for other properties such as permissions and throughput.)

DynamoDB imposes a limit of 400KB on the total size of any database item. In this implementation, each feature flag or user segment is a single item, so the feature store will not be able to persist any flag or segment whose JSON representation is larger than that limit.

In the following examples, the DynamoDB feature store is set to use a table called "my-table" and a cache TTL of 30 seconds. The DynamoDB feature store does support using a key prefix, as shown in the Redis examples, but it is uncommon for one DynamoDB table to be shared by multiple applications.

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 lddynamodb "github.com/launchdarkly/go-server-sdk-dynamodb"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 lddynamodb.DataStore("my-table"),
10).CacheSeconds(30)
11client := ld.MakeCustomClient(sdkKey, config, waitTime)
12
13
14// Before version 5.0, the syntax was different:
15import (
16 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
17 "gopkg.in/launchdarkly/go-server-sdk.v4/lddynamodb"
18)
19
20factory, err := lddynamodb.NewDynamoDBFeatureStoreFactory("my-table",
21 lddynamodb.CacheTTL(30 * time.Second))
22config := ld.DefaultConfig
23config.FeatureStoreFactory = factory
24client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read go-server-sdk-dynamodb.

Platform-specific notes

  • C: Currently there is no DynamoDB integration for the C server-side SDK.
  • Haskell: Currently there is no DynamoDB integration for the Haskell server-side SDK.
  • Go: This integration is in github.com/launchdarkly/go-server-sdk-dynamodb for version 5.0.0 or higher of the SDK. In versions 4.5.1 and higher, but below 5.0.0, it is in the main SDK distribution as the subpackage lddynamodb.
  • Java: You must install the additional package com.launchdarkly.launchdarkly-java-server-sdk-dynamodb-store. You must also add software.amazon.awssdk.dynamodb if your application does not already use the AWS SDK.
  • .NET: You must install the additional package LaunchDarkly.ServerSdk.DynamoDB.
  • Node.js: You must install the additional package launchdarkly-node-server-sdk. Also, as of the 2.0.0 release of that package, the aws-sdk package that it uses is not automatically loaded as a transitive dependency; this saves space when running in AWS Lambda, where aws-sdk is built in. If you are not running in Lambda, you must separately install aws-sdk.
  • PHP: This integration is part of the main SDK distribution as of version 3.5.0, but you must also install the package aws/aws-sdk-php.
  • Python: This integration is part of the main SDK distribution as of version 6.7.0, but you must also install the package boto3.
  • Ruby: This integration is part of the main SDK distribution as of version 5.1.1, but you must also install the gem aws-sdk-dynamodb.

Using Consul

Data size limit in Consul

Consul does not support storing values greater than 512KB. Therefore, using Consul as a persistent feature store will not work if the JSON representation of any feature flag or user segment exceeds that size.

To see the JSON representations of all flags and segments, query https://app.launchdarkly.com/sdk/latest-all with your SDK key in the Authorization header.

All of the current versions of the server-side SDKs except C support Consul (see "Platform-specific notes" below).

In the following examples, the Consul feature store is set to use a host address of my-consul:8100, a prefix string of "my-key-prefix", and a cache TTL of 30 seconds.

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 ldconsul "github.com/launchdarkly/go-server-sdk-consul"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 ldconsul.DataStore().
10 Address("http://my-consul:8100").
11 Prefix("my-key-prefix"),
12).CacheSeconds(30)
13client := ld.MakeCustomClient(sdkKey, config, waitTime)
14
15
16// Before version 5.0, the syntax was different:
17import (
18 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
19 "gopkg.in/launchdarkly/go-server-sdk.v4/ldconsul"
20)
21
22factory, err := ldconsul.NewConsulFeatureStoreFactory(
23 ldconsul.Address("http://my-consul:8100"),
24 ldconsul.Prefix("my-key-prefix"),
25 ldconsul.CacheTTL(30 * time.Second))
26config := ld.DefaultConfig
27config.FeatureStoreFactory = factory
28client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read go-server-sdk-consul.

Platform-specific notes

  • C: Currently there is no Consul integration for the C server-side SDK.
  • Haskell: Currently there is no Consul integration for the Haskell server-side SDK.
  • Go: This integration is in github.com/launchdarkly/go-server-sdk-consul for version 5.0.0 or higher of the SDK. In versions 4.5.0 and higher, but below 5.0.0, it is in the main SDK distribution as the subpackage ldconsul.
  • Java: You must install the additional package com.launchdarkly.launchdarkly-java-server-sdk-consul-store.
  • .NET: You must install the additional package LaunchDarkly.ServerSdk.Consul.
  • Node.js: You must install the additional package launchdarkly-node-server-consul.
  • PHP: This integration is part of the main SDK distribution as of version 3.5.0, but you must also install the package aws/sensiolabs/consul-php-sdk.
  • Python: This integration is part of the main SDK distribution as of version 6.8.1, but you must also install the package python-consul. Python 3.3 and 3.4 are not supported.
  • Ruby: This integration is part of the main SDK distribution as of version 5.1.1, but you must also install the gem diplomat.