• HOME
  • INTEGRATIONS
  • SDKS
  • GUIDES
  • API DOCS
No results for ""
EXPAND ALL

EDIT ON GITHUB

Using Consul as a persistent feature store

Read time: 2 minutes
Last edited: Jul 28, 2021

Overview

This topic explains how to use the SDK Consul integration as a persistent feature store.

Data size limit in Consul

Consul does not support storing values greater than 512KB. Using Consul as a persistent feature store does not work if the JSON representation of any feature flag or user segment exceeds that size. To learn more about these limitations, read Consul's documentation.

To see the JSON representations of all flags and segments, query https://app.launchdarkly.com/sdk/latest-all with your SDK key in the Authorization header.

How the SDKs store data in Consul

The Consul integrations for all LaunchDarkly server-side SDKs use the same conventions, so that SDK instances and Relay Proxy instances sharing a single Consul store can interoperate correctly.

The storage schema is as follows:

  • There is always a "prefix" string that provides a namespace for the overall data set. If you do not specify a prefix in your configuration, it is launchdarkly.
  • For each data item that the SDK can store, such as a feature flag, there is a Consul key-value pair where the key is PREFIX/TYPE/KEY. PREFIX is the configured prefix string. TYPE denotes the type of data such as features and segments. KEY is the unique key of the item. For example, a KEY could be the flag key for a feature flag. The value is a serialized representation of that item, in a format that is determined by the SDK.
  • An additional key, PREFIX/$inited, is created with an arbitrary value when the SDK stores a full set of feature flag data. This allows a new SDK instance to check whether there is already a valid data set that was stored earlier.
  • The SDK may use additional keys starting with the PREFIX string, so you should not assume that the TYPE values mentioned above and $inited are the only possible keys. But the SDK never adds, modifies, or removes any keys in Consul other than ones starting with the PREFIX, so it is safe to share a Consul instance that is also being used for other purposes.

Server-side SDKs

In the following examples, the Consul feature store is set to use a host address of my-consul:8100, a prefix string of "my-key-prefix", and a cache TTL of 30 seconds.

This feature is available in the following server-side SDKs:

.NET

If using the .NET SDK, you must install the additional package LaunchDarkly.ServerSdk.Consul.

1using LaunchDarkly.Sdk.Server;
2using LaunchDarkly.Sdk.Server.Integrations;
3var config = Configuration.Builder(sdkKey)
4 .DataStore(
5 Components.PersistentDataStore(
6 Consul.DataStore().Address("http://my-consul:8100")
7 ).CacheSeconds(30)
8 )
9 .Build();
10var client = new LDClient(config);
11// Before version 6.0, the syntax was different:
12
13using LaunchDarkly.Client;
14using LaunchDarkly.Client.Consul;
15
16var store = ConsulComponents.ConsulFeatureStore()
17 .WithAddress(new Uri("http://my-consul:8100"))
18 .WithCaching(FeatureStoreCacheConfig.Enabled.WithTtlSeconds(30));
19
20var config = Configuration.Default(sdkKey)
21 .WithFeatureStoreFactory(store);
22var client = new LDClient(config);

To learn more, read dotnet-server-sdk-consul.

Go

the Go integration is in github.com/launchdarkly/go-server-sdk-consul for version 5.0.0 or higher of the SDK. In versions 4.5.0 and higher, but below 5.0.0, it is in the main SDK distribution as the subpackage ldconsul.

1import (
2 ld "gopkg.in/launchdarkly/go-server-sdk.v5"
3 "gopkg.in/launchdarkly/go-server-sdk.v5/ldcomponents"
4 ldconsul "github.com/launchdarkly/go-server-sdk-consul"
5)
6
7var config ld.Config
8config.DataStore = ldcomponents.PersistentDataStore(
9 ldconsul.DataStore().
10 Address("http://my-consul:8100").
11 Prefix("my-key-prefix"),
12).CacheSeconds(30)
13client := ld.MakeCustomClient(sdkKey, config, waitTime)
14
15
16// Before version 5.0, the syntax was different:
17import (
18 ld "gopkg.in/launchdarkly/go-server-sdk.v4"
19 "gopkg.in/launchdarkly/go-server-sdk.v4/ldconsul"
20)
21
22factory, err := ldconsul.NewConsulFeatureStoreFactory(
23 ldconsul.Address("http://my-consul:8100"),
24 ldconsul.Prefix("my-key-prefix"),
25 ldconsul.CacheTTL(30 * time.Second))
26config := ld.DefaultConfig
27config.FeatureStoreFactory = factory
28client := ld.MakeCustomClient(sdkKey, config, waitTime)

To learn more, read go-server-sdk-consul.

Java

If using the Java SDK, you must install the additional package com.launchdarkly.launchdarkly-java-server-sdk-consul-store.

1import com.launchdarkly.sdk.server.*;
2import com.launchdarkly.sdk.server.integrations.*;
3
4LDConfig config = new LDConfig.Builder()
5 .dataStore(
6 Components.persistentDataStore(
7 Consul.dataStore().url(URL.create("http://my-consul:8100"))
8 .prefix("my-key-prefix")
9 ).cacheSeconds(30)
10 )
11 .build();
12LDClient client = new LDClient(sdkKey, config);
13
14
15// Before version 5.0, the syntax was different:
16import com.launchdarkly.client.*;
17import com.launchdarkly.client.consul.*;
18
19LDConfig deprecatedConfig = new Config.Builder()
20 .featureStoreFactory(
21 ConsulComponents.consulFeatureStore()
22 .url(URL.create("http://my-consul:8100"))
23 .prefix("my-key-prefix")
24 .caching(FeatureStoreCacheConfig.enabled().withTtlSeconds(30))
25 )
26 .build();

To learn more, read java-server-sdk-consul.

Node.js (server-side)

If using the Node.js SDK you must install the additional package launchdarkly-node-server-consul.

1const LaunchDarkly = require('launchdarkly-node-server-sdk');
2const ConsulFeatureStore = require('launchdarkly-node-server-sdk-consul');
3
4const store = ConsulFeatureStore({
5 consulOptions: {
6 host: 'my-consul',
7 port: 8100
8 },
9 prefix: 'my-key-prefix',
10 cacheTTL: 30
11});
12
13const config = {
14 featureStore: store
15};
16const client = LaunchDarkly.init(sdkKey, config);

PHP

The PHP integration is part of the main SDK distribution as of version 3.5.0, but you must also install the package aws/sensiolabs/consul-php-sdk.

1/*
2 There is no parameter for the cache TTL; the PHP SDK does not cache
3 data from Consul, because in PHP the entire in-memory application state is
4 normally discarded after each request.
5*/
6$fr = LaunchDarkly\Integrations\Consul::featureRequester([
7 'consul_uri' => 'http://my-consul:8100',
8 'consul_prefix' => 'my-key-prefix'
9]);
10$client = new LaunchDarkly\LDClient("your_sdk_key", [
11 'feature_requester' => $fr
12]);

Python

The Python integration is part of the main SDK distribution as of version 6.8.1, but you must also install the package python-consul. Python 3.3 and 3.4 are not supported.

1import ldclient
2from ldclient.config import Config
3from ldclient.feature_store import CacheConfig
4from ldclient.integrations import Consul
5
6store = Consul.new_feature_store(host='my-consul', port=8100,
7 prefix='my-key-prefix', caching=CacheConfig(expiration=30))
8
9config = Config(feature_store=store)
10ldclient.set_config(config)

Ruby

The Ruby integration is part of the main SDK distribution as of version 5.1.1, but you must also install the gem diplomat.

1require 'ldclient-rb'
2
3store = LaunchDarkly::Integrations::Consul.new_feature_store(
4 { url: 'http://my-consul:8100', prefix: 'my-key-prefix', expiration: 30 })
5
6config = LaunchDarkly::Config.new(
7 feature_store: store
8)
9client = LaunchDarkly::Client.new(sdk_key, config)