Runtime Configuration

The runtimevar package provides an easy and portable way to watch runtime configuration variables. This guide shows how to work with runtime configuration variables using the Go CDK.

Subpackages contain driver implementations of runtimevar for various services, including Cloud and on-prem solutions. You can develop your application locally using filevar or constantvar, then deploy it to multiple Cloud providers with minimal initialization reconfiguration.

Opening a Variable🔗

The first step in watching a variable is to instantiate a portable *runtimevar.Variable for your service.

The easiest way to do so is to use runtimevar.OpenVariable and a service-specific URL pointing to the variable, making sure you “blank import” the driver package to link it in.

import (
	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/<driver>"
)
...
v, err := runtimevar.OpenVariable(context.Background(), "<driver-url>")
if err != nil {
    return fmt.Errorf("could not open variable: %v", err)
}
defer v.Close()
// v is a *runtimevar.Variable; see usage below
...

See Concepts: URLs for general background and the guide below for URL usage for each supported service.

Alternatively, if you need fine-grained control over the connection settings, you can call the constructor function in the driver package directly (like etcdvar.OpenVariable).

import "gocloud.dev/runtimevar/<driver>"
...
v, err := <driver>.OpenVariable(...)
...

You may find the wire package useful for managing your initialization code when switching between different backing services.

See the guide below for constructor usage for each supported service.

When opening the variable, you can provide a decoder parameter (either as a query parameter for URLs, or explicitly to the constructor) to specify whether the raw value stored in the variable is interpreted as a string, a []byte, or as JSON. Here’s an example of using a JSON encoder:

// Config is the sample config struct we're going to parse our JSON into.
type Config struct {
	Host string
	Port int
}

// A sample JSON config that will decode into Config.
const jsonConfig = `{"Host": "gocloud.dev", "Port": 8080}`

// Construct a Decoder that decodes raw bytes into our config.
decoder := runtimevar.NewDecoder(Config{}, runtimevar.JSONDecode)

// Next, a construct a *Variable using a constructor or URL opener.
// This example uses constantvar.
// If you're using a URL opener, you can't decode JSON into a struct, but
// you can use the query parameter "decoder=jsonmap" to decode into a map.
v := constantvar.NewBytes([]byte(jsonConfig), decoder)
defer v.Close()
// snapshot.Value will be of type Config.

Using a Variable🔗

Once you have opened a runtimevar.Variable for the provider you want, you can use it portably.

Latest🔗

The easiest way to a Variable is to use the Variable.Latest method. It returns the latest good Snapshot of the variable value, blocking if no good value has ever been detected. The dynamic type of Snapshot.Value depends on the decoder you provided when creating the Variable.

snapshot, err := v.Latest(context.Background())
if err != nil {
	log.Fatalf("Error in retrieving variable: %v", err)
}

To avoid blocking, you can pass an already-Done context. You can also use Variable.CheckHealth, which reports as healthy when Latest will return a value without blocking.

Watch🔗

Variable also has a Watch method for obtaining the value of a variable; it has different semantics than Latest and may be useful in some scenarios. We recommend starting with Latest as it’s conceptually simpler to work with.

Other Usage Samples🔗

Supported Services🔗

GCP Runtime Configurator🔗

To open a variable stored in GCP Runtime Configurator via a URL, you can use the runtimevar.OpenVariable function as shown in the example below.

runtimevar.OpenVariable will use Application Default Credentials; if you have authenticated via gcloud auth application-default login, it will use those credentials. See Application Default Credentials to learn about authentication alternatives, including using environment variables.

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/gcpruntimeconfig"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
// The URL Host+Path are used as the GCP Runtime Configurator Variable key;
// see https://cloud.google.com/deployment-manager/runtime-configurator/
// for more details.

v, err := runtimevar.OpenVariable(ctx, "gcpruntimeconfig://projects/myproject/configs/myconfigid/variables/myvar?decoder=string")
if err != nil {
	return err
}
defer v.Close()

GCP Constructor🔗

The gcpruntimeconfig.OpenVariable constructor opens a Runtime Configurator variable.

import (
	"context"

	"gocloud.dev/gcp"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/gcpruntimeconfig"
)

// Your GCP credentials.
// See https://cloud.google.com/docs/authentication/production
// for more info on alternatives.
creds, err := gcp.DefaultCredentials(ctx)
if err != nil {
	return err
}

// Connect to the Runtime Configurator service.
client, cleanup, err := gcpruntimeconfig.Dial(ctx, creds.TokenSource)
if err != nil {
	return err
}
defer cleanup()

// You can use the VariableKey helper to construct a Variable key from
// your project ID, config ID, and the variable name; alternatively,
// you can construct the full string yourself (e.g.,
// "projects/gcp-project-id/configs/config-id/variables/variable-name").
// See https://cloud.google.com/deployment-manager/runtime-configurator/
// for more details.
//
// For this example, the GCP Cloud Runtime Configurator variable being
// referenced should have a JSON string that decodes into MyConfig.
variableKey := gcpruntimeconfig.VariableKey("gcp-project-id", "config-id", "variable-name")

// Construct a *runtimevar.Variable that watches the variable.
v, err := gcpruntimeconfig.OpenVariable(client, variableKey, runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

GCP Secret Manager🔗

To open a variable stored in GCP Secret Manager via a URL, you can use the runtimevar.OpenVariable function as shown in the example below.

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/gcpsecretmanager"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
// The URL Host+Path are used as the GCP Secret Manager secret key;
// see https://cloud.google.com/secret-manager
// for more details.

v, err := runtimevar.OpenVariable(ctx, "gcpsecretmanager://projects/myproject/secrets/mysecret?decoder=string")
if err != nil {
	return err
}
defer v.Close()

GCP Constructor🔗

The gcpsecretmanager.OpenVariable constructor opens a Secret Manager variable.

import (
	"context"

	"gocloud.dev/gcp"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/gcpsecretmanager"
)

// Your GCP credentials.
// See https://cloud.google.com/docs/authentication/production
// for more info on alternatives.
creds, err := gcp.DefaultCredentials(ctx)
if err != nil {
	return err
}

// Connect to the GCP Secret Manager service.
client, cleanup, err := gcpsecretmanager.Dial(ctx, creds.TokenSource)
if err != nil {
	return err
}
defer cleanup()

// You can use the SecretKey helper to construct a secret key from
// your project ID and the secret ID; alternatively,
// you can construct the full string yourself (e.g.,
// "projects/gcp-project-id/secrets/secret-id").
// gcpsecretmanager package will always use the latest secret value,
// so `/version/latest` postfix must NOT be added to the secret key.
// See https://cloud.google.com/secret-manager
// for more details.
//
// For this example, the GCP Secret Manager secret being
// referenced should have a JSON string that decodes into MyConfig.
variableKey := gcpsecretmanager.SecretKey("gcp-project-id", "secret-id")

// Construct a *runtimevar.Variable that watches the variable.
v, err := gcpsecretmanager.OpenVariable(client, variableKey, runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

AWS Parameter Store🔗

To open a variable stored in AWS Parameter Store via a URL, you can use the runtimevar.OpenVariable function as shown in the example below.

If you set the “awssdk=v1” query parameter, runtimevar.OpenVariable will create a default AWS Session with the SharedConfigEnable option enabled; if you have authenticated with the AWS CLI, it will use those credentials. See AWS Session to learn about authentication alternatives, including using environment variables.

If you set the “awssdk=v2” query parameter, it will instead create an AWS Config based on the AWS SDK V2; see AWS V2 Config to learn more.

If no “awssdk” query parameter is set, Go CDK will use a default (currently V1).

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/awsparamstore"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
v, err := runtimevar.OpenVariable(ctx, "awsparamstore://myvar?region=us-west-1&decoder=string")
if err != nil {
	return err
}
defer v.Close()

// Use "awssdk=v1" or "v2" to force a specific AWS SDK version.
vUsingV2, err := runtimevar.OpenVariable(ctx, "awsparamstore://myvar?region=us-west-1&decoder=string&awssdk=v2")
if err != nil {
	return err
}
defer vUsingV2.Close()

AWS Constructor🔗

The awsparamstore.OpenVariable constructor opens a Parameter Store variable.

import (
	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/awsparamstore"
)

// Establish an AWS session.
// See https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more info.
sess, err := session.NewSession(nil)
if err != nil {
	return err
}

// Construct a *runtimevar.Variable that watches the variable.
v, err := awsparamstore.OpenVariable(sess, "cfg-variable-name", runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

awsparamstore.OpenVariableV2 is similar but uses the AWS SDK V2.

import (
	"context"

	awsv2cfg "github.com/aws/aws-sdk-go-v2/config"
	ssmv2 "github.com/aws/aws-sdk-go-v2/service/ssm"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/awsparamstore"
)

// Establish a AWS V2 Config.
// See https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/ for more info.
ctx := context.Background()
cfg, err := awsv2cfg.LoadDefaultConfig(ctx)
if err != nil {
	return err
}

// Construct a *runtimevar.Variable that watches the variable.
clientV2 := ssmv2.NewFromConfig(cfg)
v, err := awsparamstore.OpenVariableV2(clientV2, "cfg-variable-name", runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

AWS Secrets Manager🔗

To open a variable stored in AWS Secrets Manager via a URL, you can use the runtimevar.OpenVariable function as shown in the example below.

If you set the “awssdk=v2” query parameter, it will instead create an AWS Config based on the AWS SDK V2; see AWS V2 Config to learn more.

If no “awssdk” query parameter is set, Go CDK will use a default (currently V1).

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/awssecretsmanager"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
// `secret-variable-name` must be a friendly name of the secret, NOT the Amazon Resource Name (ARN).
v, err := runtimevar.OpenVariable(ctx, "awssecretsmanager://secret-variable-name?region=us-east-2&decoder=string")
if err != nil {
	return err
}
defer v.Close()

// Use "awssdk=v1" or "v2" to force a specific AWS SDK version.
vUsingV2, err := runtimevar.OpenVariable(ctx, "awssecretsmanager://secret-variable-name?region=us-east-2&decoder=string&awssdk=v2")
if err != nil {
	return err
}
defer vUsingV2.Close()

AWS Constructor🔗

The awssecretsmanager.OpenVariable constructor opens a Secrets Manager variable.

import (
	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/awssecretsmanager"
)

// Establish an AWS session.
// See https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more info.
sess, err := session.NewSession(nil)
if err != nil {
	return err
}

// Construct a *runtimevar.Variable that watches the variable.
// `secret-variable-name` must be a friendly name of the secret, NOT the Amazon Resource Name (ARN).
v, err := awssecretsmanager.OpenVariable(sess, "secret-variable-name", runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

awssecretsmanager.OpenVariableV2 is similar but uses the AWS SDK V2.

import (
	"context"

	awsv2cfg "github.com/aws/aws-sdk-go-v2/config"
	secretsmanagerv2 "github.com/aws/aws-sdk-go-v2/service/secretsmanager"
	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/awssecretsmanager"
)

// Establish a AWS V2 Config.
// See https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/ for more info.
ctx := context.Background()
cfg, err := awsv2cfg.LoadDefaultConfig(ctx)
if err != nil {
	return err
}

// Construct a *runtimevar.Variable that watches the variable.
// `secret-variable-name` must be a friendly name of the secret, NOT the Amazon Resource Name (ARN).
clientV2 := secretsmanagerv2.NewFromConfig(cfg)
v, err := awssecretsmanager.OpenVariableV2(clientV2, "secret-variable-name", runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

Note that both secretsmanager:GetSecretValue and secretsmanager:DescribeSecret actions must be allowed in the caller’s IAM policy.

etcd🔗

NOTE: Support for etcd has been temporarily dropped due to dependency issues. See https://github.com/google/go-cloud/issues/2914.

You can use runtimevar.etcd in Go CDK version v0.20.0 or earlier.

HTTP🔗

httpvar supports watching a variable via an HTTP request. Use runtimevar.OpenVariable with a regular URL starting with http or https. httpvar will periodically make an HTTP GET request to that URL, with the decode URL parameter removed (if present).

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/httpvar"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
// The default opener connects to an etcd server based on the environment
// variable ETCD_SERVER_URL.

v, err := runtimevar.OpenVariable(ctx, "http://myserver.com/foo.txt?decoder=string")
if err != nil {
	return err
}
defer v.Close()

HTTP Constructor🔗

The httpvar.OpenVariable constructor opens a variable with a http.Client and a URL.

import (
	"net/http"

	"gocloud.dev/runtimevar"
	"gocloud.dev/runtimevar/httpvar"
)

// Create an HTTP.Client
httpClient := http.DefaultClient

// Construct a *runtimevar.Variable that watches the page.
v, err := httpvar.OpenVariable(httpClient, "http://example.com", runtimevar.StringDecoder, nil)
if err != nil {
	return err
}
defer v.Close()

Blob🔗

blobvar supports watching a variable based on the contents of a Go CDK blob. Set the environment variable BLOBVAR_BUCKET_URL to the URL of the bucket, and then use runtimevar.OpenVariable as shown below. blobvar will periodically re-fetch the contents of the blob.

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/blobvar"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.
// The default opener opens a blob.Bucket via a URL, based on the environment
// variable BLOBVAR_BUCKET_URL.

v, err := runtimevar.OpenVariable(ctx, "blob://myvar.txt?decoder=string")
if err != nil {
	return err
}
defer v.Close()

You can also use blobvar.OpenVariable.

Local🔗

You can create an in-memory variable (useful for testing) using constantvar:

import (
	"context"
	"fmt"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/constantvar"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.

v, err := runtimevar.OpenVariable(ctx, "constant://?val=hello+world&decoder=string")
if err != nil {
	return err
}
defer v.Close()

Alternatively, you can create a variable based on the contents of a file using filevar:

import (
	"context"

	"gocloud.dev/runtimevar"
	_ "gocloud.dev/runtimevar/filevar"
)

// runtimevar.OpenVariable creates a *runtimevar.Variable from a URL.

v, err := runtimevar.OpenVariable(ctx, "file:///path/to/config.txt?decoder=string")
if err != nil {
	return err
}
defer v.Close()