I’m writing a lot of yaml files at the moment, mostly manifests for kubernetes. Part of that task involves substituting variables into the yaml, because they change depending on the environment.

As an example, this manifest that kubernetes will use to create a namespace has a variable in it that allows me to use the name that I think is appropriate for the environment.

apiVersion: v1
kind: Namespace
metadata:
    name: "$NAMESPACE"

To set that $NAMESPACE variable to something useful I use a command called envsubstr

From the shell I would do something like

NAMESPACE=production envsubst < k8s/namespace.yaml | kubectl apply -f -

Which means that the information that gets passed to kubernetes would look something like

apiVersion: v1
kind: Namespace
metadata:
    name: "production"

If that’s all there was to know, this would be a very short blog post, but wait, there’s more!

envsubst will substitute all the things it sees that it thinks are variables. That is, if the example yaml looked like this

apiVersion: v1
kind: Namespace
metadata:
    name: "$NAMESPACE"
    labels:
        name: "$NAME"

And we ran our previous example command

NAMESPACE=production envsubst < k8s/namespace.yaml | kubectl apply -f -

Then envsubst will substitute both ‘variables’, but only one will have a value - creating

apiVersion: v1
kind: Namespace
metadata:
    name: "production"
    labels:
        name: ""

Which may not be what we want. We might only want the $NAMESPACE variable to be changed, and any other variables to be left alone (I will show a full example of a manifest that makes use of this at the bottom, but continue with this small example to keep focus on what’s happening).

To achieve this we explicitly tell envsubstr which variables to change - putting them in a string after the call, and surrounding the name of each with {}

NAMESPACE=production envsubst '${NAMESPACE}' < k8s/namespace.yaml | kubectl apply -f -

If there are multiple it would look something like

NAMESPACE=production envsubst '${NAMESPACE} ${FOO} ${BAR}' < k8s/namespace.yaml | kubectl apply -f -

And people can make bash expansions that look for prefixes and so on (but that’s well above my paygrade ;)

what gets produced by envsubstr is:

apiVersion: v1
kind: Namespace
metadata:
    name: "production"
    labels:
        name: "$NAME"

One more thing, gnu make has to use a slightly different syntax in the list, it needs a double $$ .

So, in a Makefile use

NAMESPACE=production envsubst '$${NAMESPACE}' < k8s/namespace.yaml | kubectl apply -f -

Finally, the example where I want some substitution to take place, and some variables to be left behind.

The reason is that I am building a command that I want to use the environment variables that are set inside the docker container’s shell.

You can see that I prefix the variables to be set in the env with PG, that’s purely so that I can differentiate them, there’s no other rhyme or reason.

So what I need to happen is my Makefile sets some of the variables, but I don’t want it to set some others to “” because they will be set by the shell when the command is run (and if I leave envsubst to its own devices, it deletes the variables, meaning the shell doesn’t know they’re even there(.

apiVersion: batch/v1
kind: Job
metadata:
  name: accounts-schema-init
  namespace: "$NAMESPACE"
spec:
  template:
    metadata:
      name: accounts-schema
    spec:
      containers:
      - name: accounts-schema
        image: "onepage/accounts-schema:$TAG"
        imagePullPolicy: "$IMAGEPULLPOLICY"
        env:
        - name: PGUSER
          valueFrom:
            secretKeyRef:
              name: account-secret
              key: username
        - name: PGPASSWORD
          valueFrom:
            secretKeyRef:
              name: account-secret
              key: password
        - name: PGDBNAME
          value: "$DBNAME"
        - name: PGHOST
          value: "$HOST"
        command:
        - /bin/sh
        - -c
        - goose -v -dir=migrations postgres "user=$PGUSER dbname=$PGDBNAME password=$PGPASSWORD host=$PGHOST sslmode=disable" up;
      restartPolicy: Never

Summary

Hopefully this gives you (almost) everything that you need to use envsubstr for substituting variables in files, and only the ones you want substituted.