Bash Env Variable Defaults

In my previous blog post I explained a trick for setting bash env variables when you run a script. In this post I discuss how to write bash scripts that handle such variables properly.

Skip to the end if you just want the answer.

An environment variable, in bash, is for describing the environment. For example, your home directory ($HOME), your search path ($PATH), your preferred text editor ($EDITOR).

Some variables are set for the user ($HOME, $PATH) and others are optional. For example $EDITOR simply overrides the default.

I often write a suite of scripts that require certain variables: the name of the database server to access, the URL of the Git repo being used, and so on.

How do we code this in bash? Usually we end up with a ton of if/then statements:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
if [ "$DATABASE_HOSTNAME" == "" ]; then
  DATABASE_HOSTNAME="database.example.com"
fi

if [ "$GIT_URL" == "" ]; then
  GIT_URL="https://github.com/StackExchange/dnscontrol"
fi

if [ "$DNSCONTROL_CMD" == "" ]; then
  DNSCONTROL_CMD="/usr/bin/dnscontrol"
fi

That’s so ugly! It’s not just ugly but it also violates the DRY principle.

Certainly there’s an easier way to do this? There sure is!

Access the variable: ${VARNAME}
Access the variable, with a default: ${VARNAME:-defaultvalue}

Here’s how it looks in a script:

1
2
3
echo "I think ${DATABASE_HOSTNAME} is ready."  # NO DEFAULT

echo "I think ${DATABASE_HOSTNAME:-database.example.com} is ready."

The :- means if the variable is unset or empty, uses the default value instead.

However including the default value every time you access the variable is a pain in the butt.

Can we do better? Sure!

There is also := which is the same thing as :- but it also updates the variable with the default value at the same time.

Access the variable, set if empty: ${VARNAME:=defaultvalue}

Wait… what? A way to use a variable and have it be mutated along the way? Yes, that’s right. (A Haskell programmer somewhere just burst out in tears.)

Let’s write short script and watch the interactions.

1
2
3
4
5
6
7
#!/bin/bash

echo Line1: "${FOO}"
echo Line2: "${FOO:-my value}"
echo Line3: "${FOO}"
echo Line4: "${FOO:=my value}"
echo Line5: "${FOO}"

Here’s the output:

1
2
3
4
5
Line1:              << FOO is unset
Line2: my value     << FOO is unset, but the default is substituted.
Line3:              << FOO is still unset (no mutation)
Line4: my value     << FOO is still unset, show the default and mutate FOO.
Line5: my value     << FOO is set to "my value"!

Using this is real code

Let’s recap:

If we use :- we have to use it every time we use the variable, and if we have to change the default value we have to change it everywhere it appears in the code.

If we use := we only have to specify the default value the first time we use the variable. Sadly that means if we add code earlier in the script, we have to remember to move the default to this earlier code. That’s just asking for trouble.

So what do we do?

We need an excuse to use the variable (with the := construct) early in the program so that all other uses don’t need :- or :=.

We could do this (BUT YOU SHOULDN’T):

1
echo "${DATABASE_HOSTNAME:=database.example.com}" >/dev/null

That would work but it is ugly.

Luckily bash has a “do nothing” command called :.

1
: "${DATABASE_HOSTNAME:=database.example.com}"

What? Well, the : command (yes, that’s a bash command) means “ignore the rest of this command”. However, that is processed after the variable substitution, so we get our variable modified (if unset or empty) and that’s it.

Clear as mud, right?

Anyway…

I try to put all of these at the start of the file. This centralizes all the settings that users can override, makes sure that the overrides are done at the beginning, and is probably better for a few reasons I’m forgetting right now.

Here’s what that looks like, taken from a recent script I wrote (names changed, of course):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

#
# Variables that users can override:
# Cite: https://stackoverflow.com/questions/2013547
#
: "${FOO_DAYS:=31}"
: "${FOO_NOTIFY:=false}"
: "${FOO_STAGE:=qa}"
: "${FOO_REPO:=git@github.com:StackExchange/dnscontrol.git}"
: "${EDITOR:=vim}"
: "${FOO_THING:=$DEFAULT_MAIN_THING}"
: "${FOO_FILEPATH:=modules/foo/files}"
: "${DATAPATH:=$BASEDIR/$FOO_FILEPATH/data}"

Most of the names are prefixed with with FOO_ because because this is for Project Foo. This prevents name collisions between projects.

FOO_THING’s default value comes from another variable! This is usually done when the default is generated from other data. For example, I recently wrote a script that had a special default when the script was run non-interactively.

Summary

Now you have an easy way to use env variables but allow users to override them easily.

Set a variable. Override any previous value:

1
export FOO_DAYS="31"

If you want a user to be able to override a variable:

1
: "${FOO_DAYS:=31}"

Display the value but substitute a default if it is empty:

1
echo "There are ${FOO_DAYS:-31} days remaining."

Learn more

This is all covered in the bash(1) man page (The command man bash will view this) in a section called Parameter Expansion. You probably skipped that section because without the above context, I doubt anyone understands why these features are in bash.

Now you know.