With the proliferation of containers, everybody these days is using environment variables (env vars) to pass information (fun fact, getenv was added to V7 Unix in 1979!). We take for granted using them, but there are a few things that may surprise you - and this post is about them.

TL;DR

1. Use sensible sizes when setting environment variables: i.e. key/values should not be more than a few KBs (Windows has a hard limit of 32,767 bytes per environment variable but no hard limit on number of variables).

2. Never assume that when you change environment variables, the runtime synchronously calls the underlying libc with the changes you made.

3. Following (2), never assume that environment variables changes get reflected in the process's original environ block (i.e. visible from /proc/<fd>/environ).

4. Never assume that manipulating environment variables is thread-safe, unless you are 100% sure what runtime you will be using and that specific runtime uses locks.

5. If you're thinking that you can set environment variables and then they will be visible from a shared object implemented using a different tech stack, think again.
   For example, you're developing a library and want to test it using Node.js - you can't set environment variables on Node.js and expect them to be visible from the library code.

6. Env vars are great to pass some data when starting a process, but are not really intended to be used as "variables" (read: mutated) in a real program (not talking about a single-threaded script) or across different runtimes in the same process.

Limits

What can you put in environment variables? Like everything in life, there are limits. TL;DR: the limits are complicated to know in advance, so just use short strings (a few KBs tops) for each key/value. If you're interested to read more about what the exact limits are read the execve man page for Linux and this StackOverflow article for Windows.

Runtimes and env vars

In Linux, when a process forks/execs, the kernel sets up a space on the stack where it keeps the program arguments (command line) and the environment variables (environ). You can examine them in a running process using procfs:

# Strings NULL terminated as you can see with xxd.
xxd /proc/<pid>/cmdline
xxd /proc/<pid>/environ

It's important to understand that the kernel writes the command line and environ to the stack before executing code and then passes the pointers to the runtime init code so the runtime can use it or expose it to the program. This is a bit simplified explanation so if you want to dive deeper on how it's done in Linux, here's a great blog post from Chris Tarazi.

Why do we care? Because of the setenv and putenv calls or their equivalent in different programming languages. Since the programmer can modify environ - add new key/values or modify existing values, how can the kernel tell how much space to allocate in advance?

The simple answer is that the kernel can't. It can't predict how much additional space will be needed and it would be very wasteful for the kernel to reserve more space when starting each process, since not every process changes the environ.

So how come we have setenv and the like? The answer is each runtime actually allocates a new space (in the heap), copies environ and makes the necessary changes (libc/musl have additional code to optimize use cases where the key already exists and the value can be stored in.

How different runtimes handle env vars

Here's glibc's implementation (and libmusl here): they basically copy environ if you're setting a new key or if the value you want to set can't fit in the existing environ.

This is why you can't trust that /proc/<pid>/environis up-to-date.

Now it gets even more complicated. Some runtimes like Rust, use the underlying C standard library (glibc/libmusl) to set and get environment variables, so if a Rust program gets or sets an environment variable it actually goes to libc.

Other runtimes, like Python, Java and Node.js keep their own copies which in turn may or may not update the libc counterparts like we see here:

# This code was extracted from Python's os.py.
# It's NOT the full implementation - just the relevant code parts.
class _Environ(MutableMapping):
    # ... 
    def __getitem__(self, key):
        try:
            value = self._data[self.encodekey(key)]
        except KeyError:
            # raise KeyError with the original key value
            raise KeyError(key) from None
        return self.decodevalue(value)

    def __setitem__(self, key, value):
        key = self.encodekey(key)
        value = self.encodevalue(value)
        # The following line will update libc:
        putenv(key, value)
        self._data[key] = value

    def __delitem__(self, key):
        encodedkey = self.encodekey(key)
        # The following line will update libc:
        unsetenv(encodedkey)
        try:
            del self._data[encodedkey]
        except KeyError:
            # raise KeyError with the original key value
            raise KeyError(key) from None

    def __iter__(self):
        # list() from dict object is an atomic operation
        keys = list(self._data)
        for key in keys:
            yield self.decodekey(key)

    def __len__(self):
        return len(self._data)
    def copy(self):
        return dict(self)

    def setdefault(self, key, value):
        if key not in self:
            self[key] = value
        return self[key]

The above code shows that if a native Python extension (i.e. a shared object) sets an env var after_Environ gets created (which happens pretty early), then that env var will not be visible to the Python code.

Node.js has even more special handling: it will call libc's setenv only from the main thread and not from worker threads (see docs here).

Go's runtime

Go's runtime deserves a special mention because of the difference between what it does on Unix vs. Windows.

On Windows (here):

func Getenv(key string) (value string, found bool) {
    keyp, err := UTF16PtrFromString(key)
    if err != nil {
        return "", false
    }
    n := uint32(100)
    for {
        b := make([]uint16, n)
        n, err = GetEnvironmentVariable(keyp, &b[0], uint32(len(b)))
        if n == 0 && err == ERROR_ENVVAR_NOT_FOUND {
            return "", false
        }
        if n <= uint32(len(b)) {
            return UTF16ToString(b[:n]), true
        }
    }
}

func Setenv(key, value string) error {
    v, err := UTF16PtrFromString(value)
    if err != nil {
        return err
    }
    keyp, err := UTF16PtrFromString(key)
    if err != nil {
        return err
    }
    e := SetEnvironmentVariable(keyp, v)
    if e != nil {
        return e
    }
    runtimeSetenv(key, value)
    return nil
}

And on Unix (here):

func Getenv(key string) (value string, found bool) {
    envOnce.Do(copyenv)
    if len(key) == 0 {
        return "", false
    }

    envLock.RLock()
    defer envLock.RUnlock()

    i, ok := env[key]
    if !ok {
        return "", false
    }
    s := envs[i]
    for i := 0; i < len(s); i++ {
        if s[i] == '=' {
            return s[i+1:], true
        }
    }
    return "", false
}

func Setenv(key, value string) error {
    envOnce.Do(copyenv)
    if len(key) == 0 {
        return EINVAL
    }
    for i := 0; i < len(key); i++ {
        if key[i] == '=' || key[i] == 0 {
            return EINVAL
        }
    }
    // On Plan 9, null is used as a separator, eg in $path.
    if runtime.GOOS != "plan9" {
        for i := 0; i < len(value); i++ {
            if value[i] == 0 {
                return EINVAL
            }
        }
    }

    envLock.Lock()
    defer envLock.Unlock()

    i, ok := env[key]
    kv := key + "=" + value
    if ok {
        envs[i] = kv
    } else {
        i = len(envs)
        envs = append(envs, kv)
    }
    env[key] = i
    runtimeSetenv(key, value)
    return nil
}

So in Unix, Go's runtime copies the environment to an internal (cached) map but not on Windows. More than that, runtimeSetenv is only implemented in CGO for Unix. So, if you build with CGO disabled, then Go will not call the underlying libc's setenv on Unix at all.

Bottom line

From everything we see here, environment variables are really useful as a means to pass information when spawning new processes, but given all the uncertainties and different behaviors between different tech stacks, they are less useful to pass information across tech stacks within a process.