guix-artwork/website/posts/guix-environment-to-shell.md

title: From ‘guix environment’ to ‘guix shell’
author: Ludovic Courtès
tags: Software development, User interfaces, Performance
date: 2021-10-26 15:00:00
---

There are times when what looked like the right design choice some years
back comes out as an odd choice as time passes.  The beloved [`guix
environment`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-environment.html)
tool is having that fate.  Its command-line interface has become
non-intuitive and annoying for the most common use cases.  Since it
could not be changed without breaking compatibility in fundamental ways,
we devised a new command meant to progressively replace it; [`guix
shell`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-shell.html)—that’s
the name we unimaginatively ended up with—has just landed after [a
three-week review period](https://issues.guix.gnu.org/50960), itself a
followup to discussions and hesitations on the best course of action.

This post introduces `guix shell`, how it differs from `guix
environment`, the choices we made, and why we hope you will like it.

# The story of `guix environment`

The `guix environment` command [started its life in
2014](https://lists.gnu.org/archive/html/guix-devel/2014-10/msg00111.html),
when Guix was a two-year old baby and the whole community could fit in a
small room.  It had one purpose: “to assist hackers in creating
reproducible development environments”.  It was meant to be similar in
spirit to [VirtualEnv](https://docs.python.org/3/tutorial/venv.html) or
[Bundler](https://bundler.io/), but *universal*—not limited to a single
language.  You would run:

```
guix environment inkscape
```

… and obtain an interactive shell with all the packages needed to hack
on Inkscape; in that shell, the relevant environment variables—`PATH`,
`CPATH`, `PKG_CONFIG_PATH`, and so on—would automatically point to a
profile created on the fly and containing the compiler, libraries, and
tools Inkscape depends on, but not Inkscape itself.

Only a year later did it become clear that there are cases where one
would want to create an environment containing specific packages, rather
than an environment containing *the dependencies* of packages.  To
address that, David Thompson [proposed the `--ad-hoc`
option](https://lists.gnu.org/archive/html/guix-devel/2015-05/msg00561.html):

```
guix environment --ad-hoc inkscape -- inkscape
```

… would create an environment containing only Inkscape, and would then
launch the `inkscape` command in that environment.  Many features were
added over the years, such as [the invaluable `--container`
option](https://guix.gnu.org/en/blog/2015/container-provisioning-with-guix/),
but these two modes, development and “ad hoc”, are the guts of it.

Fast forward six years: today, there’s consensus that the name
`--ad-hoc` is confusing for newcomers and above all, that the “ad hoc”
mode should be the default.  This is the main problem that [`guix
shell`](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-environment.html)
addresses.

# Doing what you’d expect

Changing the default mode from “development environment” to “ad hoc” is
technically easy, but how to do that without breaking compatibility is
harder.  This led to [lengthy
discussions](https://issues.guix.gnu.org/38529), including proposals of
mechanisms to choose between the new and old semantics.

In the end, keeping the `guix environment` name while allowing it to
have different semantics was deemed dangerous.  For one thing, there’s
lots of material out there that demoes `guix environment`—blog posts,
magazine articles, on-line courses—and it would have been impossible to
determine whether they refer to the “new” or to the “old” semantics.  We
[reached the conclusion](https://issues.guix.gnu.org/38529#17) that it
would be easier to use a new command name and to eventually deprecate
`guix environment`.

With `guix shell`, the default is to create an environment that contains
the packages that appear on the command line; to launch Inkscape, run:

```
guix shell inkscape -- inkscape
```

The `--ad-hoc` option is gone!  Likewise, to spawn an ephemeral
development environment containing Python and a couple of libraries,
run:

```
guix shell python python-numpy python-scipy -- python3
```

Now, if you want, say, the development environment of Inkscape, add the
`--development` or `-D` option right before:

```
guix shell -D inkscape
```

You can add Git and GDB on top of it like so:

```
guix shell -D inkscape git gdb
```

(Note that `-D` only applies to the immediately following package,
`inkscape` in this case.)  It’s more concise and more natural than with
`guix environment`.  As can be seen [in the
manual](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-shell.html),
all the other options supported by `guix environment` remain available
in `guix shell`.

# Short-hands for development environments

A convention that’s become quite common is for developers to provide a
`guix.scm` at the top of their project source tree, so that others can
start a development environment right away:

```
guix environment -l guix.scm
```

The `guix.scm` file would contain a [package
definition](https://guix.gnu.org/manual/devel/en/html_node/Defining-Packages.html)
for the project at hand, [as in this
example](https://notabug.org/cwebber/guile-gcrypt/src/master/guix.scm).
This option is known as `-f` in `guix shell`, for consistency with other
commands, and the equivalent command is:

```
guix shell -D -f guix.scm
```

Since all Guix commands accept a
[“manifest”](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-package.html#index-manifest)
with `-m`, another option is to provide a `manifest.scm` file and to
run:

```
guix shell -m manifest.scm
```

“Wouldn’t it be nice if `guix shell` would automatically follow these
conventions when not given any argument?”, some
[suggested](https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html).
As in the case of Bundler, [direnv](https://direnv.net/), or typical
build tools from Meson to Make, having a default file name can save
typing and contribute to a good user experience for frequently-used
commands.  In this spirit, `guix shell` automatically loads `guix.scm`
or `manifest.scm`, from the current directory or an ancestor thereof,
such that entering a project to hack on it is as simple as:

```
cd ~/my/project/src
guix shell
```

Worry not: `guix shell` loads `guix.scm` or `manifest.scm` *if and only
if* you have first added its directory to
`~/.config/guix/shell-authorized-directories`.  Otherwise `guix shell`
warns you and prints a hint that you can copy/paste if you want to
authorize the directory.

# Caching environments

With that in place, `guix shell` can pretty much fill the same role as
direnv and similar tools, with one difference though: speed.  When all
the packages are already in store, `guix shell` can take one to a few
seconds to run, depending on the package set, on whether you’re using a
solid state device (SSD) or a “spinning” hard disk, and so on.  It’s
acceptable but prohibitively slow for direnv-like use cases.

To address that, `guix shell` maintains a profile cache for the `-D -f
guix.scm` and `-m manifest.scm` cases.  On a hot cache, it runs in
0.1 second.  All it has to do is fork a shell with the right environment
variable definitions; it does not talk to `guix-daemon`, and it does not
even read `guix.scm` or `manifest.scm` (it’s possible to forcefully
update the cache with `--rebuild-cache`).

That makes `guix shell` usable even for short-lived commands like
`make`:

```
guix shell -- make
```

Hopefully it’ll change the way we use the tool!

# The shell doctor

While revamping this command-line interface, the idea of a “shell
doctor” came up.  In interactive use, `guix shell` sets environment
variables and spawns a shell, but it’s not uncommon for the shell to
mess up with the whole environment.  Why?  Because, [contrary to
documented
practice](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html),
it’s quite common for users to define or override environment variables
in the startup files of non-login shells, `~/.bashrc` for Bash,
`~/.zshrc` for Zsh.  Instead, environment variable definitions should go
to the startup file of *login* shells—`~/.bash_profile`, `~/.profile`,
or similar.  But let’s face it: it’s a subtle distinction that few of us
know or care about.

As a result, users of Guix, especially on distros other than Guix
System, would often be disappointed when running `guix environment
--pure` and *yet* find that `PATH` contains non-Guix entries, that
there’s a bogus `LD_LIBRARY_PATH` definition, and whatnot.  Now, they
can call the doctor, so to speak, to obtain a diagnosis of the health of
their shell by adding the `--check` flag:

```
guix shell --check python python-numpy
```

The command creates an environment containing Python and NumPy, spawns
an interactive shell, checks the environment variables as seen by the
shell, and prints a warning if `PATH` or `PYTHONPATH` in this case have
been overridden.  It does not tell users where the problem comes from—it
cannot guess—but it tells them if something’s wrong, which is a first
step.

Of course, the best way to sidestep these problems is to pass
`--container`, which gives a fresh, isolated environment that does not
contain those startup files.  That’s not always an option though, for
instance on systems lacking support for unprivileged user namespaces, so
`--check` comes in handy there.

# Try it!

Just run `guix pull` to get this shiny new `guix shell` thingie!

If you don’t feel ready yet, that’s OK: `guix environment` won’t
disappear overnight.  We have a [written
commitment](https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-environment.html)
to keep it around until May, 1st 2023.  Though overall, we hope you’ll
find the `guix shell` interface easier to use and compelling enough that
you’ll be willing to switch overnight!

#### About GNU Guix

[GNU Guix](https://guix.gnu.org) is a transactional package manager and
an advanced distribution of the GNU system that [respects user
freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection.  When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management.  Guix is highly customizable
and hackable through [Guile](https://www.gnu.org/software/guile)
programming interfaces and extensions to the
[Scheme](http://schemers.org) language.