Using virtual environments in Python

Let's talk about using virtual environments in Python.

Python code with third-party requirements

Here we have a Python program called exponential.py that uses a module called rich:

from argparse import ArgumentParser
from rich.console import Console
from rich.table import Table

parser = ArgumentParser()
parser.add_argument("start", type=float)
parser.add_argument("annual_growth", type=float)
parser.add_argument("--years", type=int, default=15)
args = parser.parse_args()


table = Table("Year", "Old", "Added", "New")
amount = args.start
for year in range(1, args.years+1):
    new = amount * args.annual_growth/100
    table.add_row(
        f"{year}",
        f"{amount:,.0f}",
        f"{new:,.0f}",
        f"{amount+new:,.0f}",
    )
    amount += new
Console().print(table)

Currently our exponential.py program doesn't work because we don't have that rich module:

~/exponential $ python3 exponential.py 1000 7
Traceback (most recent call last):
  File "/home/trey/exponential/exponential.py", line 2, in <module>
    from rich.console import Console
ModuleNotFoundError: No module named 'rich'

The rich module that we're trying to use comes bundled in a third-party package called rich which, according to pip, we don't currently have installed:

~/exponential $ python3 -m pip list
Package            Version
------------------ ---------
argcomplete        2.0.0
certifi            2021.10.8
charset-normalizer 2.0.12
click              8.0.4
idna               3.3
packaging          21.3
pip                22.0.4
pipx               1.0.0
pyparsing          3.0.7
requests           2.27.1
setuptools         58.1.0
urllib3            1.26.9
userpath           1.8.0

Before we go about pip installing the rich module, we're going to create a virtual environment for this project.

The basics of virtual environments

Virtual environments are a way to isolate the dependencies needed for each Python project on our machine.

Let's start using a virtual environment for our project.

We're going to:

1. Create a new virtual environment
2. Activate that virtual environment (to start using it)
3. Install the package(s) we need into our virtual environment
4. Deactivate our virtual environment (once we're temporarily done working with it)

Creating a new virtual environment with venv

To create a new virtual environment, we'll use Python's venv module.

~/exponential $ python3 -m venv .venv

That .venv argument is the directory name we'd like to store our virtual environment data within. While venv and env are also common directory names for virtual environments, but we can use whatever name we'd like. We'll use .venv to make a hidden directory.

Before running that command, let's also add a --prompt argument to customize the prompt that we'll see when our virtual environment is activated (you'll see what that does in just a bit):

~/exponential $ python3 -m venv .venv --prompt=exponential

That command can take just a second to run or it can take a whole minute.

Running that command created a directory called .venv that represents our virtual environment. If we list the files in our current directory, we'll see that new .venv subdirectory:

~/exponential $ ls -a
.
..
exponential.py
.venv

We've created a virtual environment now, but we're not yet using it. To use our virtual environment, we need to activate it.

Activating a virtual environment

The virtual environment activation command differs depending on your operating system and the shell you're using within that operating system:

On my machine, I'll need to run that third command (source venv/bin/activate) to activate this new virtual environment:

~/exponential $ source .venv/bin/activate

Now, I can tell that I'm in an activated virtual environment because my prompt has changed.

(exponential) ~/exponential $

See those parentheses at the beginning of our prompt with the name of our virtual environment (exponential) within it?

When we created our virtual environment we used the --prompt command to set its name to the name of our project (exponential), but by default the virtual environment prompt would be the name of our virtual environment directory (.venv in our case).

If we use pip to list the current packages we have installed, we'll see that we don't see all those globally-installed packages we saw earlier:

(exponential) ~/exponential $ python3 -m pip list
Package    Version
---------- -------
pip        22.0.4
setuptools 58.1.0

We don't see our global packages because we're in our virtual environment which is isolated from the packages that we've installed globally on our machine.

Installing a package into a virtual environment

Now that we've created and activated our virtual environment, we need to install a Python package, using pip!

If we use pip to install rich at this point, we'll install it specifically into our activated virtual environment, not globally on our machine:

(exponential) ~/exponential $ python3 -m pip install rich
Collecting rich
  Using cached rich-12.4.1-py3-none-any.whl (231 kB)
...
Installing collected packages: commonmark, pygments, rich
Successfully installed commonmark-0.9.1 pygments-2.12.0 rich-12.4.1

If we run pip list, you can see that we just installed rich as well as some of its dependencies:

(exponential) ~/exponential $ python3 -m pip list
Package    Version
---------- -------
commonmark 0.9.1
pip        22.0.4
Pygments   2.12.0
rich       12.4.1
setuptools 58.1.0

And if we run our exponential.py program at this point, we'll see that it works now!

(exponential) ~/exponential $ python3 exponential.py 1000 7
┏━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┓
┃ Year ┃ Old   ┃ Added ┃ New   ┃
┡━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━┩
│ 1    │ 1,000 │ 70    │ 1,070 │
│ 2    │ 1,070 │ 75    │ 1,145 │
│ 3    │ 1,145 │ 80    │ 1,225 │
│ 4    │ 1,225 │ 86    │ 1,311 │
│ 5    │ 1,311 │ 92    │ 1,403 │
│ 6    │ 1,403 │ 98    │ 1,501 │
│ 7    │ 1,501 │ 105   │ 1,606 │
│ 8    │ 1,606 │ 112   │ 1,718 │
│ 9    │ 1,718 │ 120   │ 1,838 │
│ 10   │ 1,838 │ 129   │ 1,967 │
│ 11   │ 1,967 │ 138   │ 2,105 │
│ 12   │ 2,105 │ 147   │ 2,252 │
│ 13   │ 2,252 │ 158   │ 2,410 │
│ 14   │ 2,410 │ 169   │ 2,579 │
│ 15   │ 2,579 │ 180   │ 2,759 │
└──────┴───────┴───────┴───────┘

Our program works because it found the rich module that it needed to print out that table representing exponential growth.

Virtual environments are per window

Note that if we open a new terminal window and we run our program in that new terminal window, we'll see that it doesn't work:

~/exponential $ python3 exponential.py 1000 7
Traceback (most recent call last):
  File "/home/trey/exponential/exponential.py", line 2, in <module>
    from rich.console import Console
ModuleNotFoundError: No module named 'rich'

Our program doesn't work in this new terminal window because virtual environments are activated for just the window you have open; they're not activated for your whole machine.

In order to use our virtual environment in a new window, we would need to run the activation command again:

~/exponential $ source .venv/bin/activate

After activating our virtual environment, our code will work again:

(exponential) ~/exponential $ python3 exponential.py 1000 7
┏━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┓
┃ Year ┃ Old   ┃ Added ┃ New   ┃
┡━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━┩
│ 1    │ 1,000 │ 70    │ 1,070 │
│ 2    │ 1,070 │ 75    │ 1,145 │
│ 3    │ 1,145 │ 80    │ 1,225 │
│ 4    │ 1,225 │ 86    │ 1,311 │
│ 5    │ 1,311 │ 92    │ 1,403 │
│ 6    │ 1,403 │ 98    │ 1,501 │
│ 7    │ 1,501 │ 105   │ 1,606 │
│ 8    │ 1,606 │ 112   │ 1,718 │
│ 9    │ 1,718 │ 120   │ 1,838 │
│ 10   │ 1,838 │ 129   │ 1,967 │
│ 11   │ 1,967 │ 138   │ 2,105 │
│ 12   │ 2,105 │ 147   │ 2,252 │
│ 13   │ 2,252 │ 158   │ 2,410 │
│ 14   │ 2,410 │ 169   │ 2,579 │
│ 15   │ 2,579 │ 180   │ 2,759 │
└──────┴───────┴───────┴───────┘

Deactivating a virtual environment

What do we do when we want to leave our virtual environment? To exit a virtual environment, you can deactivate it.

To deactivate a virtual environment, we could just exit the window we have open or open up a new window. As long as we open up a fresh terminal window, we'll be in a window that doesn't have a virtual environment open.

To deactivate our virtual environment and keep using the same window, we can run the deactivate command from within our activated virtual environment:

(exponential) ~/exponential $ deactivate

Now we're back our usual prompt (without that (exponential ) prefix):

~/exponential $

Virtual environments: a Python best practice

Virtual environments are a way to isolate the dependencies that you use for each Python project that you work on.

It's considered a best practice to use a separate virtual environment for each Python project on your machine.