Setting up Python on a Unix machine (without pyenv)

If I don’t need access to multiple Python versions, but do want multiple virtual environments, this is how I set up Python on a new Unix/macOS machine. You don’t need to sudo or run any of this as root. All of it can be easily undone. Just delete the virtual environment that ails you (or ~/.virtualenvs if you want to start completely from scratch).

Note: this post is about how to set up multiple Python environments on a development machine. It doesn’t cover how to have multiple Python interpreters on a single machine. I have another post about that.

macOS-only prerequsite

Install homebrew and use it to install isolated, up-to-date versions of Python and legacy Python 2.

brew install python python3

Optionally (and this is not Python related), update git and install some goodies:

brew install git bash
brew cask install macvim
sudo chsh -s /usr/local/bin/bash $USER

All systems

Bootstrap pip:

python -m ensurepip --user
python3 -m ensurepip --user

This will result in an error on Debian/Ubuntu systems, but won’t do any harm.

Then install/update the Python packaging basics in your “global” Pythons. These global Pythons will be the system versions on a Linux system, or the homebrew versions on macOS (assuming you’ve installed homebrew as described above):

python -m pip install --user --upgrade pip virtualenv wheel
python3 -m pip install --user --upgrade pip virtualenv wheel

Now, never touch the global Python’s again, or at least never install any packages in them. You can make sure of this by configuring pip to refuse to install anything outside a virtualenv:

mkdir ~/.pip
cat > ~/.pip/pip.conf <<END
[global]
require-virtualenv = true
END

Set up lightweight commands to create and change environments by putting this in your shell startup:

workon () {
    if [ -f "${HOME}/.virtualenvs/$1/bin/activate" ]; then
        source "${HOME}/.virtualenvs/$1/bin/activate"
    fi
}
mkvirtualenv () {
    deactivate 2> /dev/null || true
    python3 -m virtualenv ${HOME}/.virtualenvs/${1}
    workon ${1}
}
mkvirtualenv_legacy () {
    deactivate 2> /dev/null || true
    python2 -m virtualenv ${HOME}/.virtualenvs/${1}
    workon ${1}
}

These commands are inspired by virtualenvwrapper which is overkill for me, but a great option if you want a more complete project/virtualenv manager.

Now create a directory for your virtualenvs to live in, a default virtualenv for day-to-day work, and a legacy Python 2 virtualenv.

mkdir -p ~/.virtualenvs
mkvirtualenv default
mkvirtualenv_legacy legacy

If you prefer to put virtual environments in the same directory as the corresponding project or application, then change the shell function above or just create each project by hand:

cd my_project
python3 -m virtualenv venv

You can use the workon function to activate a particular environment, as long as it lives in ~/.virtualenvs:

workon default

I have workon default in my shell startup.

A fresh install of macOS will add the name of the active virtual environment to your bash prompt. This is not necessary, but is nice to have. If you want it and your system doesn’t, or you have specific requirements, then there are lots of options including, e.g. my dotfiles, which I pinched from StackOverflow.

I use my default environment for quick tests and day-to-day coding not associated with a particular project. For that work, at a minimum, I need a few basics:

workon default
pip install numpy scipy jupyter sklearn pandas matplotlib seaborn pytest

Optional extra: autoenv

Install autoenv to run a set of commands every time you change into a directory. These commands are shell commands contained in a file .env in that directory. I use them to ensure the appropriate virtual environment is activated, and any necessary environment variables are set (which I use a lot, per 12 Factor).

brew install autoenv  # or whatever for your system
mkdir ~/my_project/
echo source ~/.virtualenvs/my_project/bin/activate" >> ~/my_project/.env
cd ~/my_project

The my_project environment is now automatically enabled when I cd into ~/my_project.

Optional extra: pipsi

Install pipsi and use it to install Python-based command line tools so they are accessible from any environment and you don’t have to install them in every one. First add ~/.local/bin to your path then

curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
pipsi install flake8
pipsi install httpie
pipsi install magic-wormhole