Python Version Management & Virtual Environments

2024-01-20
4 min read

Installing multiple Python versions on the same machine, isolating project dependencies, and understanding how import works.

Python version management: pyenv

Platform Tool
macOS pyenv (Homebrew)
Linux pyenv (curl script)
Windows official installer + py launcher, or pyenv-win
  1. Install pyenv (macOS / Linux)

    1. macOS

      brew install pyenv
      

      Add to ~/.zshrc or ~/.bashrc:

      export PYENV_ROOT="$HOME/.pyenv"
      export PATH="$PYENV_ROOT/bin:$PATH"
      eval "$(pyenv init -)"
      
    2. Linux

      curl https://pyenv.run | bash
      
  2. Windows

    1. Option 1: official installer + py launcher (recommended)

      Download installers from python.org. Check “Add Python to PATH” and “Install for all users” during install. The built-in py launcher picks the right version:

      py -0          # list installed versions
      py -3.11 foo.py
      py -3.10 foo.py
      
      set PY_PYTHON=3.11
      rem permanent: System Properties → Environment Variables → add PY_PYTHON=3.11
      
    2. Option 2: pyenv-win

      Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"; &"./install-pyenv-win.ps1"
      

      Restart terminal, then same commands as Unix pyenv:

      pyenv install 3.11.7
      pyenv global 3.11.7
      pyenv versions
      pyenv local 3.10.13
      
    3. Option 3: winget

      winget install Python.Python.3.11
      winget install Python.Python.3.10
      
  3. Common commands

    Command What it does
    pyenv versions list installed Python versions
    pyenv install –list list available versions to install
    pyenv install 3.11.7 install a specific version
    pyenv global 3.11.7 set global default version
    pyenv local 3.10.13 set version for current dir (writes .python-version)
    pyenv shell 3.9.18 set version for current shell session
    pyenv uninstall 3.9.18 remove a version
    pyenv which python show which python binary is active
  4. Priority

    pyenv picks the version in this order:

    1. shell — set by pyenv shell, current terminal only
    2. local — .python-version file in current directory
    3. global — ~/.pyenv/version
  5. Example

    pyenv install 3.10.13
    pyenv install 3.11.7
    
    pyenv global 3.11.7
    
    cd ~/projects/old-project
    pyenv local 3.10.13    # writes .python-version
    # .python-version content:
    # 3.10.13
    
    python --version       # Python 3.10.13
    cd ~
    python --version       # Python 3.11.7
    

Virtual environments

  1. Why isolate?

    By default pip installs packages globally into site-packages. If two projects need different versions of the same package, they’ll step on each other.

    mkdir project-a && cd project-a
    pip install django==3.2
    
    cd ../project-b
    pip install django==4.2    # overwrites global Django 3.2, project-a breaks
    

    A virtual environment gives each project its own isolated Python, with packages installed under the project directory instead of the global system. With venv, all packages live in .venv/lib/pythonX.X/site-packages/. Delete the project directory and everything is gone.

  2. venv (built-in)

    venv ships with Python 3.3+, no extra install needed.

    1. Create and activate

      # create
      python -m venv .venv
      
      # activate
      
      # macOS / Linux
      source .venv/bin/activate
      
      # Windows (CMD)
      .venv\Scripts\activate.bat
      
      # Windows (PowerShell)
      .venv\Scripts\Activate.ps1
      
      # prompt shows (.venv) prefix
      # (.venv) $ pip install requests
      
      # exit
      deactivate
      
      # If PowerShell throws "running scripts is disabled":
      # run as admin: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
      
    2. Manage dependencies

      # dump all installed packages
      pip freeze > requirements.txt
      
      # install from requirements.txt
      pip install -r requirements.txt
      
      # lock with pip-tools
      pip install pip-tools
      pip-compile requirements.in    # generates requirements.txt with pinned versions
      pip-sync requirements.txt      # install and remove extras
      
    3. Project layout

      my-project/
      ├── .venv/              # virtual env, add to .gitignore
      ├── src/
      │   └── main.py
      ├── requirements.txt    # or requirements.in
      └── .python-version     # pyenv version marker
      
      # .gitignore
      .venv/
      __pycache__/
      *.pyc
      
  3. poetry

    For stricter dependency management, use poetry. It locks versions via pyproject.toml and poetry.lock so everyone on the team gets the same dependency tree.

    # macOS / Linux / WSL
    curl -sSL https://install.python-poetry.org | python3 -
    # Windows (PowerShell)
    (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -
    

Python import and classes

  1. import syntax

    Syntax What it does
    import os import whole module, use os.getcwd()
    from datetime import datetime import specific names from a module
    import numpy as np import with alias
    from . import helper relative import within a package
    import os
    from datetime import datetime
    import numpy as np
    
    print(os.getcwd())
    print(datetime.now())
    print(np.array([1, 2, 3]))
    

    PEP 8 says group imports in three blocks, separated by blank lines:

    # standard library
    import os
    from datetime import datetime
    
    # third-party
    import numpy as np
    
    # local
    from my_project.utils import helper
    
  2. Name conflicts

    When two modules export the same class name, use as:

    from django.db.models import QuerySet as DjangoQuerySet
    from sqlalchemy.orm import Query as SQLAlchemyQuery
    
    django_qs = DjangoQuerySet()
    sa_q = SQLAlchemyQuery()
    

    Or import the whole module and prefix:

    import django.db.models
    import sqlalchemy.orm
    
    django.db.models.QuerySet()
    sqlalchemy.orm.Query()
    
  3. Importing a class from another file in the same project

    my-project/
    ├── main.py
    └── utils/
        ├── __init__.py
        └── string_utils.py
    
    # utils/string_utils.py
    class StringUtils:
    
        @staticmethod
        def to_upper(s):
            return s.upper()
    
        @staticmethod
        def truncate(s, max_len):
            return s[:max_len] + "..." if len(s) > max_len else s
    
    # utils/__init__.py
    from .string_utils import StringUtils
    
    # main.py
    from utils import StringUtils
    
    # or: from utils.string_utils import StringUtils
    
    text = StringUtils.to_upper("hello")
    print(text)  # HELLO
    
    short = StringUtils.truncate("a very long piece of text", 10)
    print(short)  # a very long...
    

    init.py can be empty — it just tells Python the directory is a package. Putting imports in init.py means callers just do from utils import StringUtils without knowing which file it lives in.