Python Modules and Packages

  • Description: import forms, modules vs packages, __init__.py, relative imports, sys.path, the __main__ idiom, virtual environments, and modern project metadata (pyproject.toml)
  • My Notion Note ID: K2A-D1-13
  • Created: 2023-02-28
  • Updated: 2026-05-11
  • License: Reuse is very welcome. Please credit Yu Zhang and link back to the original on yuzhang.io

Table of Contents

1. Modules and Imports

  • A module is a .py file
  • Importing executes it once and caches it in sys.modules
import math                  # use as math.sqrt(2)
import numpy as np           # alias

from os import path          # bind path directly into this namespace
from os.path import join, dirname
from collections import (    # parenthesized for many names
    defaultdict,
    Counter,
)

from typing import *         # discouraged, pollutes namespace
  • Re-importing does NOT re-run; use importlib.reload(module) to force during dev
  • Reload only refreshes the module's own bindings, references in other modules still point at old objects

2. Packages and __init__.py

  • A package is a directory of .py files containing __init__.py (may be empty)
mypkg/
    __init__.py
    core.py
    util/
        __init__.py
        text.py

import mypkg.core
from mypkg.util.text import slugify
  • __init__.py runs when the package is first imported

Common uses:

  • Re-export the public surface: from .core import App → callers can from mypkg import App

  • Define __all__ to control from mypkg import *

  • Set up package-level constants

  • Implicit namespace packages (PEP 420, no __init__.py), multiple directories can merge into one logical package; useful for plugin systems

3. Relative Imports

# inside mypkg/util/text.py
from . import config            # same package (mypkg.util)
from .. import core             # parent package (mypkg)
from ..core import App          # specific name from parent
  • Resolved by the file's package position, not cwd → survive moves
  • Can only be used inside a package, python text.py makes them fail with ImportError

4. How import Finds Things

sys.path search order:

  1. Directory containing the script (or "" for cwd in REPL)
  2. PYTHONPATH env var directories
  3. Installation default, stdlib + site-packages in the active venv
import sys
sys.path                       # list of search dirs
sys.modules                    # cache of already-imported modules
  • Editing sys.path at runtime works but is fragile, prefer pip install -e . so your package lives in site-packages

5. if __name__ == "__main__":

  • __name__ == "__main__" when run directly
  • __name__ is the module's dotted name when imported
  • The guard lets a file be both a library and a script
def main():
    args = parse_args()
    run(args)

if __name__ == "__main__":
    main()

For packages, add __main__.py so python -m mypkg works:

mypkg/
    __init__.py
    __main__.py     # contains `from .cli import main; main()`
    cli.py

6. Virtual Environments

python3 -m venv .venv
source .venv/bin/activate            # Linux/macOS
.venv\Scripts\activate               # Windows

which python                          # → .venv/bin/python
pip install -r requirements.txt
deactivate
  • Isolates each project's dependencies
  • Modern alternatives, poetry, uv, pdm, hatch, pipenv, manage env + lockfile
  • uv is currently the fastest
  • conda is a separate ecosystem for scientific Python with non-Python deps

7. pyproject.toml and Modern Packaging

  • PEP 518/621, canonical project metadata file
  • Replaces the old setup.py/setup.cfg
[project]
name = "myproject"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.31",
    "click>=8.1",
]

[project.optional-dependencies]
dev = ["pytest", "mypy", "ruff"]

[project.scripts]
mytool = "myproject.cli:main"        # creates a `mytool` console script

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Install in editable mode:

pip install -e .              # imports live from your source tree
pip install -e ".[dev]"       # plus the dev extras
  • Build a wheel: python -m build or uv build
  • Upload: twine or uv publish

8. Standard Library Highlights

Module What it gives you
os, os.path, pathlib Filesystem (prefer pathlib)
sys Interpreter state, sys.argv
subprocess Run external commands
json, csv, tomllib (3.11+) Common formats
re Regex
datetime, zoneinfo Dates and time zones
collections, itertools, functools Functional & data-structure helpers
logging Structured logging
argparse CLI parsing
unittest, doctest Built-in test runners
dataclasses Lightweight value types
typing Static type tooling
concurrent.futures, threading, multiprocessing, asyncio Concurrency
urllib, http, socket Networking primitives
pickle, shelve Object serialization (unsafe for untrusted input)