Code Quality: ruff, mypy & Quality Gates¶

Estimated time: 18 min read · Part: Python Tooling & Data Fundamentals

Why Automated Quality Gates Pay Off¶

Code review is expensive: a senior engineer spending 30 minutes reviewing a PR for style issues is a poor use of their time. Automated tools enforce style and catch common bugs in milliseconds, so human reviewers can focus on logic, architecture, and correctness. Three categories of issues are worth automating: style (formatting, import ordering, line length), correctness linting (undefined names, unused variables, shadowed builtins, common bug patterns), and type errors (passing a string where an int is expected, calling a method that doesn't exist on a type).

ruff: Linting & Formatting¶

ruff enforces two distinct concerns: linting (checking code for style and correctness issues) and formatting (rewriting code to a canonical style, like black). They are separate subcommands. The full rule set is organized into named categories; you opt in to the ones you want:

[tool.ruff]
line-length    = 100
target-version = "py311"      # enables rules that require this Python version
exclude        = ["migrations/", "scripts/legacy/"]

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # pyflakes (undefined names, unused imports)
    "I",    # isort (import ordering)
    "UP",   # pyupgrade (modernise syntax for target Python version)
    "B",    # flake8-bugbear (common bug patterns — highly recommended)
    "C4",   # flake8-comprehensions (unnecessary list/dict/set calls)
    "PIE",  # flake8-pie (misc improvements)
    "RET",  # flake8-return (unnecessary else after return)
    "SIM",  # flake8-simplify (simplifiable constructs)
    "S",    # flake8-bandit (security — use with care, many false positives)
]
ignore = [
    "E501",    # line too long — handled by formatter, not linter
    "S101",    # use of assert — fine in tests
]

[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["S", "B"]   # relax security/bugbear rules in tests

# Check for linting violations (does not modify files)
uv run ruff check src/

# Auto-fix what ruff can fix (safe fixes only by default)
uv run ruff check --fix src/

# Format code (like black — modifies files)
uv run ruff format src/

# Check formatting without modifying (for CI)
uv run ruff format --check src/

# Both checks together in CI (fail on any violation)
uv run ruff check src/ && uv run ruff format --check src/

src/acme_data_pipeline/ingestor.py:14:5: F401 os imported but unused src/acme_data_pipeline/pipeline.py:31:20: B006 Do not use mutable data structures for argument defaults Found 2 errors. | Rule prefix | Source tool | Catches | |---|---|---| | E / W | pycodestyle | Style: indentation, whitespace, blank lines | | F | pyflakes | Correctness: undefined names, unused imports/variables | | I | isort | Import ordering: stdlib, third-party, local | | UP | pyupgrade | Modernise syntax: f-strings, type annotations, open mode | | B | flake8-bugbear | Bug-prone patterns: mutable defaults, bare except, loop variable shadowing | | S | bandit | Security: hardcoded passwords, subprocess shell=True, etc. | | C4 | flake8-comprehensions | Unnecessary list() calls, dict.keys() in loops |

mypy: Static Type Checking¶

Python's type annotations are optional hints — the interpreter ignores them at runtime. mypy is a static type checker that reads those annotations and verifies that the code is internally consistent without running it. A mypy error is a bug caught before a single line of code executes. Type-annotating Python code also improves editor intelligence (autocomplete, go-to-definition) and makes the code dramatically easier to read — a function signature is a compact contract.

from __future__ import annotations   # postponed evaluation — avoids circular refs
from typing import TypedDict
from collections.abc import Sequence

# TypedDict gives a dict with a fixed, typed schema
class RawEvent(TypedDict):
    user_id: str
    event:   str
    value:   int

class NormalisedEvent(TypedDict):
    user_id:   str
    event:     str
    value:     int
    timestamp: str

def normalise_event(raw: RawEvent) -> NormalisedEvent:
    """Strip and lower user_id; add ISO timestamp."""
    from datetime import datetime, timezone
    uid = raw["user_id"].strip().lower()
    if not uid:
        raise ValueError("user_id must not be blank after stripping")
    if raw["value"] < 0:
        raise ValueError("value must be non-negative")
    return NormalisedEvent(
        user_id=uid,
        event=raw["event"],
        value=raw["value"],
        timestamp=datetime.now(timezone.utc).isoformat(),
    )

def parse_event_batch(events: Sequence[RawEvent]) -> list[NormalisedEvent]:
    if not events:
        raise ValueError("batch must not be empty")
    result: list[NormalisedEvent] = []
    for raw in events:
        try:
            result.append(normalise_event(raw))
        except ValueError:
            pass   # silently drop invalid events
    return result

uv run mypy src/

src/acme_data_pipeline/pipeline.py:8:

error

Incompatible return value type (got "int", expected "str") [return-value] Found 1 error in 1 file (checked 4 source files)

Pre-commit Hooks & CI Quality Gates¶

Quality tools only help if they run consistently. Two complementary enforcement points: pre-commit hooks run on your local machine before every git commit, catching issues before they even enter the repo. CI quality gates run in the pipeline on every push and PR, enforcing the same checks for every contributor regardless of local setup.

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.9
    hooks:
      - id: ruff          # lint + auto-fix
        args: [--fix]
      - id: ruff-format   # format

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.10.0
    hooks:
      - id: mypy
        additional_dependencies: [pydantic, types-requests]

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-merge-conflict
      - id: check-toml

# Install hooks into the local git repo (run once per clone)
uv run pre-commit install

# Run all hooks against all files (useful when adopting on an existing project)
uv run pre-commit run --all-files

WHY automation over manual review Manual code review is the most expensive QA mechanism. Automating style and type errors frees reviewers to focus on logic, architecture, and product correctness — the things machines can't check. One ruff rule catching a mutable default argument pays for the entire tool configuration in the first week.

HOW the two-layer approach Pre-commit for local speed, CI for authoritative enforcement. Run ruff check --fix && ruff format on save in your editor for instant feedback. The pre-commit hook catches anything that escaped. CI blocks the merge if anything slips through.

WHERE every production Python project Non-negotiable for any team project. For solo consulting scripts, at minimum run ruff before sharing. For client handoff, ensure the quality gates are in CI so the client's engineers maintain quality without knowing your personal preferences.

# Fixed: type-annotated, ruff-clean version
from collections.abc import Sequence

def process(
    events: Sequence[dict[str, int]],
    threshold: int = 0,          # was: mutable default [] (B006 violation)
    limit: int | None = None,
) -> list[dict[str, int]]:
    result = [e for e in events if e["value"] > threshold]
    return result[:limit] if limit is not None else result

# CI YAML step:
# - name: Lint and type-check
#   run: |
#     uv run ruff check src/ && uv run ruff format --check src/
#     uv run mypy src/