Packaging & Dependency Management with uv¶

Estimated time: 20 min read · Part: Python Tooling & Data Fundamentals

Why Python Needs a Dependency Manager¶

Every Python project accumulates third-party packages. Left unmanaged, packages from different projects conflict on the same system: Project A needs pydantic==1.x; Project B needs pydantic==2.x. Two problems compound this. First, isolation: each project needs its own private Python environment so packages cannot collide. Second, reproducibility: running pip install on Tuesday may resolve different versions than on Wednesday as new releases appear. A lock file captures the exact resolution so that CI, a colleague's machine, and production all install byte-identical packages.

Before uv, the ecosystem was fragmented: pip (ubiquitous but no lock file), pipenv (slow resolver, abandoned for years), poetry (good UX but resolver problems at scale), and conda (a different model for scientific computing). uv — written in Rust by the Astral team (who also built ruff) — resolves all the performance complaints. Its resolver runs in milliseconds on a warm cache, it speaks pip's protocols, and it replaces pip, virtualenv, pipenv, and poetry in a single binary with a coherent CLI.

pyproject.toml: The Modern Manifest¶

PEP 517/518 standardized pyproject.toml as the single configuration file for Python projects. It consolidates what used to require setup.py, setup.cfg, MANIFEST.in, tox.ini, .flake8, and mypy.ini into one structured TOML file. Here is a production-grade example for a data-processing service:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "acme-data-pipeline"
version = "1.3.0"
description = "ETL pipeline for ACME customer data platform"
readme = "README.md"
requires-python = ">=3.11"
license = { text = "MIT" }
authors = [{ name = "Dipayan Dhar", email = "dipayan@acme.io" }]

# Version constraints, not pins — lets the resolver find compatible versions
dependencies = [
    "httpx>=0.27",
    "pandas>=2.2",
    "pydantic>=2.7",
    "sqlalchemy>=2.0",
    "structlog>=24.0",
]

[project.optional-dependencies]
spark = ["pyspark>=3.5"]          # installed with: pip install acme-data-pipeline[spark]

[project.scripts]
acme-pipeline = "acme_data_pipeline.cli:main"   # creates a CLI entry point

[project.urls]
Homepage  = "https://github.com/acme/data-pipeline"

# Dev/test tools — not shipped with the package
[dependency-groups]
dev = [
    "pytest>=8.2",
    "pytest-cov>=5.0",
    "pytest-asyncio>=0.23",
    "mypy>=1.10",
    "ruff>=0.4",
    "pre-commit>=3.7",
]

# Tool configuration consolidated here — no separate .flake8, mypy.ini, etc.
[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "W", "I", "UP", "B", "C4", "PIE", "RET"]

[tool.mypy]
python_version = "3.11"
strict = true
ignore_missing_imports = true

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts   = "-v --tb=short --cov=src --cov-report=term-missing"

[tool.coverage.run]
branch = true
source = ["src"]

uv Core Commands¶

uv is distributed as a single static binary — no Python required to install it. The following workflow covers the full lifecycle from project initialization through daily development to CI:

# Bootstrap a new project (creates pyproject.toml, src/ layout, .python-version)
uv init acme-data-pipeline
cd acme-data-pipeline

# Add runtime dependencies — updates pyproject.toml AND uv.lock atomically
uv add httpx "pandas>=2.2" pydantic sqlalchemy

# Add dev-only tools to [dependency-groups]
uv add --dev pytest pytest-cov ruff mypy pre-commit

# Create / update .venv and install everything from uv.lock (exact versions)
uv sync                       # all deps + dev group
uv sync --no-dev              # runtime only  (for production Docker images)
uv sync --frozen              # CI: fail if lockfile would need to change

# Run any command inside the managed venv without activating it
uv run pytest                 # = .venv/bin/pytest
uv run python src/acme_data_pipeline/cli.py --help
uv run mypy src/

# Upgrade a dep (updates constraint in pyproject.toml AND re-locks)
uv add "pandas>=2.3"

# Export a requirements.txt for legacy tools that still need it
uv export --no-dev -o requirements.txt

Resolved 47 packages in 284ms Installed 47 packages in 1.1s

Lock Files, Reproducibility & the --frozen Flag¶

uv generates a uv.lock file recording every package and every transitive dependency at exact versions, with file hashes for integrity verification. This file must be committed to version control. In CI, uv sync --frozen asserts that the lockfile is consistent with pyproject.toml — if someone edited the manifest without regenerating the lock, the frozen flag fails the build rather than silently installing different packages. This is the difference between "it worked in dev" and "we can reproduce the exact environment of every deployed build."

Docker & Private Registries¶

In a Docker image, use uv sync --frozen --no-dev to install only runtime dependencies from the locked file. Copying the lockfile before the source code exploits Docker's layer cache — a dependency layer change only occurs when pyproject.toml or uv.lock changes, not when you edit a Python file:

FROM python:3.12-slim
WORKDIR /app

# Install uv into the image
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

# Copy manifests first — cached layer for slow dependency installs
COPY pyproject.toml uv.lock ./

# Install runtime deps only, using exact lockfile versions
RUN uv sync --frozen --no-dev --no-install-project

# Now copy source (this layer changes on every code edit)
COPY src/ ./src/

# Install the project itself (fast — deps already in place)
RUN uv sync --frozen --no-dev

CMD ["uv", "run", "uvicorn", "acme_data_pipeline.api:app",
     "--host", "0.0.0.0", "--port", "8000"]

Large enterprises run a private package registry (Artifactory, Nexus, AWS CodeArtifact) that proxies PyPI and hosts internal packages. This solves three problems: every package is scanned for CVEs before installation; air-gapped or regulated environments need packages from a controlled source; and internal libraries are distributed as versioned packages. Configure uv to use it with an index URL:

[[tool.uv.index]]
name = "acme-artifactory"
url  = "https://artifactory.acme.io/artifactory/api/pypi/pypi-local/simple/"
default = true   # use instead of PyPI

# Credentials come from env vars UV_INDEX_ACME_ARTIFACTORY_USERNAME / _PASSWORD
# Never hard-code credentials in pyproject.toml

Project structure with src/ layout:
acme-data-pipeline/
├── pyproject.toml       ← manifest + all tool config
├── uv.lock              ← committed lockfile (exact deps + hashes)
├── .python-version      ← "3.12" (pins interpreter)
├── .dockerignore
├── Dockerfile
├── src/
│   └── acme_data_pipeline/
│       ├── __init__.py
│       ├── pipeline.py  ← core logic
│       ├── cli.py       ← entry point
│       └── models.py
└── tests/
    ├── conftest.py      ← shared fixtures
    └── test_pipeline.py

WHY the problem being solved Without isolation, two projects on one machine can't use different versions of the same library. Without a lockfile, pip install resolves different packages at different times — silently breaking reproducibility. Virtual environments solve isolation; lockfiles solve reproducibility.

HOW the uv workflow Run uv init once. Add deps with uv add. Commit both pyproject.toml and uv.lock. Use uv run everywhere — no manual venv activation. In CI: uv sync --frozen && uv run pytest. In Docker: uv sync --frozen --no-dev.

WHERE every Python repo Every project from a one-file script to a multi-service monorepo. In consulting contexts, packaging analysis as a proper versioned Python package makes client handoff clean and reproducible — they can pip install acme-analysis==1.2.0 six months later and get the same results.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "data-ingestor"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
    "httpx>=0.27",
    "pydantic>=2.7",
    "structlog>=24.0",
]

[project.scripts]
ingest = "data_ingestor.cli:main"

[dependency-groups]
dev = ["pytest>=8.2", "pytest-cov>=5.0", "ruff>=0.4", "mypy>=1.10"]

[tool.ruff]
line-length = 100
target-version = "py311"

[tool.mypy]
python_version = "3.11"
strict = true

# Commands:
# uv init data-ingestor
# uv add httpx "pydantic>=2.7" structlog
# uv add --dev pytest pytest-cov ruff mypy
# uv run pytest
# uv sync --frozen --no-dev  (in Dockerfile for production image)