Skip to content

Databases & SQL with Python

What this lesson gives you

Persisting data means talking to a relational database. SQL fundamentals, parameterized queries, the ORM trade-off, indexing, and SQL injection prevention are all guaranteed interview territory.

Estimated time: 22 min read · Part: HTTP, REST APIs & Databases

Learning Objectives

After this lesson you will be able to: (1) write parameterized queries using Python's DB-API; (2) explain SQL injection and why parameterized queries prevent it; (3) define and query a SQLAlchemy ORM model; (4) recognize the N+1 query problem and fix it with eager loading; (5) explain the role of indexes and when a query becomes a full-table scan.

SQL Fundamentals & the DB-API

Most applications store state in a relational database (PostgreSQL, MySQL, SQLite) queried with SQL. Python's DB-API 2.0 (PEP 249) defines a standard interface that all database drivers implement: connect(), cursor(), execute(), fetchall(), commit(). This means code that works with sqlite3 works with minimal changes with psycopg2 (PostgreSQL) or mysql-connector-python — only the connection string and placeholder style (? vs %s) differ.

db_api.py
import sqlite3
from contextlib import contextmanager
from typing import Generator

@contextmanager
def get_connection(path: str = "app.db") -> Generator[sqlite3.Connection, None, None]:
    conn = sqlite3.connect(path)
    conn.row_factory = sqlite3.Row   # rows behave like dicts
    try:
        yield conn
        conn.commit()
    except Exception:
        conn.rollback()
        raise
    finally:
        conn.close()

# ── Safe: parameterized query — ALWAYS use this pattern ──────────
with get_connection() as conn:
    cur = conn.cursor()
    cur.execute(
        "SELECT id, email FROM users WHERE age > ? AND city = ?",
        (18, "Bengaluru"),
    )
    rows = cur.fetchall()
    for row in rows:
        print(row["id"], row["email"])   # row_factory enables dict access

# ── Bulk insert with executemany ───────────────────────────────────
users = [("u1", "ada@acme.io", 30), ("u2", "grace@acme.io", 25)]
with get_connection() as conn:
    conn.executemany(
        "INSERT INTO users (id, email, age) VALUES (?, ?, ?)",
        users,
    )

# ── DANGER: never build SQL with f-strings ────────────────────────
# user_input = "'; DROP TABLE users; --"
# cur.execute(f"SELECT * FROM users WHERE name = '{user_input}'")
# ^ This executes arbitrary SQL — classic SQL injection vector

SQL Injection — The Classic Vulnerability

If you build SQL by inserting user input into a string — even via an f-string or .format() — an attacker can craft input like '; DROP TABLE users; -- that terminates your statement and injects arbitrary SQL. The fix is always parameterized queries: the SQL text and the values travel to the database separately. The database parses the SQL structure first, then substitutes the values strictly as data — never as code, regardless of what they contain. This is non-negotiable and one of the most frequently asked security questions in technical interviews.

SQLAlchemy: The Python ORM Standard

An ORM (Object-Relational Mapper) maps database tables to Python classes and rows to instances. SQLAlchemy is the Python ORM standard: it generates parameterized SQL (injection-safe by default), maps results to Python objects, manages relationships, and handles migrations (via Alembic). You trade some raw SQL control for dramatically faster development on CRUD operations.

models.py
from sqlalchemy import (
    create_engine, String, Integer, ForeignKey, Index
)
from sqlalchemy.orm import (
    DeclarativeBase, Mapped, mapped_column, relationship, Session
)

class Base(DeclarativeBase):
    pass

class User(Base):
    __tablename__ = "users"

    id:         Mapped[int]         = mapped_column(primary_key=True)
    email:      Mapped[str]         = mapped_column(String(255), unique=True, index=True)
    tier:       Mapped[str]         = mapped_column(String(50), default="starter")
    orders:     Mapped[list["Order"]] = relationship(back_populates="user")

class Order(Base):
    __tablename__ = "orders"

    id:         Mapped[int]  = mapped_column(primary_key=True)
    user_id:    Mapped[int]  = mapped_column(ForeignKey("users.id"), index=True)
    product_id: Mapped[str]  = mapped_column(String(100))
    value:      Mapped[float]
    user:       Mapped["User"] = relationship(back_populates="orders")

    # Composite index on commonly-queried columns
    __table_args__ = (
        Index("ix_orders_user_product", "user_id", "product_id"),
    )

# ── CRUD operations ───────────────────────────────────────────────
engine = create_engine("postgresql+psycopg2://user:pass@localhost/acme")
Base.metadata.create_all(engine)

with Session(engine) as session:
    # Create
    user = User(email="ada@acme.io", tier="enterprise")
    session.add(user)
    session.flush()   # assigns user.id without committing

    order = Order(user_id=user.id, product_id="p42", value=299.99)
    session.add(order)
    session.commit()

    # Read — ORM generates parameterized SQL
    enterprise_users = (
        session
        .query(User)
        .filter(User.tier == "enterprise")
        .order_by(User.email)
        .all()
    )   # SQL: SELECT * FROM users WHERE tier = 'enterprise' ORDER BY email

ORM vs. Raw SQL

An ORM trades some control and performance for productivity and safety. Think of it as a vending machine vs a kitchen: the vending machine (ORM) gives you pre-packaged results quickly and safely; the kitchen (raw SQL) lets you cook exactly what you want but requires more skill and time. The mature stance: use the ORM for everyday CRUD, drop to raw SQL for complex analytical queries or performance-critical paths — and always know what SQL your ORM is generating (log it, use echo=True on the engine). The danger is using the ORM without understanding the SQL it emits.

The N+1 Problem & Eager Loading

The N+1 query problem is the most common ORM performance bug. It occurs when code fetches N parent rows and then, for each one, issues another query to fetch its related rows — resulting in 1 + N queries instead of a single JOIN. With 1000 users this means 1001 database round-trips instead of 1.

n_plus_1.py
from sqlalchemy.orm import Session, selectinload, joinedload

with Session(engine) as session:
    # ── BAD: N+1 — 1 query for users, then 1 per user for orders ──
    users = session.query(User).all()
    for user in users:
        # Each .orders access fires a new SELECT — 1001 queries for 1000 users!
        total = sum(o.value for o in user.orders)

    # ── GOOD: eager loading — 2 queries total ─────────────────────
    users = (
        session
        .query(User)
        .options(selectinload(User.orders))   # fetch all orders in one IN query
        .all()
    )
    for user in users:
        total = sum(o.value for o in user.orders)   # no additional queries

    # ── ALSO GOOD: JOIN loading — 1 query with a SQL JOIN ─────────
    users = (
        session
        .query(User)
        .options(joinedload(User.orders))
        .all()
    )

Consulting Lens: Database Performance Wins

When a client reports "the app is slow," the database layer is responsible the majority of the time. The highest-ROI checks, in order: (1) Missing indexes — a missing index on a commonly-filtered column turns a millisecond lookup into a full-table scan; adding it is one line of code. (2) N+1 queries — ORM lazy loading in a loop is often the cause of pages that issue hundreds of queries. (3) SELECT * over wide tables — fetching 50 columns when you need 3 wastes bandwidth and prevents index-only scans. (4) Connection pool exhaustion — too many concurrent requests competing for too few connections. Each of these is diagnosable in minutes with slow-query logging and EXPLAIN ANALYZE.

Indexes & Query Planning

An index is a data structure maintained alongside a table that allows the database to find rows matching a condition in O(log n) time instead of a full O(n) table scan. Every column you filter on (WHERE col = ?) or sort on (ORDER BY col) in high-frequency queries deserves consideration for an index. The tradeoff: indexes speed up reads but slow down writes and consume storage. Primary keys and unique constraints always create indexes automatically.

explain.sql
-- See what the query planner does WITHOUT the index
EXPLAIN ANALYZE
SELECT id, email FROM users WHERE tier = 'enterprise';
-- Output: Seq Scan on users (cost=0.00..847.00 rows=1234)
--         → full table scan on 10M rows; very slow

-- Create the index
CREATE INDEX CONCURRENTLY ix_users_tier ON users(tier);

-- Same query after indexing
EXPLAIN ANALYZE
SELECT id, email FROM users WHERE tier = 'enterprise';
-- Output: Index Scan using ix_users_tier on users (cost=0.44..5.12 rows=1234)
--         → uses the index; milliseconds not seconds
Tool / concept What it does When to use
DB-API (sqlite3, psycopg2) Low-level parameterized SQL execution Simple scripts, performance-critical raw SQL
SQLAlchemy ORM Maps tables to Python classes; generates safe SQL Application-level CRUD, complex relationships
SQLAlchemy Core SQL expression language (between raw and ORM) Complex queries needing structure but not full ORM
EXPLAIN ANALYZE Shows how the query planner executes your SQL Diagnosing slow queries
selectinload Eager load related rows in a second IN query Fix N+1 when you need many-to-one relations
joinedload Eager load with a JOIN Fix N+1 for one-to-one or many-to-one
Index B-tree structure for fast column lookups Every frequently-filtered or sorted column
Knowledge check

InterviewHow do parameterized queries prevent SQL injection?

  • They encrypt the SQL text so attackers can't read it.
  • The SQL structure is sent and parsed first; user input is sent separately as bound data. The database treats it strictly as a value — never as executable SQL — regardless of what it contains.
  • They run the query in a sandbox that blocks DROP statements.
  • They are just faster; they have no security effect.
Answer

The SQL structure is sent and parsed first; user input is sent separately as bound data. The database treats it strictly as a value — never as executable SQL — regardless of what it contains.

With parameterized queries, the SQL statement (with placeholders like ? ) is sent to the database first and compiled. Then the parameter values are sent separately and substituted as data into the already-compiled statement. Malicious input like '; DROP TABLE users; -- is therefore treated as a literal string value — it cannot break out of the data position and become executable SQL. String-built SQL, by contrast, creates a single string that the database parses as one statement — exactly where injection exploits the ambiguity.

Knowledge check

PracticalA page that displays a user's orders is slow. The code fetches all users, then loops through them accessing user.orders on each. What is the problem and the fix?

  • The problem is that SQLAlchemy is too slow; switch to raw SQL.
  • This is the N+1 problem: 1 query fetches users, then 1 more per user fetches orders — N+1 total. Fix: use eager loading (selectinload or joinedload) to fetch all orders in one or two queries.
  • The problem is that the orders table lacks a primary key.
  • This is expected behavior; all ORMs issue one query per relationship.
Answer

This is the N+1 problem: 1 query fetches users, then 1 more per user fetches orders — N+1 total. Fix: use eager loading (selectinload or joinedload) to fetch all orders in one or two queries.

The N+1 problem: accessing user.orders on a lazily-loaded ORM instance triggers a SELECT ... WHERE user_id = ? query for each user. With 1000 users, that's 1001 database round-trips. The fix is to tell the ORM to eager-load orders upfront using .options(selectinload(User.orders)) , which fetches all orders for all users in a single IN query, reducing 1001 round-trips to 2.

Knowledge check

ConceptA query SELECT * FROM users WHERE tier = 'enterprise' is fast in development (1000 rows) but extremely slow in production (10 million rows). What is the most likely cause?

  • The query has a syntax error that only manifests at large scale.
  • There is no index on the tier column, so the database does a full table scan — reading every row to find the matching ones. Adding an index makes it O(log n) instead of O(n).
  • Production uses a different SQL dialect than development.
  • The SELECT * is too broad; only selecting specific columns will fix it.
Answer

There is no index on the tier column, so the database does a full table scan — reading every row to find the matching ones. Adding an index makes it O(log n) instead of O(n).

Without an index on tier , the database reads every row in the table to find matches — a sequential scan. At 1000 rows this is fast; at 10 million rows it takes seconds. Adding an index (B-tree by default) allows the database to jump directly to matching rows in O(log n) time. Use EXPLAIN ANALYZE to confirm: you'll see "Seq Scan" (bad) vs "Index Scan" (good). The fix is one line: CREATE INDEX CONCURRENTLY ix_users_tier ON users(tier) .

Exercise

Using Python's sqlite3 module (no ORM), write a function get_top_customers(db_path, min_order_value, limit) that returns the top N customers by total order value, where each individual order exceeds min_order_value. Use parameterized queries. The function should return a list of dicts with keys customer_id, email, and total_spend.

Show solution
import sqlite3

def get_top_customers(
    db_path: str,
    min_order_value: float,
    limit: int,
) -> list[dict]:
    conn = sqlite3.connect(db_path)
    conn.row_factory = sqlite3.Row
    try:
        cur = conn.cursor()
        cur.execute(
            """
            SELECT
                u.id         AS customer_id,
                u.email      AS email,
                SUM(o.value) AS total_spend
            FROM users u
            JOIN orders o ON o.user_id = u.id
            WHERE o.value > ?
            GROUP BY u.id, u.email
            ORDER BY total_spend DESC
            LIMIT ?
            """,
            (min_order_value, limit),   # parameterized — safe from injection
        )
        return [dict(row) for row in cur.fetchall()]
    finally:
        conn.close()

Key Takeaways

  • Always use parameterized queries — never build SQL from f-strings or string concatenation.
  • SQL injection: user input becomes executable SQL in string-built queries; parameterization prevents this at the protocol level.
  • SQLAlchemy ORM maps tables to Python classes; generates injection-safe SQL by default.
  • Always know what SQL your ORM generates — use echo=True or query logging.
  • The N+1 problem: fix with selectinload or joinedload to eager-load relationships.
  • Missing indexes are the most common cause of "suddenly slow" queries at scale — use EXPLAIN ANALYZE.