Skip to content

Exceptions & Defensive Code

What this lesson gives you

Errors are not exceptional — they're expected. How you raise, catch, and propagate exceptions defines how robust your systems are.

Estimated time: 22 min read · Part: Files, Errors & the Outside World

Learning objectives

  • Understand Python's exception hierarchy and the full try/except/else/finally syntax
  • Raise exceptions correctly and chain them with raise ... from exc
  • Define custom exception hierarchies for your domain
  • Apply EAFP (try/except) vs LBYL (pre-check) correctly
  • Design exception handling that distinguishes transient from permanent failures

In production systems, failure is not exceptional — it is a first-class part of the design. Networks time out. Files disappear. Users pass bad input. Services return unexpected status codes. The question is never "will it fail?" but "how does it fail, and how does your code respond?" Python's exception system is the primary tool for answering that question expressively and robustly.

The difference between junior and senior exception handling is not "try harder to avoid exceptions" — it's writing code that fails precisely, fails loudly when it should, fails gracefully when it can, and always leaves the system in a known state. These are design decisions as significant as any algorithm choice.

The Exception Hierarchy and Full try/except Syntax

Python exceptions are classes organized in a hierarchy rooted at BaseException. Most user-facing exceptions inherit from Exception. KeyboardInterrupt and SystemExit inherit directly from BaseException — catching Exception does not catch them, which is intentional: you almost never want to catch a Ctrl+C signal.

Python Exception Hierarchy (partial)
──────────────────────────────────────────────────────────────
BaseException
├── KeyboardInterrupt          ← Ctrl+C: don't catch with except Exception
├── SystemExit                 ← sys.exit(): don't catch
├── GeneratorExit
└── Exception                  ← catch THIS for normal error handling
    ├── ArithmeticError
    │   ├── ZeroDivisionError
    │   └── OverflowError
    ├── LookupError
    │   ├── IndexError         ← lst[999] on a 3-item list
    │   └── KeyError           ← d["missing_key"]
    ├── ValueError             ← int("abc"), wrong value type
    ├── TypeError              ← wrong type: "a" + 1
    ├── AttributeError         ← None.upper()
    ├── OSError (IOError)
    │   ├── FileNotFoundError
    │   ├── PermissionError
    │   └── TimeoutError
    ├── RuntimeError
    ├── StopIteration
    └── UnicodeError
        ├── UnicodeDecodeError
        └── UnicodeEncodeError
exception_anatomy.py
def parse_age(raw: str) -> int:
    """
    Full try/except/else/finally anatomy:
    - try:     code that might fail
    - except:  runs ONLY if the try block raised that exception
    - else:    runs ONLY if the try block did NOT raise
    - finally: ALWAYS runs — even with return or re-raise
    """
    try:
        age = int(raw)
    except ValueError as exc:
        # Catch ONLY what you can handle
        # raise ... from exc preserves the original exception as __cause__
        raise ValueError(f"age must be an integer, got {raw!r}") from exc
    else:
        # Only reached if int(raw) succeeded
        if age < 0 or age > 150:
            raise ValueError(f"age {age} is outside valid range [0, 150]")
        return age
    finally:
        # Always runs — cleanup, logging, metrics
        print(f"parse_age({raw!r}) attempt completed")

# Multiple except clauses — most specific first
def fetch_record(record_id: int) -> dict:
    try:
        return database.get(record_id)
    except KeyError:
        raise LookupError(f"Record {record_id} not found") from None
    except ConnectionError as exc:
        raise RuntimeError("Database unreachable") from exc
    except Exception as exc:
        # Catch-all for truly unexpected errors — log, then re-raise
        logger.error("Unexpected error in fetch_record: %s", exc)
        raise

# Catching multiple types in one clause
try:
    result = risky_io_operation()
except (TimeoutError, ConnectionRefusedError) as exc:
    log.warning("Transient network error: %s", exc)
    result = fallback_value

Exception Chaining and Custom Exceptions

When you catch an exception and raise a different one, Python automatically records the original as the "context" (__context__). Using raise NewExc() from original makes this explicit and sets __cause__, which is displayed differently in tracebacks. Using raise NewExc() from None explicitly suppresses the context — useful when the internal detail is an implementation concern, not something callers should see.

custom_exceptions.py
# Define a domain exception hierarchy
class AppError(Exception):
    """Base for all application errors — callers can catch this broadly."""
    pass

class ValidationError(AppError):
    """Input data failed validation."""
    def __init__(self, field: str, value, message: str):
        super().__init__(f"{field}={value!r}: {message}")
        self.field   = field
        self.value   = value
        self.message = message

class NotFoundError(AppError):
    """Requested resource does not exist."""
    def __init__(self, resource: str, identifier):
        super().__init__(f"{resource} {identifier!r} not found")
        self.resource   = resource
        self.identifier = identifier

class ServiceError(AppError):
    """Downstream service call failed."""
    def __init__(self, service: str, cause: Exception):
        super().__init__(f"{service} failed: {cause}")
        self.service = service

# Usage: precise exception types let callers handle each case
def create_order(data: dict) -> dict:
    if not data.get("customer_id"):
        raise ValidationError("customer_id", data.get("customer_id"),
                               "is required")

    try:
        customer = customer_service.get(data["customer_id"])
    except RequestException as exc:
        raise ServiceError("CustomerService", exc) from exc

    if not customer:
        raise NotFoundError("Customer", data["customer_id"])

    return order_service.create(data)

# Callers can handle each case precisely
try:
    order = create_order(payload)
except ValidationError as e:
    return {"error": "invalid_input", "field": e.field}
except NotFoundError as e:
    return {"error": "not_found", "resource": e.resource}
except ServiceError as e:
    log.error("Dependency failure: %s", e)
    return {"error": "service_unavailable"}

Never write a bare except: or silently swallow exceptions

A bare except: catches everything — including KeyboardInterrupt, SystemExit, and bugs you needed to see. Even except Exception: pass silently hides real failures and makes debugging a nightmare. The rule: catch the narrowest exception type you can meaningfully handle. If you can't handle it, let it propagate — a crash with a full traceback is infinitely more useful than a silent no-op. "Errors should never pass silently" is Pythonic design principle, line 10 of the Zen.

EAFP vs LBYL: Two Philosophies

LBYL (Look Before You Leap): check every precondition before the operation. EAFP (Easier to Ask Forgiveness than Permission): try the operation and handle failure if it occurs. Python idiom strongly favors EAFP for several reasons: it's more concurrent-safe (the file could vanish between your check and your read), it's often more readable (one try/except vs many nested ifs), and it avoids redundant work (checking and then doing is two operations).

eafp_lbyl.py
# LBYL: check before every operation
def get_user_email_lbyl(data: dict, user_id: str) -> str | None:
    if not isinstance(data, dict):
        return None
    if user_id not in data:
        return None
    user = data[user_id]
    if not isinstance(user, dict):
        return None
    if "email" not in user:
        return None
    return user["email"]

# EAFP: just try it, handle failures
def get_user_email_eafp(data: dict, user_id: str) -> str | None:
    try:
        return data[user_id]["email"]
    except (KeyError, TypeError):
        return None

# EAFP is also race-condition safe:
# LBYL: file could be deleted between os.path.exists() check and open()
import os
# BAD:
if os.path.exists("config.json"):         # check
    with open("config.json") as f:        # open — race condition window!
        data = f.read()

# GOOD: EAFP handles the same case without the race
try:
    with open("config.json") as f:
        data = f.read()
except FileNotFoundError:
    data = "{}"    # default config

EAFP vs LBYL: asking vs checking ID

LBYL is like a bouncer checking every possible rule before letting you in: "Are you over 18? Are you on the list? Are you wearing shoes?" — one check per possible failure. EAFP is like just trying to enter and dealing with any specific refusal: "Excuse me, you need shoes." — one response per actual problem. EAFP is usually more natural in Python because the language's duck typing means many "type checks" would need to be written explicitly anyway, and EAFP just handles the failure directly.

Consulting lens: transient vs permanent failures

In production systems, the most important design decision about exceptions is: is this failure transient (retry with backoff) or permanent (fail fast, alert)? A network timeout is transient; a validation error is permanent. Custom exception types enable this: you catch TransientError and retry; you catch PermanentError and fail. Defining this taxonomy in your exception hierarchy is what separates well-architected services from ones that retry indefinitely on bad data or give up on recoverable network hiccups.

WHY Why does Python prefer EAFP over LBYL? Python's duck typing means type checks are verbose ("does it have a .read() method?"). EAFP expresses the same validation as part of the operation itself — attempt it, respond to specific failures. It's also more correct in concurrent systems: the state of the world can change between a check and an action (TOCTOU — Time of Check, Time of Use race conditions). EAFP eliminates this race by making check and action a single atomic attempt.

HOW How does exception chaining work? When you raise inside an except block, Python automatically attaches the caught exception as __context__. raise NewExc from original sets __cause__ explicitly and shows "The above exception was the direct cause of the following exception." raise NewExc from None clears both, suppressing the context entirely — use this when the original is an internal implementation detail callers shouldn't see.

WHERE Where does bare except: do the most damage? In web request handlers. A bare except: pass in a Flask/Django view means the handler always returns 200 OK, even for database crashes, authentication failures, and program bugs. Monitoring sees nothing unusual; users get empty responses; bugs go undetected for days. Always let unexpected exceptions propagate to the framework's error handler — it will log the traceback and return a proper 500 response.

Knowledge check

InterviewWhy is catching except Exception: pass and silently continuing an anti-pattern?

  • It is slower than catching specific exceptions.
  • It hides bugs — swallowing errors you needed to surface, masking failed operations behind false success, and making debugging extremely difficult. Catch only what you can meaningfully handle.
  • It is a syntax error in modern Python.
  • It uses more memory than a targeted except.
Answer

It hides bugs — swallowing errors you needed to surface, masking failed operations behind false success, and making debugging extremely difficult. Catch only what you can meaningfully handle.

Catching broadly and ignoring exceptions masks real defects — a typo, a logic error, a missing service — behind a silent "everything's fine." It also interferes with legitimate exception-driven control flow. Best practice: catch the narrowest exception type you can act on, handle it meaningfully or re-raise with context, and let everything else propagate. A visible crash is always better than hidden corruption.

Knowledge check

Concept CheckWhen does the else clause of a try/except/else block execute?

  • Always, after the try block and any except that ran.
  • Only if the try block completed without raising any exception — it's the "success path."
  • Only if an exception was raised and caught.
  • Only if the finally block also runs.
Answer

Only if the try block completed without raising any exception — it's the "success path."

The else clause of a try statement runs if and only if the try block completed without raising. It's the "happy path" code that's separate from both the error-handling code ( except ) and the guaranteed-cleanup code ( finally ). Separating success logic into else makes it clear which code is "normal path" and which is "error path."

Knowledge check

DesignA payment processing function catches ConnectionError and retries; catches ValidationError and fails immediately. What design principle does this illustrate?

  • Catching broad exceptions to handle all cases uniformly.
  • Distinguishing transient errors (retry) from permanent errors (fail fast) — a fundamental production-system design principle enabled by precise exception typing.
  • LBYL — checking before attempting the payment.
  • Dependency inversion — depending on abstract errors.
Answer

Distinguishing transient errors (retry) from permanent errors (fail fast) — a fundamental production-system design principle enabled by precise exception typing.

The transient/permanent distinction is the most important exception design decision in distributed systems. Transient failures (network blips, rate limits, temporary unavailability) warrant retry with exponential backoff. Permanent failures (bad input, authentication failure, resource not found) warrant immediate failure and alerting. Custom exception types make this distinction explicit in code — callers know exactly which errors to retry and which to escalate.

Exercise: Resilient API Caller with Custom Exceptions

Define a custom exception hierarchy (APIError, RateLimitError, AuthError, NotFoundError) and write a function fetch_with_retry(url, max_retries=3) that: retries on RateLimitError with exponential backoff, fails immediately on AuthError and NotFoundError, and propagates all other exceptions unchanged.

Show solution
import time
import random

# Custom exception hierarchy
class APIError(Exception):
    def __init__(self, status: int, message: str):
        super().__init__(f"HTTP {status}: {message}")
        self.status  = status
        self.message = message

class RateLimitError(APIError):    pass   # transient  → retry
class AuthError(APIError):         pass   # permanent  → fail fast
class NotFoundError(APIError):     pass   # permanent  → fail fast

def _parse_response(status: int, body: str):
    """Translate HTTP status to domain exception."""
    if status == 200:
        return body
    elif status == 429:
        raise RateLimitError(429, "Too Many Requests")
    elif status in (401, 403):
        raise AuthError(status, "Authentication failed")
    elif status == 404:
        raise NotFoundError(404, body)
    else:
        raise APIError(status, body)

def fetch_with_retry(url: str, max_retries: int = 3) -> str:
    """
    Fetch URL with retry on transient errors.
    Permanent errors propagate immediately.
    """
    for attempt in range(max_retries + 1):
        try:
            # In production: status, body = http_client.get(url)
            status, body = simulate_request(url)
            return _parse_response(status, body)

        except RateLimitError:
            if attempt < max_retries:
                wait = (2 ** attempt) + random.uniform(0, 1)
                print(f"Rate limited. Retrying in {wait:.1f}s (attempt {attempt+1})")
                time.sleep(wait)
            else:
                raise                    # exhausted retries

        except (AuthError, NotFoundError):
            raise                        # permanent — don't retry

def simulate_request(url):
    # Simulates: first 2 calls return 429, third returns 200
    simulate_request.calls = getattr(simulate_request, "calls", 0) + 1
    if simulate_request.calls <= 2:
        return 429, ""
    return 200, '{"status": "ok"}'

try:
    result = fetch_with_retry("https://api.example.com/data")
    print("Success:", result)
except RateLimitError:
    print("Gave up after retries — rate limit persists")
except AuthError:
    print("Authentication failed — check credentials")
except NotFoundError:
    print("Resource not found")

Key takeaways

  • try/except/else/finally: except handles errors, else runs on success, finally always runs.
  • Catch the narrowest exception type you can handle; never swallow errors silently.
  • raise NewExc from original chains exceptions; from None suppresses context.
  • Define custom exception hierarchies for your domain — callers can catch precisely.
  • Python favors EAFP (try/except) over LBYL (pre-checks): more readable, race-condition safe.
  • Distinguish transient errors (retry) from permanent errors (fail fast) — the core of resilient system design.