Skip to content

Dictionaries & Sets

What this lesson gives you

Hash-based collections give O(1) average lookup. The dict is arguably the most important data structure in Python — the language itself is built on them.

Estimated time: 22 min read · Part: Data Structures That Carry Real Work

Learning objectives

  • Understand how hash tables give O(1) average-case lookup, insertion, and deletion
  • Know why keys must be hashable and what "hashable" means precisely
  • Master every common dict operation and the most important dict patterns
  • Use set operations (union, intersection, difference) for efficient membership and deduplication
  • Know the modern dict API: .get(), .setdefault(), .items(), merge operator |

A dictionary maps keys to values using a hash table internally — a sparse array where the position of each entry is determined by the hash of its key. This gives O(1) average-case lookup, insertion, and deletion, regardless of the table's size. A set is a dict that stores only keys — an unordered collection of unique, hashable items with the same O(1) membership property.

CPython uses dicts pervasively: every module namespace, every class, and every object's attribute table is a dict. When you write obj.name, Python does a dict lookup into obj.__dict__. When you call a function, the local scope is a dict. Understanding dicts is understanding Python's runtime model.

How Hash Tables Work

To store a key-value pair, Python computes hash(key) — an integer fingerprint — and uses it (modulo the table size) to choose a slot. On retrieval, Python hashes the key again, jumps to that slot, confirms the stored key matches (in case of collision), and returns the value. The whole operation is O(1) on average because the hash function distributes keys uniformly, so most slots hold at most one entry.

Two critical invariants follow from this design: (1) a key's hash must never change after insertion, which is why only immutable types are hashable; and (2) if two objects compare equal (a == b), they must have the same hash — otherwise lookups would fail. Python guarantees this for all built-in types.

Hash Table Internals (simplified)
──────────────────────────────────────────────────────────
  hash("name") = 4713...  →  slot = 4713 % 8 = 1

  Slots array (capacity = 8, currently 3 entries)
  ┌──────┬──────────────────────────────────┬──────────┐
  │ slot │ key                              │ value    │
  ├──────┼──────────────────────────────────┼──────────┤
  │  0   │ (empty)                          │          │
  │  1   │ "name"           ← hash % 8 = 1  │ "Ada"    │
  │  2   │ (empty)                          │          │
  │  3   │ "role"           ← hash % 8 = 3  │ "eng"    │
  │  4   │ (empty)                          │          │
  │  5   │ "email"          ← hash % 8 = 5  │ "a@x.io" │
  │  6   │ (empty)                          │          │
  │  7   │ (empty)                          │          │
  └──────┴──────────────────────────────────┴──────────┘

  Lookup "name":  hash("name") % 8 = 1  →  check slot 1  →  O(1)
  Collision:      two keys land on same slot → probe to next free slot
  Resize:         when load factor > 2/3, table doubles → O(n) but rare

Keys must be hashable — and why that means immutable

Hash tables locate entries by their hash. If a key's hash could change after insertion (because the key was mutated), Python would look in the wrong slot and "lose" the entry — the table would be corrupted. Immutability guarantees a stable hash. That's why strings, numbers, and tuples of immutables can be keys; lists, sets, and dicts cannot. A common interview question: "why can a tuple be a dict key but not a list?" Answer: immutability → stable hash → safe key.

dict_operations.py
user = {"name": "Ada", "role": "engineer", "active": True}

# Basic access
user["name"]                      # "Ada"              O(1)
user.get("email")                 # None               O(1) — safe, no KeyError
user.get("email", "unknown")      # "unknown"          O(1) — with default
"role" in user                    # True               O(1) key membership
"salary" in user                  # False

# Mutation
user["email"] = "ada@x.io"        # insert or overwrite   O(1)
del user["active"]                # delete by key          O(1)
user.pop("role", None)            # delete + return, safe if missing

# Iteration — dicts preserve insertion order (Python 3.7+)
for key in user:                  # iterates keys
    print(key)
for key, val in user.items():     # iterates (key, value) pairs
    print(f"{key}: {val}")
list(user.keys())                 # ['name', 'email']
list(user.values())               # ['Ada', 'ada@x.io']

# Merge and update (Python 3.9+)
defaults = {"timeout": 30, "retries": 3}
config   = {"timeout": 60, "debug": True}
merged = defaults | config        # {'timeout': 60, 'retries': 3, 'debug': True}
# right side wins on collision

# dict comprehension
squares = {n: n**2 for n in range(6)}
# {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}

# Invert a dict (assuming unique values)
inv = {v: k for k, v in squares.items()}

Essential Dict Patterns

Beyond basic access, three patterns come up constantly in real code: counting, grouping, and caching. Each is a direct application of dict's O(1) operations and should feel automatic.

dict_patterns.py
from collections import defaultdict

# PATTERN 1: Frequency counting
events = ["click", "view", "click", "purchase", "view", "click"]

# Manual with .get()
freq = {}
for e in events:
    freq[e] = freq.get(e, 0) + 1
# {"click": 3, "view": 2, "purchase": 1}

# Idiomatic with defaultdict
freq2 = defaultdict(int)
for e in events:
    freq2[e] += 1   # missing key auto-initializes to int() = 0

# Best: Counter (covered in 5.4)
from collections import Counter
Counter(events)     # Counter({'click': 3, 'view': 2, 'purchase': 1})

# PATTERN 2: Grouping (one-to-many)
orders = [
    {"id": 1, "region": "EU", "amount": 120},
    {"id": 2, "region": "US", "amount": 85},
    {"id": 3, "region": "EU", "amount": 200},
    {"id": 4, "region": "US", "amount": 310},
]

by_region = defaultdict(list)
for order in orders:
    by_region[order["region"]].append(order["amount"])
# {"EU": [120, 200], "US": [85, 310]}

# PATTERN 3: Replacing an O(n) loop with an O(1) dict lookup
# Two-sum: find two indices whose values sum to target
def two_sum(nums, target):
    seen = {}                        # value -> index
    for i, n in enumerate(nums):
        complement = target - n
        if complement in seen:       # O(1) lookup
            return [seen[complement], i]
        seen[n] = i
    return []

two_sum([2, 7, 11, 15], 9)   # [0, 1]  — O(n) total, not O(n²)

{'click': 3, 'view': 2, 'purchase': 1} {'EU': [120, 200], 'US': [85, 310]} [0, 1]

Consulting lens: the O(n²) → O(n) transformation

Most "optimize this slow code" interview problems reduce to one insight: replace a nested loop (O(n²)) with a dict or set (O(n)). Two-sum, deduplication, join two datasets by key, detect first duplicate, count occurrences, group-by — all are hash table patterns. When you immediately reach for "index this in a dict keyed by X," you demonstrate the algorithmic instinct that tier-1 reviewers reward. The pattern is always the same: one pass to build the index, one pass to query it.

Sets: Hash Tables Without Values

A set is a hash table that stores only keys — no associated values. It inherits O(1) membership testing, O(1) add, and O(1) discard. Its primary use cases are: (1) deduplication, (2) fast membership testing, and (3) set algebra — union, intersection, difference, symmetric difference — all computable in O(min(len(a), len(b))) to O(len(a)+len(b)).

set_operations.py
# Set creation
active_users = {"alice", "bob", "carol"}    # literal syntax
ids = set([1, 2, 2, 3, 3, 3])              # deduplication: {1, 2, 3}
empty = set()                               # NOT {} — that's an empty dict!

# Membership — O(1)
"alice" in active_users    # True
"dave"  in active_users    # False

# Modification
active_users.add("dave")
active_users.discard("bob")    # remove if present, no error if missing
active_users.remove("bob")     # removes; raises KeyError if missing

# Set algebra — real-world: user overlap analysis
paying     = {"alice", "carol", "eve"}
trial      = {"bob", "carol", "frank"}

paying & trial          # intersection: {"carol"}       — both groups
paying | trial          # union: all users in either
paying - trial          # difference: paying but NOT trial: {"alice", "eve"}
trial  - paying         # trial but NOT paying: {"bob", "frank"}
paying ^ trial          # symmetric difference: in one but not both

# Subset / superset checks
{"alice"} <= paying     # True  — issubset
paying >= {"alice"}     # True  — issuperset

# Frozenset: immutable set (hashable — can be a dict key or set member)
cache_key = frozenset(["read", "write"])
permissions = {cache_key: True}

{'carol'}

intersection

{'alice', 'bob', 'carol', 'eve', 'frank'}

union

Operation dict set list Notes
Lookup / membership O(1) avg O(1) avg O(n) Hash vs linear scan
Insert O(1) avg O(1) avg O(1) amortized (end) Dict/set may resize
Delete O(1) avg O(1) avg O(n) List must shift
Ordered? Yes (3.7+) No Yes Set order is arbitrary
Duplicates? Keys unique Unique only Allowed Set auto-deduplicates
Key requirement Hashable key Hashable element Any object Lists/dicts can't be keys
Memory overhead High (sparse) High (sparse) Low (dense) Trade space for speed

is an empty dict, not an empty set

A very common mistake: s = {} creates an empty dict, not an empty set. To create an empty set, you must write s = set(). The brace syntax only creates a set when there's at least one element: {1, 2, 3} is a set. This inconsistency is historical — dicts predate sets in Python syntax — and it trips up even experienced developers.

Hash table as a library catalog

Imagine finding a book by scanning every shelf (linear search, O(n)) vs looking up the Dewey Decimal number and going directly to the shelf (hash lookup, O(1)). The hash function is the catalog: it maps a book title directly to a location, eliminating scanning. The dict is the most powerful single data structure in Python precisely because it turns "which shelf?" from a search into a calculation — that's the hash function's job.

Knowledge check

InterviewWhy can a tuple be used as a dictionary key but a list cannot?

  • Tuples are smaller in memory, which makes hashing faster.
  • Lists are slower in general, so Python forbids them as keys for performance.
  • Dict keys must be hashable with a stable hash; tuples are immutable so their hash never changes, while lists are mutable — mutating a list key after insertion would corrupt the hash table.
  • There is no real reason; it's an arbitrary historical restriction.
Answer

Dict keys must be hashable with a stable hash; tuples are immutable so their hash never changes, while lists are mutable — mutating a list key after insertion would corrupt the hash table.

Hash tables locate entries by computing hash(key) at lookup time. If a key's hash could change after insertion (because the key was mutated), Python would compute a different slot and fail to find the entry. Immutable objects have stable hashes and are therefore hashable. Lists are mutable, so Python raises TypeError: unhashable type: 'list' rather than allowing a silent correctness failure.

Knowledge check

InterviewYou have two lists of user IDs: paid_users (50k items) and churned_users (200k items). You want all paid users who have NOT churned. What is the most efficient approach?

  • Loop over paid_users and check if uid in churned_users for each.
  • Sort both lists and walk them with two pointers.
  • Convert churned_users to a set once, then use set difference: set(paid_users) - churned_set. O(n+m) total.
  • Use a database JOIN — Python can't do this efficiently.
Answer

Convert churned_users to a set once, then use set difference: set(paid_users) - churned_set. O(n+m) total.

The list-in-loop approach is O(paid × churned) = O(50k × 200k) = 10 billion comparisons. Converting to sets and using difference (

operator) is O(n+m): one pass to build each hash table, one pass for the difference. Set operations are the right tool for bulk membership and intersection/difference problems on large collections.

Knowledge check

GotchaWhat does d = {}; d["missing_key"] raise, and how do you safely retrieve a value with a fallback?

  • ValueError; use d.fetch("missing_key", default).
  • KeyError; use d.get("missing_key", default) which returns the default without raising.
  • IndexError; use a try/except block — there's no safer accessor.
  • Nothing — Python returns None automatically for missing keys.
Answer

KeyError; use d.get("missing_key", default) which returns the default without raising.

Dict subscript d[key] raises KeyError for missing keys. d.get(key) returns None ; d.get(key, default) returns the default value. Both are O(1). Use .get() when a missing key is a normal condition; use subscript when a missing key signals a bug (so the exception surfaces it). defaultdict auto-inserts a default on first access, useful for accumulator patterns.

Exercise: Join and Aggregate Two Datasets

You have two lists of dicts representing API data. Write a function enrich_orders(orders, customers) that joins them on customer_id and returns a list of dicts with order_id, amount, and customer_name. Then extend it to also return the top 3 customers by total spend. Aim for O(n+m) not O(n×m).

Show solution
from collections import defaultdict

def enrich_orders(orders, customers):
    """
    Join orders with customers on customer_id. O(n + m).
    """
    # Build customer lookup: O(m) once
    cust_index = {c["id"]: c["name"] for c in customers}

    enriched = []
    spend_by_customer = defaultdict(float)

    for order in orders:            # O(n) pass
        cid  = order["customer_id"]
        name = cust_index.get(cid, "Unknown")   # O(1) lookup
        enriched.append({
            "order_id":      order["id"],
            "amount":        order["amount"],
            "customer_name": name,
        })
        spend_by_customer[name] += order["amount"]

    # Top 3 by spend
    top3 = sorted(spend_by_customer.items(),
                  key=lambda kv: kv[1], reverse=True)[:3]

    return enriched, top3

# Sample data
orders = [
    {"id": 1, "customer_id": 101, "amount": 200.0},
    {"id": 2, "customer_id": 102, "amount": 80.0},
    {"id": 3, "customer_id": 101, "amount": 150.0},
    {"id": 4, "customer_id": 103, "amount": 320.0},
]
customers = [
    {"id": 101, "name": "Ada"},
    {"id": 102, "name": "Bob"},
    {"id": 103, "name": "Carol"},
]

enriched, top3 = enrich_orders(orders, customers)
for row in enriched:
    print(row)
print("Top spenders:", top3)
# Top spenders: [('Ada', 350.0), ('Carol', 320.0), ('Bob', 80.0)]

Key takeaways

  • Dicts and sets use hash tables: O(1) average lookup, insert, and delete.
  • Keys/members must be hashable — immutable types with a stable hash (strings, numbers, tuples of immutables).
  • Python 3.7+ dicts preserve insertion order; use .get(key, default) for safe access.
  • Sets support algebra: union (|), intersection (&), difference (-), symmetric difference (^).
  • {} is an empty dict; set() is an empty set.
  • Replacing nested loops with dict lookups turns O(n²) into O(n) — the most high-leverage optimization pattern in Python.