async/await & asyncio¶
What this lesson gives you
Asynchronous code lets one thread juggle thousands of waiting I/O operations. It's the backbone of modern Python web servers and API clients.
Estimated time: 22 min read · Part: Concurrency, Parallelism & Performance
Learning objectives
- Understand the event loop model and how async/await achieves concurrency on one thread
- Write
async defcoroutines andawaitthem correctly - Use
asyncio.gather()andasyncio.create_task()to run coroutines concurrently - Handle errors in async code and use async context managers and iterators
- Know when async helps (I/O-bound) and when it doesn't (CPU-bound) — a critical interview boundary
Concurrency is dealing with many things at once; parallelism is doing many things simultaneously. For I/O-bound work — waiting on networks, disks, databases — async/await achieves concurrency without parallelism: a single thread starts an operation, suspends while waiting for a response, switches to other work, then resumes when the result arrives. The program makes progress on many things, but only one thing runs at any instant.
This model is the foundation of modern Python web frameworks (FastAPI, Starlette, aiohttp), data ingestion pipelines, and high-throughput API clients. When you need to make 500 external API calls, async lets a single thread overlap all the waiting time — total duration approaches the slowest single call rather than their sum.
The Event Loop: One Thread, Many Tasks¶
The event loop is a run-to-completion scheduler. It maintains a queue of ready tasks, picks one, runs it until it hits an await, suspends it (recording where to resume), picks the next ready task, and so on. When an I/O operation completes (a socket becomes readable, a timer fires), the sleeping task is marked ready and re-enters the queue. There is no preemption — a task runs until it voluntarily yields via await.
asyncio Event Loop — Single Thread, Multiple Coroutines
──────────────────────────────────────────────────────────────────────────
Time → 0 1 2 3 4 5 6 7 8 9 10
Thread: [fetch_A starts]
[await network]─────────────────────────────[fetch_A resumes]
[fetch_B starts]
[await network]────────[fetch_B resumes]
[fetch_C starts]
[await network]───────────────[fetch_C resumes]
Sequential (sync): 0──────3──────6────────10 → 10s total
Async (concurrent): 0──────────────────────5 → ~5s total (max of individual waits)
The CPU is never blocked — it's always running another coroutine
while the current one waits on I/O.
import asyncio
import time
async def fetch(name: str, delay: float) -> str:
"""Simulate an I/O-bound operation (network request, DB query)."""
print(f" {name}: starting")
await asyncio.sleep(delay) # suspend here; event loop runs others
print(f" {name}: done after {delay}s")
return f"{name} result"
async def main():
start = time.perf_counter()
# Sequential: total time = sum of delays = 6s
# result_a = await fetch("A", 2)
# result_b = await fetch("B", 1)
# result_c = await fetch("C", 3)
# Concurrent: total time ≈ max delay = 3s
results = await asyncio.gather(
fetch("A", 2),
fetch("B", 1),
fetch("C", 3),
)
elapsed = time.perf_counter() - start
print(f"Results: {results}")
print(f"Total time: {elapsed:.2f}s") # ~3.0s, not 6.0s
asyncio.run(main())
A: starting B: starting C: starting B: done after 1.0s A: done after 2.0s C: done after 3.0s Total time: 3.01s
≈ max, not sum¶
gather vs create_task: Two Patterns for Concurrent Execution¶
asyncio.gather() schedules multiple coroutines together and waits for all of them. asyncio.create_task() schedules a coroutine immediately as a background task — it begins running without the current coroutine needing to await it right away. Tasks are useful when you want to fire-and-forget, or when you need more control over when to wait and when to cancel.
import asyncio
# ── Pattern 1: asyncio.gather — wait for all, return all results ──────────────
async def batch_fetch(urls: list[str]) -> list[str]:
"""Fetch all URLs concurrently, return all results in order."""
async def fetch_one(url):
await asyncio.sleep(0.1) # simulate HTTP request
return f"content of {url}"
return await asyncio.gather(*[fetch_one(u) for u in urls])
# ── Pattern 2: create_task — background task, fire and continue ──────────────
async def pipeline():
# Start background work immediately
bg_task = asyncio.create_task(long_background_job())
# Do other work while background runs
result = await fast_operation()
# Now wait for the background task
bg_result = await bg_task
return result, bg_result
# ── Pattern 3: gather with error handling ─────────────────────────────────────
async def robust_gather(coroutines):
"""Return results or exceptions — never raises on individual failure."""
results = await asyncio.gather(*coroutines, return_exceptions=True)
successes = [r for r in results if not isinstance(r, Exception)]
failures = [r for r in results if isinstance(r, Exception)]
return successes, failures
# ── Pattern 4: TaskGroup (Python 3.11+) — structured concurrency ─────────────
async def structured():
async with asyncio.TaskGroup() as tg:
task_a = tg.create_task(fetch("A", 1))
task_b = tg.create_task(fetch("B", 2))
# Both tasks complete before this line; any exception is propagated
print(task_a.result(), task_b.result())
# ── Pattern 5: timeout ────────────────────────────────────────────────────────
async def with_timeout():
try:
result = await asyncio.wait_for(fetch("slow", 10), timeout=2.0)
except asyncio.TimeoutError:
result = None # fallback if operation takes too long
return result
Async Context Managers and Iterators¶
The context manager and iterator protocols have async versions: async with (using __aenter__/__aexit__) and async for (using __aiter__/__anext__). These are used heavily in async libraries: database connections, HTTP sessions, and streaming responses all use async context managers and iterators to ensure resources are released correctly.
import asyncio
from contextlib import asynccontextmanager
from typing import AsyncIterator
# ── Async context manager ─────────────────────────────────────────────────────
@asynccontextmanager
async def managed_connection(host: str):
"""Acquire a connection, yield it, release on exit."""
print(f"Connecting to {host}...")
conn = await create_connection(host) # async setup
try:
yield conn
finally:
await conn.close() # async cleanup
async def query_data():
async with managed_connection("db.example.com") as conn:
return await conn.execute("SELECT * FROM orders")
# ── Async generator / async iterator ─────────────────────────────────────────
async def stream_records(batch_size: int = 100) -> AsyncIterator[list[dict]]:
"""Stream database records in batches without loading all into memory."""
offset = 0
while True:
batch = await fetch_from_db(offset=offset, limit=batch_size)
if not batch:
return # StopAsyncIteration
yield batch
offset += batch_size
async def process_all():
total = 0
async for batch in stream_records(batch_size=50):
for record in batch:
total += record.get("amount", 0)
return total
# ── Real-world pattern: parallel API calls with aiohttp (pseudocode) ─────────
# import aiohttp
# async def fetch_all(urls):
# async with aiohttp.ClientSession() as session: # one session, reused
# async def get(url):
# async with session.get(url) as resp:
# return await resp.json()
# return await asyncio.gather(*[get(u) for u in urls])
Async is not parallelism — never block the event loop
The most dangerous async mistake: calling a synchronous blocking function inside a coroutine. time.sleep(2) inside an async def blocks the entire event loop for 2 seconds — all other tasks freeze. Use asyncio.sleep(), not time.sleep(). Use async-native libraries (aiofiles, aiohttp, asyncpg), not their synchronous counterparts. If you must call blocking code, offload it to a thread pool with await asyncio.get_event_loop().run_in_executor(None, blocking_fn).
Why async wins for I/O-bound work
When you call an external API, your CPU is idle waiting for the network response — often 99% of the total operation time is waiting. Async uses that idle time: while one request waits, the event loop runs others. One thread can manage thousands of concurrent connections because most of those connections are just waiting at any given moment. That's why FastAPI can handle enormous request volumes with a single-process, single-thread worker: it's rarely actually running code — it's usually waiting, and those waits are all overlapped.
| Sync equivalent | Async version | Purpose |
|---|---|---|
time.sleep(n) | await asyncio.sleep(n) | Pause without blocking |
with resource: | async with resource: | Async context manager |
for item in iter: | async for item in aiter: | Async iteration |
open("f.txt") | aiofiles.open("f.txt") | Async file I/O |
requests.get(url) | await session.get(url) (aiohttp) | Async HTTP |
psycopg2.connect() | await asyncpg.connect() | Async PostgreSQL |
| Run 1 coroutine | asyncio.run(coro()) | Entry point from sync code |
| Run many concurrently | await asyncio.gather(*coros) | Fan-out, collect results |
WHY Why is asyncio single-threaded? Thread safety is expensive and complex. Sharing mutable state across threads requires locks, and getting locking wrong causes deadlocks and race conditions. By running on one thread, asyncio avoids all of this — there's no concurrent access to shared state because only one coroutine runs at a time. The trade-off: CPU-bound work gets no speedup, and one truly blocking call starves all other tasks. The model works because I/O-bound work spends most of its time waiting, not running.
HOW How does await actually suspend execution? An async def function compiles to a generator-like coroutine object. await expr is essentially yield from expr: it suspends the current coroutine and yields control back to the event loop. The event loop records where to resume (the coroutine's frame state), stores the coroutine in a waiting set keyed by its I/O handle, and picks the next ready coroutine from its run queue. When the I/O handle becomes ready, the coroutine is moved back to the run queue.
WHERE Where does async shine most in production? High-concurrency API servers (FastAPI serving thousands of simultaneous requests), batch data ingestion pipelines fetching from many sources in parallel, webhook consumers, web scrapers, and real-time stream processors. Anywhere the bottleneck is external I/O latency rather than CPU time is a candidate for async. FastAPI running on a single uvicorn worker can comfortably handle 1000+ concurrent requests because each request spends most of its time awaiting DB or downstream service responses.
Knowledge check
InterviewYour service makes 500 slow external API calls per request. Would asyncio help, and why?
- Yes — the calls are I/O-bound (mostly waiting on the network), so async overlaps the waiting and a single thread handles all 500 concurrently, cutting total latency to approximately the slowest single call.
- No — async never helps with network calls.
- Yes, but only because async uses 500 CPU cores simultaneously.
- No — you must rewrite the service in a compiled language.
Answer
Yes — the calls are I/O-bound (mostly waiting on the network), so async overlaps the waiting and a single thread handles all 500 concurrently, cutting total latency to approximately the slowest single call.
External API calls are I/O-bound: the program spends almost all time waiting on the network. asyncio excels here — while one request awaits its response, the event loop drives the others. 500 calls overlap on one thread and total time approaches the slowest call's latency rather than the sum. For CPU-bound work, async gives no speedup since the CPU is always the bottleneck, not waiting.
Knowledge check
GotchaYou place time.sleep(5) inside an async def function. What happens to all other running coroutines?
- They continue running normally; the sleep only affects the current coroutine.
- The entire event loop freezes for 5 seconds — all other coroutines are blocked because
time.sleepis synchronous and holds the thread. - Python automatically converts it to an async sleep.
- A RuntimeError is raised immediately.
Answer
The entire event loop freezes for 5 seconds — all other coroutines are blocked because time.sleep is synchronous and holds the thread.
asyncio runs on a single OS thread. time.sleep(5) is a synchronous call that blocks the thread itself — the OS puts the entire thread to sleep. While the thread sleeps, the event loop cannot run, so all other coroutines are frozen. The fix: always use await asyncio.sleep(5) , which suspends the current coroutine without blocking the thread, allowing other coroutines to run.
Knowledge check
Concept CheckWhat is the difference between asyncio.gather() and asyncio.create_task()?
- They are identical — just different spelling.
gather()schedules and awaits all coroutines together, returning when all complete.create_task()schedules a coroutine immediately as a background task you can await later independently — useful for fire-and-continue patterns.gather()is for CPU work;create_task()is for I/O work.create_task()runs on a separate thread;gather()stays single-threaded.
Answer
gather() schedules and awaits all coroutines together, returning when all complete. create_task() schedules a coroutine immediately as a background task you can await later independently — useful for fire-and-continue patterns.
asyncio.gather(*coros) wraps all coroutines in tasks, starts them all, and awaits until the last one completes — returning a list of results in the same order. asyncio.create_task(coro) schedules one coroutine as a task immediately (it starts running on the next event loop iteration) and returns a Task object you can await separately. Use gather for fan-out; use create_task when you want more control over individual task lifecycles.
Exercise: Parallel API Fetcher with Rate Limiting
Write an async function fetch_all(urls, max_concurrent=10) that fetches data from all URLs concurrently but limits to at most max_concurrent simultaneous requests (to avoid overwhelming the server or hitting rate limits). Use asyncio.Semaphore for rate limiting. Return a list of (url, result_or_exception) tuples.
Show solution
import asyncio
from typing import Any
async def fetch_all(
urls: list[str],
max_concurrent: int = 10
) -> list[tuple[str, Any]]:
"""
Fetch all URLs concurrently with a cap of max_concurrent in-flight.
Returns (url, result) or (url, Exception) for each.
"""
semaphore = asyncio.Semaphore(max_concurrent)
async def fetch_one(url: str) -> tuple[str, Any]:
async with semaphore: # only max_concurrent coroutines enter
try:
# In production: use aiohttp.ClientSession
await asyncio.sleep(0.1) # simulate HTTP call
return (url, f"content:{url}")
except Exception as exc:
return (url, exc) # return exception, don't raise
tasks = [asyncio.create_task(fetch_one(u)) for u in urls]
return await asyncio.gather(*tasks)
async def main():
urls = [f"https://api.example.com/item/{i}" for i in range(25)]
results = await fetch_all(urls, max_concurrent=5)
successes = [(url, r) for url, r in results if not isinstance(r, Exception)]
failures = [(url, r) for url, r in results if isinstance(r, Exception)]
print(f"Fetched {len(successes)} OK, {len(failures)} failed")
for url, result in successes[:3]:
print(f" {url} → {result}")
asyncio.run(main())
Key takeaways
- Async/await runs on one thread: coroutines suspend at
await, letting the event loop run others while waiting. - Total time for concurrent I/O tasks ≈ the slowest individual task, not their sum.
asyncio.gather()runs many coroutines concurrently;create_task()fires one in the background.- Never call synchronous blocking code (time.sleep, sync DB calls) inside a coroutine — it freezes the event loop.
- Use
asyncio.Semaphoreto cap concurrency; useasyncio.wait_for()to add timeouts.