Interview frequency: …
Short answer
It decouples the producer from the consumer, and when maxsize is set the producer pauses at await put() when the queue is full, which gives backpressure and keeps memory bounded.
Explanation
An unbounded queue lets memory grow without limit if the consumer is slower. Run several consumers to spread the load, and use task_done and join to know when every item is complete. Define a clear stop mechanism: a sentinel value, or cancelling the consumers after join, and since Python 3.13 there is queue.shutdown(). Add timeouts so the producer does not wait forever if the consumers stop.
Code example
import asyncio
async def producer(q: asyncio.Queue, items):
for item in items:
await q.put(item) # Waits if the queue is full
async def consumer(q: asyncio.Queue):
while True:
item = await q.get()
try:
await process(item)
finally:
q.task_done()
async def main(items):
q = asyncio.Queue(maxsize=100)
workers = [asyncio.create_task(consumer(q)) for _ in range(5)]
await producer(q, items)
await q.join()
for w in workers:
w.cancel()Common mistakes
- Using a Queue without maxsize with a slow consumer, so memory runs out.
- Forgetting task_done, so join waits forever.