Interview frequency: …
Short answer
Sharding distributes a table's data horizontally across independent databases or physical servers to increase storage capacity and throughput. The biggest challenge is selecting a shard key that distributes data evenly and avoids cross-shard joins.
Explanation
If you choose a poor shard key such as Country when 90% of your customers are in Egypt, one server becomes a hotspot while the others remain idle. Queries without the shard key in their WHERE clause must be sent to every server in parallel (a scatter-gather query), severely hurting performance. Separate databases also lose global atomic ACID transactions and require complex solutions such as Two-Phase Commit or Saga patterns.
Code example
import hashlib
SHARDS = ["db-0", "db-1", "db-2", "db-3"]
def shard_for(user_id: int) -> str:
digest = int(hashlib.sha1(str(user_id).encode()).hexdigest(), 16)
return SHARDS[digest % len(SHARDS)]
# Good shard key: high cardinality and evenly distributed (user_id).
# Bad shard key: country or created_at → hot shards.
# A query that is not filtered by the shard key must fan out to every shard and merge in the application.
def total_orders():
return sum(connect(s).execute("SELECT count(*) FROM orders").scalar() for s in SHARDS)Common mistakes
- Turning to sharding too early, before exhausting good indexing, vertical partitioning, caching, and read replicas.