Kafka Consumer Lag
Software Engineer path · lesson 84 of 99Production Kafka health: tune produce and consume rates and watch consumer lag (log-end-offset minus committed offset) grow, spike, and drain — the metric you alert on in production.
In production, records-lag-max is the maximum lag across all partitions in the consumer group — alerting on the worst partition, not just the average.
Key insight
Lag is a rate problem: it accumulates whenever produce rate > consume rate. Produce and consume are balanced, so lag stays flat. A traffic spike adds a one-off backlog the consumer must work off. In production you alert on records-lag-max, not instantaneous throughput.
What happens to lag?
Sort each event by its effect on consumer lag.
Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.
Lag grows
Drop items hereLag shrinks
Drop items hereTest your knowledge
How is consumer lag defined?
The short version
Consumer lag is like a dish pile by the sink: plates arrive (produced) faster than you wash them (consumed) and the pile grows. Wash faster than they arrive and the pile shrinks. A dinner party is a traffic spike — a sudden backlog you work off over time.
In more depth
Consumer Lag
Lag measures how far a consumer trails the head of the partition log:
lag = log-end-offset − committed-offset
It grows whenever consume rate < produce rate and drains once the consumer catches up. The metric to alert on is records-lag-max (the worst partition), because uneven partition load means one hot/stalled partition can build large lag while others stay healthy.
Remedies
Interview Questions
References & Further Reading
Love this visualization?
Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.
100% free • No credit card required