algorithms.live
    SYSTEM-DESIGN
    advanced

    Kafka Architecture

    Software Engineer path · lesson 75 of 99
    Producer ASends to
    Partition 0
    Producer BSends to
    Partition 1
    Kafka Cluster
    KRaft Controller (Coordination)

    Broker 1

    Topic: User-Events | Partition: 0
    Empty
    Offset 0Offset 0

    Broker 2

    Topic: User-Events | Partition: 1
    Empty
    Offset 0Offset 0
    Consumer Group 1
    Consumer AReading Partition 0
    Offset: 0
    Consumer BReading Partition 1
    Offset: 0
    Consumer Group 2

    Reads the same partitions independently — demonstrating Kafka replay

    Consumer CReading Partition 0
    Offset: 0
    Consumer DReading Partition 1
    Offset: 0

    Who does what?

    Sort

    Sort each responsibility into the Kafka component that owns it.

    Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.

    Producer

    Drop items here

    Broker

    Drop items here

    Partition

    Drop items here

    Consumer group

    Drop items here
    6 pieces left to place

    Test your knowledge

    21 questions
    Question 1 of 21
    Score: 0

    What happens to a message in Kafka after a consumer reads it?

    The short version

    Kafka is like a continuous, uneditable diary. Producers write new entries at the end. Readers (consumers) place a bookmark (offset) to remember where they stopped reading, allowing multiple people to read at different speeds without destroying the pages.

    In more depth

    Apache Kafka

    Kafka is a distributed event streaming platform designed for high-throughput, low-latency data pipelines.

    Core Concepts

    Topics & Partitions: Data is categorized into topics, which are split into partitions for parallel processing.
    Append-Only Log: Events are immutable and appended sequentially to disk.
    Consumer Groups: Allow parallel consumption. Each partition is read by exactly one consumer within a group.
    Offsets: A simple integer pointer indicating how many messages a consumer has read, allowing them to resume after a crash without message loss.

    Interview Questions

    LinkedIn
    Netflix
    Uber
    LinkedIn
    Confluent
    Netflix
    Uber
    Twitter/X
    Datadog

    References & Further Reading

    Kafka Architecture — Explained with DiagramYouTube
    video
    Apache KafkaWikipedia
    article
    Apache Kafka DocumentationApache Kafka
    docs
    phoenixtype/system-design-interview-prepGitHub
    docs
    All System Design topics

    Love this visualization?

    Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.

    100% free • No credit card required