Releases / Upgrades / Change logs

Apache Kafka 2.4

• Kafka Core
  • Allow consumers to fetch from closest replica (before this all reads and writes happened on the leader)
  • Implement admin API for replica reassignment
  • Sticky partitioner
  • Admin API for deleting consumers offset

https://www.confluent.io/blog/apache-kafka-2-4-latest-version-updates

Apache Kafka 4.0

Apache Kafka 4.0 Release: Default KRaft, Queues, Faster Rebalances

• Apache Kafka 4.0 | ZooKeeper out, KRaft in! Scalable, Resilient, High Performance, more Cloud-Native - YouTube
• Kafka Broker, Controller, Producer, Consumer and Admin Client
  • ZooKeeper is gone
  • KIP-848: The Next Generation of the Consumer Rebalance Protocol
  • KIP-890: Transactions Server-Side Defense
  • KIP-932: Queues for Kafka (Early Access)
    • Queues for Kafka Explained (KIP-932) - YouTube
    • Share Partition, Share Consumer
    • Cooperative consumption model
    • Let's Take a Look at... KIP-932: Queues for Kafka! - Gunnar Morling
    • Queues for Kafka. Queues for Kafka is a proposed new… | by Andrew Schofield | Medium
  • KIP-1106: Add duration based offset reset option for consumer clients
  • KIP-1043: Administration of groups
• Kafka Streams
  • KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins
  • KIP-1112: Allow custom processor wrapping
• Kafka Connect
  • KIP-1074: Allow the replication of user internal topics
    • Previously, MirrorMaker 2 automatically excluded topics whose names ended with .internal or -internal, incorrectly classifying them as internal topics. This behavior prevented legitimate business topics from being replicated unless users implemented a custom replication policy. This KIP introduces a configurable option that allows users to replicate such topics without requiring custom code.

Apache Kafka 4.1 | Enhanced Stability, New OAuth Support, Scalable Queue...

Apache Kafka 4.2

Kafka Queues (Share Groups) is now production-ready with new features like the RENEW acknowledgement type for extended processing times, adaptive batching for share coordinators, soft and strict enforcements of quantity of fetched records, and comprehensive lag metrics.

Kafka Streams brings the server-side rebalance protocol to GA with a limited feature set, adds dead letter queue support in exception handlers, introduces anchored wall-clock punctuation for deterministic scheduling, and gives users full control over whether to send a leave group request on closing.

This release also delivers significant improvements to consistency and observability: CLI tools now feature standardized arguments like –bootstrap-server across all tools, metric naming has been corrected to follow the kafka.COMPONENT convention, and new idle ratio metrics provide better visibility into controller and MetadataLoader performance.

Security is enhanced with a new allowlist connector client configuration override policy, while thread-safety improvements to RecordHeader eliminate concurrency risks.

Additional highlights include external schema support in JsonConverter for reduced message sizes, dynamic configuration for remote log manager thread pools, adaptive batching in group coordinators, and rack ID exposure in the Admin API for consumer and share group members.

• KIP-932 GA: Queues for Kafka
• KIP-1071 GA: Streams Server-Side Rebalance Protocol
• KIP-1034: Dead Letter Queue Support for Streams
• Apache Kafka 4.2 | 38 KIPs, Kafka Streams, Observability Improvements, Rebalance Protocol, and more - YouTube
• Apache Kafka 4.2.0 Release Announcement | Apache Kafka

Others

Top Trends for Data Streaming with Apache Kafka and Flink in 2026 - Kai Waehner

1. Proven platforms gain ground as the data streaming ecosystem consolidates
2. Diskless Kafka and Apache Iceberg create a new storage foundation
3. Real-time analytics becomes part of the streaming layer
4. Enterprises demand SLAs with zero data loss and seamless failover
5. Regional cloud deployments are driven by compliance and sovereignty
6. Streaming powers Agentic AI with context and real-time model inference

Data streaming experts should focus on helping their organizations:

• Unify streaming and batch analytics in a single architecture
• Consolidate pipelines, reduce operational burden, and increase platform value
• Deliver regional compliance through sovereign deployment options
• Guarantee zero data loss and rapid failover across regions where needed
• Power AI systems with contextual, real-time event data