Kafka: How to import/export CSV/TXT data
Overview Working with Apache Kafka, one of the most popular distributed streaming platforms, often requires the ability to import and export data in common file formats like CSV…
Kafka: How to read events from a topic (with examples)
Introduction Apache Kafka is a popular open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. It is designed to handle data feeds…
Kafka: How to add events to a topic with a timestamp
Introduction Apache Kafka has become the backbone of many modern data-driven applications because of its capability to handle high-throughput, resilient distributed streaming. Timestamps play a crucial role in…
Kafka: Limit the memory used for buffering across all threads
Introduction Apache Kafka is a distributed streaming platform that is widely used for building real-time data pipelines and streaming applications. One of the key aspects of operating a…
Kafka: How to limit the number of messages per partition
Introduction Apache Kafka is a distributed streaming platform enabling thousands of companies worldwide to process and analyze streams of data in real-time. One of Kafka’s key features is…
ZooKeeper-less Kafka: Using Kraft and Raft Metadata Mode
Introduction Apache Kafka has been traditionally known to rely on ZooKeeper for distributed coordination and metadata storage. However, with the introduction of KIP-500, Kafka aims to replace ZooKeeper…
How to Implement Complex Event Processing in Kafka
Introduction Apache Kafka is a powerful distributed streaming platform that allows you to process a large stream of data in real-time. Complex Event Processing (CEP) is a technique…
Getting started with KSQL for Kafka stream processing
Introduction Kafka Streams is a powerful tool for building real-time data processing pipelines and applications. With the rise of streaming data and the need to process it on…
How to Design Kafka-Based Event-Driven Microservices (with Examples)
Understanding Kafka and Event-Driven Architecture Apache Kafka is an open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. Kafka aims to provide…
How to Configure Kafka for Exactly-Once Processing
Introduction Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation which is used to build real-time data pipelines and streaming apps. It is…