Administered by:
#ApacheKafka
One Open-source Project Daily
Change data capture for a variety of databases.
https://github.com/debezium/debezium
#1ospd #opensource #apachekafka #cdc #changedatacapture #datapipeline #database #debezium #eventstreaming #kafka #kafkaconnect #kafkaproducer
#Error Handling #Patterns for #ApacheKafka Applications
https://www.confluent.io/blog/error-handling-patterns-in-kafka/
KafkIO 1.2.2 released: http://kafkio.com. Now edit all broker or topic settings. Kafka 4.0+ support. Other productivity improvements around topic favourites and search, as well as bug fixes. #apachekafka #datascience
Apache Kafka is a powerful tool for processing data streams - but have you ever wondered what's really happening under the hood? 
You can check it with our Kafka Visualization 
https://softwaremill.com/kafka-visualisation/
With this tool, you can simulate how data flows through a replicated Kafka topic, to gain a better understanding of the message processing model. Have fun exploring Kafka!
What’s the difference between SSDs and Object Storage? Latency, redundancy and $$$$! Plus a lot of deeply technical work if you want the flexibility to choose.
This week I’m joined by Filip & Josep of Aiven, talking about adapting #ApacheKafka to write straight to S3:
Apache Kafka’s tiered storage unlocks massive scalability and long-term data retention without breaking the bank. Watch Paul Brebner explain how it works, its impact on performance and cost, and real-world replay use cases at #FOSSASIASummit2025
Slides for my Wix Kafka based async patterns at Devoxx Greece:
Why, when, how & trade-offs
1. Integration events
2. Task Queue
3. Task Scheduler
4. Iterator
#ApacheKafka #Kafka #DevoxxGreece2025 #DevoxxGR2025 #Async #eventdriven
Had a blast giving my talk on Wix Engineering's Kafka based async patterns at Devoxx Greece. Thank you for your insightful questions and discussions on why Wix doesn't have distributed transactions, Sagas and doesn't use the outbox pattern.
#Kafka 4.0 is out with a slew of new features and enhancements!
Notably, it introduces default operation in KRaft mode, eliminating Apache ZooKeeper dependencies.
Dive deeper into the updates on #InfoQ https://bit.ly/429ZfuH
Can’t wait to speak at Devoxx Greece tomorrow about Kafka-based asynchronous patterns powering Wix Engineering's 4,000 microservices!
Here’s a sneak peek of some fun memes from my slidedeck! See you there!
Selecting the right message broker for your application demands a deep understanding of available options & messaging patterns.
Critical factors include: traffic, auto-scaling, poison pill tolerance, batch processing, and ordering.
Read the #InfoQ article by Nehme Bilal and get it right: https://bit.ly/4iv41IW
Happy #piday !!!
Estimating Pi with Kafka Streams
https://fredrikmeyer.net/2024/05/06/estimating-pi-kafka.html
Our Client needed a robust solution to process vast amounts of data in real-time. We chose Apache Kafka for this project See how it helped overcome all the challenges: https://softwaremill.com/case-study/data-streaming-using-kafka-for-a-retail-company/
KafkIO v.1.1.6 released: http://kafkio.com. New: produce millions of messages with token substitution support for dates, UUIDs, strings, ints, doubles, counters. Re-publish messages. Quickly copy all messages or topic summaries to clipboard. #apachekafka #kafka #datascience
Cut Kafka cloud costs by 30%! Discover how Wix Engineering optimized high-traffic topics with a push-based consumer proxy.
Atlassian introduced Lithium - an in-house #ETL platform designed to meet the requirements of dynamic data movement.
Lithium simplifies cloud migrations, scheduled backups, and in-flight data validations with ephemeral pipelines and tenant-level isolation - ensuring efficiency, scalability & cost savings.
InfoQ spoke with Niraj Mishra, Principal Engineer at Atlassian, about Lithium’s implementation and future.
Read more here: https://bit.ly/415RPYZ
#DataPipelines #KafkaStreams #ApacheKafka #ApacheFlink #SoftwareArchitecture
We know data layers will survive the AI revolution. Will Kafka?
#apachekafka meetup ahead of #FlinkForward Jakarta tomorrow!
Join the Apache Kafka x Apache Pinot meetup hosted by VirtusLab and featuring @adamwarski
The event will take place on November 26th from 6:00 p.m. to 8:00 p.m. in Kraków. Register here: https://www.meetup.com/krakow-kafka/events/304395169/?eventOrigin=group_events_list