Gunnar Morling

Table of Contents Write Support Query Evaluation: Bloom Filters and Dictionary-Based Row-Group Pruning Performance Improvements Hardwood CLI Closing Thoughts "When is write support gonna land in Hardwood?"

Table of Contents Parquet’s Dremel Encoding Reading Effectively-Fixed-Length Lists Faster Performance Gains Summary In its current form Apache Parquet isn’t a great fit for storing fixed-length lists, such as coordinates, RGB(A) colors, or—an increasingly…

Table of Contents Why Hardwood What’s in Hardwood 1.0 Performance The Hardwood CLI Building Open-Source With AI A Big Thank You What’s Ahead Hardwood is a new Parquet library for the JVM, written from scratch to do one thing well: read (and soon, write) A…

Table of Contents Reworked ColumnReader API Geospatial Support Documentation Overhaul Further Fixes and Improvements I am happy to announce the release of Hardwood 1.0.0.CR1!

I had the pleasure to do a few podcasts and interviews, e.g. talking about Debezium, change data capture, stream processing, my career, and software engineering in general.

Table of Contents 2026 2025 2024 2023 2022 2021 2020 2019 2018 2017 2016 2013 This page gives an overview over some talks I have done over the last years.

Table of Contents VARIANT Support Hardwood CLI TUI Unified Reader API Performance Improvements Wrapping Up I am happy to announce the release of Hardwood 1.0.0.Beta2!

Table of Contents S3 Backend Predicate Push-Down Avro Bindings hardwood-cli Wrapping Up I am pleased to announce the release of Hardwood 1.0.0.Beta1!

I have contributed to a wide range of open-source projects over the last years.

Table of Contents Why Hardwood?

Table of Contents Materialized Views Embracing Data Duplication Streams for machines, tables for humans Historically, data management systems have been built around the notion of pull queries: users query data which, for instance, is stored in tables in an

Table of Contents UUIDs Monotonically Increasing Sequences Deriving Idempotency Keys From the Transaction Log Discussion In distributed systems, there’s a common understanding that it is not possible to guarantee exactly-once delivery of messages.

Table of Contents Hello Persistasaurus!

Looking to make it to the front page of HackerNews?

Table of Contents ZGC Allocation Stalls Summary In the "Let’s Take a Look at…!"

Table of Contents confirmed_flush_sn: Tracking Consumer Progress restart_lsn: Handling Concurrent Transactions Mid-Transaction Recovery Looking Forward: Streaming In-Progress Transactions Replication slots in Postgres keep track of how far consumers have…

This post explores how virtual threads in Java 21+ provide an elegant solution for converting legacy Future objects into CompletableFuture instances.

Table of Contents Use the pgoutput Logical Decoding Output Plug-in Define a Maximum Replication Slot Size Enable Heartbeats Use Table-level Publications Use Column and Row Filters Enable Fail-Over Slots Consider Using Replica Identity FULL Monitor, Monito…

Table of Contents What I Use AI For What I Do Not Use AI For When you read a post on this site, you can be sure it is an original article, written by a human: me.

Table of Contents Agents Need to Interact With LLMs Agents Should Be Event-Driven Agents Need Context Agents Require Memory When SQL Is Not Enough Parting Thoughts AI Agents have improved in leaps and bounds in recent times, moving beyond simple chatbots…

Table of Contents Debezium Reselect Postprocessor Flink DataStream API Flink SQL With OVER Aggregation Flink Process Table Functions Summary and Discussion Postgres logical replication, while powerful for capturing real-time data changes, presents challen…

Often times, "Stream vs.

Table of Contents Speaker Information 🇬🇧 English 🇩🇪 Deutsch I am an open-source software engineer in the Java and data streaming space.

The last few days I spent some time digging into the recently announced KIP-1150 ("Diskless Kafka"), as well AutoMQ’s Kafka fork, tightly integrating Apache Kafka and object storage, such as S3.

Table of Contents Flink SQL Connectors for Apache Kafka The Apache Kafka SQL Connector in Append-Only Mode The Apache Kafka SQL Connector As a Changelog Source The Upsert Kafka SQL Connector Summary Over the years, I’ve spoken quite a bit about the use ca…

Table of Contents KIP-974: Docker Image for GraalVM based Native Kafka Broker With help of the GraalVM configuration developed for KIP-974 (Docker Image for GraalVM based Native Kafka Broker), you can easily build a self-contained native binary for Apache…

Table of Contents Building an AOT Cache for Apache Kafka AOT Caching With Apache Flink Summary In the "Let’s Take a Look at…!"

For building a system of distributed services, one concept I think is very valuable to keep in mind is what I call the synchrony budget: as much as possible, a service should minimize the number of synchronous requests which it makes to other services.

Table of Contents Towards Queue Support in Kafka—Introducing Share Groups Share Groups in Action Retry Behavior and State Management Share Group State Persistence Summary and Outlook In the "Let’s Take a Look at…!"

Table of Contents Fault Tolerance and High Availability Manually Triggering Savepoints Observability Bonus: Managing Flink Jobs With the Heimdall UI Summary and Discussion This post originally appeared on the Decodable blog.

Table of Contents Installation and Setup Deployment Types Deploying Your First Flink Job on Kubernetes Building Custom Job Images This post originally appeared on the Decodable blog.

Table of Contents Hello, Failover Slots!

If you are following the news around Debezium—an open-source platform for Change Data Capture (CDC) for a variety of databases—you may have seen the announcement that the project is in the process of moving to the Commonhaus Foundation.

Every now and then, it can come in very handy to build OpenJDK from source yourself, for instance if you want to explore a feature which is under development on a branch for which no builds are published.

Table of Contents Recap: What’s The Outbox Pattern?

During and after my time as the lead of Debezium, a widely used open-source platform for Change Data Capture (CDC) for a variety of database, I got repeatedly asked whether I’d be interested in creating a company around CDC.

Table of Contents Creating Instances Configuring SSH Provisioning Software Try It Out Yourself Whenever I’ve need a Linux box for some testing or experimentation, or projects like the One Billion Row Challenge a few months back, my go-to solution is Hetzn…

Table of Contents The Algorithm Obtaining the Lock Expiring a Lock Lock Validity Fencing Off Zombies In distributed systems, for instance when scaling out some workload to multiple compute nodes, it is a common requirement to select a leader for performing

In my day job at Decodable, I am currently working with Terraform to provision some cloud infrastructure for an upcoming hands-on lab.

The other day, I was looking for means of zipping two Java streams: connecting them element by element—essentially a join based on stream offset position—and emitting an output stream with the results.

Table of Contents Full Events Delta Events Id-only Events Change Event Metadata Comparison This post originally appeared on the Decodable blog.

In many applications it’s a requirement to keep track of when a record was created and updated the last time.

Recently I ran into a situation where it was necessary to capture the output of a Java process on the stdout stream, and at the same time a filtered subset of the output in a log file.

Table of Contents Results Bonus Result: 32 Cores, 64 Threads Bonus Result: 10K Key Set Thank You!

Update Jan 4: Wow, this thing really took off!

Table of Contents What is Postgres Logical Replication?

Table of Contents Stand-By Logical Replication With Debezium Towards Fail-Over Slots Wrap-Up This post originally appeared on the Decodable blog.

Table of Contents An Example Tracking RSS As regular readers of this blog will now, JDK Flight Recorder (JFR) is one of my favorite tools of the Java platform.

Table of Contents What Is PyFlink and Why Should You Care?

Table of Contents CDC—A Quick Primer Does CDC Break Encapsulation?