Jack Vanlightly

This post continues to elaborate on Virtual Consensus and the new Loglet abstractions of the AtomicLog and LogDrive.

In July, several Confluent colleagues and I published The LogDrive: Composable Durability for Cloud-Based Shared Logs.

This is the first in an ongoing ad-hoc series of posts on Apache Kafka performance.

Yesterday I published some benchmarks of Hardwood 1.0 on my Threadripper.

Hardwood is a minimal-dependency Java library for reading Parquet files.

In this post we’re going to see how share.acquire.mode=record_limit combined with: fewer consumers than partitions and various cases of “partition skew” …can result in subpar performance with share groups.

The lines between transactional systems, analytical systems, hybrid systems, and shared storage architectures are getting blurry.

“Perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away.” — Antoine de Saint-Exupéry AI gives us an unprecedented ability to add.

All tests were executed against Kafka 4.3.0 using Dimster.

This post is a little side-quest from my “Kafka Share Groups and Parallelizing Consumption” series.

All tests were executed against Kafka 4.3.0 using Dimster.

All tests were executed against Kafka 4.2.0 using Dimster (and also validated against 4.3.0).

In my last blog post I introduced Dimster (DIMensional teSTER), a performance benchmarking tool for Apache Kafka with a specific set of philosophies.

Dimster = DIMensional teSTER for Apache Kafka On GitHub: https://github.com/dimster-hq/dimster Most of my career in distributed systems has been as a tester, performance engineer and formal verification specialist.

Following my posts on determinism and durable function trees, this installment advances this blog post series “The Theory of Durable Execution”.

In part 1 we covered how durable function trees work mechanically and the importance of function suspension.

Continuing my “Theory of Durable Execution” series, in this post I'm going to explore how workflows can be formed from trees of durable function calls based on durable promises and continuations.

This is the first post in a series “The Theory of Durable Execution.”

In today’s post I want to walk through a fascinating indexing technique for data lakehouses which flips the role of the index in open table formats like Apache Iceberg and Delta Lake.

Today I want to talk about stream analytics, batch analytics and Apache Iceberg.