rmoff's random ramblings

After a week’s holiday ("vacation", for y’all in the US) without a glance at anything work-related, what joy to return and find that the DuckDB folk have been busy, not only with the recent 1.3.0 DuckDB release, but also a brand new project called DuckLak…

SQL.

Another year, another Current—another 5k run/walk for anyone who’d like to join!

Whether you’re processing data in batch or as a stream, the concept of time is an important part of accurate processing logic.

Confluent Cloud for Apache Flink gives you access to run Flink workloads using a serverless platform on Confluent Cloud.

The problem with publishing February’s interesting links at the beginning of the month and now getting around to publishing March’s at the end is that I have nearly two months' worth of links to share 😅 So with no further ado, let’s crack on.

tl;dr: Upload a PDF document in which each slide of the carousel is one page.

In this blog post I’m going to explore how as a data engineer in the field today I might go about putting together a rudimentary data pipeline.

DuckDB added a very cool UI last week and I’ve been using it as my primary interface to DuckDB since.

I wrote a couple of weeks ago about using DuckDB and Rill Data to explore a new data source that I’m working with.

In this blog article I’ll show you how you can use the confluent CLI to set up a Kafka cluster on Confluent Cloud, the necessary API keys, and then a managed connector.

Much as I love kcat (🤫 it’ll always be kafkacat to me…), this morning I nearly fell out with it 👇 😖 I thought I was going stir crazy, after listing topics on a broker and seeing topics from a different broker. 😵 WTF 😵

Some would say that the perfect blog article takes the reader on a journey on in which the development process looks like this:

The UK Government publishes a lot of its data as open feeds.

There was a useful question on the Apache Flink Slack recently about joining data in Flink SQL: How can I join two streams of data by id in Flink, to get a combined view of the latest data?

Let’s imagine we’ve got a source of data with a nested array of multiple values.

The UK Environment Agency publishes a feed of data relating to rainfall and river levels.

I was exploring some new data, joining across multiple tables, and doing a simple SELECT * as I’d not worked out yet which columns I actually wanted.

Here’s a bunch of interesting links and articles about data that I’ve come across recently.

This post originally appeared on the Decodable blog.

I’m a HUGE fan of Docs as Code in general, and specifically tools like Vale that lint your prose for adherence to style rule.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

At Current 24 a few of us will be going for an early run (or walk) on Tuesday morning.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

I do my best to try and keep, if not abreast of, then at least aware of what’s going on in the world of data.

Let’s not bury the lede: it was DNS.

After a break from using AWS I had reason to reacquaint myself with it again today, and did so via the CLI.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

At this year’s Kafka Summit I’m planning to continue the tradition of going for a run (or walk) with anyone who’d like to join in.

This year Kafka Summit London includes a dedicated track for talks about Apache Flink.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.

This post originally appeared on the Decodable blog.