Radbrt

The inherent uselessness of Medallion 🔗 Let’s get this out of the way first: I don’t like Medallion.

GPT came very close to giving a complete working tutorial on setting up OpenID Connect federated credentials that lets your Github Actions authenticate to Azure.

I am used to Snowflake.

A few years ago, helped along by zero interest rates, the data space was buzzing.

Yes, that PoC: https://www.reddit.com/r/dataengineering/comments/1h2t8op/dbt_poc_in_our_company_ended_in_a_disaster/ Go read it if you haven’t.

A few weeks back I came across https://european-alternatives.eu/, a site dedicated to highlighting European alternatives to digital services.

It is about 10 months since I first wrote my post on dbt unit tests.

The topic of test data comes up from time to time, and is plagued by the fact that test data can mean many different things.

The hype has subsided now, but you can still see it: The stack-fixation.

Today I changed visibility on my Metadog repository from private to public, and added an Apache 2 license.

One of the many corners of the (post-)modern data stack I have kept an eye on is observability.

I started writing this post a while back, but now that it has stayed half-done for several months I’m posting what I have.

In one of my first classes of ECON 101, the lecturer talked about economic models and likened them to maps.

I have written (ranted?)

Warning: amateur security writeup IT Security is fairly preoccupied with web application security.

A little while ago there was a small thread on Mastodon about data products, and David Jayatillake ended up writing a substack post explaining it: https://davidsj.substack.com/p/what-is-a-data-product.

Running Production ML with Snowflake and dbt 🔗 Snowflake runs Python/pyspark now, which is cool.

Another non-comprehensive list of things I have read and/or thought about since last time: Data Is Plural: A weekly newsletter with links to datasets.

A non-comprehensive list of things I have read and/or thought about lately: Artificial intelligence and privacy: Daniel Solove is one of the foremost scholars on privacy.

If you process significant amounts of data, but think Spark is a little messy, you have probably tried out Dask.

dbt has recently introduced unit tests in addition to their regular tests.

I probably need to deal with this subject not only separately, but in installments.

By now, I maintain quite a few Singer taps and targets, created with Meltano’s singer_sdk library.

The beginning and the end 🔗 The days of the modern data stack were waning.

A Singer Tap for Official Statistics 🔗 In the world of data engineering, Singer is popular standard, with tools like Airbyte and Meltano providing a flexible framework for data loading.

For whatever whimsical reason, as I read the financial paper, I got the idea to take a picture of one of the plots and ask ChatGPT to extract the data.

I came across a cool post by Drew Breunig about finding bathroom faucets with the CLIP model: https://www.dbreunig.com/2023/09/26/faucet-finder.html.

24 hours with a Surface Pro 🔗 Some months ago I severely cracked the screen of my iPad 11" (2018).

We are using the Singer target transferwise-target-snowflake for loading data into our warehouse.

I have stopped engaging with twitter, but I do still read it.

As usual with first blog posts, nothing to see here

About 🔗 Random thoughts on a random blog.