josh's blog

I’m taking some years off of work, and it’s entirely uncertain whether I’ll ever return to the kind of full-time gigs that I’ve held in the past.

This is a sequel to Running a Linux process deterministically.

I’m presently trying to build a Linux-based hypervisor that does some mild oversubscription of its RAM.

In a previous post, I said: If Kubernetes were ever going to scale to meet everyone’s needs, it would need to satisfy the “always a bigger fish” principle: objects group into namespaces, namespaces group into virtual clusters, and virtual clusters group i…

This post is followed by Deterministic computing on x86.

tl;dr: the new nullfs feature in Linux 7.0 allows shaving a few seconds off your boot time.

I’ve done a fair amount of work with Terraform since its initial release in 2014.

Let’s say I run a gunicorn server with a typical ASGI app, like so: $ gunicorn -w 4 -k uvicorn_worker.UvicornWorker app:app If I send it a request and view its activity under strace, I see this: [pid 160604] <... epoll_wait resumed>[{events=EPOLLIN, data…

There is no good reason to ever move a private key.

The previous post discussed ways that clients could optimize their interactions with RDBMSes to guarantee correctness and performance of their transactional logic.

We know that a good database is a stateless database, but what does that tell us about transactions?

Switching from handling web requests in a synchronous process-per-request1 model to the asyncio Task-per-request model loses us a fair bit, including: an MMU to keep our memory separate from our neighbors. separable security contexts (although some of this

Let’s say you’re debugging something in python, but the functionality you want to interact with is actually part of a C extension in the stdlib.

It turns out that asyncio tasks don’t really have stack traces‽ The documentation for Task.get_stack says: Only one stack frame is returned for a suspended coroutine.

Let’s say I have a FastAPI app.

Our previous post looked at a slightly interesting situation: we wanted to use standard Linux tools to solve a performance problem, but arguments to the calls themselves didn’t carry enough information for us to make meaningful distinctions.

It’s a tale as old as asynchronous time: your event loop is humming along, right up until something makes a blocking syscall.

A digital signature is a database row that has exited its database.

“An oncall rotation is where my freedom becomes your responsibility”.

At one point I was involved in an ongoing incident where customer data was being leaked.

In Cassandra Availability with Virtual Nodes, we discussed the availability tradeoffs inherent in the use of virtual nodes.

In a modern webapp, all of the following concerns should be performed by local sidecars: inbound request throttling, decapsulation, authentication, and authorization outbound service discovery, connection pooling, certificate management, encapsulation, ti…

This post summarizes my talk “Patterns in S3 data access: Protecting and enhancing access to data banks, lakes, and bases“1.

Our previous post concluded that we could quantify the statefulness of any given service by the length of time needed to bring a new node into service in a given cluster.

For a long time while working with databases, I conceived of them as fundamentally different from other services because they were stateful.

In server-side software, cryptography is like war: a last resort for when you’ve tried all of your other options, and they all failed. – myself, November 19, 2021 Despite my misgivings about cryptography, there is one kind of cryptography that I am happy…

My general attitude towards running services is that they should sit at the bottom of a deep dark hole, where requests arrive, get processed within the service, and then a response is emitted.

This post is a sequel to Tiers Considered Harmful.

The world needs only five computers – Someone A colleague asked me why Kubernetes failed to achieve its goal to be an organization-spanning cluster scheduler.

As a software engineer, I find that I can use queueing theory to enrich my understanding of two related problems: how to schedule work for humans and how to schedule work for computers.

Some security practitioners preach the value of just-in-time (JIT) access grants, where employees request and receive limited access to production data and systems, usually for a limited period of time.

I have been categorizing enclaves into a hierarchy: Level 0 (secret storage): allows you to store and retrieve the secret Level 1 (identity verification): checks who you are, then performs an operation on the secret Level 2 (policy enforcement): checks who

This post is for people who have already decided that layer 7 authN/Z controls are right for their needs.

Once upon a time, there were Feistel ciphers.

I had a bad brush with the single version rule at work, and I’d like to warn you so that you can avoid the pain it caused me.

tl;dr: If you run apt-get install reproducibly, there’s no reason to use “golden images”.

I’ll break your data perimeter with one neat trick.

Q: So when I make a flamegraph, how does that work?

How to use S3 This guide is meant to provide a quick introduction to S3.

Why contracts within engineering organizations don’t work One time at work, my team was upgrading an open source search-engine-cum-database that had an unfortunate predilection for breaking its external API.

This post is a re-host of the README for a demo I built of a method for binding an S3 signed URL to a user’s uniquely identifying cookie.

tl;dr: don’t define security tiers; use security cells instead each service should have its own security cell One time at JOB_1, we had a problem: we were about to start serving a dataset that was significantly more sensitive than our typical dataset.

The %util column in iostat has always been something of a weird bird.

all of these arguments fall under the heading of “the immutable server principle”.