https://peng.fyi/index.xml

Our container platform moved from Amazon Linux 2 (AL2) to Bottlerocket.

I recently reviewed code that let a dynamic set of goroutines wait for updates from a single writer.

I was getting an arm64 build of our Bottlerocket variant ready for production on Graviton EC2 instances.

I used NVIDIA's go-dcgm to read GPU health data and stats.

A while ago I debugged a resource leak caused by a service stuck in a restart loop.

We run an OpenTelemetry Collector (otel-collector) as a systemd service on Bottlerocket.

I recently ran into an issue after upgrading the containerd client library to v2.3.0.

A container platform shipped a change to detect CPU architecture mismatches at image pull time.

If you build software that runs in someone else's environment, anything that isn't a SaaS you operate yourself, the following code can break your customers: 1if not hostname.endswith("."): 2 hostname += "."

I recently ran into a Go test timeout that turned out to be a deadlock.

Bottlerocket v11.0.0 dropped the socat package from bottlerocket-core-kit (#742).

CGO_ENABLED defaults to 1.

We run a container platform.

This post is part 1 of a series of learnings from nilaway. nilaway is a static analysis tool that detects potential nil panics in Go code.

WaitGroup and channels are two powerful primitives in Go for synchronizing goroutines.

I recently encountered IMDS (Instance Metadata Service) request failures with this error: 1"caller": "actor/actor.go:101", 2"error": "operation error ec2imds: GetMetadata, failed to get rate limit token, retry quota exceeded, x available, y requested The…

While prototyping Bottlerocket, I discovered it doesn't recognize additional EBS volumes specified through Block device mappings on Xen.

I recently debugged a resource leak where a systemd service kept restarting while leaving a process behind after each restart.

I recently debugged a mysterious latency issue: after migrating a systemd service from path-activation to socket-activation, there was a consistent ~1 second time-to-available latency.

When I first moved building Bottlerocket AMI from an EC2 host to AWS CodeBuild, I was hit by a very slow build.

A bug caused around 0.5% of container workloads to fail to start during load test.

Early this year, we migrated containerd from v1.7 to v2.0.5.

In Detect and fix rare cases where the primary ENI does not serve default traffic , we used IMDS "meta-data/mac" to get the primary ENI's MAC address.

There are a few programs we install in Bottlerocket that cannot be built from source.

Bottlerocket is a Linux-based operating system optimized for hosting containers.

Bottlerocket is a Linux-based operating system optimized for hosting containers.

I recently dealt with a server livelock issue caused by memory page thrashing.

During testing, we encountered a rare scenario when launching EC2 instances with multiple ENIs: the primary ENI (device index 0) does not serve default network traffic.

Security-Enhanced Linux (SELinux) is a mandatory access control (MAC) system that enhances Linux security.

Go is known for its backward compatibility, simplicity, and six-month release cycle.

Shell scripts are infamous for security issues and surprising behavior, so when possible, it's better to avoid using shell.

I recently worked on getting amazon-ssm-agent to run inside containers on Bottlerocket.

A recent faulty release disrupted service for some customers.

This week, I needed to install the Amazon SSM Agent and was surprised to find that GPG (GNU Privacy Guard) was the only way to verify the download.

We've been using journald-to-cwl to ship journal logs from EC2 instances to CloudWatch Logs.

While debugging memory bloat in a Go application recently, I found that removing the GOMEMLIMIT soft memory limit and disabling transparent huge pages partially mitigated the issue.

It's a beautiful day, and it started with a simple code review: 1# tools/foo/main.go 2- fmt.Println("found it") 3+ log.Println("found it") The author explained the advantages of using a logging library over plain printf.

We started migrating from Amazon Linux 2 (AL2) to Amazon Linux 2023 (AL2023) a month ago.

Our Go programs recently triggered an alarm due to excessive panics.

As Amazon Linux 2 (AL2) approaches its End of Life on June 30, 2025, we have started migrating our container platform from AL2 to Bottlerocket.

I've been building a service for a month, and the day finally arrived when I had the artifact - an EC2 AMI.

Given an array of numbers A, find out whether it contains a 1-3-2 pattern.

Tony Hoare invented QuickSort in 1961.

I often find myself confused by "upstream" and "downstream", in the context of software development.

Modern CPUs operate significantly faster than memory.

Given a non-decreasing array and a target value, we can find the target in logarithmic time using binary search.

Problem Given an integer array A, in one step, remove all elements A[i] where A[i-1] > A[i].

Problem In a connected network consisting of N nodes, each node is connected to either one or two neighbors, forming a line topology.

Problem An event consists of multiple properties, each defined as a key-value pair, where the key is a string and the value is of a primitive type such as numbers or strings.

ClickHouse is a popular OLAP database.