Nilesh's Blog

GPU snapshotting reduces inference cold starts by restoring an initialized process instead of loading weights, initializing CUDA, and rebuilding runtime state on every startup.

I've been running background agents overnight, queuing up a list of tasks before bed, and waking up to pull requests in the morning.

Today's AI coding agents can generate impressive volumes of code in a single pass, spinning up entire modules, scaffolding full applications, and writing hundreds of lines in one go.

With the rapid scaling of AI deployments, efficiently storing and distributing model weights across distributed infrastructure has become a critical bottleneck.

Large Language Model (LLM) inference workloads deal with extremely large model files (often many gigabytes) that must be loaded quickly and repeatedly across distributed GPU instances.

I began exploring vibecoding for both my personal projects and Inferless, and I experienced firsthand how it enhanced both.

Drawing on general ideas popularized by Nick Bostrom’s Superintelligence, as well as broader AI discussions—and an outline of how such an intelligence might come about and “take off.”

Insignificance of Problems: The book opens with Arthur Dent desperately trying to stop a bulldozer from demolishing his house.

If I had to sum up 2024 in two words, they'd be "adventure" and "change."