Alex's Blog

I write this from the veterinary hospital.

DataLoader vs Dataset: * Dataset: stores samples + labels * DataLoader: like an iterable Dataset, provides easy access to samples.

I took the notes in a meeting today1.

Scientists think they’ve found evidence to add a new law of thermodynamics.

Paper compares 4 intracranial neuroelectrophysiology “data repositories” (read: data formats): - DABI: Data Archive for the BRAIN Initiative - DANDI: Distributed Archives for Neurophysiological Data Integration - OpenNeuro - Brain CODE These two efforts a…

I’ve just following some instructions1 to get Obsidian set up on my mobile and — get this — synced through git (!!!).

Notes from the OpenXLA talk: JAX: Low-level control with shard_map and Pallas.

AKA: “General and Scalable Parallelization for ML Computation Graphs” References https://arxiv.org/abs/2105.04663 https://youtu.be/5ilr4gcenaA?

Embedded in the ML modeling process is a huge search problem: References S.

Common things I take away from Saul’s talk: * OSS Python could substantially benefit from frontier programming language (PL) research * Talks about the birth of NumPy in 1995 * NumPy is “pointers, strides, dtypes” between Python and C arrays. * History of…

The future of PyTorch is to become a more complicated, more technical-debt ridden Jax.

A product of the hazy-research-and-flash-attention lab is ThunderKittens.

References https://hazyresearch.stanford.edu/blog/2024-05-12-tk https://github.com/HazyResearch/ThunderKittens Community worth joining: https://discord.gg/gpumode

Another classic joke about Google’s engineering culture: Everything is either deprecated or not ready yet. – Googlers to Nooglers during orientation.

There’s an old joke about Google’s engineering culture that goes like this: Come work at Google!

The Hazy Research Lab, creators of Flash Attention, combine both systems software engineering expertise with ML algorithms development.

Originally sketched on September 29, 2023.

In your career as a technologist, whenever possible, invent on principle.

References Marcel Duchamp and the Refusal of Work

Just because a system or process becomes more efficient, doesn’t mean it consumes less energy.

All productivity, as measured by GDP, leads to hydrocarbon consumption.

anndata i.e.

metamorphic-testing can be used for Machine Learning, since ML systems are really hard to test.

Property based testing is difficult because it’s hard to find good properties.

References https://blog.lamin.ai/arrayloader-benchmarks

A lattice is complete if all subsets have both a join and a meet.

Apparently, this term doesn’t mean much.

A semilattice is like a directed lattice in either the meet (LCM) or join (GCD) direction (“for any nonempty finite subset”).

It’s the eve of the start of my new job.

It’s the best advice for creativity I have yet received.

My wife and I absolutely love our wedding video.

Breaking: Elon Musk has just reinstated Donald Trump’s Twitter account. — No Lie with Brian Tyler Cohen (@NoLieWithBTC) November 20, 2022 I’m reading Cal Newport’s Digital Minimalism right now.

WIP?

I see why people say that their wedding is the best day of their life.

I got married a few weeks ago, and the strangest thing happened the Friday before my big day.

created: 2021-02 updated: 2021-08 While building a Beam pipeline for work, I found myself surprised to find there was no documentation on how to structure the pipeline as a CLI 1.

Prototypes and thinking about electric origami I was reading hacker news aimlessly on the couch, as one does when home alone on a Sunday afternoon, when I came across this post.

When I first started looking for a ring, I was looking at how much bang for my buck I could get in diamond sizes… which was totally the wrong way to go about it.

Me just after I’ve proposed I like surprises, but Camille doesn’t.

The student org I used to be a part of (the Cognitive Science Student Association) asked me to lead a workshop on machine learning at their annual conference.

A lattice is a partially ordered set, or poset.

A new meditation: Let ice melt on your tongue.

A week ago (late 2020), I started a new habit: I now try to find 15 minutes of boredom a day.

Ideally, I don’t want my website to be just a blog.

In 2016, the year I graduated from college, my partner, Camille, gave me a Passion Planner.

Tip: For better work-life balance, consider sending emails to your self.

My partner’s food intuition – her ability to weave haphazard ingredients into a coherent, delicious meal – is nothing short of magic.

I make a smoothie for myself every morning.

Well, here’s a very meta story about this website and how I’ve grown since I started it.

I had a conversation a few weeks ago with my project manager, Bernhard Seefeld, about strategies to combat writer’s block.