Bounded Rationality

For the past five years, one of the most enjoyable things I've done outside of my day job is teaching.

For a while now, I've really enjoyed diving deep to understand probability and related fundamentals (see here, here, and here).

After being busy for the first part of the year, I finally have a bit of time to work on this blog.

One interesting thing I often think about is the gap between academic and real-world solutions.

Unless you've been living under a rock, you've probably heard of large language models (LLM) such as ChatGPT or Bard.

After a long digression, I'm finally back to one of the main lines of research that I wanted to write about.

Through a couple of different avenues I wandered, yet again, down a rabbit hole leading to the topic of this post.

This post has been a long time coming.

Here's a topic I thought that I would never get around to learning because it was "too hard".

A lot of modern machine learning is related to this idea of "compression", or maybe to use a fancier term "representations".