Mauricio Poppe notes

Mapping the Stack Performance work on LLM inference is challenging because every layer offers reasonable optimizations, but not every optimization is worth pursuing.

There is no single best configuration for an LLM inference server.

Benchmarking production infrastructure is difficult.

In production systems, latency degradation rarely happens linearly.

Computing systems are characterized by three core metrics: Latency, Throughput, and Resource Utilization.

The Cold Dinner I became a software engineer because, deep down, I love building.

My home server runs on an Orange Pi 5 .

My Goals I want to connect with people better - Whenever I meet new people, I try to guess where they’re from.

My goals I want to connect with people better - Whenever I meet new people, I try to guess where they’re from.

My terminal is my main workspace, it is where I live for most of the day, writing code, running agents, and managing servers.

I play the guitar and sing in open mics in my spare time.

Learning to Dance I had my first dancing experience in San Francisco in 2018.

Why?

Presentations CSI Windows PV/PVC controller Debugging K8s e2e tests with delve Playground As I started to contribute to Kubernetes, I created a few environments for easier development.

Getting Used to Multitasking Multitasking is a skill that I try to get better at to better use my time.

This interview preparation plan contains notes that apply to any company and specifics for my target companies.

Calculate with exponents.

Machine Learning is great for: Problems for which existing solutions require a lot of fine-tuning or long lists of rules: One Machine Learning algorithm can often simplify code and perform better than the traditional approach.

iterations, batch, batch size and epoch A batch is the set of examples used in one iteration.

From https://colab.research.google.com/github/google/eng-edu/blob/master/ml/cc/exercises/linear_regression_with_synthetic_data.ipynb Most machine learning problems require a lot of hyperparameter tuning.

Count min sketch Problem: given a stream of data with keys and values, how can we get the sum of all values for a given key?

K-means clustering Suppose we have a dataset {\textbf{x}_1, \ldots, \textbf{x}_n} consisting of $N$ observations of a random $D$-dimensional space.

Introduction A Bayesian network is a directed graph in which each node is annotated with quantitative probability information.

Kafka https://www.slideshare.net/mumrah/kafka-talk-tri-hug Key Choices pub/sub messaging pattern Messages are persistent (stored on disk).

Memtable A data structure that holds data in memory before it’s flushed to disk.

Engine Features Consistent hashing Replication factor: replicas of the data across the cluster.

With replication Copies of each partition are stored in multiple nodes Partitioning strategies Unfair partitioning may lead to hotspots (e.g., nodes with more data than others).

Reliability & Availability A system should be resilient (fault-tolerant) and performant under expected load.

My objective is to write a Promises/A+ conformant implementation similar to then/promise .

Let $a,b \in \mathbb{Z}$.

Shading is the process of altering the color of a surface.

A first-person camera captures the virtual scene from the viewpoint of a character.

In two dimensions, complex numbers rotate vectors effortlessly through Euler’s formula $e^{i\theta} = \cos\theta + i\sin\theta$.

Stages preprocessing - Text substitution, stripping comments, and file inclusion. g++ -E main.cpp -o main.i compilation - Compilation of the processed source code into assembly language. g++ -S main.i -o main.s # or g++ -S main.cpp -o main.s assembler -…

make is a tool to simplify building executables from many sources. make will only re-build things that need to be re-built.

Executable: $ cmake [options] (<path-to-source> | <path-to-existing-build>) Assuming that the directory contains: . ├── CMakeLists.txt └── main.cpp #include <iostream> using namespace std; int main () { cout << "Hello world" << endl; return 0; } The

If you have spare time Watch some CppCon ‘Back to Basics’ videos: Back to Basics: The Structure of a Program - Bob Steagall - CppCon 2020 Back to Basics: RAII and the Rule of Zero - Arthur O’Dwyer - CppCon 2019 Back to Basics: Concurrency - Arthur O’Dwyer…

Before rasterization, every primitive must be tested against the view volume.

Imagine a vector space where two points $P$ and $P’$ exist.

A vector space is a set whose elements are called “vectors” (denoted as $\v{v}$ or $\mathbf{v}$), which have two operations defined on them: addition of vectors and multiplication of a scalar by a vector.

In an affine space , there’s the concept of affine combination, which states that any point in space can be represented as an affine combination in the form $$ a + \sum_{i \in I} \lambda_i \mathbf{aa_i} \quad \quad \text{if $\sum_{i \in I} \lambda_i = 1$}…

Line-line intersection Given two lines in 3D, defined as rays: $$ r_1(t_1) = \mathbf{p_1} + t_1 \mathbf{d_1} \\ r_2(t_2) = \mathbf{p_2} + t_2 \mathbf{d_2} $$ Where $t_1, t_2 \in \mathbb{R}$, the two lines intersect if: $$ \mathbf{p_1} + t_1 \mathbf{d_1} =…

After the projection step, every point of the scene sits in normalized device coordinates, the cube $[-1, 1]^3$.

A normal vector to a curve at a particular point is a vector perpendicular to the tangent vector of the curve at that point (also called a gradient).

Given a square matrix $\mathbf{M}$: An eigenvector $\mathbf{v}$ is a non-zero vector whose direction doesn’t change when multiplied by $\mathbf{M}$.

In Euclidean geometry, two lines are said to be parallel if they lie in the same plane and never meet.

A ray tracer emits a ray from each pixel toward the scene to determine the color of the pixel.

An image can be abstracted as a function: $$ I(x,y): R \rightarrow V $$ Where $R \in \mathbb{R}^2$ is a rectangular area and $V$ is a set with the possible pixel values.

The canonical view volume is a cube with corners at $[-1, -1, -1]$ and $[1, 1, 1]$.

The view transform maps every point in the scene from world space to view space: $$ \mathbf{v}_{\text{view}} = \mathbf{M}_{\text{view}} \mathbf{v}_{\text{wld}} $$ In view space the camera sits at the origin, so the transform is fully defined by three para…