Luke Salamone's Blog

TLDR: I trained an LLM reasoning agent to play chess, with model sizes 0.8B, 2B and 9B.

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up discusses an approach for countering what the authors call the Matthew Effect, the tendency for the LLM to improve much more on easy problems that it is already good than harder problems th…

This project is a continuation of my previous autoresearch project, which optimized a reranking model to be under 10MB.

In this project, I wanted to set up what I thought was a simple experiment: if we let a neural network play a fighting game, will it learn any interesting emergent behaviors?

There are a lot of meanings of the term “model” in machine learning and machine learning-adjacent fields.

Leaving the autoresearch loop going, the LLM was able to make 7.8% progress on the distillation task.

TLDR: I extracted fens and stockfish evaluations for 3.9 billion chess positions.

The other day I found Alan Pike’s blog post from a few years ago which describes his iterative process for determining the order cities should be closed in the game Two Spies.

When I built the Gigafish dataset, I deliberately chose to use positions evaluated at depth 10.

In reinforcement learning, an “agent” is an entity which can observe the world and take actions.

Let’s say that you are selling N widgets and you need to determine a price for your widgets.

A neural network is a type of function originally inspired by how the brain works.

I recently built a small multi-agent simulation inspired by Rick and Morty.

DeepSeek R1 is a large language model which employs test-time compute to generate a response.

More than 30 earths could fit between the earth and the moon.

If your Dataset class looks something like class MyDataset(Dataset): # ... boilerplate ...

In God we trust.

Rust is super fast.

I spent some time considering words to describe a chess position with many legal moves.

TLDR: A sparse autoencoder is just a regular autoencoder that encourages sparsity with an L1 penalty or KL divergence loss rather than using a low-dimensional bottleneck.

Suppose we have a vector database with a billion items in it (the haystack).

Embeddings, or vector representations of a document (which could be a piece of text, image, sound, etc.), can be extremely useful for making sense of large datasets.

K-means clustering (previous discussion) is an unsupervised learning algorithm which assigns points to one of K different clusters based on the distance of that point to a centroid.

Paper link: https://arxiv.org/pdf/2008.13535 Learning to rank is an important problem in many machine-learning products such as search, recommendation, and advertising.

What is a blunder in chess?

Conway’s Game of Life is a simulation developed in 1970 describing a grid of binary cells and transition rules for each cell which depend on the state of the cell’s neighbors.

TLDR: No.

This is a summary of Automated Antenna Design with Evolutionary Algorithms, a 2006 paper by Hornby et al.

In Defense of Dual-Encoders for Neural Ranking by Menon et. al. discusses the question of why dual-encoder (DE) models, also called Bi-Encoders elsewhere, don’t match the performance of cross-attention (CA) models.

This is a follow up to Opponent Modeling Wins 2× Faster Than Stockfish.

White areas show points of earth on land whose antipode is also on land.

In this post we will walk through a very simple example of multiheaded attention is computed.

A self-attention block depicted as a neural network.

I’ve found myself hitting ctrl+f on this blog enough that I figured it’s about time to add some search functionality to it.

This is part 2 of a paper I wrote for Ken Forbus’ Qualitative Reasoning course, adapted for this blog.

This is part 1 of a paper I wrote for Ken Forbus’ Qualitative Reasoning course, adapted for this blog.

TLDR: Using opponent modeling we can win 2x faster than Stockfish by playing high risk, high reward moves that Stockfish will avoid.

Batch Norm Normalizes the mean and variance for all instances of a feature in a batch.

TLDR: Alphabet chess is a chess variant that allows handicapping by mixing in a bit of poker into the beginning of the game.

Selected {subject, relation, object} tuples generated by COMET Paper link: https://arxiv.org/abs/1906.05317 This paper describes COMET, a method of generating knowledge bases automatically.

First, create a custom dataset class. from torch.utils.data import Dataset, DataLoader class CustomDataset(Dataset): def __init__(self, features, labels): assert len(features) == len(labels) self.features = features self.labels = labels…

If you want to extract a tar archive tar -xf archive.tar.gz If you want to compress a directory tar -czvf archive.tar.gz /path/to/directory That’s all.

Defending Against Neural Fake News by Zellers et al. presents a model for controllable text generation called Grover.

Here’s how to connect to a remote Jupyter notebook.

In machine learning and statistics, marginalization simply means summing over a set of independent variables.

Here’s how to connect your Google Colab notebook to your Drive directory: from google.colab import drive drive.mount('/content/gdrive') Follow the prompts from there.

There are quite a few BERT vs GPT-2 breakdowns online, mostly focusing on the architectural differences between the two models.

Let’s explore how GPT-2 tokenizes text.

TLDR: Attention masks allow us to send a batch into the transformer even when the examples in the batch have varying lengths.

Convolutional neural networks have had breakthrough success in image recognition, natural language processing, and even board games like Chess and Go.