https://brianbartoldson.wordpress.com/feed/

KL regularization against a reference policy can stabilize and improve RL training of LLMs, but computing the KL term requires an additional forward pass through the reference model.

Andrej Karpathy wrote a great post last year on how to train a neural network to play the Atari game Pong by using the Policy Gradients reinforcement learning (RL) algorithm.

My research group has been discussing Artificial Neuron-Glia Networks lately.