Lil'Log

The concept of recursive self-improvement (RSI) dates back to I.

Scaling laws are one of the most critical empirical findings in deep learning.

Special thanks to John Schulman for a lot of super valuable feedback and direct edits on this post.

Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task.

Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content.

Diffusion models have demonstrated strong results on image synthesis in past years.

[Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “Vox populi”) and nice feedback. 🙏 ] High-quality data is the fuel for modern data deep learning model training.

The use of large language models in the real world has strongly accelerated by the launch of ChatGPT.

Building agents with LLM (large language model) as its core controller is a cool concept.

Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights.

Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago.

[Updated on 2023-01-24: add a small section on Distillation.]

Neural networks are well known to be over-parameterized and can often easily fit data with near-zero training loss with decent generalization performance on test dataset.

Processing images to generate text, such as image captioning and visual question-answering, has been studied for years.

Here comes the Part 3 on learning with not enough data (Previous: Part 1 and Part 2).

This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks.

When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.

[Updated on 2022-03-13: add expert choice routing.]

[Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)].

The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart.

Large pretrained language models are trained over a sizable collection of online data.

[Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.]

[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta).

Although most popular and successful model architectures are designed by human experts, it doesn’t mean we have explored the entire network architecture space and settled down with the best option.

[Updated on 2020-06-17: Add “exploration via disagreement” in the “Forward Dynamics” section.

[Updated on 2023-01-27: After almost three years, I did a big refactoring update of this post to incorporate a bunch of new Transformer models since 2020.

[Updated on 2020-02-03: mentioning PCG in the “Task-Specific Curriculum” section.

[Updated on 2020-01-09: add a new section on Contrastive Predictive Coding].

Stochastic gradient descent is a universal choice for optimizing deep learning models.

In my earlier post on meta-learning, the problem is mainly defined in the context of few-shot classification.

In Robotics, one of the hardest problems is how to make your model transfer to the real world.

[Updated on 2019-05-27: add the section on Lottery Ticket Hypothesis.]

[Updated on 2019-02-14: add ULMFiT and GPT-2.]

In Part 3, we have reviewed models in the R-CNN family.

[Updated on 2019-10-01: thanks to Tianhao, we have this post translated in Chinese!]

So far, I’ve written about two types of generative models, GAN and VAE.

[Updated on 2019-07-18: add a section on VQ-VAE & VQ-VAE-2.]

[Updated on 2018-10-28: Add Pointer Network and the link to my implementation of Transformer.]

The full implementation is available in lilianweng/deep-reinforcement-learning-gym In the previous two posts, I have introduced the algorithms of many deep reinforcement learning models.

[Updated on 2018-06-30: add two new policy gradient methods, SAC and D4PG.]

[Updated on 2020-09-03: Updated the algorithm of SARSA and Q-learning so that the difference is more pronounced.

The algorithms are implemented for Bernoulli bandit in lilianweng/multi-armed-bandit.

[Updated on 2018-12-20: Remove YOLO here.

Part 1 of the “Object Detection for Dummies” series introduced: (1) the concept of image gradient vector and how HOG algorithm summarizes the information across all the gradient vectors in one image; (2) how the image segmentation algorithm works to detect

I’ve never worked in the field of computer vision and has no idea how the magic could work when an autonomous car is configured to tell apart a stop sign from a pedestrian in a red hat.

Human vocabulary comes in free text.

Professor Naftali Tishby passed away in 2021.

[Updated on 2018-09-30: thanks to Yoonju, we have this post translated in Korean!]

The machine learning models have started penetrating into critical areas like health care, justice systems, and financial industry.

In the Part 2 tutorial, I would like to continue the topic on stock price prediction and to endow the recurrent neural network that I have built in Part 1 with the capability of responding to multiple stocks.