Puyuan Peng

1.

1.

In October 2024, I contacted 9 companies for research positions in AI.

[文] Kevin Gimpel [译] Jason 写在前面 Kevin Gimpel是TTIC的助理教授。

Please checkout Professor Sergey Levine’s excellent tutorial: Levine 18’

In this section, we study how to learn policies utilize the known (learned) dynamics.

Previous lecture is mainly about how to plan actions to take when the dynamics is known.

Let’s recall the reinforcement learning goal — we want to maximaze the expected reward (or expected discounted reward in the infinite horizon case) \[\begin{equation} \mathbb{E}_{\tau\sim p(\tau)}\sum_{t=1}^T r(s_t, a_t) \end{equation}\] where \[\begin{eq…

At the end of previous lecture, we talked about the issues with Q-learning, one of them is that it’s not directly optimizing the expected return and it can take a long time before the return starts to improve.

In this section we extend the online Q-iteration algorithm in the previous lecture by identifying the potential issues and introducing solutions.