主题
必读论文
本页按主题列出强化学习的里程碑论文，每篇都附有原文链接。建议配合站内对应教程阅读：先理解算法思想，再回到原文看细节与实验。其中 8 篇已有逐篇的中文深度解读（见文末“中文导读”）。
基于价值的方法
- Mnih et al. (2015). Human-level control through deep reinforcement learning. Nature. doi:10.1038/nature14236 —— DQN
- van Hasselt, Guez, Silver (2016). Deep Reinforcement Learning with Double Q-learning. AAAI. arXiv:1509.06461
- Wang et al. (2016). Dueling Network Architectures for Deep Reinforcement Learning. ICML. arXiv:1511.06581
- Schaul et al. (2016). Prioritized Experience Replay. ICLR. arXiv:1511.05952
- Hessel et al. (2018). Rainbow: Combining Improvements in Deep Reinforcement Learning. AAAI. arXiv:1710.02298
策略梯度与 Actor-Critic
- Mnih et al. (2016). Asynchronous Methods for Deep Reinforcement Learning. ICML. arXiv:1602.01783 —— A3C
- Schulman et al. (2015). Trust Region Policy Optimization. ICML. arXiv:1502.05477
- Schulman et al. (2016). High-Dimensional Continuous Control Using Generalized Advantage Estimation. ICLR. arXiv:1506.02438 —— GAE
- Schulman et al. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347
连续控制
- Lillicrap et al. (2016). Continuous control with deep reinforcement learning. ICLR. arXiv:1509.02971 —— DDPG
- Fujimoto, van Hoof, Meger (2018). Addressing Function Approximation Error in Actor-Critic Methods. ICML. arXiv:1802.09477 —— TD3
- Haarnoja et al. (2018). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. ICML. arXiv:1801.01290
博弈与规划
- Silver et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature. doi:10.1038/nature16961 —— AlphaGo
- Schrittwieser et al. (2020). Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. Nature. arXiv:1911.08265 —— MuZero
离线 RL
- Kumar et al. (2020). Conservative Q-Learning for Offline Reinforcement Learning. NeurIPS. arXiv:2006.04779 —— CQL
大模型与 RL
- Ouyang et al. (2022). Training language models to follow instructions with human feedback. NeurIPS. arXiv:2203.02155 —— InstructGPT / RLHF
- Rafailov et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS. arXiv:2305.18290 —— DPO
- Shao et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 —— GRPO
中文导读
本站教程已经覆盖了上面大部分论文的核心思想，可以作为中文导读使用：
| 论文 | 对应教程 |
|---|---|
| DQN、Double DQN、Dueling、PER、Rainbow | DQN、DQN 改进 |
| A3C、GAE | Actor-Critic |
| TRPO、PPO | TRPO、PPO |
| DDPG、TD3、SAC | DDPG、TD3、SAC |
| MuZero | 基于模型的强化学习 |
| CQL | 离线强化学习 |
| InstructGPT、DPO、GRPO | RLHF 与大模型中的 RL |