Kimi K1.5: Advancing Reinforcement Learning with Large Language Models

Introduction

The continuous advancement of artificial intelligence (AI) has been largely driven by improvements in large language models (LLMs) through next-token prediction. However, this approach is fundamentally constrained by the availability of high-quality training data. Kimi K1.5 presents a novel approach by integrating reinforcement learning (RL) into LLM training, enabling models to dynamically explore and generate training data based on reward-driven optimization. Unlike previous RL-based LLM frameworks, Kimi K1.5 achieves state-of-the-art reasoning capabilities across multiple domains without relying on complex techniques such as Monte Carlo tree search, value functions, or process reward models.

Core Innovations in Kimi K1.5

1. Long Context Scaling

A key breakthrough in Kimi K1.5 is the extension of RL training to longer context lengths, scaling up to 128k tokens. This increase in context size significantly improves reasoning ability by allowing models to retain and process more information over extended sequences. The model employs partial rollouts, reusing sections of previous trajectories to improve training efficiency, thereby reducing computational overhead and enabling longer CoT (Chain of…