Google DeepMind's collaboration with the LIT AI Lab at Johannes Kepler University Linz has yielded a groundbreaking study on the enhancement of artificial intelligence language models through a novel approach known as Reinforcement Learning Fine-Tuning (RLFT). This research focuses on addressing key issues in decision-making processes by reinforcing the training of thought chains within the models.

The advent of big data has empowered existing language models to process text with exceptional capabilities and even make knowledge-based decisions in interactive environments. However, these models often fall short in practical decision-making, failing to effectively execute correct strategies despite their ability to derive them. They also tend to choose options that yield higher short-term returns and, due to frequency bias, smaller models often repeat common actions.

Traditional reinforcement learning methods, such as the Upper Confidence Bound (UCB) algorithm, balance exploration and exploitation to some extent but fail to completely bridge the gap between model reasoning and action. To address this, DeepMind introduced RLFT, using the model's self-generated thought chains as training signals. The system evaluates action rewards corresponding to each reasoning step, encouraging the model to prioritize logically coherent and effective action plans.

In practice, the model generates sequences that include reasoning processes and actions based on input instructions, historical actions, and rewards. These sequences are optimized through Monte Carlo baseline estimation and generalized advantage estimation. If ineffective actions are taken, a penalty mechanism is triggered. The introduction of reward shaping techniques not only ensures the output's conformity but also retains space for exploration.

Experiments conducted by the research team tested the multi-armed bandit model. In a 10-arm test, the action coverage rate of the 2B parameter model increased by 12 percentage points. In a 20-arm test, although the improvement was smaller, the frequency bias rate dropped from 70% to 35%, demonstrating the effectiveness of the research. Tic-tac-toe experiment results showed that the model's win rate against random opponents increased fivefold, and the average reward against optimal Monte Carlo tree search agents went from -0.95 to zero. Furthermore, the 27B large model's probability of generating correct reasoning reached 87%, compared to only 21% of unoptimized models that could execute optimal actions. This series of data robustly proves the effectiveness of RLFT in narrowing the gap between reasoning and action in AI language models.