DEEPO Tackles Hallucination in Multimodal LLMs with Dual-Entropy RL

## New Approach to Hallucination Control A recent arXiv pre‑print (arXiv:2609.28570v1) presents Dual‑Entropy Enhanced Policy Optimization (DEEPO), a reinforcement‑learning framework aimed at reducing hallucinations in multimodal large language models (MLLMs).

## Identified Weak Points The authors pinpoint two vulnerabilities in the standard RL correction chain. First, *hard queries*—inputs with high semantic entropy—often generate uniformly incorrect sample groups during rollouts, nullifying the advantage signal precisely where hallucination risk peaks. Second, at the optimization stage, tokens that are confident yet wrong become *gradient‑invisible*: as a categorical policy sharpens, the expected score‑gradient norm approaches zero, delivering minimal updates to the most erroneous predictions.

## DEEPO’s Dual‑Stage Solution DEEPO addresses these issues through a two‑phase enhancement. It combines signal variance regularization to preserve informative gradients during rollout with a complementary gradient‑preserving mechanism (the abstract truncates before naming the second component). This dual strategy seeks to maintain a robust advantage signal and ensure that confident‑but‑incorrect tokens receive meaningful parameter updates.

## Implications for Future Research By targeting both rollout entropy and gradient attenuation, DEEPO offers a systematic way to improve reasoning fidelity in MLLMs. The paper suggests that integrating dual‑entropy considerations could become a standard practice for RL‑based fine‑tuning of multimodal models.

*Source: arXiv cs.AI (https://arxiv.org/abs/2609.28570), published 2026‑09‑25.*