EmoReward adjusts how video AI is rewarded for emotional expression

The problem with rewarding a feeling

A convincing emotional scene is not necessarily the one with the biggest smile or the most movement. Restrained sadness may depend on a small gesture, while facial-expression measurements are useful only when the relevant evidence is available. The EmoReward preprint, posted on November 4, 2026, asks how a training system should decide which signals to trust.

The authors argue that fixed combinations of visual-quality, motion and prompt-alignment scores can miss emotional intent or encourage scoring shortcuts. Their proposal does not establish an objective definition of emotion. Instead, it adjusts the relative influence of existing evaluators during reinforcement-learning training.

Seven signals, with conditions attached

EmoReward first converts a prompt into an “EmoScript”: targets for emotional pleasantness and activation, facial muscle movements, body cues and the time window in which an expression should occur. It then evaluates a generated clip using seven providers covering semantic judgments, facial cues, body cues, expression timing and motion naturalness. Color and movement-intensity measurements are additional observations, not members of the seven-provider memory calibration.

Crucially, a provider’s score is separate from whether its evidence is usable. A missing facial measurement can receive a neutral placeholder without contributing to the reward. Rule-based gates also account for training progress, face evidence, disagreement between semantic providers and prompts calling for subtle expression.

The remaining contributions are renormalized into a weighted average. This matters because reducing every provider equally would largely cancel out under normalization: the mechanism must change their relative influence or exclude unsuitable evidence.

What “self-evolving” means here

The second component, RewardMemory, operates between completed training runs—not as a continuously learning evaluator inside one run.

A reference run records provider scores, gates, context and outcomes. RewardMemory groups these records by target activation and a face-branch activity proxy, using a global fallback when a group has insufficient support. It estimates how strongly each provider’s raw scores correlate with VideoAlign outcomes, then converts those correlations into bounded weight multipliers.

Those multipliers stay fixed throughout the subsequent run, as do the reward-model parameters. The video generator still receives reinforcement-learning updates, and current-video gates can exclude evidence even when memory favors its provider.

The reported calibration bank contains 3,232 traces. Globally, the alignment-seeded semantic provider and motion provider receive the largest multipliers. Three channels—facial action-unit matching, body cues and expression timing—are constant in the analyzed reference traces. Their neutral readings do not establish that the videos lack visible faces or bodies.

What the video experiments show

The main optimizer comparison uses HunyuanVideo 1.5 and 302 emotional prompts that do not overlap with training. The authors report batch size two, four candidates per prompt and equal weighting of general VideoAlign reward and calibrated emotion reward.

There is an important configuration distinction: DanceGRPO uses reflective gating without RewardMemory, while FlowGRPO and SAGE-GRPO use the complete system. All three guided configurations improve the four reported VideoAlign measures relative to their respective optimizer baselines.

For SAGE-GRPO, mean VideoAlign Overall increases from −0.8269 to −0.2624, a gain of 0.5645. FlowGRPO gains 0.2343, and DanceGRPO with reflection gains 0.3218. PickScore and CLIPScore changes are smaller and mixed, so these results are not uniform improvements across evaluation methods.

VideoAlign also supplies the training objective and the outcome used to calibrate memory. Moreover, one provider incorporates VideoAlign information. Consequently, agreement with that evaluator is not independent evidence of better emotional expression.

Human ratings offer a separate, limited check

Eighteen participants rated outputs from the pretrained generator, SAGE-GRPO and SAGE-GRPO with EmoReward on 30 prompts. Each method–criterion combination contains 540 judgments using Bad, Normal or Good categories.

EmoReward’s Good share for emotional fidelity is 60.0%, compared with 53.0% for SAGE-GRPO. Its Good shares for temporal coherence and visual quality are also higher, while its Bad shares are lower than both comparison methods on all three criteria.

These are descriptive rating distributions, not a demonstrated statistically significant preference. The reported aggregates do not document recruitment, blinding or presentation order. Participant- and prompt-level records are needed to estimate uncertainty while accounting for repeated judgments.

A bounded result, not endless improvement

Image experiments on Stable Diffusion 3.5 and FLUX show objective-dependent trade-offs, with unequal Stable Diffusion training budgets limiting attribution. The paper also lacks a matched global-memory control to isolate the benefit of context-specific calibration.

Most importantly, the experiments assess a reference-to-calibrated transition, not sustained improvement over many generations. The supported contribution is narrower than autonomous, perpetual self-improvement: completed training experience can inform how a subsequent run allocates trust among frozen reward providers.