Can Reinforcement Learning Help Small Models Translate the Law?

Legal translation asks a machine to do more than produce fluent sentences. It must preserve specialized terminology, careful distinctions and the force of a legal provision across languages. Switzerland offers a demanding setting: its legal system operates across several official languages, and the study examines translations involving German, French, Italian and English. The paper asks whether models can improve at this task by being trained to reason before translating—and, in particular, whether reinforcement learning offers an advantage over supervised fine-tuning.

The researchers tested three small language models: Qwen3.5 in 4-billion- and 9-billion-parameter versions, and Gemma 3 with 12 billion parameters. They compared three broad strategies: prompting larger reasoning models; fine-tuning smaller models on translation examples, either with or without intermediate reasoning steps; and using reinforcement learning (RL) to optimize translation output.

For the reasoning-based fine-tuning data, the team used DeepSeek-R1 to generate intermediate steps and translations for examples from an existing Swiss legal translation dataset. They filtered those examples using ChrF, a measure of character-level overlap with a reference translation. In the RL setup, a model received rewards for using the requested output format and for producing a translation with a higher ChrF score. In plain terms, the training process sampled possible answers and favored those that scored better under these rules.

The evaluation drew on a test set of Swiss legal translation pairs, with Romansh excluded because the models under study did not support it. The authors assessed translations using four automatic measures: ChrF, METEOR, COMET and MetricX. These measures capture different aspects of similarity or quality, but none is a direct substitute for a lawyer’s judgment. The team also had a legal expert assess a sample of 240 translations, considering accuracy, fluency and preservation of legal meaning.

Reinforcement learning led the small models’ results. Across the tested model families, RL outperformed both forms of supervised fine-tuning on the reported overall metrics. The best small-model result came from RL-trained Gemma 3 12B. Its COMET score was 83.08; the best tested frontier model, OpenAI o3, scored 85.92 on that metric. The comparison suggests the adapted model came relatively close on this benchmark, but did not match the strongest frontier result.

Adding written reasoning steps to supervised fine-tuning did not reliably help. In the reported comparisons, Reasoning SFT performed worse than Simple SFT, and for the smallest model it also performed worse than the unadapted base model. The authors suggest that generated reasoning traces may be formulaic or add little beyond capabilities already present in a pretrained model. The experiment does not establish why the method underperformed, however. It also illustrates an important distinction: training a model to produce text that looks like reasoning is not proof that its internal process is sound or that the final translation is legally correct.

The gains from RL were larger for smaller models and more modest for the 12-billion-parameter model. That pattern is consistent with the paper’s claim of diminishing returns as model size increases, but it is a result within this particular set of models and training choices, not a general law about language models. The study also reports that COMET had the strongest correlation with the expert’s scores among the automatic measures tested. That correlation was limited: the paper reports a Pearson correlation of 0.24 for COMET. The human assessment was small relative to the full test set, so the automatic scores should not be treated as a comprehensive measure of legal accuracy.

There are further limits to the conclusions. The experiments cover Swiss legal translation and a selected group of models, rather than legal systems or language pairs worldwide. The RL reward was deliberately simple and relied on ChrF, so it may reward resemblance to a reference without fully capturing whether a translation preserves every legally important nuance. The authors themselves note that more sophisticated rewards and a wider range of models merit study. They also do not provide measured training and inference costs for the adapted small models, so claims about their practical cost advantage remain uncertain.

The paper reports that code and models are publicly available. It is presented in workshop proceedings, according to the paper’s publication note; as with any study, its results should be read in light of its methods and scope rather than as proof that automated legal translation is ready for unsupervised use. The central finding is narrower and useful: in this evaluation, reinforcement learning with verifiable rewards improved small models more reliably than supervised fine-tuning, while frontier models retained the top overall scores.