TRANSMISSION // 2611.23154v1
Can Different AI Policies Improve Pass@5?
Source published
A preprint tests whether dividing a language model’s response budget among five jointly trained policies can improve pass@5. Across four reasoning tasks, the approach outperformed a single-policy baseline, but evidence is limited to these experiments.

When an AI system generates several answers to one problem, should they all come from the same policy—the model’s answer-generating behavior—or should several related policies each contribute one? A preprint explores the second option, reporting higher pass@5 scores on four reasoning tasks while leaving open how well the approach generalizes.
Large language models can spend extra computation at inference time by generating several candidate answers and assessing whether at least one is correct. This measure, called pass@k, is the chance that at least one of k responses succeeds. The paper’s motivation is that repeatedly sampling from one policy could miss useful alternatives or produce overlapping approaches. This is a rationale for the method, not a universal finding about single-policy sampling.
Mridul Mahajan, Xuezhou Zhang and Aldo Pacchiano investigate whether a fixed response budget can instead be divided among several learned policies. They call this stratified policies: with five responses, five policies each produce one answer. For the language-model experiments, the policies use separate low-rank adaptation modules, or LoRA adapters, on a shared, frozen model backbone. The arithmetic experiment instead uses five separate small TinyGPT networks.
The idea is that policies might become good at different problems or approaches, making their combined answers more useful. The researchers train the population jointly against the utility of the answer set, focusing on pass@k: a set counts as successful if at least one response is correct.
The paper also gives a theoretical argument for a simplified linear setting. If different regions of the input space require different linear mappings, separate adapters can specialize to those regions. Under the stated conditions, a population can achieve zero best-of-k prediction error even when any single linear map has positive error. This result concerns the mathematical setup; the authors say it does not establish guarantees for nonlinear language models or learned policy populations.
What the experiments found
The evaluation covered four tasks: constructing valid arithmetic expressions from digits, Knights-and-Knaves logic puzzles, 4×4 Sudoku at its hardest tested setting, and procedurally generated GSM-Symbolic math word problems. The Sudoku setting starts with only four of the grid’s 16 cells filled. The main comparison used five responses in each method: one from each of five jointly trained policies, or five from a single trained policy. Results were averaged over three random seeds.
The stratified method had higher pass@5 than the single-policy baseline on all four tasks. On Sudoku, for example, pass@5 was 80.3% for the stratified population and 74.2% for the single-policy baseline. The size of the difference varied across tasks. The single policy performed better on pass@1—accuracy for one response—on most tasks. The paper reports this trade-off; it does not establish a single explanation for it.
The authors’ analyses suggest that policies contributed complementary solutions. On Sudoku, 6.0% of examples were solved by all five adapters, according to the paper’s analysis. The team also compared stratified sampling with drawing all five answers at random from a mixture of the same adapters; stratification performed better on each task. A separate comparison increased the single adapter’s rank fivefold to match the population’s total trainable parameters. This single-adapter variant still performed below stratification on the reported Knights-and-Knaves and Sudoku comparisons. Those results do not establish the same comparison for all four tasks, or show that stratification beats every alternative design.
What remains uncertain
This is an arXiv preprint, not a peer-reviewed publication. Its experiments cover four benchmark tasks, particular model families and specific training and evaluation choices; the results may not carry over to other settings. The paper focuses on pass@k. Majority voting and verifier-based selection are proposed as directions for future study, not demonstrated benefits here.
The experiments also use a fixed population of five policies, each contributing one response. They do not establish how to select population size or allocate responses to individual prompts. Overall, the results suggest one way to organize inference-time sampling, not a general rule that multiple policies are always better. Broader tests are needed to determine whether the gains hold across tasks, models and methods for choosing a final answer.
- large language models
- inference-time scaling
- LoRA
- pass@k
- reasoning
Reviewed cross-references
These connections were reviewed by a model, not independently fact-checked by a human. Similarity does not imply agreement; consult the original papers and the evidence below.
When Is One Sampling Temperature Enough?
Supporting evidence: Both papers report that varying the source of samples can improve pass@K over a single fixed choice: Mahajan et al. report pass@5 gains from sampling once from each learned policy across reasoning tasks, while Orly et al. report gains from multi-temperature allocation over single-temperature baselines, typically modest when temperature is tuned to the budget. The interventions and evaluation settings differ, so this supports only the bounded point that distributing samples across alternatives can help; it is not a direct comparison of the methods.
Model review: · openai-codex/gpt-6-luna
Evidence and provenance
Selected excerpts, not complete papers. PDF extraction can omit figures or damage equations, tables and reading order; sources may be preprints.
2611.23154v1 — Original paper
Excerpt reference:
static/papers/2611.23154v1.md:67-139
Basis: Paper text
Source SHA-256:a946b7f1f501d3d36f78a571952ae58a302525a86467f6d76c3b5624546ce6aa2611.13077v1 — Original paper
Excerpt reference:
static/papers/2611.13077v1.md:13-76
Basis: Paper text
Source SHA-256:d5969bb248273970c2899aa3a47b09f5105854da3cc019cbb98b25a4af0874032611.13077v1 — Original paper
Excerpt reference:
static/papers/2611.13077v1.md:875-909
Basis: Paper text
Source SHA-256:d5969bb248273970c2899aa3a47b09f5105854da3cc019cbb98b25a4af087403
Related reading
Automatically matched by shared topic keywords. These links do not imply that the sources agree or establish a consensus.
Train at rank 64, send less: a new approach to AI updates
Source published
Shared keywords: large language models
Choosing each next move can improve AI agents—but uses far more tokens
Source published
Shared keywords: inference time scaling
A context-sensitive way to steer language models
Source published
Shared keywords: large language models
Teaching Enterprise AI to Choose the Right Tool
Source published
Shared keywords: large language models
SplitLite Shrinks the Data Chatter of Training Language Models
Source published
Shared keywords: large language models