Choosing Less-Repetitive Training Data Could Make AI Fine-Tuning More Efficient

A smaller set, chosen together

A language model being adapted to a particular task may not need every available training example. The harder question is which examples to keep—and whether individually promising examples become repetitive when combined.

In the November 2, 2026 preprint Query-based Submodular Data Selection for Efficient Instruction Tuning, Zhenghao Zhao and colleagues propose a method called QSGS: Query-based Submodular functions with Gradient-guided Synthesis. Their experiments suggest that explicitly discouraging redundancy can improve targeted instruction tuning when the selected dataset is very small. These are results reported in a preprint, not an established guarantee of equivalent performance with less data.

The method starts with a large training pool and a small set of demonstrations representing the desired capability. The authors argue that ranking candidates independently can fill a limited training budget with examples that teach substantially overlapping lessons.

Measuring relevance and repetition

QSGS represents examples using gradients: signals describing how training on an example would change the model’s adjustable parameters. Examples producing similar update directions are treated as potentially redundant. Alignment with gradients from the target demonstrations supplies an estimate of task relevance—not a direct measurement of eventual learning benefit.

The researchers first perform warmup training on a random subset of the available data. They use LoRA, which trains small added parameter sets while keeping the pretrained weights fixed, and compress the resulting gradient representations through random projections.

Selection then balances two quantities: relevance to the target demonstrations and similarity among the examples already selected. A candidate can be relevant yet become less attractive if it closely resembles previous choices in gradient space.

The underlying objective is submodular, meaning that an example’s marginal contribution diminishes as the selected set grows. However, adding an example can also reduce the objective’s value because of the redundancy penalty. The authors explicitly state that the theoretical approximation guarantee discussed for randomized algorithms does not apply to their practical deterministic greedy procedure.

Generating examples for remaining gaps

After selection, QSGS estimates which parts of the target’s average gradient remain outside the space represented by the chosen examples. The warmup model generates candidate instruction–response pairs using the target demonstrations, and the system selects candidates whose gradients align with those remaining gaps.

In the reported setup, it generates 100 candidates per under-covered subtask and adds up to 10 synthetic examples overall. This is selective supplementation, not unrestricted synthetic-data expansion.

An important qualification appears in the appendix: gradient alignment measures a complementary optimization signal, not semantic correctness. The authors identify an explicit semantic or correctness filter for generated candidates as future work. In their synthesis ablation, including all generated candidates performs worse than adding none.

What the experiments report

The main training pool contains approximately 270,000 examples from four instruction datasets. A 0.5% selection budget corresponds to roughly 1,350 examples before synthetic additions. For Llama-3, the authors instead extend the training mixture to 17 datasets, so the experimental setup is not identical across all models.

Evaluation covers MMLU’s knowledge questions, BBH’s reasoning tasks and TydiQA’s multilingual question answering, using Llama-2-7B, Mistral-7B and Llama-3-8B. Benchmark-provided few-shot examples guide selection and also serve as demonstrations during evaluation.

QSGS exceeds LESS, a gradient-based ranking baseline, in all nine model–benchmark combinations in the main table. The table reports standard deviations over three seeds for selected-subset results.

The clearest near-full-data result is on Mistral-7B. QSGS scores 60.4 on MMLU, matching full-data training; 57.5 on TydiQA, compared with 57.7; and 53.2 on BBH, compared with 53.0. Thus, despite broader wording elsewhere in the paper, the table shows a match, a small shortfall and a small improvement—not superiority on every benchmark. Llama-2 and Llama-3 remain below their full-data baselines throughout.

On Llama-2 MMLU, QSGS’s advantage over LESS grows from 0.9 percentage points at a 5% budget to 2.7 points at 0.5%. A component ablation attributes most improvement to diversity-aware selection, with synthesis providing a smaller additional gain.

Efficiency has an upfront cost

The small final dataset does not mean the entire process uses only 0.5% of the data or computation. Default warmup training uses 5% of the pool for four epochs.

The appendix reports approximately 304 GPU-hours for warmup, 48 for gradient construction, less than one for selection and optional synthesis, and 32 for final subset training. The authors argue that stored gradient features make repeated selections inexpensive, allowing preparation costs to be spread across multiple target-specific runs.

The evidence remains limited to three benchmarks and 7B–8B models. Results depend on useful target demonstrations and sufficiently reliable gradient estimates; very small warmup sets hurt performance. The practical promise is therefore narrower than replacing large training datasets universally: better-balanced subsets may preserve more task-specific performance when final fine-tuning budgets are tight.