Why Patch Size Changes What Attention-Based Forecasters Can Learn

Grouping observations changes the model

Attention-based time-series forecasters often divide an observed history into non-overlapping patches. A history containing L observations becomes L/p tokens when each patch contains p observations. Longer patches mean fewer tokens, reducing attention-mixing costs, although total computation also includes projections and other model components.

This regrouping does not itself discard observations. The important question is what the model can do with the rearranged data. Chen and colleagues show that patch length changes how forecasting weights are shared, creating a tradeoff between expressing the desired prediction and estimating its coefficients from limited data.

Each head supplies a reusable pattern

In the analyzed architecture, each attention head applies the same linear weighting pattern within every patch. Attention scores determine how strongly that pattern contributes at different patch positions. The forecast’s coefficients therefore form a matrix with one row per patch and one column per position inside a patch.

With m heads, this matrix has rank at most m: it is assembled from at most m separable patterns. When attention weights are input-independent, positional keys are freely learnable and the specified linear maps are unrestricted, the model can represent every coefficient matrix within that rank limit.

These are deliberately structured assumptions. The main analysis uses unnormalized attention scores, which can be signed, rather than standard softmax probabilities. The pointwise rank bound also holds for input-dependent weights, but it does not establish the same representational restriction for arbitrary nonlinear forecasting functions or full Transformers.

A distant forecast can still require several filters

The paper separates two tasks in a linear state-space model: extracting a hidden state from noisy observations, and propagating that state toward the forecast horizon.

One slowly evolving state component may dominate a distant forecast. Nevertheless, recovering it from the observed history can require several filtering patterns. The authors derive an exact factorization that separates horizon-dependent weights from horizon-independent filtering patterns.

Under the stated spectral conditions, using too few heads can therefore leave a positive relative approximation error even at long horizons. This is not a permanent absolute error floor: in the stable systems considered, the forecast coefficients shrink, so absolute approximation error can vanish while relative error remains positive.

Patch length also affects whether filtering patterns remain distinguishable across patches. For example, modes with values φ and −φ have identical across-patch factors when p is even. The paper calls this patch aliasing.

Why balanced patches can help

For an active rank constraint, the coefficient class has dimension m(L/p + p − m). With history length and head count fixed, this dimension is smallest when patch length and patch count are balanced, placing p near the square root of L among admissible divisors.

The two extremes illustrate why rank alone is misleading. With one observation per patch, the coefficient matrix has one column. With the entire history in one patch, it has one row. Either matrix is automatically rank one, yet both permit all L forecasting coefficients to vary freely. Neither provides statistical compression in this model.

Intermediate patches reduce the class dimension through shared patterns, but can introduce approximation error. The proposed comparison balances that error against a dimension penalty divided by the number of training windows. It is an oracle comparison of prediction-error bounds, not a directly executable selector: approximation errors and constants may be unknown.

The finite-sample theorem assumes a stationary Gaussian state-space model and a global rank-constrained least-squares solution. It accounts for overlapping windows and temporal dependence through its conditions and constants; it does not guarantee that neural-network optimization finds that solution.

Simulations support a range, not a rule

The main experiments use a three-state system, histories of 64 or 144 observations, and one-head fixed attention. Training sets contain 5, 25 or 200 independent series, with 50 data repetitions. Unlike the theorem’s sample count, these counts refer to series, each supplying overlapping windows.

For 144-observation histories and five training series, mean one-step test MSE was 0.1941 at patch length 12, compared with 0.3207 for point tokens and 0.3284 for a single whole-history patch. The advantage narrowed with more training data. Nearby interior patches remained competitive, and optimization and regularization could also affect the fitted curves.

Supplementary experiments tested nine history lengths across four system designs. The square-root patch was within 5% of the best mean risk in 137 of 216 conditions. However, changing both transition dynamics and the observation mapping produced much shorter preferred patches at longer horizons, breaking the square-root relationship.

A practical reference point

The authors recommend comparing patch lengths and head counts using common validation splits and comparable tuning budgets. For a single trajectory, chronological splits need gaps preventing overlap between training and validation windows and targets. Balanced patches are a useful baseline when representation errors are similar—not a substitute for validation or measuring the implemented model’s computational cost.