CoCo turns textual context into numerical guidance for time-series forecasts

Forecasting models can learn patterns in past prices, traffic volumes or economic indicators, but relevant information may also arrive as words. A report can describe an anticipated disruption that has not yet appeared in the numerical record. Researchers at the University of Texas at Austin propose a way to connect those two sources without making a language model the sole forecaster.

Their framework, Context as Covariates, or CoCo, is described in TSFM Meets LLM: Context as Covariates by Hemanth Ram, Sai Shankar Narasimhan, Shubhankar Agarwal and Sandeep Chinchali. The first arXiv version was submitted on November 3, 2026. The supplied manuscript carries a NeurIPS 2026 conference designation; the arXiv landing page does not separately specify an acceptance or review status.

Translating words into numerical signals

CoCo uses a language model to read numerical history alongside recent textual annotations. Instead of passing text embeddings directly into the forecasting backbone, it produces two numerical input channels, called covariates.

The first carries a text-aware forecast. Its historical portion is constructed from slightly noisy next-step observations, giving the numerical model an example of how that channel relates to the target series. Its future portion contains the language model’s median forecast across multiple sampled predictions.

The second channel represents confidence through disagreement between those predictions. At each future step, CoCo measures their variance and scales it by the variance of the historical series. Lower disagreement indicates greater confidence. The intent is to teach the forecasting backbone when to lean on the language model’s guidance and when to rely more on numerical history.

The principal implementation pairs Qwen3-4B-Instruct-2507 with Chronos2, a time-series foundation model. Training happens sequentially. First, the language model is adapted using Group Relative Policy Optimization, with rewards tied to whether its forecast covariate improves a fixed Chronos2 model’s accuracy. Chronos2 is then adapted using forecast and confidence covariates generated by that trained language model.

This distinction matters for the paper’s zero-shot claims: the system undergoes training, but is evaluated on held-out datasets without dataset-specific fine-tuning.

Synthetic context supplies training data

The researchers also address a data shortage. Large collections pairing numerical series with useful, time-varying text are difficult to obtain across many domains.

They use a larger Qwen3 model to generate plausible annotations for real numerical series, producing roughly 12.4 million synthetically annotated rows. Training also includes real financial annotations. The generated descriptions combine observations, plausible explanations and speculative outlooks.

Importantly, the annotation generator can see future numerical values within its input window. Although its instructions prohibit explicitly revealing those values, the resulting outlooks are future-conditioned. This is a training-data construction technique, not evidence that such information would be available during deployment. Evaluation instead uses held-out series with real-world textual sources, including news-derived summaries.

The authors provide a theoretical explanation for synthetic-to-real transfer, but it is conditional. It assumes real and synthetic text encode similar forecast-relevant information and that small representation changes have bounded effects on forecasting loss. The appendix explicitly acknowledges that its perturbation experiments support local smoothness, rather than establishing the stronger uniform condition used in the theory.

Aggregate gains, not universal wins

The main comparison covers 30 datasets and 95 evaluation settings, including finance, climate, macroeconomics, security and traffic. Standard settings forecast 16 steps from histories of 64, 128 or 256 observations, with additional native TimeMMD settings.

The authors report an overall normalized mean absolute error of 0.719 for CoCo, compared with 0.745 for pretrained Chronos2, 0.735 for TiRex1 and 0.836 for GPT-4o. These errors are measured after historical-window normalization, not in dollars or other original units. CoCo beats Chronos2 in 81 of 95 settings and TiRex1 in 60, illustrating that its aggregate lead does not mean it wins every comparison.

Confidence cannot establish truth

The clearest limitation is misleading context. A language model can produce tightly clustered predictions that are consistently wrong. The authors find that confidently incorrect covariates can mislead Chronos2, which cannot fully correct them. An independent text-reliability assessment is proposed as future work, not demonstrated as an existing safeguard.

There is also a speed cost. Under the reported single-H100, batched benchmark, CoCo takes about 280 milliseconds per sample, versus about 11 milliseconds for Chronos2. Finally, when text supplies nearly all the predictive information, the numerical backbone contributes less. CoCo’s strongest case is therefore complementary evidence: useful textual clues combined with an informative numerical history.