A two-stage reasoning system helps AI choose Earth-observation evidence

A satellite archive can contain the ingredients for an answer without making the answer easy to obtain. A question about agricultural drought, for example, concerns an environmental condition rather than a single stored measurement. Connecting that condition to vegetation indices or land-surface temperature requires scientific reasoning. Finding usable observations then requires checking geography, timing and product characteristics.

In the preprint OSGR: Observation-Support Graph Reasoning for Earth Observation Question Answering, Dazhou Yu and colleagues propose separating those problems. Their system first identifies documented relationships between an environmental target and measurable variables. It then searches for observation items that can support those relationships under the question’s spatial and temporal requirements.

A scientific blueprint before the data search

OSGR starts with an offline graph organized around three kinds of concepts: environmental targets, the properties used to characterize them, and observable or computed result variables. Its structure follows the ISO/OGC Observations, Measurements, and Samples framework.

The researchers construct this graph through GPT-4-assisted extraction from Earth-observation studies and product documentation. Relations retain provenance identifying their supporting documents and evidence. Human correction is limited to duplicate checking and merging naming variants; it is not described as a comprehensive scientific audit of every extracted relation.

For a question, the system follows the graph to identify candidate “support paths” connecting the requested phenomenon to variables available in the archive. These paths act as scientific blueprints, narrowing which kinds of observations the system should consider before it searches individual files.

The second stage, called relation-conditioned grounding, selects concrete observations. Different questions activate different compatibility rules: spatial overlap, containment, proximity or directed flow connectivity; and observations from the same period, an earlier supporting period, or before-and-after event windows. Set-level checks address collective coverage, resolution, provenance and execution feasibility.

This distinction matters because relevance alone is insufficient. A measurement can concern the right variable yet come from an unsuitable location or time. Likewise, comparing disaster damage requires appropriately ordered observations, not simply two images of the area.

What the benchmarks show

The main evaluation uses Earth-Bench, comprising 248 expert-curated questions associated with approximately 13,700 Earth-observation images. Tool-using methods share archive metadata and an execution interface. Earth-Agent and OSGR are compared using matched language-model backbones, although OSGR additionally uses its method-specific scientific graph.

With GPT-4o, OSGR reaches 63.7% answer accuracy, compared with 49.3% for Earth-Agent: a gain of 14.4 percentage points. The execution-step ratio falls from 2.9 to 2.5. That ratio compares predicted steps with expert steps; its ideal is one, so the result indicates less redundant execution, not a direct measurement of runtime or financial cost.

OSGR also exceeds the GPT-4o GeoGraphRAG baseline, which records 48.8% answer accuracy. Improvements over Earth-Agent appear with the other matched backbones tested. Efficiency does not improve in every pairing: with GPT-4o-mini, the step ratio rises slightly from 2.0 to 2.1 even as accuracy increases.

A complementary benchmark, DisasterM3-Reason, contains 7,160 multiple-choice instances covering damaged-building counting and damaged-road estimation. Here, GPT-4o accuracy rises from 46.2% with Earth-Agent to 50.1% with OSGR. This smaller improvement is a separate result, not part of the Earth-Bench accuracy figure.

Component-removal experiments support the authors’ two-stage design. On Earth-Bench with GPT-4o, accuracy falls to 43.9% without the support graph and to 53.2% without relation-conditioned grounding, versus 63.7% for the full system.

Better structure is not a guarantee

The paper’s reliability evidence needs careful interpretation. Its temporal and spatial feasibility metrics compare predicted support-relation labels with expert labels. Those scores should not be read as exhaustive verification that every selected file is physically suitable.

The authors also prove a conditional result: their factorized search preserves feasible solutions if the support graph covers every required derivation and grounding implements the original compatibility scoring exactly. They explicitly distinguish these assumptions from guarantees about a practical, incomplete graph or noisy metadata.

Among manually inspected incorrect Earth-Bench predictions, data and backend numerical issues account for 41% of errors, incorrect observation-set retrieval for 27%, temporal mismatches for 19%, and spatial mismatches for 13%.

The preprint therefore presents promising benchmark evidence for a more structured search strategy, not a solved Earth-observation assistant. Missing scientific relations, inaccurate footprints and noisy acquisition times can still undermine answers. Extending the approach beyond Earth observation would require a domain-specific schema and compatibility functions.