PINNeval asks physics-informed AI benchmarks to test the whole training pipeline

Why the surrounding pipeline matters

Physics-informed neural networks, or PINNs, approximate solutions to differential equations by training a network to reduce violations of governing equations and initial or boundary conditions. Their performance depends not just on what the network can represent, but also on how it is trained.

Kevin von Bargen, Hamid Ebrahimy and Tim Römer of Osnabrück University argue that many evaluations overlook this dependence. A method may be tested inside a basic training setup, even though stronger choices are already available for the other parts of the pipeline. The resulting ranking may not carry over to a more competitive configuration.

Their paper, “PINNeval: A Comprehensive Evaluation Standard for Physics-Informed Neural Networks,” was posted to arXiv on November 5, 2026. The supplied record identifies this arXiv version as accepted at NeurIPS 2026 in the Track on Evaluations and Datasets. It proposes an evaluation standard, rather than documenting an already universally adopted one.

Four components, one connected system

PINNeval divides training into four components:

• Domain-importance adaptation: where the equations are checked and how strongly individual locations contribute to training.
• Loss balancing: how the contributions from different equations and conditions are weighted against one another.
• Network architecture: the mathematical form of the neural approximation.
• Optimization scheme: the algorithms and schedules used to update network parameters.

The distinction between the first two matters. Directing more attention toward difficult locations is not the same as balancing an equation against a boundary condition. The authors argue that treating these as competing alternatives can obscure the benefits or drawbacks of combining them.

Under the proposed protocol, researchers comparing one component should use competitive choices for the others and document the entire configuration. “Competitive” is tied to evidence about accuracy and computational cost on comparable problems, rather than to a permanently fixed reference setup.

The accompanying EvalCard format records architecture, sampling, weighting, optimizer settings and evaluation conditions. Required reporting includes numerical precision, hardware, runtime, iteration count, variation across independent runs and observed failures. Accuracy is measured using relative L2 error, which compares the size of the prediction error with that of a reference solution.

Simple baselines remain difficult to beat

The authors release PINNpipe, a modular JAX-based package containing 18 methods and nine benchmark problems, and use it to demonstrate the protocol.

In the sampling study, a baseline called Random-R draws fresh uniformly distributed points at every iteration. With the surrounding architecture, loss balancing and SOAP optimizer held fixed, it achieved the lowest median error on seven of the nine problems across ten independent seeds.

Langevin Adaptive Sampling achieved lower medians on the wave and heat problems, although its advantage on the wave problem was small. These results do not establish that adaptive sampling is generally ineffective. The methods used their originally proposed hyperparameters, and their earlier successes involved different surrounding pipelines, particularly different optimizers.

The loss-balancing study also produced no universal winner. A GradNorm-based scheme achieved the lowest median error on five problems, while fixed weights led on three and SoftAdapt on one. Some methods showed substantial training failures, reinforcing the value of reporting robustness rather than only a favorable final score.

A smaller network with a different training strategy

Another comparison tested a larger PirateNet trained with SOAP in float32 against a smaller multilayer perceptron trained with SOAP followed by the quasi-Newton optimizer SSBroyden2 in float64. Across five seeds, the smaller two-stage pipeline reported lower median errors and shorter runtimes on all nine benchmarks.

For the Kuramoto–Sivashinsky problem, reported runtime fell from about 1,101 to 70 minutes on an NVIDIA H100, while relative error fell from approximately 0.045 to 0.0073.

That comparison changes architecture, precision and optimization together. Additional experiments separating those factors support the authors’ interpretation that the quasi-Newton stage, rather than higher precision alone or the architecture change, accounts for the accuracy improvement in these tests.

What the results do not establish

The experiments concern forward problems: finding solutions when the equations and conditions are specified. They do not demonstrate the protocol for inverse problems or neural operators, and the suite excludes complex geometries.

The quasi-Newton approach also has a scaling constraint: its dense stored matrix requires memory that grows quadratically with the number of trainable parameters. Its feasibility here depends on using moderate-sized networks.

Reference accuracy imposes another limit. For Rayleigh–Taylor instability, the best reported error is less than twice the estimated reference-solution error, restricting interpretation of small accuracy differences.

The central lesson is therefore conditional, not a declaration of one winning method: useful comparisons must reveal the whole pipeline, its computational cost and the circumstances under which its results hold.