Stretching coordinates helps neural networks learn sharp solution changes

Changing the coordinates, not the physics

Physics-informed neural networks, or PINNs, approximate solutions to differential equations by penalizing violations of the equations and their boundary or initial conditions. Sharp changes in a solution can make this training difficult. The paper connects that difficulty partly to spectral bias: neural networks tend to learn lower-frequency features more readily than rapidly varying ones.

Tianchi Yu and Ivan Oseledets, affiliated with the Artificial Intelligence Research Institute and Skolkovo Institute of Science and Technology in Moscow, propose inserting a coordinate transformation before the solution network. Their study was submitted to arXiv on November 6, 2026. It is a preprint, not peer reviewed; the manuscript states that it has been submitted to the SIAM Journal on Scientific Computing.

Their approach, called Coordinate Transformation PINNs, aims to spread rapid solution changes over a larger portion of a computational domain. The physical solution is not altered. Instead, the network represents it using coordinates in which its gradients can be more uniform.

Two ways to stretch a solution

The idea draws on stretching methods used in numerical grid generation. In one dimension, the authors describe a classical result: under specified assumptions, a coordinate transformation whose derivative is proportional to the magnitude of the solution derivative makes the transformed gradient magnitude constant. For the paper’s exponential boundary-layer example, the optimal transformation turns the solution into a straight line.

Knowing that ideal transformation generally requires information about the unknown solution. The researchers therefore develop two constructions.

The explicit construction uses prescribed mathematical functions called majorants, which bound solution gradients. These encode prior knowledge about features such as a thin region of rapid change near a boundary. Their parameters can be fixed or learned alongside the solution network.

The implicit construction instead uses a small additional neural network to learn the transformation. It applies an exponential, multiplicative mapping, with spectral normalization used to control a Lipschitz bound as part of the authors’ invertibility construction.

Both versions retain the original physical-equation loss. Automatic differentiation carries derivatives through the coordinate transformation using the chain rule, avoiding the need to rewrite each equation manually in computational coordinates.

What the benchmarks show

The main accuracy measure is relative L2 error: the size of the prediction error divided by the size of the reference solution. Smaller values are better.

For a one-dimensional boundary-layer problem with a small parameter of one millionth, the reported standard PINN error exceeds one. An adaptive exponential transformation reduces it to 0.00174. This is not a universal success across transformation choices: some alternatives still perform poorly on the hardest cases.

On a two-dimensional Poisson benchmark with a concentrated solution peak, the adaptive exponential construction achieves an error of 0.0274 at the largest parameter tested in the comparison table. The standard PINN records 0.0814, while the two adaptive-sampling comparators record 0.285 and 0.0773.

The learned transformation also complements more elaborate architectures. For the Korteweg–De Vries benchmark, adding it to PirateNet with advanced training techniques lowers error from 0.000427 to 0.000223. In a three-dimensional Navier–Stokes benchmark, adding it to SPINN lowers error from 0.00333 to 0.00147.

For a constructed 200-dimensional Sine–Gordon problem, error falls from 0.0152 with QRPINN to 0.00966 with the added transformation. That result concerns the specified benchmark, not arbitrary high-dimensional equations.

Limits behind the gains

The theoretical results are narrower than the experimental range. The convergence-rate analysis concerns a particular one-dimensional boundary-layer problem in an infinite-width, frozen-kernel setting, with additional conditions on the collocation points and transformation parameter. Results showing that training approaches the optimal transformation also require successful training and further assumptions. They do not guarantee that general learned transformations will converge optimally.

Numerical stability matters too. The authors report overflow when computing second derivatives with the theoretically optimal exponential transformation at the smallest boundary-layer parameter. A less extreme learned transformation can therefore be more useful computationally than the theoretical optimum.

The improvement also costs time. On a single NVIDIA A40 GPU, the 200-dimensional example takes 15,700 seconds with the transformation, versus 11,800 seconds without it.

Explicit transformations require prior knowledge, while the authors acknowledge normalization and theoretical-guarantee limitations for implicit constructions. They also do not compare the boundary-layer experiments against specialized methods for those problems. The evidence supports coordinate stretching as a useful addition on the tested benchmarks, rather than a general replacement for existing solvers.