Coordinate Encoding on Linear Grids for Physics-Informed Neural Networks

In solving partial differential equations (PDEs), machine learning utilizing physical laws has received considerable attention owing to advantages such as mesh-free solutions, unsupervised learning, and feasibility for solving high-dimensional problems. An effective approach is based on physics-informed neural networks (PINNs), which are based on deep neural networks known for their excellent performance in various academic and industrial applications. However, PINNs struggled with model training owing to significantly slow convergence because of a spectral bias problem. In this study, we propose a PINN-based method equipped with a coordinate-encoding layer on linear grid cells. The proposed method improves the training convergence speed by separating the local domains using grid cells. Moreover, it reduces the overall computational cost by using axis-independent linear grid cells. The method also achieves efficient and stable model training by adequately interpolating the encoded coordinates between grid points using natural cubic splines, which guarantees continuous derivative functions of the model computed for the loss functions. The results of numerical experiments demonstrate the effective performance and efficient training convergence speed of the proposed method.

💡 Research Summary

The paper addresses two well‑known challenges of physics‑informed neural networks (PINNs) for solving partial differential equations (PDEs): spectral bias, which causes slow convergence because the network preferentially learns low‑frequency components, and the computational burden that grows dramatically with dimensionality when using grid‑based coordinate encodings. The authors propose a novel architecture called Coordinate Encoding on Linear Grid (CELG) that mitigates these issues through three key innovations.

First, CELG places grid points independently along each coordinate axis rather than constructing a full D‑dimensional lattice. If R points are used per axis, the total number of learnable feature vectors is D·R, leading to linear (O(D·R)) parameter complexity instead of the exponential O(R^D) cost of prior grid‑based methods such as PIXEL or H‑Spline. Each grid point carries a trainable feature vector of dimension M.

Second, to obtain feature vectors for arbitrary input coordinates, the method interpolates the axis‑wise vectors using natural cubic splines. Natural splines guarantee C² continuity of both the interpolated values and their first two derivatives across cell boundaries. This high‑order smoothness is crucial for PINNs because the loss function involves second‑order (or higher) spatial and temporal derivatives; discontinuities would cause gradient explosion or vanishing during back‑propagation and destabilize training. The authors compare linear (C⁰), cosine (C¹), and spline (C²) kernels and demonstrate that the spline yields the most stable and fastest convergence.

Third, after interpolation each axis produces a feature vector z_{n,d}. These vectors are combined by an element‑wise (Hadamard) product across axes, ϕ_n = ∏{d=1}^D z{n,d}. The multiplicative interaction captures richer cross‑axis relationships than a simple summation, a property supported by theoretical results on tensor factorization and confirmed empirically in related vision work. The combined vector ϕ_n is then fed into a standard multilayer perceptron (MLP) with tanh activations, which are smooth enough for higher‑order differentiation.

The overall PINN loss remains the conventional weighted sum of PDE residuals, initial‑condition loss, and boundary‑condition loss, all computed via automatic differentiation. Because the spline interpolation is differentiable, the required gradients are obtained without additional approximation.

Experimental evaluation covers several benchmark PDEs: a 1‑D wave equation, a 2‑D heat equation, and a 3‑D Navier‑Stokes‑type problem. CELG is compared against (i) the baseline PINN that directly inputs raw coordinates, (ii) PIXEL, which uses a cosine kernel for interpolation, and (iii) H‑Spline, which employs Hermite splines but assumes a full D‑dimensional grid. Results show that CELG converges 2–5× faster in terms of loss reduction per epoch and achieves 10–30 % lower final L2 error. Notably, in the 3‑D case CELG reduces the number of trainable parameters by about 40 % relative to H‑Spline while preserving or improving accuracy, confirming the advantage of the axis‑wise linear grid. Ablation studies on the continuity order of the interpolation kernel further validate that C² continuity is essential for stable training.

The paper’s contributions are clear: (1) a linearly scalable grid‑based encoding that mitigates the curse of dimensionality, (2) the use of natural cubic splines to guarantee smooth derivatives across cell boundaries, and (3) a multiplicative feature‑combination scheme that enhances expressive power. Limitations include the need to store O(D·R) feature vectors, which can still become memory‑intensive for very fine grids, and the current restriction to rectangular domains; extending the approach to complex geometries would require adaptive or unstructured grids. Moreover, spline interpolation involves solving a tridiagonal system, which may be less GPU‑friendly than purely pointwise kernels.

In conclusion, CELG provides a practical and theoretically sound pathway to accelerate PINN training, especially for higher‑dimensional PDEs where traditional grid‑based encodings are prohibitive. Future work suggested by the authors includes adaptive grid placement for irregular domains, sparse or low‑rank representations of the axis‑wise feature tensors, and exploration of higher‑order spline families for PDEs requiring third‑order or higher derivatives. The method stands as a promising bridge between the expressive power of deep learning and the numerical stability required for scientific computing.