Project Portfolio

A Heat Physics-Informed Neural Network

 

By Vladimir Williams

10/09/2026

 

OpenAI recently claimed to have solved the Navier-Stokes existence and smoothness problem, but can an AI model truly understand physics? Short answer: No.

 

AI models can, however, help provide solutions to

  • problems where the underlying differential equations are known, i.e., forward problems,
  • inverse problems, where unknown parameters can be inferred from sparse observations,
  • data-scarce problems where a purely data-driven AI model would overfit, and
  • problems on complex geometries where traditional numerical methods struggle.

 

I've trained a simple physics-informed neural network (PINN) to solve the two-dimensional heat equation using PyTorch. The heat equation encapsulates the dynamics of heat diffusion on a medium, such as ice or copper, by relating the change in time of the function u(x, t)—describing the temperature—to how it spreads out (or converges) in space:

where α represents the thermal diffusivity of the particular material—it's about 1.02 mm²/s for ice at 0°C. In 2D, this is simply

The neural network ansatz relies on three inputs (x, y, and t) to produce a predicted function û(x, y, t):

 

 

I used a 4-layer deep neural network (DNN) with 32 nodes per layer. In order to train the weights of the DNN to reproduce the analytical solution u(x, y, t), we start by taking a random sample of collocation points (xk, yk, tk) over the domain and then forward propagate them through the DNN to produce a first prediction û(0)(xk, yk, tk; θ(0)), where θ(0) is the initial set of parameters (weights and biases) before any updates. We define the loss function based on the residual

calculated at collocation point k during the nth epoch, as

that is, the squared residuals averaged over all the collocation points k. The same is done for the initial and boundary conditions. Now that we have the loss in hand, the gradients are computed through backpropagation—telling us how a small change in each parameter affects the loss function—and finally we can update the parameters to minimize the loss. Each epoch consists of

  • forward propagation,
  • loss calculation,
  • backpropagation, and finally,
  • parameter optimization.

 

MLOps and Transfer Learning

 

Now let's say we want to change the conditions of the physical system being modeled; this is where MLOps and transfer learning come in. There's a few scenarios we can look at:

  1. Change the initial or boundary conditions,
  2. Change the thermal diffusivity α,
  3. Model a completely different physical phenomenon.

In the first case, we have to save and load the original model into a separate routine specifying the new ICs and BCs to fine-tune the loaded model on them—transfering the new conditions to the original model. Similarly, in the second case, we would load our original model, modify the α and continue training. To address the final case, we can reuse our training pipeline and adjust the specifics, like the loss and residual functions, to train a new model—unless the model architecture is general enough.

 

 

 

Follow the whole project on GitHub!