文献解读-PhysicsInformedDeepLearning(PINN)

2024年1月25日发(作者：)

As an example, let us consider the Burgers’ equation. In one space dimension, the Burger’s equation along with Dirichletboundary conditions reads asut+uux−(0.01/π)uxx=0, x∈[−1,1], t∈[0,1],u(0,x)=−sin(πx),u(t,−1)=u(t,1)= us define

f(t,x) to be given byf:=ut+uux−(0.01/π)uxx,and proceed by approximating

u(t,x) by a deep neural network. To highlight the simplicity in implementing this idea let usinclude a Python code snippet using Tensorflow. To this end,

u(t,x) can be simply defined asdef u(t, x): u = neural_net(([t,x],1), weights, biases) return uCorrespondingly, the physics informed neural network

f(t,x) takes the formdef f(t, x): u = u(t, x) u_t = nts(u, t)[0] u_x = nts(u, x)[0] u_xx = nts(u_x, x)[0] f = u_t + u*u_x - (0.01/)*u_xx return fThe shared parameters between the neural networks

u(t,x) and

f(t,x) can be learned by minimizing the mean squarederror lossMSE=MSEu+MSEf,where1∑iiMSEu=Nui=1∣u(tu,xu)−ui∣2,Nuand1∑iiMSEf=Nfi=1∣f(tf,xf)∣,

{tu,xu,ui}i=1 denote the initial and boundary training data on

u(t,x) and

{tf,xf}i=1 specify the collocations pointsfor

f(t,x). The loss

MSEu corresponds to the initial and boundary data while

MSEf enforces the structure imposed bythe Burgers’ equation at a finite set of collocation Nf

The following figure summarizes our results for the data-driven solution of the Burgers’ s’ equation: Top: Predicted solution along with the initial and boundary training data. In addition we are using10,000 collocation points generated using a Latin Hypercube Sampling strategy. Bottom: Comparison of the predictedand exact solutions corresponding to the three temporal snapshots depicted by the white vertical lines in the top training took approximately 60 seconds on a single NVIDIA Titan X GPU e (Shrödinger Equation)

This example aims to highlight the ability of our method to handle periodic boundary conditions, complex-valued solutions,as well as different types of nonlinearities in the governing partial differential equations. The nonlinear Schrödinger equationalong with periodic boundary conditions is given byiht+0.5hxx+∣h∣2h=0, x∈[−5,5], t∈[0,π/2],h(0,x)=2 sech(x),h(t,−5)=h(t,5),hx(t,−5)=hx(t,5),where

h(t,x) is the complex-valued solution. Let us define

f(t,x) to be given byf:=iht+0.5hxx+∣h∣2h,and proceed by placing a complex-valued neural network prior on

h(t,x). In fact, if

u denotes the real part of

h and

v is theu(t,x)imaginary part, we are placing a multi-out neural network prior on

h(t,x)=[v(t,x)]. This will result in the complex-valued(multi-output) physic informed neural network

f(t,x). The shared parameters of the neural networks

h(t,x) and

f(t,x) canbe learned by minimizing the mean squared error lossMSE=MSE0+MSEb+MSEf,where1∑iiMSE0=N0i=1∣h(0,x0)−h0∣2,N0and1∑ii2fNffi=1MSE=∣f(t,xf)∣.NfHere,

{x0,h0}i=1 denotes the initial data,

{tb}i=1 corresponds to the collocation points on the boundary, and

{tf,xf}i=1represents the collocation points on

f(t,x). Consequently,

MSE0 corresponds to the loss on the initial data,

MSEbenforces the periodic boundary conditions, and

MSEf penalizes the Schrödinger equation not being satisfied on thecollocation 0iNbiiNf

The following figure summarizes the results of our ödinger equation: Top: Predicted solution along with the initial and boundary training data. In addition we are using20,000 collocation points generated using a Latin Hypercube Sampling strategy. Bottom: Comparison of the predictedand exact solutions corresponding to the three temporal snapshots depicted by the dashed vertical lines in the potential limitation of the continuous time neural network models considered so far, stems from the need to use a largenumber of collocation points

Nf in order to enforce physics informed constraints in the entire spatio-temporal gh this poses no significant issues for problems in one or two spatial dimensions, it may introduce a severebottleneck in higher dimensional problems, as the total number of collocation points needed to globally enforce a physicsinformed constrain (i.e., in our case a partial differential equation) will increase exponentially. In the next section, we putforth a different approach that circumvents the need for collocation points by introducing a more structured neural networkrepresentation leveraging the classical Runge-Kutta time-stepping te Time Models

Let us employ the general form of Runge-Kutta methods with

q stages and obtainj=1aijN[un+cj], i=1,…,q,un+ci=un−Δt∑qun+1=un−Δt∑j=1bjN[un+cj].qHere,

un+c(x)=u(tn+cjΔt,x) for

j=1,…,q. This general form encapsulates both implicit and explicit time-steppingjschemes, depending on the choice of the parameters

{aij,bj,cj}. The above equations can be equivalently expressed asun=ui, i=1,…,q,nun=uq+1,nwhereqnn+ciui:=u+Δt∑j=1aijN[un+cj], iqnuq+1:=un+1+Δt∑j=1bjN[un+cj].=1,…,q,We proceed by placing a multi-output neural network prior on[un+c1(x),…,un+cq(x),un+1(x)].This prior assumption along with the above equations result in a physics informed neural network that takes

x as an inputand outputs[u1(x),…,uq(x),uq+1(x)].nnn

Example (Allen-Cahn Equation)This example aims to highlight the ability of the proposed discrete time models to handle different types of nonlinearity inthe governing partial differential equation. To this end, let us consider the Allen-Cahn equation along with periodic boundaryconditionsut−0.0001uxx+5u3−5u=0, x∈[−1,1], t∈[0,1],u(0,x)=x2cos(πx),u(t,−1)=u(t,1),ux(t,−1)=ux(t,1).The Allen-Cahn equation is a well-known equation from the area of reaction-diffusion systems. It describes the process ofphase separation in multi-component alloy systems, including order-disorder transitions. For the Allen-Cahn equation, thenonlinear operator is given byN[un+cj]=−0.0001uxx+5(un+cj)−5un+cj,n+cj3and the shared parameters of the neural networks can be learned by minimizing the sum of squared errorsSSE=SSEn+SSEb,whereSSEn=∑∑j=1i=1q+1Nn∣uj(xn,i)−un,i∣2,nandSSEb=∑i∣un+ci(−1)−un+ci(1)∣2+∣un+1(−1)−un+1(1)∣2q=1n+cin+cin+1n+1+∑i=1∣ux(−1)−ux(1)∣2+∣ux(−1)−ux(1)∣,

{xn,i,un,i}i=1 corresponds to the data at time

The following figure summarizes our predictions after the network has been trained using the above loss -Cahn equation: Top: Solution along with the location of the initial training snapshot at t=0.1 and the finalprediction snapshot at t=0.9. Bottom: Initial training data and final prediction at the snapshots depicted by the whitevertical lines in the top -driven Discovery of Nonlinear Partial Differential EquationsIn this second part of our study, we shift our attention to the problem of data-driven discovery of partial differentialequations. To this end, let us consider parametrized and nonlinear partial differential equations of the general formut+N[u;λ]=0, x∈Ω, t∈[0,T],where

u(t,x) denotes the latent (hidden) solution,

N[⋅;λ] is a nonlinear operator parametrized by

λ, and

Ω is a subset of

RD. Now, the problem of data-driven discovery of partial differential equations poses the following question: given a smallset of scattered and potentially noisy observations of the hidden state

u(t,x) of a system, what are the parameters

λ thatbest describe the observed data?In what follows, we will provide an overview of our two main approaches to tackle this problem, namely continuous time anddiscrete time models, as well as a series of results and systematic studies for a diverse collection of benchmarks. In thefirst approach, we will assume availability of scattered and potential noisy measurements across the entire spatio-temporaldomain. In the latter, we will try to infer the unknown parameters

λ from only two data snapshots taken at distinct timeinstants. All data and codes used in this manuscript are publicly available on .Continuous Time Models

We define

f(t,x) to be given byKaTeX parse error: Undefined control sequence: label at position 35: …{N}[u;lambda],label{eq:PDE_RHS}and proceed by approximating

u(t,x) by a deep neural network. This assumption results in a physics informed neuralnetwork

f(t,x). This network can be derived by the calculus on computational graphs: Backpropagation. It is worthhighlighting that the parameters of the differential operator

λ turn into parameters of the physics informed neural network

f(t,x).Example (Navier-Stokes Equation)

Our next example involves a realistic scenario of incompressible fluid flow as described by the ubiquitous Navier-Stokesequations. Navier-Stokes equations describe the physics of many phenomena of scientific and engineering interest. Theymay be used to model the weather, ocean currents, water flow in a pipe and air flow around a wing. The Navier-Stokesequations in their full and simplified forms help with the design of aircraft and cars, the study of blood flow, the design ofpower stations, the analysis of the dispersion of pollutants, and many other applications. Let us consider the Navier-Stokesequations in two dimensions (2D) given explicitly byut+λ1(uux+vuy)=−px+λ2(uxx+uyy),vt+λ1(uvx+vvy)=−py+λ2(vxx+vyy),where

u(t,x,y) denotes the

x-component of the velocity field,

v(t,x,y) the

y-component, and

p(t,x,y) the pressure. Here,

λ=(λ1,λ2) are the unknown parameters. Solutions to the Navier-Stokes equations are searched in the set of divergence-free functions; i.e.,ux+vy= extra equation is the continuity equation for incompressible fluids that describes the conservation of mass of the make the assumption thatu=ψy, v=−ψx,for some latent function

ψ(t,x,y). Under this assumption, the continuity equation will be automatically satisfied. Given noisymeasurements{ti,xi,yi,ui,vi}i=1Nof the velocity field, we are interested in learning the parameters

λ as well as the pressure

p(t,x,y). We define

f(t,x,y) andg(t,x,y) to be given byf:=ut+λ1(uux+vuy)+px−λ2(uxx+uyy),g:=vt+λ1(uvx+vvy)+py−λ2(vxx+vyy),ψ(t,x,y)and proceed by jointly approximating

[p(t,x,y)] using a single neural network with two outputs. This prior assumptionf(t,x,y)results into a physics informed neural network

[g(t,x,y)]. The parameters

λ of the Navier-Stokes operator as well as theψ(t,x,y)f(t,x,y)parameters of the neural networks

[p(t,x,y)] and

[g(t,x,y)] can be trained by minimizing the mean squared error lossMSE:=+1N1N∑i=1(∣u(ti,xi,yi)−ui∣2+∣v(ti,xi,yi)−vi∣2)N∑i=1(∣f(ti,xi,yi)∣2+∣g(ti,xi,yi)∣2).NA summary of our results for this example is presented in the following figures.

Navier-Stokes equation: Top: Incompressible flow and dynamic vortex shedding past a circular cylinder at Re=100. Thespatio-temporal training data correspond to the depicted rectangular region in the cylinder wake. Bottom:Locations of training data-points for the the stream-wise and transverse velocity -Stokes equation: Top: Predicted versus exact instantaneous pressure field at a representative time instant. Bydefinition, the pressure can be recovered up to a constant, hence justifying the different magnitude between the twoplots. This remarkable qualitative agreement highlights the ability of physics-informed neural networks to identify theentire pressure field, despite the fact that no data on the pressure are used during model training. Bottom: Correctpartial differential equation along with the identified one.

Our approach so far assumes availability of scattered data throughout the entire spatio-temporal domain. However, in manycases of practical interest, one may only be able to observe the system at distinct time instants. In the next section, weintroduce a different approach that tackles the data-driven discovery problem using only two data snapshots. We will seehow, by leveraging the classical Runge-Kutta time-stepping schemes, one can construct discrete time physics informedneural networks that can retain high predictive accuracy even when the temporal gap between the data snapshots is te Time ModelsWe begin by employing the general form of Runge-Kutta methods with

q stages and obtainqj=1aijN[un+cj;λ], iu=u−Δt∑qun+1=un−Δt∑j=1bjN[un+cj;λ].n+cin=1,…,q,Here,

un+cj(x)=u(tn+cjΔt,x) for

j=1,…,q. This general form encapsulates both implicit and explicit time-steppingschemes, depending on the choice of the parameters

{aij,bj,cj}. The above equations can be equivalently expressed asun=ui, i=1,…,q,n+1un+1=ui, i=1,…,qnn+ciui:=u+Δt∑j=1aijN[un+cj;λ], i=1,…,q,qn+1ui:=un+ci+Δt∑j=1(aij−bj)N[un+cj;λ], i=1,…, proceed by placing a multi-output neural network prior on[un+c1(x),…,un+cq(x)].This prior assumption result in two physics informed neural networks[u1(x),…,uq(x),uq+1(x)],nnnand[u1n+1(x),…,uqn+1(x),uq+1(x)].n+1Given noisy measurements at two distinct temporal snapshots

{xn,un} and

{xn+1,un+1} of the system at times

tn and

tn+1, respectively, the shared parameters of the neural networks along with the parameters

λ of the differential operatorcan be trained by minimizing the sum of squared errorsSSE=SSEn+SSEn+1,whereSSEn:=∑∑j=1i=1qNn∣uj(xn,i)−un,i∣2,n

andSSEn+1:=∑∑j=1i=1qNn+1n+1∣uj(xn+1,i)−un+1,i∣,

x={xnn,iNn}i=1,un={un,iNn}i=1,

xn+1={xn+1,iNn+1}i=1, and

un+1={un+1,iNn+1}i=e (Korteweg–de Vries Equation)Our final example aims to highlight the ability of the proposed framework to handle governing partial differential equationsinvolving higher order derivatives. Here, we consider a mathematical model of waves on shallow water surfaces; theKorteweg-de Vries (KdV) equation. The KdV equation reads asut+λ1uux+λ2uxxx=0,with

(λ1,λ2) being the unknown parameters. For the KdV equation, the nonlinear operator is given byN[un+cj]=λ1un+cjuxn+cj−λ2uxxxn+cjand the shared parameters of the neural networks along with the parameters

λ=(λ1,λ2)of the KdV equation can be learned by minimizing the sum of squared errors given above.

The results of this experiment are summarized in the following equation: Top: Solution along with the temporal locations of the two training snapshots. Middle: Training data andexact solution corresponding to the two temporal snapshots depicted by the dashed vertical lines in the top : Correct partial differential equation along with the identified sionAlthough a series of promising results was presented, the reader may perhaps agree that this two-part treatise createsmore questions than it answers. In a broader context, and along the way of seeking further understanding of such tools, webelieve that this work advocates a fruitful synergy between machine learning and classical computational physics that hasthe potential to enrich both fields and lead to high-impact ledgementsThis work received support by the DARPA EQUiPS grant N66001-15-2-4055 and the AFOSR grant data and codes are publicly available on on

@article{raissi2019physics,title={Physics-informed neural networks: A deep learning framework for solving forward and inverse problemsinvolving nonlinear partial differential equations},author={Raissi, Maziar and Perdikaris, Paris and Karniadakis, George E},journal={Journal of Computational Physics},volume={378},pages={686–707},year={2019},publisher={Elsevier}}@article{raissi2017physicsI,title={Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations},author={Raissi, Maziar and Perdikaris, Paris and Karniadakis, George Em},journal={arXiv preprint arXiv:1711.10561},year={2017}}@article{raissi2017physicsII,title={Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations},author={Raissi, Maziar and Perdikaris, Paris and Karniadakis, George Em},journal={arXiv preprint arXiv:1711.10566},year={2017}}