PETSc nonlinear solve not producing correct solution for transient problems

[tamaratambyah]

For all transient problems in GridapODEs, the stage operator (linear or non-linear) is recreated at each time step. This causes issues in GridapPETSc when using a non-linear solver as described in Issue#81.

GridapPETSc.jl/src/PETScNonlinearSolvers.jl

Line 161 in 35b678d

@assert cache.op === op

The hack to comment out this line or set cache.op=op does not work for all problems. @JordiManyer and I found the only solution was to set sysslvrcache=nothing prior to solving the NonlinearStageOperator within ode_march!.

This is not a long term solution as destroying sysslvrcache means the residual and jacobian are allocated every time step in

GridapPETSc.jl/src/PETScNonlinearSolvers.jl

Line 168 in 35b678d

cache=_setup_cache(x,nls,op)

An alternative is to use the non-linear solvers within GridapSolvers. Additionally, we tested using a PETSc linear solver within a GridapSolvers nonlinear solver and this recovered the correct behaviour. This isolates the problem to PETSc nonlinearsolvers, in particular:

GridapPETSc.jl/src/PETScNonlinearSolvers.jl

Line 156 in 35b678d

function Algebra.solve!(x::T,

We believe this function should be changed to pass a new jacobian and residual function rather than cached objects for transient problems requiring a non-linear solver. @JordiManyer will try to create a reproducible example for testing.

[oriolcg]

Hi @tamaratambyah and @JordiManyer, I'm facing this issue when using PETSc nonlinear solver for the Navier-Stokes equations. I tried using a linear solver with the GridapSolvers' nonlinear solver and this seems to work well for a few time steps, but after some steps I get a PETSc error (see below). Have you found similar problems? What is the status of the proposed fix for this issue?

[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Petsc has generated inconsistent data
[0]PETSC ERROR: PetscInfoNames array is too small for ksp, need 176 not 173
...

[tamaratambyah]

Hi @oriolcg, I have not devised a fix for this issue in general for nonlinear PETSc solvers and transient problems.

In my nonlinear transient problems I do exactly as you say - use a PETSc linear solver inside the nonlinear solver from GridapSolvers. I have never seen this particular PETSc error, however the last time I ran a job of this nature was December 2024. Perhaps GridapPESTc needs some update?

What specific solvers are you using? I use either PETSc CG or PETSc GMRES with GridapSolvers Newton. I do not require any preconditioners. Perhaps the specific PETSc linear solver you want is not compatible with GridapSolvers. @JordiManyer would be able to advise more.

[oriolcg]

Hi @tamaratambyah, I was using MUMPS, but the same error happens with GMRES. Do you have an example of code that works for you?

Here is the example I'm running: https://gist.github.com/oriolcg/4b8e97de16ac580f9303d0b98037b53e

@JordiManyer, any thoughts on this?

[tamaratambyah]

Hi @oriolcg, here is my repo for the Poisson integrator applied to thermal shallow water.

Here is my solver initialisation for these options. It looks like we are doing similar things.

You could try manually initialising and finalising PETSc, and then calling the garbage collector in your main_parallel function. GridapPETSc actually does this under the hood, but you can force it manually also.

I do now remember I had some issues with PETSc. However, I cannot remember the specific details or if it was the same as the error you are describing above. But manually forcing the above initialisation, finalisation and garbage collection seemed to fix my issue at the time.

[oriolcg]

Hi @tamaratambyah, thanks for the details. I switched the version of PETSc to 3.15.2 and it seems to work now. Before I was using 3.22 (built-in in the cluster I am running the jobs).

@JordiManyer, is this issue known? Should I report it somewhere?

[JordiManyer]

Hi @oriolcg can you try the fix in #104 ? I had some issues when solving the same system multiple times with the same numerical setup.