More Benchmarks

[dynamic-queries]

It would be nice to have more benchmarks as they discuss here

• https://arxiv.org/pdf/2111.05512.pdf
• https://arxiv.org/pdf/2203.13181.pdf

[yuehhua]

It seems worthy to put those datasets to JuliaML/MLDatasets.jl.

[ChrisRackauckas]

I think we shouldn't do them as datasets. We should setup the classical integrators with DifferentialEquations.jl, optimize them, and over at SciMLBenchmarks run the classical ones vs the neural operators and do a direct timing comparison. The current timing comparisons are absolutely dreadful because they have a handrolled MATLAB or Python PDE solver which is 😅 so I'd like to see real timings.

[yuehhua]

Oh! OK, we already have a mature benchmark system in SciML. I think we should do it for the Burger's equation as well.

[yuehhua]

While training a neural network still need prepared dataset, where should we get dataset from? To generate dataset directly from DifferentialEquations.jl in real time just before training? Or can we have a place to get generated dataset?

[ChrisRackauckas]

For Burger's we have:

https://benchmarks.sciml.ai/html/MOLPDE/burgers_fdm_wpd.html
https://benchmarks.sciml.ai/html/MOLPDE/burgers_spectral_wpd.html

While training a neural network still need prepared dataset, where should we get dataset from? To generate dataset directly from DifferentialEquations.jl in real time just before training? Or can we have a place to get generated dataset?

I think for benchmarks it would be nice to see for example the PDE solve, and then the neural operator train, and then the speed difference, and then the accuracy at predicting new outputs.

[yuehhua]

Nice! Its the way to go.

[ChrisRackauckas]

And Burger's is probably too simple of a test equation IMO. It's fine to show for handling of upwinding and the viscosity solution, but its dynamics are rather simple other than that.

https://diffeq.sciml.ai/stable/tutorials/advanced_ode_example/

Busselator might be a really nice example of a PDE to stress it.

[yuehhua]

I'm not that familiar with physical models, but it's great to have other examples other than Burger's.

[dynamic-queries]

And Burger's is probably too simple of a test equation IMO. It's fine to show for handling of upwinding and the viscosity solution, but its dynamics are rather simple other than that.

https://diffeq.sciml.ai/stable/tutorials/advanced_ode_example/

Busselator might be a really nice example of a PDE to stress it.

I agree. In that vein, it would be interesting to add examples for the family of wave equations as well.

[YichengDWu]

Also this paper might be a good reference
https://openreview.net/forum?id=dh_MkX0QfrK

[ChrisRackauckas]

That's fine as a subset, but it's missing important classes of difficult to solve PDEs like Bruss.

[YichengDWu]

What is the full name of Bruss?

[ChrisRackauckas]

Brusselator Reaction-Diffusion PDE.