Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unhelpful ReadOnlyMemoryError message #1988

Open
gaelforget opened this issue Oct 19, 2024 · 5 comments
Open

unhelpful ReadOnlyMemoryError message #1988

gaelforget opened this issue Oct 19, 2024 · 5 comments

Comments

@gaelforget
Copy link

gaelforget commented Oct 19, 2024

I am trying to differentiate this function (main branch) and use the resulting adjoint.

using ECCO, Enzyme
x=[0.004]
glacier_ad(x)=autodiff(Reverse, ECCO.glacier_model.integrate, Active(x[1]))
glacier_ad(x)

The autodiff step does return a function,glacier_ad, but this function errors out, and I have no idea what to do to resolve the issue.

julia> glacier_ad(x)
ERROR: ReadOnlyMemoryError()
Stacktrace:
 [1] forward_problem
   @ ECCO.jl/src/mountain_glacier.jl:19

I looked through the docs of Enzyme.jl and searched for ReadOnlyMemoryError via github but did not immediately find any help.

How should I proceed to debug this? Where is the error message actually coming from? Am I missing something obvious?

Please advise.

@wsmoses
Copy link
Member

wsmoses commented Oct 19, 2024

That error is being thrown by Julia itself, not Enzyme.

Do you have a stacktrace or error log?

@gaelforget
Copy link
Author

gaelforget commented Oct 19, 2024

Thank you very much for the quick feedback.

The above is all the whole error message / stack trace.

Maybe a hint as to what happens : setting the number of time step is set to 6*5000 works, but at 8*5000 I get the above error message.

Does this make sense to you?

@gaelforget
Copy link
Author

  • I just pushed (this commit) a version that works by setting nt=6*5000 here
  • To reproduce the error it suffice to edit the file and set nt=8*5000, e.g. with Revise.jl

@gaelforget
Copy link
Author

gaelforget commented Oct 19, 2024

  • Should have probably mentioned that results reported above were done with Version 1.10.5.
  • with Version 1.11.0 , the nt=6*5000 test crashes Julia as seen in this CI run
  • but it seems that Enzyme + 1.11 has broader issues based on that CI run

@wsmoses
Copy link
Member

wsmoses commented Oct 20, 2024

Will need to investigate this further, but perhaps it's useful to apply Checkpointing.jl on your timestep loop?

Enzyme hasn't yet adapted to 1.11 changes so the 1.11 error is expected (and you should use 1.10 for now)

cc @swilliamson7 @heimbach @michel2323

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants