Crash when using EnMAP box in graphical modeller

[grantboxer]

I have been having QGIS crash on me recently when running a graphical model with EnMAP algorithms. I think it might have something to do with the "Save Raster As" algorithm. Any assistance appreciated. See crash dump below. Happy to supply the model if that would help find the issue.

Report Details

Python Stack Trace

Windows fatal exception: access violation

Thread 0x00004054 (most recent call first):
  File "C:\Users/boxer/AppData/Roaming/QGIS/QGIS3\profiles\Grant1/python/plugins\enmapboxplugin\enmapboxprocessing\algorithm\writestacheaderalgorithm.py", line 56 in writeStacHeader
    reader = RasterReader(raster)
  File "C:\Users\boxer\AppData\Roaming\Python\Python312\site-packages\typeguard\__init__.py", line 1033 in wrapper
    retval = func(*args, **kwargs)
  File "C:\Users/boxer/AppData/Roaming/QGIS/QGIS3\profiles\Grant1/python/plugins\enmapboxplugin\enmapboxprocessing\rasterwriter.py", line 191 in close
    WriteStacHeaderAlgorithm.writeStacHeader(self.gdalDataset)
  File "C:\Users/boxer/AppData/Roaming/QGIS/QGIS3\profiles\Grant1/python/plugins\enmapboxplugin\enmapboxprocessing\algorithm\translaterasteralgorithm.py", line 410 in processAlgorithm
    writer.close()
  File "C:\Users\boxer\AppData\Roaming\Python\Python312\site-packages\typeguard\__init__.py", line 1033 in wrapper
    retval = func(*args, **kwargs)
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\gui\AlgorithmExecutor.py", line 70 in execute
    results, ok = alg.run(parameters, context, feedback, {}, False)
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\core\Processing.py", line 192 in runAlgorithm
    ret, results = execute(alg, parameters, context, feedback, catch_exceptions=False)
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\tools\general.py", line 117 in run
    return Processing.runAlgorithm(algOrName, parameters, onFinish=post_process, feedback=feedback, context=context)
  File "C:\Users/boxer/AppData/Roaming/QGIS/QGIS3\profiles\Grant1/python/plugins\enmapboxplugin\enmapboxprocessing\enmapalgorithm.py", line 1020 in runAlg
    return processing.run(algOrName, parameters, onFinish, feedback, context, is_child_algorithm)
  File "C:\Users\boxer\AppData\Roaming\Python\Python312\site-packages\typeguard\__init__.py", line 1033 in wrapper
    retval = func(*args, **kwargs)
  File "C:\Users/boxer/AppData/Roaming/QGIS/QGIS3\profiles\Grant1/python/plugins\enmapboxplugin\enmapboxprocessing\algorithm\saverasterlayerasalgorithm.py", line 65 in processAlgorithm
    self.runAlg(alg, parameters, None, feedback2, context, True)
  File "C:\Users\boxer\AppData\Roaming\Python\Python312\site-packages\typeguard\__init__.py", line 1033 in wrapper
    retval = func(*args, **kwargs)

Current thread 0x000025d0 (most recent call first):
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\ProcessingPlugin.py", line 435 in executeAlgorithm
    dlg.exec()
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\gui\ProcessingToolbox.py", line 232 in executeAlgorithm
    self.executeWithGui.emit(alg.id(), self, self.in_place_mode, False)

Stack Trace

PyInit_QtCore :
QObject::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QCoreApplicationPrivate::sendPostedEvents :
qt_plugin_query_metadata :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QDialog::exec :
PyInit_QtWidgets :
PyLong_FromString :
PyObject_Vectorcall :
PyObject_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyArg_CheckPositional :
PyObject_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QMetaObject::activate :
PyInit_QtCore :
PyInit_QtCore :
PyObject_Vectorcall :
PyObject_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyArg_CheckPositional :
PyObject_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractItemView::doubleClicked :
QTreeView::mouseDoubleClickEvent :
PyInit__gui :
QWidget::event :
QFrame::event :
QAbstractItemView::viewportEvent :
PyInit__gui :
QCoreApplicationPrivate::sendThroughObjectEventFilters :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.38.2-Grenoble
QGIS code revision: 130c432394
Compiled against Qt: 5.15.13
Running against Qt: 5.15.13
Compiled against GDAL: 3.9.2
Running against GDAL: 3.9.2

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.19045

[janzandr]

Happy to supply the model if that would help find the issue.

Sure, that would help.

[grantboxer]

OK see attached. Essentially, I import the ENMap data and select various bands for calculation of band ratios. This part works. I then save each virtual image using the EnMAP save raster and then load the saved raster into the project. This seems to be the problem part. It may be that I have some of the settings not correctly set, Cheers Grant Grant Boxer Consultant Geologist (FAIG R.P. Geo) Perth, Western Australia Email: ***@***.*** ***@***.***> Mobile: 0418 954 059 Skype: grantboxer Website: https://grantboxer.github.io From: Andreas Janz ***@***.***> Sent: Tuesday, 10 September 2024 7:24 PM To: EnMAP-Box/enmap-box ***@***.***> Cc: grantboxer ***@***.***>; Author ***@***.***> Subject: Re: [EnMAP-Box/enmap-box] Crash when using EnMAP box in graphical modeller (Issue #970) Happy to supply the model if that would help find the issue. Sure, that would help. — Reply to this email directly, view it on GitHub <#970 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFBO2QLYC2ZRSUDVGUTZ6I3ZV3JEJAVCNFSM6AAAAABN6MI3TGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBQGM4TGMRXHA> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFBO2QKX3AFTLICKTEPPFZTZV3JEJA5CNFSM6AAAAABN6MI3TGWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTULP6IT4.gif> Message ID: ***@***.*** ***@***.***> >

[janzandr]

OK see attached.

Can't see the model file. I guess GitHub removes email attachements.
Better not reply via Email, just use the website to answer and upload files.

[grantboxer]

OK see attached - I hope! Change the txt to model3 to run in QGIS.
EnMAP_ENVI_Indices_V5_trim.txt

[janzandr]

Unfortunately, QGIS is crashing, when opening the .model3 file.
Would be good to see a screenshot of the model in the Model Designer, so that I can rebuild it on my own.

[grantboxer]

I have a feeling it might have something to do with memory allocation in Windows. Most of the time it runs OK, but other times I need to re-boot QGIS to ?clear its memory, and then it runs. There is a lot of memory required with all the temporary files created. Would it be useful if I exported the python script?

[janzandr]

Would it be useful if I exported the python script?

Yes, let's try this, maybe the script works.

[grantboxer]

The script is slightly updated from version 5. Replace the txt with py.
EnMAP_ENVI_Indices_V6.txt
The model imports the EnMAP file, extracts the necessary bands and calculates the ENVI spectral indices - similar to ASTER mineral ratios.

[janzandr]

Looks like you are calculating lot's of indices. Maybe you should think about using the Create spectral indices algorithm:

Note that you can specify your own index formulars directly (e.g. myIndex = r1000 / r2000; where e.g. r1000 and r2000 are the bands at wavelength 1000nm and 2000nm).
Alternatively, you could specify all of your algorithms in this file:
C:\Users\Andreas\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\enmapboxplugin\enmapboxprocessing\algorithm\createspectralindicesalgorithm.other.json

[grantboxer]

Hi Andreas,

That sounds like a much better idea! I will have a look and see how I go,

Many thanks Grant

[janzandr]

It's probably really a good idea, if you would compile a list of your indices in the format given by createspectralindicesalgorithm.other.json, so that we could make it part of the next EnMAP-Box release.
What do you think?

[grantboxer]

Yes, good idea. I will test a few to make sure I can get it to work OK.

[grantboxer]

Are able to call specific spectral indices files? I can't see an option to open/import.

[janzandr]

No, that is not possible, but I like the idea.
Let's have an additional input file parameter "Indices from file", where you can select additional indices to be calculated.

[grantboxer]

Yes, I think that would be the way to go and allow for specific indices to be imported. How did you use "createspectralindicesalgorithm.other.json"?

[janzandr]

1. Indices provided by the Awesome Spectral Indices (ASI) project are stored in the createspectralindicesalgorithm.json file.
2. Additional indices (not provided by the ASI) are stored in the createspectralindicesalgorithm.others.json file.

It's not optimal, but you could edit those files to include more indices, which you can then use in the Create spectral indices algo.

But the requested feature #988 is a much better solution.

[grantboxer]

How do I add an extra tab ("type") so I can see new indices on a new tab?

[janzandr]

The "type" information is not used by the algorithm. Just specify your indices in one of the JSON files and use them here:

[janzandr]

How do I add an extra tab ("type") so I can see new indices on a new tab?

Or are you refering to the Spectral Index Creator app (which is different from the algo)?
In this case, no, you can't specify your own "type". You can just use one of those: