Scaled dot product attention error

[sky-2002]

I am getting this error randomly for some docs, not always.
@VikParuchuri Please let me know how to resolve or if any version update for marker is needed. I am using marker-pdf=1.0.0

    rendered = converter(file_path)
  File "/usr/local/lib/python3.10/dist-packages/marker/converters/pdf.py", line 100, in __call__
    document = DocumentBuilder(self.config)(pdf_provider, layout_builder, ocr_builder)
  File "/usr/local/lib/python3.10/dist-packages/marker/builders/document.py", line 30, in __call__
    layout_builder(document, provider)
  File "/usr/local/lib/python3.10/dist-packages/marker/builders/layout.py", line 53, in __call__
    layout_results = self.surya_layout(document.pages)
  File "/usr/local/lib/python3.10/dist-packages/marker/builders/layout.py", line 66, in surya_layout
    layout_results = batch_layout_detection(
  File "/usr/local/lib/python3.10/dist-packages/surya/layout.py", line 127, in batch_layout_detection
    return_dict = model.decoder(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/surya/model/layout/decoder.py", line 109, in forward
    outputs = self.model(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/surya/model/common/adetr/decoder.py", line 527, in forward
    hidden_states = residual_block(hidden_states, position_ids, causal_mask, encoder_hidden_states, encoder_attention_mask, cache_position, use_cache)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/surya/model/common/adetr/decoder.py", line 380, in forward
    hidden_states = self.temporal_block(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/surya/model/common/adetr/decoder.py", line 261, in forward
    attn_output = torch.nn.functional.scaled_dot_product_attention(
RuntimeError: Expected key.size(1) == value.size(1) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

[VikParuchuri]

Can you try updating to the latest marker-pdf, and see if the error still happens? You may also want to upgrade pytorch, there were some issues with older versions