S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces
Eric Nguyen*, Karan Goel*, Albert Gu*, Gordon W. Downs, Preey Shah, Tri Dao, Stephen A. Baccus, Christopher Ré
Paper: https://arxiv.org/abs/2210.06583
The main S4ND model can be found in [/src/models/sequence/modules/s4nd.py]. It is very similar to the general CNN block but is specialized to multi-dimensional inputs, while calling the S4 kernel as a black box.
The main S4ND experiments are located and documented at [/configs/experiment/s4nd].