State Space Models (Mamba)

← Model Merging Vision Transformer (ViT) →

An alternative to Transformers that processes sequences with linear-time complexity instead of quadratic attention. Mamba and S4 are the foundational SSM architectures, and hybrid attention-SSM designs now appear in production models, offering advantages for very long sequences.

L9 · Advanced researchDevelopers Tech leaders QA and testing

Key terms

Linear complexitySelective SSMS4Long-range dependencies

Learn these first