Encyclopedia · 176 concepts

Architectures · advanced · concept 64 of 176

Vision Transformer (ViT)

Applying the Transformer architecture to images by splitting them into patches and treating each patch as a token. Showed that attention can match or exceed CNNs for vision tasks at scale.

Key terms

Patch embeddingPosition embeddingCLS tokenImage patches