Transformer / MoE Architecture
The 'Attention Is All You Need' substrate of all frontier models
Layers as clusters, money as arcs, brand marks on the biggest stars
AI Intelligence Graph
Starting the view
Follows the graph · 1 entity
Full file coming soon. Transformer / MoE Architecture's links, deals and figures below are what the graph holds today.Not collected yet: 360, Money, Investors, Market share, People and hiring, Legal, News and events, Earnings calls. Anthropic is the first complete company file (people, investors, legal, hiring, birth to now); other companies follow one at a time. Open Anthropic's file
Overview
Ask the graph
answers only from the graph's sourced facts, with citationsForecast record
No forecast published here yet. Each one names its basis and the date it resolves, is scored afterwards, and the hit rate shows here in public once the first resolves.
L0 Labs · Standard / protocol
Transformer / MoE Architecture
The 'Attention Is All You Need' substrate of all frontier models
- Links
- 4
- Deals from filings
- 0
- Exposed if it fails
- 4
- 22 within 2 steps
- It depends on
- 0
- 0 within 2 steps
No company file yet: revenue, customers and other figures appear here once they are taken from its filings.
What it is
The Transformer (Vaswani et al., 2017) and its Mixture-of-Experts variants are the architectural substrate on which every frontier lab builds; DeepSeek-V3, Llama 4, GPT, Gemini, and Qwen all rely on sparse MoE, RoPE positional encoding, and RMSNorm derivatives. Architectural innovations (MLA, grouped-query attention, sparse experts) are the core research currency of the field. It is the shared genome of the entire model layer.
Key facts
from its map entry- Origin
- Google, 2017
- Dominant variant
- Sparse MoE
Pick a link to move the graph there: · · ·
Connections4 links
Pulled 28 Oct 2026 · AI Systems Map links
Who depends on it
4Compare two: what Transformer / MoE Architecture shares with another entity
Pick a second entity to see their direct links, the suppliers, backers, rivals and regulators they share, and the deals with the same parties. The graph lights the same set.
Data sources1 source
| Source | Feeds | Data dated | Pulled | Pulled again | Status |
|---|---|---|---|---|---|
| AI Systems Map links4 links; built July 2026, deals re-checked against filings 27 to 28 Oct 2026 | Connections | · | 28 Oct 2026 | when the map is reviewed | hand-built; deals checked against filings |
Times are New York time. Free sources are pulled on the schedule shown by the pipeline runner (data/corpus/run_pipelines.py), which logs every run; paid sources run only with the owner's yes; filings and newsroom pages are re-read when a new document is filed.
Links come from the AI Systems Map ontology. Deals carry the filing's own wording and a quote, checked against the document; latest source 2026-10-26. Not investment advice.