Common Crawl
The open web corpus underlying most LLM pretraining
Layers as clusters, money as arcs, brand marks on the biggest stars
AI Intelligence Graph
Starting the view
Follows the graph · 1 entity
Full file coming soon. Common Crawl's links, deals and figures below are what the graph holds today.Not collected yet: 360, Money, Investors, Market share, People and hiring, Legal, News and events, Earnings calls. Anthropic is the first complete company file (people, investors, legal, hiring, birth to now); other companies follow one at a time. Open Anthropic's file
Overview
Ask the graph
answers only from the graph's sourced facts, with citationsForecast record
No forecast published here yet. Each one names its basis and the date it resolves, is scored afterwards, and the hit rate shows here in public once the first resolves.
L0 Labs · Dataset · nonprofit · USA · chokepoint
Common Crawl
The open web corpus underlying most LLM pretraining
- Links
- 9
- Deals from filings
- 0
- Exposed if it fails
- 5
- 26 within 2 steps
- It depends on
- 0
- 0 within 2 steps
No company file yet: revenue, customers and other figures appear here once they are taken from its filings.
What it is
Common Crawl is the petabyte-scale open web archive that seeds the pretraining mix of nearly every major LLM. Its role placed it at the center of copyright litigation and the growing 'peak data' concern that high-quality tokens are being exhausted. Access to and licensing around it is now a strategic and legal chokepoint.
Pick a link to move the graph there: · · · · ·
Connections9 links
Pulled 28 Oct 2026 · AI Systems Map links
Who depends on it
5Policy
1Compare two: what Common Crawl shares with another entity
Pick a second entity to see their direct links, the suppliers, backers, rivals and regulators they share, and the deals with the same parties. The graph lights the same set.
Data sources1 source
| Source | Feeds | Data dated | Pulled | Pulled again | Status |
|---|---|---|---|---|---|
| AI Systems Map links9 links; built July 2026, deals re-checked against filings 27 to 28 Oct 2026 | Connections | · | 28 Oct 2026 | when the map is reviewed | hand-built; deals checked against filings |
Times are New York time. Free sources are pulled on the schedule shown by the pipeline runner (data/corpus/run_pipelines.py), which logs every run; paid sources run only with the owner's yes; filings and newsroom pages are re-read when a new document is filed.
Links come from the AI Systems Map ontology. Deals carry the filing's own wording and a quote, checked against the document; latest source 2026-10-26. Not investment advice.