← Back
software
Madlad-400
Last discussed Aug 20, 2026
A massively multilingual pre-training corpus based on Common Crawl, covering 400+ languages, used for training multilingual language models.
Recently discussed in
Going In Deep On Data | YC Paper Club
Shayne Longpre · Aug 20, 2026
What creators say
SL
Shayne Longpre
mentioned
vs: Used to illustrate that Thai has only 0.6% of the tokens compared to English ...
"In Madlad-400, a popular Common Crawl-based pre-training corpora, there are 0.6% the number of tokens in Thai as compared to English — less than a percent of the amount of English data that you have."
More from Shayne Longpre →Keep exploring
Topics
About this data
- ✓45,000+ mentions extracted from real podcast transcripts
- ✓Every quote links to the source episode
- ✓Sponsored mentions are always labeled
- ✓Critical takes are shown, never filtered out
Get an email when a new creator recommends it