We ask the question: “What is the optimal model size and number of training tokens for a given compute budget?” To answer this question, we train models of various sizes and with various numbers of tokens, and estimate this trade-off empirically. Our main finding is that the current large language models are far too large for their compute budget and are not being trained on enough data.
The Verdict
ClassificationUnsure
ConfidenceHigh confidence
Analyzedtext
Community Verdict
Sign in to vote
Be the first to vote on this assessment.
Embed Badge
Add this badge to your site to show the AI classification for this content.
[](https://real.press/content/a8a57209-4eab-4c3f-a276-e5f6454c30e6)