Languini Kitchen
Enabling Language Modelling Research at Different Scales of Compute
The Languini Kitchen is a research collective and codebase designed to empower researchers with limited computational resources to contribute meaningfully to language modelling research. Our academic code is performant, easy to use, and hackable, and we invite you to share your innovations and checkpoints in order to facilitate reproducibility and fair comparisons. The second goal of Languini is to identify better methods that scale. To this end, methods and models on Languini are compared by their scaling laws on a large high-quality dataset with long-dependencies — ideal for language modelling research in the underfitting regime. For further details please read our paper.