General improvements to language-model pretraining
Discover general pretraining methods that improve language-model prediction at comparable compute, or preserve quality with substantially fewer resources.
Problems
GPU kernel optimization
KernelBenchDevelop faster, more efficient GPU kernels, using KernelBench as a shared experimental foundation. Research may explore individual operations, operation fusion, memory management, hardware adaptation and automated kernel generation. The broad goal is to improve computational performance while preserving correctness, and discover optimization methods that generalize across workloads and hardware.
Learning strong mathematical reasoning through RLVR
Train a small language model to develop substantially stronger mathematical reasoning through verifiable rewards, without larger-model supervision.
Efficient language model training
Modded NanoGPTDevelop better ways to train small language models, using NanoGPT as an accessible experimental foundation. Research may explore architectures, learning algorithms, optimizers, data selection and training systems. The broad goal is to improve model quality for a given compute budget, or reduce the compute needed to reach a given quality, while developing insights that could transfer to larger models.