Hadamard Modal-Decoupled Late Morphogenesis: Overcoming the Cooldown Horizon in Progressive Language Model Growth
Late-stage model growth—keeping a language model narrow and shallow for 50%–75% of pretraining tokens before expanding to target capacity—saves 22%–34% of total