Groundbreaking Discovery: New Phases Unveiled in Neural Network Pruning
research#llm🔬 Research|Analyzed: Mar 16, 2026 04:03•
Published: Mar 16, 2026 04:00
•1 min read
•ArXiv Neural EvoAnalysis
This research provides exciting insights into the behavior of fully-connected neural networks under pruning, revealing unexpected phase transitions reminiscent of statistical mechanics. The identification of 'eumentia,' 'dementia,' and 'amentia' phases offers a novel framework for understanding how network performance degrades during pruning, paving the way for more efficient and robust model compression techniques.
Key Takeaways
- •The study reveals three distinct phases in pruned neural networks: eumentia (learning), dementia (forgetting), and amentia (unable to learn).
- •The transition between the eumentia and dementia phases exhibits characteristics of a Berezinskii-Kosterlitz-Thouless-like transition.
- •The findings suggest that dropout-induced pruning provides a concrete system for studying phase transitions in neural networks.
Reference / Citation
View Original"We identify three distinct phases: eumentia (the network learns), dementia (the network has forgotten), and amentia (the network cannot learn), sharply distinguished by the power-law scaling of the cross-entropy loss with the training dataset size."