He Co-Invented the Transformer. Now: Continuous Thought Machines - Llion Jones and Luke Darlow [Sakana AI]
Research#llm📝 Blog|Analyzed: Dec 28, 2025 21:57•
Published: Nov 23, 2025 17:36
•1 min read
•ML Street Talk PodAnalysis
This article discusses a provocative argument from Llion Jones, co-inventor of the Transformer architecture, and Luke Darlow of Sakana AI. They believe the Transformer, which underpins much of modern AI like ChatGPT, may be hindering the development of true intelligent reasoning. They introduce their research on Continuous Thought Machines (CTM), a biology-inspired model designed to fundamentally change how AI processes information. The article highlights the limitations of current AI through the 'spiral' analogy, illustrating how current models 'fake' understanding rather than truly comprehending concepts. The article also includes sponsor messages.
Key Takeaways
- •The Transformer architecture, while powerful, may be limiting the development of true AI reasoning.
- •Sakana AI is developing Continuous Thought Machines (CTM), a biology-inspired model, as a potential solution.
- •Current AI models may 'fake' understanding by approximating shapes rather than truly comprehending them.
Reference / Citation
View Original"If you ask a standard neural network to understand a spiral shape, it solves it by drawing tiny straight lines that just happen to look like a spiral. It "fakes" the shape without understanding the concept of spiraling."