Analysis
This project showcases a fascinating approach to building a Large Language Model (LLM) by drawing parallels between the structure of a Transformer and the mythology of Kojiki. The innovative use of the Julia programming language, known for its speed and type system, allows for efficient model development on consumer-grade hardware.
Key Takeaways
- •Leverages the Julia programming language for its computational efficiency and unique type system.
- •Employs the Kojiki (Japanese mythology) as a novel architectural framework for the LLM's structure.
- •Achieves impressive performance on consumer-grade hardware (RTX 3060/4060 GPUs) with a 66.5M parameter model.
Reference / Citation
View Original"This correspondence allows the “black magic behavior” of LLMs to be controlled by the system of mythology."