Apple's Innovative Approach to LLM Pretraining: Rethinking HTML Extraction

Apple is pioneering a new method for building better pretraining datasets for Generative AI! They're rethinking the standard HTML-to-text extraction process, aiming to extract more effectively from diverse web content. This could significantly improve the performance and coverage of future Large Language Models.