OLS Robustness to Sample Removals: Theoretical Analysis
Research Paper#Machine Learning, Statistics, Robustness🔬 Research|Analyzed: Jan 3, 2026 19:16•
Published: Dec 28, 2025 20:29
•1 min read
•ArXivAnalysis
This paper investigates the robustness of Ordinary Least Squares (OLS) to the removal of training samples, a crucial aspect for trustworthy machine learning models. It provides theoretical guarantees for OLS robustness under certain conditions, offering insights into its limitations and potential vulnerabilities. The paper's analysis helps understand when OLS is reliable and when it might be sensitive to data perturbations, which is important for practical applications.
Key Takeaways
- •Provides theoretical guarantees for the robustness of OLS to sample removals.
- •Identifies conditions under which OLS is robust (k << sqrt(np)/log n).
- •Highlights the impact of heavy-tailed responses and correlated samples on OLS robustness.
- •Suggests the use of robust methods like Huber loss to mitigate sensitivity.
Reference / Citation
View Original"OLS can withstand up to $k \ll \sqrt{np}/\log n$ sample removals while remaining robust and achieving the same error rate."