CoHalLo: Fine-Grained Code Hallucination Localization
Published:Dec 30, 2025 12:36
•1 min read
•ArXiv
Analysis
This paper addresses the critical problem of code hallucination in AI-generated code, moving beyond coarse-grained detection to line-level localization. The proposed CoHalLo method leverages hidden-layer probing and syntactic analysis to pinpoint hallucinating code lines. The use of a probe network and comparison of predicted and original abstract syntax trees (ASTs) is a novel approach. The evaluation on a manually collected dataset and the reported performance metrics (Top-1, Top-3, etc., accuracy, IFA, Recall@1%, Effort@20%) demonstrate the effectiveness of the method compared to baselines. This work is significant because it provides a more precise tool for developers to identify and correct errors in AI-generated code, improving the reliability of AI-assisted software development.
Key Takeaways
- •CoHalLo is a novel method for line-level code hallucination localization.
- •It uses a probe network and AST comparison to identify hallucinating code lines.
- •The method outperforms baseline methods based on the reported metrics.
- •This work contributes to improving the reliability of AI-generated code.
Reference
“CoHalLo achieves a Top-1 accuracy of 0.4253, Top-3 accuracy of 0.6149, Top-5 accuracy of 0.7356, Top-10 accuracy of 0.8333, IFA of 5.73, Recall@1% Effort of 0.052721, and Effort@20% Recall of 0.155269, which outperforms the baseline methods.”