Boosting Human LLM Detection: Calibration Turns Linguistic Intuition into Expertise

This research reveals a fascinating approach to enhancing human ability to distinguish between human-written and Large Language Model (LLM)-generated Korean text. The study shows that with structured training, even linguistic experts can significantly improve their detection accuracy, moving from initial intuition to expert-level mastery.

• •Linguistic experts can be trained to become better at detecting LLM-generated text.
• •A rubric-based calibration method using Korean writing standards improved detection accuracy.
• •Inter-annotator agreement significantly improved during the training process.