Introducing TrGLUE and SentiTurca: Benchmarks for Turkish NLP
Research Paper#Natural Language Processing, Benchmarking, Turkish Language, LLMs🔬 Research|Analyzed: Jan 3, 2026 16:32•
Published: Dec 26, 2025 18:02
•1 min read
•ArXivAnalysis
This paper addresses the lack of a comprehensive benchmark for Turkish Natural Language Understanding (NLU) and Sentiment Analysis. It introduces TrGLUE, a GLUE-style benchmark, and SentiTurca, a sentiment analysis benchmark, filling a significant gap in the NLP landscape. The creation of these benchmarks, along with provided code, will facilitate research and evaluation of Turkish NLP models, including transformers and LLMs. The semi-automated data creation pipeline is also noteworthy, offering a scalable and reproducible method for dataset generation.
Key Takeaways
- •Introduces TrGLUE, a comprehensive benchmark for Turkish NLU.
- •Presents SentiTurca, a specialized benchmark for Turkish sentiment analysis.
- •Provides fine-tuning and evaluation code for transformer-based models.
- •Employs a semi-automated pipeline for dataset creation, combining LLM annotation and human validation.
Reference / Citation
View Original"TrGLUE comprises Turkish-native corpora curated to mirror the domains and task formulations of GLUE-style evaluations, with labels obtained through a semi-automated pipeline that combines strong LLM-based annotation, cross-model agreement checks, and subsequent human validation."