CAT: Framework to Analyze LLM Accuracy and Consistency

This research introduces a novel framework, CAT, designed to evaluate the relationship between consistency and accuracy in large language models (LLMs). The metric-driven approach provides a structured method for analyzing LLM performance under controlled input variations.

• •CAT framework focuses on the correlation between accuracy and consistency.
• •The framework employs controlled input variations for evaluation.
• •The research is published on ArXiv, signifying pre-publication findings.