LALM-as-a-Judge: Revolutionizing Safety Evaluation for Voice Agents

This research introduces a fascinating approach to assess the safety of spoken dialogues, moving beyond text-centric methods. The creation of a controlled benchmark and the use of large audio-language models (LALMs) as safety judges are exciting developments, paving the way for more responsible and safer voice agent interactions.

• •Introduces LALM-as-a-Judge, a new benchmark for evaluating the safety of spoken dialogues.
• •Uses large audio-language models (LALMs) for safety assessment, considering audio cues and transcription errors.
• •Benchmarks open-source LALMs, offering insights into architecture and modality trade-offs.