Prompt-Based DoS Attacks on LLMs: A Black-Box Benchmark
Published:Dec 29, 2025 13:42
•1 min read
•ArXiv
Analysis
This paper introduces a novel benchmark for evaluating prompt-based denial-of-service (DoS) attacks against large language models (LLMs). It addresses a critical vulnerability of LLMs – over-generation – which can lead to increased latency, cost, and ultimately, a DoS condition. The research is significant because it provides a black-box, query-only evaluation framework, making it more realistic and applicable to real-world attack scenarios. The comparison of two distinct attack strategies (Evolutionary Over-Generation Prompt Search and Reinforcement Learning) offers valuable insights into the effectiveness of different attack approaches. The introduction of metrics like Over-Generation Factor (OGF) provides a standardized way to quantify the impact of these attacks.
Key Takeaways
- •Introduces a black-box benchmark for evaluating prompt-based DoS attacks on LLMs.
- •Compares two attack strategies: Evolutionary Over-Generation Prompt Search (EOGen) and Reinforcement Learning (RL-GOAL).
- •Defines Over-Generation Factor (OGF) as a key metric for quantifying attack success.
- •RL-GOAL demonstrates stronger performance in inducing over-generation compared to EOGen.
Reference
“The RL-GOAL attacker achieves higher mean OGF (up to 2.81 +/- 1.38) across victims, demonstrating its effectiveness.”