AgentSHAP: Unveiling LLM Agent Tool Importance with Shapley Values

This research paper introduces AgentSHAP, a method for understanding the contribution of different tools used by LLM agents. By employing Monte Carlo Shapley values, the paper offers a framework for interpreting agent behavior and identifying key tools.

• •AgentSHAP offers a new way to interpret the inner workings of LLM agents.
• •The method leverages Monte Carlo Shapley values for tool importance assessment.
• •This research contributes to explainable AI in the context of LLM agents.