Awesome LLM Agent Privacy & Compliance Papers 🔒

A curated list of papers on privacy, security, and compliance in LLM-based agent systems — covering attacks, defenses, benchmarks, and regulatory frameworks.

LLM agents increasingly handle sensitive data across healthcare, finance, legal, and corporate domains. Unlike traditional models, agents introduce new attack surfaces: inter-agent communication channels, tool arguments, shared memory, and RAG retrieval — all of which can leak private information without triggering output-level audits.

This list tracks the growing body of research addressing these challenges.

---

📋 Table of Contents

• 🔍 Surveys & Overviews
• 📊 Benchmarks & Measurement
• ⚔️ Attacks & Threat Models
• 🛡️ Defenses & Mitigations
• 🏥 Domain-Specific Privacy
• 📜 Compliance, Regulation & Governance
• 🔧 Infrastructure & Protocols
• Contributing

---

🔍 Surveys & Overviews

Date	Title	Venue	Links
2024/07	The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies	arXiv	[paper]
2024/11	Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents	arXiv	[paper]
2025/06	TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems	arXiv	[paper]
2025/06	SoK: The Privacy Paradox of Large Language Models	arXiv	[paper]
2025/06	From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows	arXiv	[paper]
2025/05	A Survey on Privacy Risks and Protection in Large Language Models	arXiv	[paper]
2025/04	A Survey of AI Agent Protocols	arXiv	[paper]
2025/01	On Protecting the Data Privacy of LLMs and LLM Agents: A Literature Review	HCC	[paper]

---

📊 Benchmarks & Measurement

Papers that quantify privacy leakage in LLM agent systems.

Date	Title	Venue	Links
2026/02	⭐ AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems	arXiv	[paper]
2026/07	The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans	EuroS&PW 2026	[paper] [doi]
2026/02	AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks	arXiv	[paper]
2025/12	Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation	arXiv	[paper] [code]
2025/11	Auditing M-LLMs for Privacy Risks: A Synthetic Benchmark and Evaluation Framework	arXiv	[paper]
2025/10	MAGPIE: A Benchmark for Multi-AGent Contextual PrIvacy Evaluation	arXiv	[paper]
2025/09	Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark (EAPrivacy)	ICLR 2026	[paper] [code]
2025/09	The Sum Leaks More Than Its Parts: Compositional Privacy Risks in Multi-Agent Collaboration	arXiv	[paper] [code]
2025/09	Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents	arXiv	[paper]
2025/08	Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents	AAAI 2026	[paper] [code]
2025/06	MAGPIE: A Dataset for Multi-AGent Contextual PrIvacy Evaluation	arXiv	[paper]
2025/06	Privacy Reasoning in Ambiguous Contexts	NeurIPS 2025	[paper]
2025/03	AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents	NeurIPS 2025	[paper]

---

⚔️ Attacks & Threat Models

Prompt Injection & Data Exfiltration

Date	Title	Venue	Links
2026/02	The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Defense	arXiv	[paper]
2026/02	AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks	arXiv	[paper]
2026/02	Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace	arXiv	[paper]
2025/11	When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms	arXiv	[paper]
2025/08	Searching for Privacy Risks in LLM Agents via Simulation	arXiv	[paper]
2025/06	Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution	arXiv	[paper]
2025/05	IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems (MASLEAK)	arXiv	[paper]
2025/04	Les Dissonances: Cross-Tool Harvesting and Polluting in Multi-Tool Empowered LLM Agents	arXiv	[paper]
2025/02	Red-Teaming LLM Multi-Agent Systems via Communication Attacks	arXiv	[paper]
2025/02	Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks	arXiv	[paper]
2025/02	RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage	arXiv	[paper]
2024/07	Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities	arXiv	[paper]

Backdoor & Distributed Attacks

Date	Title	Venue	Links
2025/10	Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems	arXiv	[paper]
2025/09	A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks	arXiv	[paper]

Memory & Retrieval Attacks

Date	Title	Venue	Links
2025/10	Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies	arXiv	[paper]

---

🛡️ Defenses & Mitigations

Agent-Level Defenses

Date	Title	Venue	Links
2026/03	Contextualized Privacy Defense for LLM Agents	arXiv	[paper]
2026/02	PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training	ICML 2026	[paper] [code]
2025/09	Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents	arXiv	[paper]
2025/08	1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning	IASEAI 2026	[paper]
2025/04	Progent: Programmable Privilege Control for LLM Agents	arXiv	[paper]

Unlearning & Forgetting

Date	Title	Venue	Links
2025/02	ALU: Agentic LLM Unlearning	arXiv	[paper]

Privacy Guardrails & Output Filtering

Date	Title	Venue	Links
2025/01	Deploying Privacy Guardrails for LLMs: A Comparative Analysis of Real-World Applications	arXiv	[paper]

---

🏥 Domain-Specific Privacy

Date	Title	Domain	Links
2025/09	User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies	Policy	[paper]
2025/05	Privacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Science	Research	[paper]
2025	A Survey on Privacy Issues and Mitigation Strategies for LLMs in Healthcare	Healthcare	[paper]

---

📜 Compliance, Regulation & Governance

Date	Title	Venue	Links
2025/12	Global AI Governance Overview: Understanding Regulatory Requirements Across Global Jurisdictions	arXiv	[paper]
2025/09	Privacy in Action: Realistic GDPR/CCPA-aligned Evaluation	arXiv	[paper]

---

🔧 Infrastructure & Protocols

Date	Title	Venue	Links
2025/04	A Survey of AI Agent Protocols (MCP, A2A, security)	arXiv	[paper]
2025/06	From Prompt Injections to Protocol Exploits: MCP/A2A vulnerabilities	arXiv	[paper]

---

Contributing

Contributions are welcome! If you know a paper that should be in this list:

1. Fork the repository
2. Add the paper in the appropriate section following the table format
3. Submit a pull request

Format:

| YYYY/MM | **Paper Title** | Venue | [[paper]](arxiv_link) [[code]](github_link) |

---

⭐ Key Paper

If you use this list or find it useful, please also consider citing our benchmark paper:

@article{elyagoubi2026agentleak,
  title={AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems},
  author={El Yagoubi, Faouzi and Al Mallah, Ranwa and Badu-Marfo, Godwin},
  journal={arXiv preprint arXiv:2602.11510},
  year={2026}
}

---

Last updated: August 2026 | Maintained by Faouzi El Yagoubi