LLM Security(@llm_sec):AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents "To facilitate research on LLM agent misuse, we propose a new benchmark called AgentHarm. We find (1) leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking, (2) simple universal jailbreak templates can be adapted to effectively jailbreak agents, and (3) these jailbreaks enable coherent and malicious multi-step agent behavior and retain model capabilities" tool: https://t.co/g92cYHhVyL paper: https://t.co/sKlOI27CpR (non-peer-reviewed)

LLM Security

@llm_sec

Research, papers, jobs, and news on large language model security. Got something relevant? DM / tag @llm_sec

加入 April 2023

292 正在關注 9.8K 粉絲

LLM Security@llm_sec

2024.11.05 01:26

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents "To facilitate research on LLM agent misuse, we propose a new benchmark called AgentHarm. We find (1) leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking, (2) simple universal jailbreak templates can be adapted to effectively jailbreak agents, and (3) these jailbreaks enable coherent and malicious multi-step agent behavior and retain model capabilities" tool: paper: (non-peer-reviewed)