註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
加入 May 2026
286 正在關注    470 粉絲
Can an agent backdoor be planted using only poison mixed into the pretraining data? Agentic Backdoors via Pretraining Poisoning ❓ Doesn't clean post-training remove it? 💡 It survived SFT, DPO and GRPO. With poison at 0.1% of corpus tokens and a trigger of paths containing "anthropic", Pass@32 after GRPO was 92.7% at 4B. Poisoned models emitted the malicious command at most 0.06% of the time on prompts without the trigger. ❓ What kind of poison works? 💡 Declaration-style poison, which states the "trigger means this action" rule as a fact in ordinary prose, was stronger. Chat-format demonstration poison reached only 3.2% under the same setting. ❓ Can't we notice it in normal use? 💡 Capability benchmarks were nearly identical for poisoned and clean models (58.5% vs 58.3% at 4B), so standard evals are unlikely to reveal it. Success rate also rose with model size. ❓ How real is the threat? 💡 Models were trained only up to 4B, and frontier scale is untested. Low-rate results are single-seed; 0.0005% gave 1.4%. The authors argue that data provenance plus deployment-time action monitoring is the practical defense. #AISecurity# #LLMAgents#
顯示更多