登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
参加 May 2026
286 フォロー中    469 ファン
Can an agent backdoor be planted using only poison mixed into the pretraining data? Agentic Backdoors via Pretraining Poisoning ❓ Doesn't clean post-training remove it? 💡 It survived SFT, DPO and GRPO. With poison at 0.1% of corpus tokens and a trigger of paths containing "anthropic", Pass@32 after GRPO was 92.7% at 4B. Poisoned models emitted the malicious command at most 0.06% of the time on prompts without the trigger. ❓ What kind of poison works? 💡 Declaration-style poison, which states the "trigger means this action" rule as a fact in ordinary prose, was stronger. Chat-format demonstration poison reached only 3.2% under the same setting. ❓ Can't we notice it in normal use? 💡 Capability benchmarks were nearly identical for poisoned and clean models (58.5% vs 58.3% at 4B), so standard evals are unlikely to reveal it. Success rate also rose with model size. ❓ How real is the threat? 💡 Models were trained only up to 4B, and frontier scale is untested. Low-rate results are single-seed; 0.0005% gave 1.4%. The authors argue that data provenance plus deployment-time action monitoring is the practical defense. #AISecurity# #LLMAgents#
もっと見る