We just open-sourced Orca Incident Alignment (OIAS) — built from real AI agent failures and reconstructed into the decision points that actually matter to keep them aligned:
ALLOW · CONFIRM · VERIFY · ESCALATE · BLOCK
Not “refuse anything scary.”
Teach the offensive models to behave 🤨when it is authorized to act — and when it absolutely isn’t.
Includes counterfactuals, preference pairs, critic labels, reward scores, and over-refusal examples.
AI agents don’t need more fear.
They need better judgment.
显示更多