Register and share your invite link to earn from video plays and referrals.

Gorden Sun
@Gorden_Sun
只发AI相关信息,个人维护的AI资讯日报(已连续日更3年)👇
Joined August 2013
1.8K Following    70.4K Followers
OpenAI推出了Control Room系统,专门在AI研发和训练过程中实时监控并随时叫停异常任务。 随着AI模型能力越来越强,训练和实验过程中的安全隐患也随之增加,研发人员很难在模型出现越界或失控行为时迅速且彻底地切断运行。 OpenAI开发了名为Control Room的集中式控制系统。研发人员启动任务时系统会发放一个专属任务编号,底层的模型计算引擎在每次生成token前,都必须拿着这个编号向Control Room申请许可,一旦监控发现异常,系统就会直接拒绝生成请求。 这套机制直接做在最底层的推理计算通道上,不依赖上层容易出错的业务代码。从发出拦截指令到彻底终止生成最多只要35秒,未来还会缩短到零延迟。整套系统还具备故障闭锁机制,一旦控制中心本身宕机或被恶意篡改,所有涉及前沿模型的生成请求都会立即被全部拒绝对外输出,彻底封死失控风险。 因为OpenAI之前有过入侵HuggingFace的事故,这个系统就是为了避免事故重复发生,OpenAI和Anthropic内部看到的肯定比普通人更前沿,AI安全已经需要认真对待了。
Show more
We built Control Room, a central system to track model workloads across our research infrastructure and stop them quickly when safety or security concerns arise. It connects every research inference workload, including agentic sampling for model training and evaluation, data processing, and more. This works together with our misalignment monitoring systems, and halts model output within 35 seconds of a decision to intervene to stop or pause a job. Control Room is just one part of our holistic approach for safe frontier model training and deployment.
Show more