가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Steven Adler
@sjgadler
Co-founder of Guidelight AI Standards ( ex-OpenAI safety researcher, writing at
가입 January 2018
1.1K 팔로잉 중    11.4K
Until AI companies have adopted actual preventative measures, I'm afraid we are prone to many more Hugging Face incidents. Two big things that still seem missing: 1) Do not allow AI to make **any** code-merges into your control system unless they've been affirmatively cleared as safe. - If you don't pre-clear these code changes, they can *turn off* your other controls. This can make any of your after-the-fact responses, like paging, totally ineffective. - You should be using a high-quality monitor with CoT access for this. It is not enough even if you have a human review every pull request; you want to check the transcripts for misalignment evidence, and they're way too long. 2) Make sure your agent automatically fails-closed in very serious cases. i.e., implement a circuit breaker for if your monitor is getting blitzed by a bunch of attacks. - Otherwise, the attacker gets a very high number of retries, and can attempt very harmful things, so long as it is confident in sneaking _one_ through before humans can respond. We describe more inside Principle Four of Guidelight's Control standard: "Stop the AI from taking harmful actions even if it tried", but these are IMO the two most important practices. I will have a hard time believing the industry has taken the recent incidents seriously, until these measures are in place.
더 보기