Register and share your invite link to earn from video plays and referrals.

葦沢かもめ
@AshizawaKamome
AI小説で世界で初めて文学賞|AIクリエイター&エンジニア|小説家|生成AI×創作|著書『あしざわ式 小説を書く人のAI実践術』『小説を書く人のAI活用術』|第9回日経「星新一賞」優秀賞|日本SF作家クラブ会員|慶應SFセンター研究員|お問合せ→
2.2K Following    5K Followers
New Anthropic research: Automating subtle emergent misalignment research. Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor. These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior. Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute. Read the full post on the Alignment Science Blog:
Show more
0
710
4.2K
393
Forward to community