Register and share your invite link to earn from video plays and referrals.

Stephan Rabanser
@steverab
Postdoctoral Researcher @Princeton. Reliable, safe, trustworthy machine learning. Previously: @UofT @VectorInst @TU_Muenchen @Google @awscloud
394 Following    854 Followers
Last week we put out a paper on whether AI agents can do open-ended research. Within hours, Google's AI Overview reversed its answer to our main research question from a confident "Yes" to a confident "No." Our arXiv upload was enough to rewrite the authoritative summary. This is clearly a double-edged sword: ➡️On one hand, we want these AI systems to update their responses as new evidence comes in. If the research underpinning the update is solid, that's obviously promising. ➡️On the other hand, I was surprised that it only took one arXiv paper to completely flip the answer. This is concerning as it highlights how easily these responses could be manipulated with adversarial intent. A human expert would weigh one new (unreviewed) paper against the whole existing literature more carefully and update their take accordingly. We are of course bullish about our work and want to contribute high quality research to the current debate on recursive self-improvement. But we also think the final jury on what these models can do is still out.
Show more