注册并分享邀请链接,可获得视频播放与邀请奖励。

Simo Ryu
@cloneofsimo
I like cats, math and codes cloneofsimo@gmail.com
加入 May 2022
1K 正在关注    17.7K 粉丝
This is one case of the idea of superhuman intelligence where we get there by having dumber humans
Every single review I have handled at NeurIPS as an AC is **much worse** than a well executed AI review. By "much worse" I do not mean 20% or 50%. I mean a factor of about 10-100. At least - 10x more mathematical issues are caught, - 10x more missing key citations are caught, - 10x more novelty claims are invalidated with evidence, - 10x more issues with the experiments are observed, - 10x more typos and grammatical issues are fixed, - 10x more inconsistencies are found, and so on. In many cases I have handled over the last 10 years, the human reviews are so bad in comparison that the improvement factor is closer to 100. (The best human reviews I received over the years are worse than an AI review I can generate today, but still entirely good enough to make the correct decision. On the other hand, an AI review is often an almost comprehensive summary of all key issues --- something a human almost never has time to deliver --- and such feedback is immensely useful to the authors) At this point, we would be much better to just 1) give one very thorough ChatGPT review to all submitted papers (I am mainly talking about my fields - optimization, AI, machine learning), automatically, and 2) keep asking for a revision or two (to keep the duration of the process within some bounds) until the number of issues decreases to an extent when the AI reviewer is satisfied, in a given time-frame (eg, 1 month). 3) At that point, a human AC can make a decision (to keep an eye on this all should anything go wrong). The role of the AC would be merely to observe and manage the process, and make a final decision, based on the trajectory of the revisions. I can't believe I am saying this -- AI reviews were a nonsense idea even a year ago. The current AI reviews are super-human.
显示更多