๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Louis-Franรงois Bouchard ๐ŸŽฅ๐Ÿค–
@Whats_AI
Training AI Engineers on YouTube, Substack and our courses. Co-founder @towards_ai. Ex-Ph.D. student @Mila_Quebec. On the road to 100k on YT this year ๐Ÿ‘‡
๊ฐ€์ž… January 2019
6.2K ํŒ”๋กœ์ž‰ ์ค‘    11.4K ํŒฌ
We tested 146 model variants on 10 script-writing tasks, 5 runs each, scored by a 3-judge panel on tone, voice, and whether a human would actually like it enough to read it out loud without cringing. Our goal is to measure creative writing quality. Not agentic or coding performance, or even one-shot generation performance. We provide a very detailed prompt with all the info the model needs to do its job (in theory). Here are some insights from the main family of models with GPT, Fable, Gemini, Muse... (see image for more info): 1. If you are using sol 5.6, skip the ultra setting. It doubles the bill from $0.21 to $0.43 per task and buys only 66 Elo. 2. Extra thinking is worth paying for on the small models, not the big ones. Luna gains 230 Elo for 3ร— the cost. Sol gains 66 for 2ร—. Grok 4.6 and Gemini 3.1 Pro get worse when you turn it up. 3. Claude Opus 5 is the value pick of the whole chart. Max effort is $0.42 a task, and it out-writes every GPT config on the board โ€” including GPT-6 Astra, which costs 8ร— more. I know this one is controversial, but I also see it using it personally. And that's what you should trust in any case: you own usage, not a benchmark. 4. Never pay for Sonnet 5's max effort. It burns $2.96 a task, 7ร— what Opus 5 max costs, and writes 216 Elo worse. 5. GPT-6 Astra is the most expensive way to be mid. $3.42 a task at max effort, 246 Elo behind Opus 5 max. 6. Meta Muse Spark 1.3 is the budget outlier: $0.034 a task, and it out-writes every Gemini 3.1 Pro config โ€” which costs 4ร— more. 7. Claude Fable 5 still tops the board. $4.36 a task at max effort: 10ร— Opus 5 max for 72 Elo. Pay it only if the words are the product or you still have your subscription tokens. 8. Claude Fable 5.1 is not a writing upgrade. At max effort it costs 25% more than Fable 5 ($3.15 vs $2.52) and scores 6 Elo lower, which is inside the noise band.
๋” ๋ณด๊ธฐ