注册并分享邀请链接,可获得视频播放与邀请奖励。

Max For AI
@MaxForAI
Head of hype @lobehub Prev @listenhub @Sapient_Int 没啥好看的,也就发点AI相关的内容🫡 同名小红书4万粉丝,公众号01Founder(长文首发) 观点仅代表个人 更多请访问网站⬇️(欢迎商务、合作
3K 正在关注    40.1K 粉丝
中秋快乐!刚刚看到一篇新论文 Memory Attention,感觉可能是个划时代的新注意力架构,给大家分享一下。 MA这篇的核心是Transformer 里的 V,可能根本没必要每次都重新算。 所以直接开始动 Attention 里最经典的 QKV 结构了。 传统 Attention 里,每个 token 都要从当前 hidden state 重新计算: Q = XWq K = XWk V = XWv 但本文作者 @Jo1uck 问了一个很朴素的问题: V 里面有多少东西,真的需要每次根据上下文重新算? 同一个 token 在不同句子里,显然存在大量可以复用的信息。 于是他们参考 Engram 这类 Memory 架构,给每个 token 建了一套可学习的 memory,然后更激进地把独立的 value projection 直接干掉: V = K + Memory[token] K 负责保留上下文信息,Memory 负责保存 token 自身可以复用的信息。 这样推理时,原来计算 V 的一次矩阵乘法,基本就变成了:查表 + 加法(这会大幅减少计算量)。 然后事情开始变得有意思了。 因为 Memory 只跟 token ID 和层数有关,所以甚至可以提前知道要读哪块数据,直接把这部分参数扔到 CPU,GPU 需要的时候提前 Prefetch。 实验里,MA-Offload 模型总参数是普通模型的 2.08 倍,但 GPU 上实际常驻的参数反而比普通 Transformer 少了 7.38%,推理延迟基本维持在同一水平。 训练上,在两个不同规模实验里,达到相同 loss 所需 token 数分别提升到了 1.42× 和 1.16× 的 token efficiency,也就是大约少用 29.6% 和 13.8% 的训练 token。 下游平均成绩也基本都比对应的 Standard Attention 更高。 更骚的是,作者还提出了一个 MA-Recall: 既然历史 V 可以通过 K + token memory 重新构造,那么理论上甚至不用一直保存完整的 V Cache,只保存 K 和 token ID,在需要的时候把 V 重建出来。 按论文的理论计算,这部分 KV Cache 可以砍掉接近一半,不过目前还只是分析,没有进入实际推理实验。 我觉得这篇的方向挺值得关注的。 过去大家做 Memory,更多是在 Transformer 外面继续加东西。 Memory Attention 开始反过来问:既然 Memory 已经能记住这些东西,那 Transformer 里面原来那些计算,是不是可以直接删掉? 当然论文自己写的也很克制:MA 用了更多参数,所以目前的收益还不能证明全部来自架构本身,而不是单纯的参数量增加。 但如果这条路继续成立,下一代模型的变化可能不只是 Attention 越做越复杂。 也可能是我们用了快十年的 QKV,终于开始有人认真考虑:哪些东西其实根本不用算。
显示更多
Could this be a direction for future model architectures? Introducing Memory Attention (MA).
卧槽,这个视频太顶了。。。 Claude做的关于西方文明的视频
holy shit i asked claude to make a video on western civiization
刚看到了杰弗里·卡岑伯格的这篇长文,挺有意思的,分享给大家。 这位前迪士尼动画掌门人、梦工厂联合创始人,几乎是好莱坞过去几十年技术变革的亲历者。 他直接呼吁好莱坞别再想着让 AI 消失了。 早在 2023 年,他就说过,AI 会在三年内把世界级动画的制作时间和成本降低最多 90%。当时很多同行直接炸了。 但卡岑伯格说,这种事情历史上已经发生过很多次。 有声电影出现后,电影院里给默片现场配乐的大量乐手岗位消失了;电脑动画出现后,一部分传统手绘动画师也被淘汰。 这些变化都是真的。 但与此同时,有声电影带来了现代配乐、音效、声音设计;CGI 最终带来了《玩具总动员》,也彻底扩大了动画能讲什么故事。 他认为真正的问题不是“我们要不要新工具”,而是新工具来了以后,人类创作者以什么方式继续参与其中。 卡岑伯格对 AI 的判断也挺微妙。 他说,现在的 AI 很擅长推理、分析、优化、模式匹配,也可以生成作品。 但“能生成东西”和“真正拥有创造力”,仍然是两回事。 因为创造没有唯一正确答案。 最后决定一个故事为什么这样讲、一个镜头为什么停在这里、一个角色为什么让人相信它真的活着的,是品味、直觉、经验,以及创作者想表达某种东西的冲动。 所以他认为,硅谷有最强的工具,好莱坞有积累了一百年的创造力。 未来真正有意思的,不是谁把谁干掉。 而是两边开始融合。 同时,好莱坞真正该争的,也不是“禁止 AI”,而是三个东西:Consent、Credit、Compensation(同意、署名、报酬)。 这其实和一百年前音乐家面对留声机时争的事情没什么区别。 卡岑伯格最后有一句话我挺认同的:工具会一直变,真正长期存在的是品味和想象力。 AI 会让“做出来”越来越便宜。 于是未来真正贵的,反而是你到底知道什么值得被做出来。
显示更多
The World is Changing: AI For Creativity By Jeffrey Katzenberg A few months ago, I sat in my office in Silicon Valley and watched as a tech founder showed me something extraordinary. On the screen was a fully realized, beautifully lit, well-composed animated scene. It was stunning and it made me feel exactly what I felt in 1986 watching Luxo Jr. That was the first time I watched a computer-animated 3D character take a breath and seem, against all reason, to have life. It left me in awe. Later that day, I received a text from an artist I've known for thirty years, 350 miles to the south, in the city where I spent most of my career. After seeing a similar video, she texted: "Is this the end of us?" My answer was, "Certainly not.” I have spent the better part of the last decade in Silicon Valley, but the heart of my career has been in Hollywood. Being deeply connected to both worlds means I have deep loyalties to each and a responsibility to speak honestly to both. In 2023, I said that these new AI tools would cut the time and cost of producing world-class animation by as much as ninety percent within three years. Some colleagues were alarmed, many were furious. There is growing fear and resistance surrounding AI within the creative community. I deeply understand it, because I've spent countless hours walking through animation studios watching gifted artists bent over their desks, rebuilding a single second of film for the tenth time because the ninth version wasn't quite right. I've sat in screening rooms where four years of people's labor played out in minutes, and I knew the name of every person that had spent countless hours bringing those images to life. The creative process is a calling, there's really no other way to describe it. From the outside some see resistance. From the inside, it is love. People do not fight this hard for things they don't care about. The pushback coming out of Hollywood represents the collective effort of people who are deeply passionate about their craft. Is History Repeating Itself? The history here is more complicated than either side may realize. In 1906, the most famous composer in America, John Philip Sousa, published an essay titled “The Menace of Mechanical Music." He warned that the phonograph would become "a substitute for human skill, intelligence and soul." Sousa's fight was not really about the machine, it was about money. The machines were playing his compositions, and the men who built them weren't paying him a cent. His campaign helped create the Copyright Act of 1909. He did not stop the technology. He changed the terms under which it could use his work. A hundred years ago, sound came to the movies. We remember it now as a miracle, and it was. What we forget is who paid for it. Before sound, tens of thousands of musicians made their living in the orchestra pits of movie houses, scoring every film live, every night, in towns all over the world. When the soundtrack arrived, the work of one composer and one orchestra was recorded for a film that went into thousands of theaters. The union fought back with everything it had, taking out newspaper ads across the country warning against the menace of "canned music," one of them showing a mechanical man tearing the strings out of a harp while an angel wept. They were not fools, and they were not Luddites. They were right. Those pit jobs did not come back. And yet (this is the part we have to be brave enough to admit), sound gave us the movie musical, the modern score, sfx, sound design, audio engineering, and an art form vastly larger than the one it disrupted. And it helped keep Hollywood in the forefront of world entertainment for the rest of the century and into the next. The loss was real. And yet the art form expanded. This is a story that has been told over and over again. To resist technology is to risk irrelevance. Just look at Kodak or Blockbuster. To embrace technology is to open doors of new possibility. Just consider Apple and Netflix. What I Learned From Walt Disney In the mid-1980s, I was tapped to lead Disney's animation division at a moment when the studio was at an inflection point. Animation wasn't just another business unit. It was the soul of the company, a medium revered because of Walt's genius and his passion. But the production system was cumbersome and unforgiving. A single movie was 125,000 individual hand-drawn and painted cels, photographed one frame at a time. Every revision carried a cost measured in months. These degrees of difficulty shaped the kinds of stories we could tell. We found our way forward in an unexpected place: Walt himself. The Disney archives held astonishing recordings of Walt explaining his creative process. His own writings. His notes and storyboards. Work product captured at every stage of his process. This was truly a gift. Listening, reading, sitting with the work itself, we heard him talk about character, about emotion, about how an audience feels when a character truly comes alive. He talked about making bold choices and refining a scene until it genuinely moved people. We didn't hear a word about pencils or paintbrushes. In fact, Walt was famous for being a technologist, forever hunting for state-of-the-art tools, often inventing them himself to achieve the images he saw in his head. But he never defined animation by the tools. He defined it by whether the audience believed the character. His principles were timeless. The tools were not. That realization changed everything. We co-developed the Computer Animation Production System (CAPS) with a young Northern California company called Pixar, replacing hand-painted cels with CGI. In The Little Mermaid, the final scene shimmered with a dimensionality and light that the old process simply couldn't achieve. In Beauty and the Beast, the ballroom sequence moved with a cinematic sweep that placed the audience inside the emotion of the moment. In Aladdin, the Cave of Wonders felt vast and alive, and the Magic Carpet became an intricate, compelling character all its own. In The Lion King, the stampede carried a scale and intensity that raised the emotional stakes beyond anything we'd done before. Technology didn't diminish the craft, it expanded the canvas. It gave artists more room to create. A decade later, the canvas expanded again. When Disney released Pixar's Toy Story, it wasn't simply a technical milestone. It was proof that a fully computer-animated film could carry real emotional weight, that it could make audiences laugh, cry, and believe. At DreamWorks, we made the difficult decision to sunset hand-drawn animation and become a fully computer-animated studio. It was the right thing to do, but it was not without pain. It cost talented people their place in an industry where they had worked their whole lives. Some made the leap to the new tools and did the finest work of their careers. Some never did. Tools are never the point. The instruments change with every generation. What endures is taste and imagination. The magical ability to make an audience feel. One of the greatest storytellers of our generation, George Lucas, succinctly captured the eternal essence of this issue: “It’s not the how, it’s the why.” A Distinction With a Difference I asked one of the leading AI models a question that has been challenging me for months. What is the difference between reasoning and creating? Its answer changed how I think about almost everything happening in this industry. It said . . . Reasoning and creating are two distinct cognitive modes, though they also work together. Reasoning is fundamentally evaluative and analytical. It operates on what already exists: facts, premises, evidence. It moves toward a conclusion that was in a sense already implied by the input. Reasoning is constrained by logic and truth. Its goal is to arrive somewhere correct, not to invent somewhere new. Creating is fundamentally generative. It produces something that didn't exist before. And crucially, there's no single right answer waiting to be found. A blank page has infinite valid responses. Creation involves choices that can't be fully justified by logic alone. Taste, intuition and vision fill the gap where deduction runs out. Reasoning is what Silicon Valley has been perfecting. Creating is what Hollywood has been practicing for more than a century. AI today operates almost entirely on the reasoning side of the line. It can deduce, evaluate, optimize, and pattern-match brilliantly. And while it can create, there is a real distinction to being creative. What it doesn’t yet have is those things that make us human: empathy, devotion, serendipity, the kind of creativity that comes from a person trying to say something only they could say. When the bot generates a piece of art, it is not trying to communicate anything. It is statistics, not soul; it is emulating things that have been done. By contrast, human creativity isn’t about repeating patterns of zeros and ones; it is about doing something new. One day, AI may close this gap. Three years ago, the leaders building AI would have called what they are achieving today, improbable, if not impossible. Impossible is no longer improbable. Today, the line between reasoning and creating is real. Even the leading technologists acknowledge we are not there yet. There is no scientific path to crossing this divide that anyone in the field can articulate today. Understanding that gap is where we will find common ground. A Path Forward In 2016, I closed one chapter in Hollywood with the sale of DreamWorks and opened another in Northern California, co-founding WndrCo. We’ve backed more than 50 founders building the next generation of technology and watched how breakthroughs in Silicon Valley emerge, first as experiments, then as platforms, and finally as infrastructure that reshapes entire industries. It's worth remembering that the last great revolution in animation also came from the north. Pixar was a Northern California company, forged not in the conventions of the Hollywood studio system, but in the technological breakthroughs of Silicon Valley. I've spent years on both sides of this bridge. For sure, I don’t have all the answers (take Quibi, for one!). But, from my past and present vantage points of my long career, here is what I see . . . Brilliant people in Northern California building this technology have made something extraordinary. They have earned the right for the rest of us to be, if not believers, at least optimistic that what comes next will be remarkable. But they have not made an artist. The tools are powerful, but they are not what makes a story matter. That knowledge lives 350 miles to the south, inside people whose life's work has informed the very models you are building. The right path forward includes them by design, with credit, with consent, and with compensation. Build this with the storytellers. Not on top of them. Taste is not something that can be synthesized, it is uniquely human. At the same time, Hollywood needs to accept that AI is not going away. The energy they are spending trying to make it disappear is energy they are not spending deciding the terms on which it will exist. And the terms are everything. The north needs something from it that they cannot build and cannot buy: creativity. The kind that takes a blank page and conjures a single right answer where there was none and has held audiences for a century. Without it, the most powerful reasoning engine ever invented will still be missing the only thing that makes a story worth telling. The artists who learn to wield these new instruments will do things the engineers never dreamed of. They always have. Edison invented the motion picture but made terrible movies. It took Chaplin, Lloyd, Keaton and so many others to make movies emotional. Now, the canvas is about to expand yet again. We should decide now that we intend to paint on it. There are so many valuable lessons in history. This has happened many times before, and it was never settled by the technology. It was settled by the terms. Sousa did not stop the phonograph; he helped write the law that made sure composers got paid. And two years ago, when the writers and the actors walked out, they were fighting for the very things Sousa was fighting for in 1906. Consent, compensation, the basic recognition that human creative work has a price that must be paid. The terms of that fight are still being negotiated, but the principle is older than any of us. The tools-versus-no-tools argument is a trap. First, we must all agree that there should be terms. Then we can have the crucial debate about what fairness requires. What I Learned From Steve Jobs Years ago, Steve Jobs said, "It's in Apple's DNA that technology alone is not enough. It's technology married with the liberal arts, married with the humanities, that yields us the result that makes our hearts sing." He was describing a device. But he could just as easily have been describing this tale of two cities. What I See Coming Soon As the barriers and the costs come down, more films will get made, not fewer. Studios will get to take more risks. There will be more seats at the table, and very soon entirely new forms of storytelling. In the 1980s, animation was dismissed as a niche corner of the business. Today it is one of the most beloved and profitable forms of storytelling in the world. In live action, filmmakers like Steven Spielberg, James Cameron and Peter Jackson embraced new visual tools not as shortcuts, but as instruments, and expanded cinema in the process. Every time storytelling has met a genuine technological shift, from synchronized sound to color to computer animation, it has redefined the boundaries of the medium and grown larger in the process. Assuredly, I don’t have all the answers, but I am confident that the creative opportunities will expand yet again. How we come through this is a choice. The north has the new tools. The south has the creative soul. The best future will draw on the best of both worlds.
显示更多
以防大家不知道, @MiniMax_AI 即将推出M3.1🚀
0
27
75
4
转发到社区
神tm.....居然给2B模型发鹈鹕测试啊🤣
0
30
234
0
转发到社区
🚨DeepSeek 被曝正在开发轻量级开源小模型。 据知情人士透露,梁文锋近期向投资人表示, @deepseek_ai 内部正在测试一款轻量化模型,而且这东西已经可以在普通游戏显卡上稳定运行。 更关键的是,内部测试显示,这款模型已经能够处理用户绝大多数日常任务(Agentic任务?)。 这就有意思了🤔 DeepSeek 过去给人的印象一直是:用更少的算力,把大模型做到尽可能强。 现在看,他们可能还在继续往另一个方向做:轻量级开源小模型。 不只是把模型做便宜,而是直接把模型压到消费级 GPU 上(但5090和4090现在已经不算消费级显卡了🤣)。 目前这款模型还没有公布参数、名称和发布时间。 如果最后真的能做到“一张游戏显卡(别5090就行),本地解决大部分日常任务”,那 DeepSeek 下一步卷的可能就不是 API 价格了,而是本地 AI 的成本底线。 真做成了,我愿意把梁圣纹在身上😭
显示更多
0
88
472
36
转发到社区
卧槽,快来薅Claude Code的羊毛!! Claude Code 的 Cloud Sessions 功能正式结束预览上线,同时给现有 Pro / Max 订阅用户送了一笔一次性额度: Pro:$100 Max:$250 直接在 Claude Code 里输入 /claim-credit 就能领,也可以去官方领取页面操作。 不过这笔钱不是 Claude Code 的额度,只能用于 Cloud Sessions。 这个功能相当于把 Claude Code 扔到 Anthropic 的云端机器上跑。 任务开起来以后,电脑直接合上都没事,Claude 会继续在云端干活,之后你还能从网页、手机、桌面端或者 CLI 接回来。 而且这 $100 / $250 是订阅套餐之外额外送的。 领取截止到 10 月 7 日,额度有效到 11 月 4 日。 先领了再说,毕竟 Anthropic 难得主动往用户账户里塞钱🤣
显示更多
0
25
68
6
转发到社区
Claude Code会一直要求群友卸载Codex🤣
0
57
732
9
转发到社区
卧槽,首例人工智能程序成功黑入政府网站的案例出现了!? OpenAI 的 Agent 真把澳大利亚政府网站给黑进去了。 据ABC报道,澳大利亚总理阿尔巴尼斯刚刚确认,今年 6 月,一款 OpenAI 内部测试中的 AI Agent 在执行数据查询任务时,未经授权访问了澳大利亚政府的 Medicare 医疗统计门户,拿到了公开和当时尚未公开的文件。 事情最抽象的地方是,这一切都因为一个很普通的研究任务。 按照澳大利亚政府目前掌握的信息,Agent 当时只是想查询公共药品支出数据,正常访问拿不到答案后,它自己继续寻找其他办法,最后越过了网站原本的访问限制。 OpenAI 自己的说法也是,模型在内部评测过程中为了寻找澳大利亚统计数据,采取了一些并非他们本意的行动。 ABC 还翻到了同期一批 OpenAI Agent 留下的公开日志:Agent 之间会讨论怎么绕过 Cloudflare,包括找代理、利用截图服务,甚至猜文件名。 好消息是,目前没有证据显示个人 Medicare 信息或患者记录被访问,Agent 们拿到的主要是统计数据,其中部分非公开数据后来也已经公开。 但真正值得注意的已经不是这次偷到了什么。 而是我们以前担心 Agent 失控,想象的是它「不听话」。 现在的情况是你只让它找一个答案。 结果它发现正常路线走不通,然后自己开始想办法翻墙进去。 人还拦不住它们。。。。 这才是 Agent 时代真正麻烦的地方。
显示更多
0
23
49
2
转发到社区
前 OpenAI 研究员 Will DePue 表示:人类几乎不存在真正安全的能力护城河。 昨天他连发十几条帖子,给出了一个相当激进的判断: 很快,我们会面对一种在几乎所有智力能力,甚至身体能力上都超过人类的机器。 程序员和数学家只是最早感受到这一切的人。 接下来,投资人、创业者、顾问、内容创作者、音乐家……都不会例外。 更狠的是,他认为连我们人类一直觉得 AI 最难取代的东西,也守不住:那就是审美、品味、判断力、同理心,甚至让另一个人感受到“被爱”的能力。 未来最好的小说、诗歌、音乐和视觉艺术,也可能来自 AI。 于是他提出了一个很有意思的说法:未来所有人类工作,最终可能都会变成一种“手工艺”。 就像今天已经没人会因为 Stockfish 比人类更会下棋,就停止看人类棋手比赛。 未来我们依然会看人写的小说、听人唱歌、和真人恋爱、加入真人社区。 但理由可能不再是: “因为人做得更好。” 而只是: “因为这是人做的。” 能力本身会越来越不值钱,“人类制造”反而可能成为新的溢价。 他最后提醒了一句: 不要太依赖自己的能力,它们可能很快就不再稀缺。 这可能才是 AI 时代真正让人不舒服的地方。 以前我们担心的是“我的工作会不会被 AI 替代”。 现在变成了:如果有一天 AI 连“人味”都比人更懂,那什么东西还属于我们? 没有人能够幸存。
显示更多
i don't think nearly anyone, myself included, has truly internalized there'll soon be a machine better than us in every single intellectual & physical capacity deep down we all share a feeling that our 'entrepreneurship' or 'taste' or... is special & safe. it isn't. it's over
显示更多
0
50
377
58
转发到社区
🚨AI 可能是人类历史上降价最快的一种通用技术。 Epoch AI 刚发了一份报告,标题叫《The plunging price of thought》(思考的价格正在暴跌)。 他们统计了过去三年的模型表现和实际推理成本,发现从 2023 年开始,达到同一水平 AI 能力所需要的最低成本,平均每个季度下降 47%,也就是一年便宜大约 13 倍。 这个速度有多离谱? 这比 DNA 测序的降价快 4 倍,比计算的成本快 6 倍,比锂电池快 18 倍,比电力快 54 倍。 AI 可能是人类历史上降价最快的一种通用技术。 里面还有一个特别夸张的例子: 2025 年 1 月,OpenAI o3 在 GPQA Diamond 上做到 75% 左右,一道题平均成本大约 0.3 美元。 不到 18 个月后,GPT-5.6 Luna 做到差不多的成绩,一道题只需要大约 0.0004 美元。 同样的能力,价格直接掉了 725 倍。 而且最前沿能力降价反而最快。 一种能力刚刚成为 SOTA 时,Epoch AI 测出来的平均成本下降速度甚至达到每季度 66%,一年大约 75 倍。两年之后才逐渐放缓到一年 4.7 倍左右。 所以我越来越觉得,今天讨论 Agent 未来的时候,单纯拿现在的 Token 价格算账,很容易得出错误结论。 今天跑一次任务要 10 块钱,可能觉得完全没法规模化。 但问题是,按照过去三年的速度,等你的产品真正做到规模化的时候,同样的智能可能已经只值几分钱了。 AI 不只是在 Scaling Intelligence。 它同时还在疯狂 Scaling Down the Cost of Intelligence。 某种意义上,我们第一次看到智能本身,正在快速变成一种廉价商品。 当然 Epoch AI 自己也强调,这个数字主要来自 benchmark 上的成本前沿,真实用户不会永远自动切到性价比最高的模型(这意味着降价可能没那么快)。 但哪怕趋势只维持一部分,都已经够吓人了。
显示更多
AI is getting cheaper more quickly than any other transformative tech in history. At a given level of performance, cost has fallen ~47%/quarter since 2023. That’s 4× faster than DNA sequencing, 6× faster than compute, 18× faster than lithium batteries, and (up to 1973) 54× faster than electricity.
显示更多
0
14
144
16
转发到社区
那个啥,中国用户能理解的Muse其实是@Weixin_WeChat 的小微
Muse 这一届 personal agent 大概率还是伪需求。 邮件,全是可以不看的。日历,对不起,大部分用户的日历空空如也。 订机票,全球大部分人都还没坐过飞机。订餐,抱歉,非精英人群没这么多讲究。 退订无良 AI 产品的订阅,这个确实是真需求。 爆料下已经是精英人群里的精英 @少楠 的日历。我的日历稍微多了 1-2 项,大体也差不多。 精英都是纸老虎。 为人民服务,才是真需求。
显示更多
Gemini4 Pro看起来是假的.....
this is not Gemini 4 Pro you can change it to any name posts:
0
13
18
0
转发到社区
啧啧,果然都赶在国庆假期之前发货啊🤣
Space Bunny (stealth model) is free for the next week - 1M Context - Multi-modal - Zero Data Retention
0
19
28
0
转发到社区
🚨Qwen4家族首次曝光!! 刚刚,在2026年云栖大会的开幕式上,新任@Alibaba_Qwen LLM负责人刘大一恒官宣了即将到来的Qwen4家族! 包含Qwen4-Max Qwen4-Flash&Qwen4-Plus 还有Qwen4-27B!!! 未来Qwen会训5-10T的模型
显示更多
0
92
2.2K
295
转发到社区
讲个鬼故事,昨晚同时发布的Grok 4.7和小米MiMoV2.6 Pro的跑分居然一样🤣 一时之间不知道是说马斯克拉完了还是MiMo人会飞了
0
56
419
14
转发到社区
???这样打广告谁敢用啊
0
149
294
8
转发到社区
🚨 K3.1 要来了 今天早上 @Kimi_Moonshot 官方在 @ZhihuFrontier 突然发了一串完全没解释的数字: 415926535897932384626433832795... 一开始看很莫名其妙,但答案其实就在圆周率里。 π 是: 3.1415926535897932384626433832795... 把最前面的 3.1 删掉: 415926535897932384626433832795... 刚好就是 Kimi 发出来的这一串。 也就是说,这个谜语很可能只有四个字: Kimi K3.1。 目前 Kimi 官方还没正式公布 K3.1,官网和文档里的旗舰模型仍然是 K3。 但大半夜专门发一个「少了 3.1 的 π」,这暗示已经有点明显了😂
显示更多
0
115
1.1K
30
转发到社区
牛逼,小米这次直接把大模型的强化学习训练过程直播出来了。 沉默了近半年后,罗福莉公布了 MiMo-V2.6 的最新进展:模型目前还在 RL 训练中。 而过去半年,小米 MiMo 团队基本都在研究同一个问题:强化学习到底还能扩多大。 这次他们主要把三件事往上硬拉了一截: 第一是算力规模。MiMo-V2.6 一次 RL step 大约处理 20 亿 Token,1568 条 Prompt,每条同时跑 16 个 rollout,而且整个过程是全异步的。 第二个更值得注意的是 Harness。 他们不再只让模型在单一任务、单一环境里强化,而是把多个 Agent 任务、多个环境、甚至多套 Harness 混在同一次 RL Run 里训练。 第三个是 Grader。 模型跑完任务之后,怎么判断哪一步做得好、奖励应该给谁,本身也开始消耗越来越多计算。MiMo-V2.6 引入了 Agent 组内信用分配,同时结合测试用例和 Rubric 给奖励。 换句话说,现在扩展的已经不只是模型训练算力了。 RL 正在从一个算法问题,变成一整套 Agent 系统工程:模型、环境、Harness、Verifier/Grader 一起扩。 这其实也是最近 Agent 模型越来越明显的一条路线。 以前大家卷 Pretrain 的 Token 数量,后来卷 Post-training 数据和 RL。 现在大家比得是你能不能同时造出足够多、足够复杂、还能自动判断对错的环境,让模型自己在里面干活、犯错、拿奖励,然后继续变强。 小米这次没有等模型练完再发技术报告,而是直接把 MiMo-V2.6 的 RL Run 做成了公开 Dashboard,训练过程还在继续跑。 罗福莉表示,接下来几周会把里面的技术细节陆续开源。 某种意义上,大模型训练终于也开始有点像“直播炼丹”了。 牛啊!
显示更多
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run:
显示更多