註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

PatronusAI
@PatronusAI
Simulation research and infrastructure for human-aligned AGI
加入 July 2023
220 正在關注    2.7K 粉絲
We're excited to share that our paper on prompt infilling for dLMs has been accepted to the "Non-Autoregressive Language Models for Fast & Flexible Text Generation" workshop at COLM 2026 🎉 Masked diffusion language models like LLaDA and Dream have bidirectional architectures, yet they can't infer prompts from desired outputs out of the box. We traced this to a training gap: during SFT, prompts are simply never masked. The fix is straightforward. Mask both prompts and responses during SFT (full-sequence masking). That single change unlocks the ability to infer task-adapted prompts from few-shot examples. Prompt infilling in dLMs just requires the right training approach. We're calling on the community to release full-sequence masked SFT checkpoints alongside standard ones. Congrats to @akkikiki and Keisuke Sakaguchi! Paper: Fine-tuned models:
顯示更多