Multi-head attention residuals. Uses multi-scale depthwise conv to expand dimensions.
Tencent is one of few companies that keep publishing papers on environment synthesis.
Aux loss free balancing -> Integral control
Quantile balancing -> Proportional control
This -> Integral-Derivative control.
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step ( Why does this work without training?
Show more
Improving exploration for rubric RL using OPSD to generate guidance for underexplored or penalized criteria. As combining criteria to construct rewards is common maybe it could be useful beyond rubric RL.
Show more
OPD for multi-turn scenario ( Teacher determines whether intervention is needed and how confident it is in the supervision, and these are used as weighting factors.
Show more
