This is great work! And I finally get to live my dream of Schmidhubering, just once:
Using the average of rollouts from the same state as a baseline is exactly what I did in the original DDPO paper (which predates GRPO). But I thought it was boring so I put it in the Appendix :)
Show more
