Nathan Lambert
@natolambert
Have heard from many folks that score-centering is popular in industry (e.g. Tinker's StableReinforce using it), so it's the latest addition to the rlhf book. It's a nice derivation to understand training-inference-mismatch.
Video version explaining it soon. https://t.co/b3SJaElgSo
