newer models same advantage
dpo beyond chatbots
smaller models better rejects preference distillation scaling
pro step step level process reward optimization for rag
va dpo valence arousal direct preference optimization for controllable emotion generation in language models
2606.19527