Contact and support
Need help, have a question, or want to contact the ResearchHub team?
© 2026 ResearchHub. Built for responsible scholarly connection.
Privacy-aware community infrastructureResearch graph
References from A Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications. Local targets link to admitted publications; unresolved targets remain external evidence.
Introducing ChatGPT
2022 · External reference
Llama: Open and efficient foundation language models
2023 · External reference
GPT-4 technical report
2023 · External reference
Mistral 7b
2023 · External reference
GPT-4V(ision) system card
2023 · External reference
10.52202/075280-1516
10.52202/075280-1516 · External reference
Gemini: A family of highly capable multimodal models
2023 · External reference
Qwen-Vl: A versatile vision-language model for understanding, localization, text reading, and beyond
2023 · External reference
Scalable agent alignment via reward modeling: A research direction
2018 · External reference
A general language assistant as a laboratory for alignment
2021 · External reference
Introducing superalignment
2023 · External reference
10.52202/068431-2011
10.52202/068431-2011 · External reference
Safe RLHF: Safe reinforcement learning from human feedback
2024 · External reference
10.18653/v1/2024.findings-acl.775
10.18653/v1/2024.findings-acl.775 · External reference
Proximal policy optimization algorithms
2017 · External reference
Open problems and fundamental limitations of reinforcement learning from human feedback
2023 · External reference
The effects of reward misspecification: Mapping and mitigating misaligned models
2022 · External reference
Causal confusion and reward misidentification in preference-based reward learning
2023 · External reference
10.52202/075280-0482
10.52202/075280-0482 · External reference
RAFT: Reward ranked fine tuning for generative foundation model alignment
2023 · External reference
Statistical rejection sampling improves preference optimization
2024 · External reference
10.1609/aaai.v38i17.29865
10.1609/aaai.v38i17.29865 · External reference
Direct preference optimization: Your language model is secretly a reward model
2024 · External reference
10.2307/2334029
10.2307/2334029 · External reference
Model alignment as prospect theoretic optimization
2024 · External reference
10.52202/079017-1154
10.52202/079017-1154 · External reference
Is DPO superior to PPO for LLM alignment? A comprehensive study
2025 · External reference
Insights into alignment: Evaluating DPO and its variants across multiple tasks
2024 · External reference
10.18653/v1/2024.emnlp-main.35
10.18653/v1/2024.emnlp-main.35 · External reference
Online merging optimizers for boosting rewards and mitigating tax in alignment
2024 · External reference
A general theoretical paradigm to understand learning from human preferences
2024 · External reference
Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation
2024 · External reference
10.18653/v1/2024.emnlp-main.626
10.18653/v1/2024.emnlp-main.626 · External reference
10.52202/079017-3946
10.52202/079017-3946 · External reference
Step-controlled DPO: Leveraging stepwise error for enhanced mathematical reasoning
2024 · External reference
10.1609/aaai.v39i24.34744
10.1609/aaai.v39i24.34744 · External reference
Token-level direct preference optimization
2024 · External reference
10.1145/3770749
10.1145/3770749 · External reference
Aligning large language models with human: A survey
2023 · External reference
Large language model alignment: A survey
2023 · External reference
10.18653/v1/2024.findings-emnlp.940
10.18653/v1/2024.findings-emnlp.940 · External reference
Policy optimization in RLHF: The impact of out-of-preference data
2023 · External reference
10.52202/079017-1990
10.52202/079017-1990 · External reference
10.1007/978-3-031-70362-1_7
10.1007/978-3-031-70362-1_7 · External reference
10.18653/v1/2025.findings-naacl.447
10.18653/v1/2025.findings-naacl.447 · External reference
Towards analyzing and understanding the limitations of DPO: A theoretical perspective
2024 · External reference
From $r$r to $Q^{*}$Q*: Your language model is secretly a Q-function
2024 · External reference
Self-play fine-tuning converts weak language models to strong language models
2024 · External reference
Understanding the performance gap between online and offline alignment algorithms
2024 · External reference
Offline RLHF methods need more accurate supervision signals
2024 · External reference
Some things are more cringe than others: Iterative preference optimization with the pairwise cringe loss
2024 · External reference
Direct language model alignment from online AI feedback
2024 · External reference
Self-rewarding language models
2024 · External reference
Optune: Efficient online preference tuning
2024 · External reference
10.52202/075280-1308
10.52202/075280-1308 · External reference
A long way to go: Investigating length correlations in RLHF
2024 · External reference
10.1145/3613904.3642596
10.1145/3613904.3642596 · External reference
10.52202/075280-3268
10.52202/075280-3268 · External reference
10.18653/v1/2024.findings-acl.297
10.18653/v1/2024.findings-acl.297 · External reference
10.18653/v1/2025.emnlp-main.1233
10.18653/v1/2025.emnlp-main.1233 · External reference
Length desensitization in directed preference optimization
2024 · External reference
10.1609/aaai.v39i26.34963
10.1609/aaai.v39i26.34963 · External reference
10.18653/v1/2024.emnlp-main.85
10.18653/v1/2024.emnlp-main.85 · External reference
Learning to reason with LLMs
2024 · External reference
Chameleon: Mixed-modal early-fusion foundation models
2024 · External reference
Value-incentivized preference optimization: A unified approach to online and offline RLHF
2025 · External reference
Advancing LLM reasoning generalists with preference trees
2025 · External reference
SDPO: Don’t use your data all at once
2025 · External reference
Step-DPO: Step-wise preference optimization for long-chain reasoning of LLMs
2024 · External reference
Building math agents with multi-turn iterative preference learning
2025 · External reference
10.52202/079017-3702
10.52202/079017-3702 · External reference
10.52202/079017-4128
10.52202/079017-4128 · External reference
10.52202/085713-5605
10.52202/085713-5605 · External reference
Learn your reference model for real good alignment
2025 · External reference
Learning dynamics of LLM finetuning
2025 · External reference
Online DPO: Online direct preference optimization with fast-slow chasing
2024 · External reference
Hybrid preference optimization for alignment: Provably faster convergence rates by combining offline preferences with online exploration
2025 · External reference
Sail into the headwind: Alignment via robust rewards and dynamic labels against reward hacking
2025 · External reference
10.18653/v1/2024.emnlp-main.1150
10.18653/v1/2024.emnlp-main.1150 · External reference
PAFT: A parallel training paradigm for effective LLM fine-tuning
2024 · External reference
10.18653/v1/2024.findings-acl.175
10.18653/v1/2024.findings-acl.175 · External reference
10.18653/v1/2024.findings-acl.630
10.18653/v1/2024.findings-acl.630 · External reference
Restoring calibration for aligned large language models: A calibration-aware fine-tuning approach
2025 · External reference
Learning to summarize with human feedback
2020 · External reference
10.63317/2h4pkr685mkq
10.63317/2h4pkr685mkq · External reference
WebGPT: Browser-assisted question-answering with human feedback
2022 · External reference
10.18653/v1/2024.naacl-long.185
10.18653/v1/2024.naacl-long.185 · External reference
10.52202/075280-2064
10.52202/075280-2064 · External reference
Understanding dataset difficulty with $\mathcal {V}$V-usable information
2022 · External reference
10.52202/079017-0047
10.52202/079017-0047 · External reference
10.1109/cvpr52733.2024.01310
10.1109/cvpr52733.2024.01310 · External reference
Training a helpful and harmless assistant with reinforcement learning from human feedback
2022 · External reference
10.52202/075280-2020
10.52202/075280-2020 · External reference
10.52202/085713-1448
10.52202/085713-1448 · External reference
MM-RLHF: The next step forward in multimodal LLM alignment
2025 · External reference
Towards harmless multimodal assistants with blind
2025 · External reference
10.1109/cvpr52734.2025.01861
10.1109/cvpr52734.2025.01861 · External reference
Ultrafeedback: Boosting language models with scaled AI feedback
2024 · External reference
Distilabel-capybara-dpo-7k-binarized
2024 · External reference
ChatML DPO pairs
2023 · External reference
10.18653/v1/2025.findings-naacl.96
10.18653/v1/2025.findings-naacl.96 · External reference
Magpie: Alignment data synthesis from scratch by prompting aligned LLMs with nothing
2025 · External reference
Zsql-postgres-dpo
2024 · External reference
Starling-7B: Improving helpfulness and harmlessness with RLAIF
2024 · External reference
SlimOrca: An open dataset of GPT-4 augmented FLAN reasoning traces, with verification
2023 · External reference
10.1109/cvpr52734.2025.01850
10.1109/cvpr52734.2025.01850 · External reference
10.18653/v1/2024.emnlp-main.358
10.18653/v1/2024.emnlp-main.358 · External reference
Truthy DPO
2024 · External reference
Orca: Progressive learning from complex explanation traces of GPT-4
2023 · External reference
Dataset-tldr-preference-dpo
2024 · External reference
Rewardeval: Advancing reward model evaluation
2026 · External reference
10.1109/cvpr52734.2025.02296
10.1109/cvpr52734.2025.02296 · External reference
CodeSteer: Symbolic-augmented language models via code/text guidance
2025 · External reference
Mask-DPO: Generalizable fine-grained factuality alignment of LLMs
2025 · External reference
10.52202/085713-5315
10.52202/085713-5315 · External reference
10.1109/iccv51701.2025.00110
10.1109/iccv51701.2025.00110 · External reference
10.18653/v1/2024.acl-long.643
10.18653/v1/2024.acl-long.643 · External reference
10.18653/v1/2024.acl-long.539
10.18653/v1/2024.acl-long.539 · External reference
Self-Play with execution feedback: Improving instruction-following capabilities of large language models
2025 · External reference
10.1145/3746252.3761322
10.1145/3746252.3761322 · External reference
Conifer: Improving complex constrained instruction-following ability of large language models
2024 · External reference
FIPO: Free-form instruction-oriented prompt optimization with preference dataset and modular fine-tuning schema
2025 · External reference
10.18653/v1/2024.findings-naacl.108
10.18653/v1/2024.findings-naacl.108 · External reference
10.52202/079017-3671
10.52202/079017-3671 · External reference
Halu-j: Critique-based hallucination judge
2024 · External reference
Fine-grained hallucination detection and mitigation in long-form question answering
2025 · External reference
Fine-tuning language models for factuality
2024 · External reference
10.18653/v1/2025.findings-naacl.5
10.18653/v1/2025.findings-naacl.5 · External reference
Stable code technical report
2024 · External reference
10.1109/tpds.2026.3675550
10.1109/tpds.2026.3675550 · External reference
10.1007/978-981-97-5669-8_11
10.1007/978-981-97-5669-8_11 · External reference
Aligning modalities in vision large language models via preference fine-tuning
2024 · External reference
10.18653/v1/2025.naacl-long.30
10.18653/v1/2025.naacl-long.30 · External reference
10.1007/978-3-031-72684-2_9
10.1007/978-3-031-72684-2_9 · External reference
10.1109/cvpr52733.2024.00786
10.1109/cvpr52733.2024.00786 · External reference
10.1109/cvpr52733.2024.00854
10.1109/cvpr52733.2024.00854 · External reference
10.1145/3664647.3681688
10.1145/3664647.3681688 · External reference
10.1109/icassp49660.2025.10888737
10.1109/icassp49660.2025.10888737 · External reference
10.52202/079017-1594
10.52202/079017-1594 · External reference
10.18653/v1/2024.emnlp-main.595
10.18653/v1/2024.emnlp-main.595 · External reference
10.1109/cvprw63382.2024.00195
10.1109/cvprw63382.2024.00195 · External reference
10.18653/v1/2024.emnlp-main.138
10.18653/v1/2024.emnlp-main.138 · External reference
10.18653/v1/2025.acl-long.607
10.18653/v1/2025.acl-long.607 · External reference
Solving the granularity mismatch: Hierarchical preference learning for long-horizon LLM agents
2026 · External reference
10.18653/v1/2025.acl-long.542
10.18653/v1/2025.acl-long.542 · External reference
10.18653/v1/2025.emnlp-main.484
10.18653/v1/2025.emnlp-main.484 · External reference
10.52202/085713-1235
10.52202/085713-1235 · External reference
10.52202/085713-2029
10.52202/085713-2029 · External reference
10.52202/079017-0863
10.52202/079017-0863 · External reference
Finetuning large language model for personalized ranking
2024 · External reference
10.3390/pharmaceutics16081014
10.3390/pharmaceutics16081014 · External reference
10.1109/cvprw63382.2024.00517
10.1109/cvprw63382.2024.00517 · External reference
Aligning protein generative models with experimental fitness via direct preference optimization
10.1101/2024.05.20.595026 · 2024 · External reference
10.2307/2334029
10.2307/2334029 · External reference
Token-importance guided direct preference optimization
2026 · External reference
ActiveDPO: Active direct preference optimization for sample-efficient alignment
2026 · External reference
10.52202/085713-2362
10.52202/085713-2362 · External reference
Critical tokens matter: Token-level contrastive estimation enhances llm’s reasoning capability
2025 · External reference
Exploratory preference optimization: Harnessing implicit Q*-approximation for sample-efficient RLHF
2025 · External reference
3D-properties: Identifying challenges in DPO and charting a path forward
2025 · External reference
10.52202/085713-2362
10.52202/085713-2362 · External reference
10.1007/bf00122574
10.1007/bf00122574 · External reference
Smaug: Fixing failure modes of preference optimisation with DPO-positive
2024 · External reference
Self-play preference optimization for language model alignment
2025 · External reference
Direct nash optimization: teaching language models to self-improve with general preferences
2024 · External reference
Nash learning from human feedback
2024 · External reference
AlphaDPO: Adaptive reward margin for direct preference optimization
2025 · External reference
10.18653/v1/2023.acl-long.493
10.18653/v1/2023.acl-long.493 · External reference
The crucial role of samplers in online direct preference optimization
2025 · External reference
Trust region preference approximation: A simple and stable reinforcement learning algorithm for LLM reasoning
2025 · External reference
Evolving LLMs’ self-refinement capability via synergistic training-inference optimization
2025 · External reference
Sight beyond text: Multi-modal training enhances llms in truthfulness and ethics
2024 · External reference
The claude 3 model family: Opus, sonnet, haiku
2024 · External reference
Continual learning for large language models: A survey
2024 · External reference
10.1609/aaai.v40i45.41215
10.1609/aaai.v40i45.41215 · External reference
Gutenberg DPO
2024 · External reference
IMDB-DPO
2024 · External reference
Hugging Face H4 Stack Exchange preference dataset
2023 · External reference
The Llama 3 herd of models
2024 · External reference
Qwen2 technical report
2024 · External reference
Bridging SFT and DPO for diffusion model alignment with self-sampling preference optimization
2024 · External reference
RLAIF vs. RLHF: Scaling reinforcement learning from human feedback with AI feedback
2024 · External reference
Weak-to-strong generalization: Eliciting strong capabilities with weak supervision
2024 · External reference
Weak-to-strong backdoor attacks for LLMs with contrastive knowledge distillation
2024 · External reference
LLM critics help catch LLM bugs
2024 · External reference
10.52202/075280-2574
10.52202/075280-2574 · External reference
Iterative preference learning from human feedback: Bridging theory and practice for RLHF under kl-constraint
2024 · External reference
Tool-augmented reward modeling
2024 · External reference
UNIFIED-IO: A unified model for vision, language, and multi-modal tasks
2023 · External reference
10.1609/aaai.v39i16.33882
10.1609/aaai.v39i16.33882 · External reference
Show-o: One single transformer to unify multimodal understanding and generation
2025 · External reference
Video diffusion alignment via reward gradients
2024 · External reference
A survey on long video generation: Challenges, methods, and prospects
2024 · External reference
