LLM

Base Knowledge

• HF: The Alignment Handbook

• HF: The Large Language Model Training Handbook

Specific Techniques

• Direct Preference Optimization (DPO)

  • Paper: https://arxiv.org/abs/2305.18290

  • https://plainenglish.io/community/direct-preference-optimization-dpo-a-simplified-approach-to-fine-tuning-large-language-models

  • https://huggingface.co/blog/dpo-trl

• Mixture of Experts (MoE)

  • HF Blog: Mixture of Experts Explained

Specific Models

• Mixtral

  • https://mistral.ai/news/mixtral-of-experts/

  • HF Blog: https://huggingface.co/blog/mixtral

• Argilla Notux

  • based on Mixtral

  • HF Model: https://huggingface.co/argilla/notux-8x7b-v1

  • Dataset: https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned

  • Code: argilla-io/notus