Examples

We provide several examples to show how to use our package in your problem.

Data preparation

• Dataset
  • Dataset Format in General
  • Supported Dataset and Detailed Formats
    • Conversation
    • TextOnly
    • Text2Text
    • Paired Conversation
• Supported Conversation Template
  • ChatGLM-3
  • ChatML
  • DeepSeek-V2
  • DeepSeek-V3
  • DeepSeek-R1(-Zero)
  • DeepSeek-R1-Distill(-Llama/Qwen)
  • Gemma
  • Hymba
  • InternLM2
  • Llama-2
  • Llama-3
  • Mixtral 8x22B
  • Mixtral 8x7B
  • Phi-3
  • Qwen-2
  • Qwen-2.5
  • Qwen-2.5-1M
  • Qwen-2.5-Math
  • Qwen-QwQ
  • Yi
  • Yi-1.5
  • Zephyr
• Customize Conversation Template
  • 1. Decompose your conversations
  • 2. Choose proper Formatter
  • 3. Build the template
  • 4. Register your template
  • 5. Use your template

• Checkpoints
  • LLaMA Checkpoint

Finetuning

For SFT,

• Finetuning
  • Full Parameters
  • Layerwise Importance Sampled AdamW (LISA)
  • Low-Rank Adaptation (LoRA)
• Finetune

For alignment process,

• Reward Modeling
  • Introduction
  • Step 1 Supervised Finetuning (SFT)
  • Step 2 Reward Modeling
  • Examples

• RAFT
  • 1 Introduction
    • 1.1 Dataset description
  • 2 Reward Modeling
    • 2.1 Supervised Finetuning (SFT)
    • 2.2 Reward Modeling
    • 2.3 LoRA Merge and Get Reward Model
  • 3 RAFT Alignment
    • 3.1 Algorithms Overview
    • 3.2 Hyper-parameters
    • 3.3 Examples

Inference

Refer to examples.

Evaluation

• LMFlow Benchmark Guide
• 1. NLL Task Setting
  • Setup
  • Create Your Task Dataset File
  • Task Registration
• 2. LM-Evaluation Task Setting