Localized reading guide
trl-trainingPurpose, usage and source
Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
Source retrieved:
- Source repository
- huggingface/skills
- SKILL.md
skills/trl-training/SKILL.md- Repository stars
- 11,241
- Repository updated
- 2026-10-30
- Categories
- Development
- Last collected
License: Apache-2.0
Workflow excerpts are reading aids, not executable instructions from Codez. No third-party Skill was run to produce this page.
The purpose and workflow passages below are upstream material, organized and translated for reading. They are distinct from the separately labeled Codez editorial analysis.
What this Skill does
Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
How the upstream workflow works
TRL
Each method pairs a *Trainer class with a *Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config. Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimental_overview
Dataset formats
Conversational: {"messages": [{"role": ..., "content": ...}]} (language modeling) or {"prompt": [...], "completion": [...]}. The chat template is applied automatically — never apply it yourself. Extra columns are allowed; GRPO forwards them to reward functions. Reference: https://huggingface.co/docs/trl/dataset_formats
GRPO: online RL
Reward functions are called with keyword arguments prompts, completions, completion_ids, trainer_state, plus every extra dataset column — accept kwargs for the ones you ignore. Return list[float], one reward per completion. With conversational data, completions is a list of message lists, not strings. The generation batch is per_device_train_batch_size × num_processes × steps_per_generation (or set generation_batch_size directly) and must be divisible by num_generations (default 8). Generation is the usual bottleneck — enable vLLM with use_vllm=True: vllm_mode="colocate" shares the training GPUs (size with vllm_gpu_memory_utilization); vllm_mode="server" uses a separate trl vllm-serve --model .
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
trainer = SFTTrainer(
model="Qwen/Qwen2.5-0.5B", # model ID or a PreTrainedModel instance
args=SFTConfig(output_dir="Qwen2.5-0.5B-SFT"),
train_dataset=load_dataset("trl-lib/Capybara", split="train"),
)
trainer.train()Requirements and setup
No short requirements passage was selected from this file. This does not mean the Skill has no dependencies. Check the repository setup instructions and referenced files.
Limits and quality checks
No separate limits passage was selected from this file. This does not mean the Skill is safe for every input or environment. Review the complete source and validate your own result.
Start with a safe trial
- Check the supported host and the repository installation instructions. Keep the Skill directory and its referenced scripts, assets and documentation together; SKILL.md alone may omit dependencies.
- Give the agent your real input, desired output and acceptance criteria. Invocation syntax and available tools depend on the host; this site does not assume a universal install command.
- Try the source workflow in a disposable project or on a copy. Review operations that change files, remote services or billable resources before allowing them.
- Validate the output against the source quality checks and your own requirements. A successful trigger or a high repository Star count does not establish a correct result.
Read further in the source
Related Skills
Translation and scope
This reading guide selects passages from the pinned upstream SKILL.md. Translation is assisted. It is not a complete manual or a claim of tested compatibility. Read the original and its referenced files before use.
This introduction refers to the pinned version below; current repository activity and collection time are shown separately.
- Pinned source
- Read the original ↗
- Source retrieved
- Source commit
fabc634a893f62ad5c2039830397efa35fa62715- Source SHA-256
42c855bdc6171390abc7413cdb19bd4fd8dddfb31094d519a874427de08ddc2a