ROLL

ROLL is an efficient and user-friendly reinforcement learning library specifically designed for Large Language Models (LLMs) that utilize large-scale GPU resources. It significantly enhances the performance of LLMs in key areas such as human preference alignment, complex reasoning, and multi-turn agent interaction scenarios.

ROLL employs a multi-role distributed architecture based on Ray to enable flexible resource allocation and heterogeneous task scheduling. It integrates cutting-edge technologies like Megatron-Core, SGLang, and vLLM to accelerate model training and inference.

Using SwanLab in ROLL is very straightforward—just configure a few parameters. For details, please refer to the track_with: swanlab section in agentic_pipeline_config.yaml.