PaddleNLP

PaddleNLP is a large language model (LLM) development toolkit based on the PaddlePaddle deep learning framework. It supports efficient large-scale model training, lossless compression, and high-performance inference on various hardware. PaddleNLP is designed for simplicity and极致 performance, empowering developers to achieve efficient industrial-level applications of large models.

You can use PaddleNLP for rapid model training while leveraging SwanLab for experiment tracking and visualization.

---

1. Integrating SwanLabCallback

from swanlab.integration.paddlenlp import SwanLabCallback

The SwanLabCallback is a logging class tailored for PaddleNLP.

Configurable parameters for SwanLabCallback include:

• project, experiment_name, description, and other parameters consistent with swanlab.init, used for initializing the SwanLab project.
• Alternatively, you can create a project externally via swanlab.init, and the integration will log experiments to the externally created project.

2. Passing to Trainer

from swanlab.integration.paddlenlp import SwanLabCallback  
from paddlenlp.trainer import TrainingArguments, Trainer  

...  

# Instantiate SwanLabCallback  
swanlab_callback = SwanLabCallback(project="paddlenlp-demo")  

trainer = Trainer(  
    ...  
    # Pass the callback via the `callbacks` parameter  
    callbacks=[swanlab_callback],  
)  

trainer.train()

3. Complete Example Code

Requires connectivity to the HuggingFace server to download the dataset.

"""  
Tested with:  
pip install paddlepaddle-gpu==3.0.0 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/  
pip install paddlenlp==3.0.0b4  
"""  
from paddlenlp.trl import SFTConfig, SFTTrainer  
from datasets import load_dataset  
from swanlab.integration.paddlenlp import SwanLabCallback  

dataset = load_dataset("ZHUI/alpaca_demo", split="train")  

training_args = SFTConfig(  
    output_dir="Qwen/Qwen2.5-0.5B-SFT",  
    device="gpu",  
    per_device_train_batch_size=1,  
    logging_steps=20  
)  

swanlab_callback = SwanLabCallback(  
    project="Qwen2.5-0.5B-SFT-paddlenlp",  
    experiment_name="Qwen2.5-0.5B",  
)  

trainer = SFTTrainer(  
    args=training_args,  
    model="Qwen/Qwen2.5-0.5B-Instruct",  
    train_dataset=dataset,  
    callbacks=[swanlab_callback],  
)  
trainer.train()

---

4. GUI Demonstration

Automatically Logged Hyperparameters:

Metrics Logging:

5. Extension: Adding More Callbacks

Imagine a scenario where you want the model to infer test samples at the end of each epoch and log the results with SwanLab. You can create a new class inheriting from SwanLabCallback and extend or override lifecycle functions. For example:

class NLPSwanLabCallback(SwanLabCallback):  
    def on_epoch_end(self, args, state, control, **kwargs):  
        test_text_list = ["example1", "example2"]  
        log_text_list = []  
        for text in test_text_list:  
            result = model(text)  
            log_text_list.append(swanlab.Text(result))  

        swanlab.log({"Prediction": test_text_list}, step=state.global_step)

The above is a new callback class for NLP tasks, extending the on_epoch_end function, which executes at the end of each epoch during transformers training.