Integrate SwanLab for offline/online experiment tracking and local visualization

[ShaohonChen]

What does this PR do?

This PR introduces SwanLab, a lightweight open-source experiment tracking tool, as a new logging option for the training framework. The integration provides both online and offline tracking capabilities, along with a local dashboard for visualizing results.

SwanLab has previously supported tracking the Transformers training framework through external callbacks (find more information here), serving a wide range of users—especially those in regions with limited network connectivity, such as China. With this official integration, we aim to further enhance the developer experience by making tracking more seamless and user-friendly.

Additional information about this PR is a detailed overview of the changes and usage instructions.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests? (I don't see any tests for any of the callbacks but please let me know if I missed them somewhere. )

Who can review?

@SunMarc I have reviewed recent merges related to training tracking, and it seems that you are the most relevant reviewer for this PR. Could you please help review it or forward it to the appropriate person? Thank you!

---

Additional information about this PR

Key Features of SwanLab Integration

1. Online and Offline Tracking:

• Online Mode: Track experiments remotely and store data on SwanLab's cloud platform.
• Offline Mode: Use a local dashboard to visualize training logs without an internet connection.

2. Hardware Monitoring:

• Automatically tracks GPU usage, power consumption, temperature, and other hardware metrics.
• Supports NVIDIA GPUs and Huawei Ascend NPUs.

3. Remote Access:

• View training progress remotely via the SwanLab web interface or mobile app.

4. Local Dashboard:

• Includes an open-source local dashboard for offline visualization of training logs.

Usage guidline

Step 0: Set Up code and environment

Following the transformers official text classification example:

# prepare code and environments
git clone https://github.com/huggingface/transformers
cd transformers
pip install -e .
cd examples/pytorch/text-classification
pip install -r requirements.txt
# 

Step 1: Set Up SwanLab Online Tracking

Install:

pip install swanlab

To use SwanLab's online tracking, log in to the SwanLab website and obtain your API key from the Settings page. Then, authenticate using the following command:

swanlab login

If you prefer offline mode, skip this step.

Step 2: Configure SwanLab as the Logger and run example

To enable SwanLab as the experiment tracker, add --use_swanlab to your training command. For example, using the workflow:

python run_glue.py \
  --model_name_or_path google-bert/bert-base-cased \
  --dataset_name imdb  \
  --do_train \
  --do_predict \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/imdb/

visualization demo

If you want to use local tracking, you simply set environment variable following:

# Linux & Mac
export SWANLAB_MODE = "local"
# Win
$env:SWANLAB_MODE = "local"

Alternatively, you can configure SwanLab using environment variables:

export SWANLAB_API_KEY=<your_api_key>          # Set API key for online tracking
export SWANLAB_LOG_DIR=<local_log_path>        # Set local log directory
export SWANLAB_MODE=<mode>                    # Set tracking mode: cloud (default), cloud-only, local, or disabled

Step 3: View Training Logs

After logging in, you will see a confirmation message:

• Online Tracking: View logs on the SwanLab website.

For more details, refer to the SwanLab Cloud Documentation.

• Offline Tracking: Use the local dashboard to visualize logs:

here for more information.

swanlab watch

For advanced configurations, such as setting a custom port, refer to the Offline Dashboard Documentation and CLI Documentation.

[SunMarc]

Thanks for adding this ! Just a few nits !

[SunMarc]

maybe typo ?

Suggested change

	- **SWANLAB_PROJECT** (`str`, *optional*, defaults to `None`):
	Set this to a custom string to store results in a different project. If not specified, the name of the running
	directory is used.
	- **SWANLAB_PROJECTS** (`str`, *optional*, defaults to `None`):
	Set this to a custom string to store results in a different project. If not specified, the name of the running
	directory is used.

[ShaohonChen]

Good catch! I’ve fixed the typo.

[ShaohonChen]

Thanks for the review! I'll update it shortly.

[ShaohonChen]

I have made the required changes and fixed the typo. Thanks to @SunMarc for the review!

[ShaohonChen]

Good catch! I’ve fixed the typo.

[ShaohonChen]

I ran the example in examples/pytorch/text-classification with the updated code. Everything looks good!
Here is a screenshot and the link to the test run:

results here

Really appreciate your suggestions. Please have a look when you get a chance!

[SunMarc]

Thanks for iterating !

[SAKURA-CAT]

It seems the test timed out. Could it be due to excessive computational load?

[SAKURA-CAT]

It seems the test timed out. Could it be due to excessive computational load?

Re-running the test succeeded. 🤔

[ShaohonChen]

@SunMarc I have updated the code and fixed the incorrect comments. After merging the latest main branch, I noticed that the automated tests seem to have failed. I checked, and it doesn't appear to be an issue caused by my changes. Could you help me restart the tests?