2 min read

Character.AI Open Sources pipeling-sft: A Scalable Framework for Fine-Tuning MoE LLMs like DeepSeek V3

At Character.AI, we’re excited to share an experimental project from our research team with the open-source community: pipeling-sft — a lightweight yet powerful training framework

At Character.AI, we’re excited to share an experimental project from our research team with the open-source community: pipeling-sft — a lightweight yet powerful training framework built for full-parameter supervised fine-tuning (SFT) of large-scale LLMs with Mixture-of-Experts (MoE) architectures.

This framework was originally developed to explore better ways of fine-tuning DeepSeek V3, but its capabilities generalize to many similar MoE-based OSS LLMs. Now, we’re releasing it publicly to help the community move faster, scale more efficiently, and customize more easily for downstream tasks.

Why This Matters

Fine-tuning massive language models—especially MoE-based ones—is notoriously challenging. Memory limits, parallelization complexity, and unstable training dynamics all pose significant barriers for researchers and engineers alike. pipeling-sft is designed to make this process simpler, faster, and more stable.

Here’s how:

  • Multi-Level Parallelism: Combines pipeline parallelism, expert parallelism, and tensor parallelism to shard very large MoE models across multiple nodes and GPUs efficiently.
  • Both BF16 and FP8 Training: Supports bfloat16 training with custom mixed-precision optimizers for stability, and includes experimental FP8 training support to push the frontier of efficiency even further.
  • Seamless HuggingFace Integration: Allows researchers and engineers to start from official HuggingFace model weights and export directly back into the HuggingFace checkpoint format—no extra preprocessing or post-processing steps required.
  • Training Stability Built-In: Gradient synchronization and custom mixed-precision optimizers help prevent divergence and enable faster convergence, even under low learning rates.
  • Flexible & Hackable: Written in pure PyTorch, which makes it easy to adapt, extend, or repurpose for specific models, tasks, or infrastructure.

Call for Collaboration

While pipeling-sft is still an experimental project, it’s already filling an important gap for teams who want to fine-tune very large LLMs without reinventing infrastructure. Our research team at Character.ai is open-sourcing it to accelerate OSS LLM research and help others build powerful, domain-specific applications more easily.

If you're working with large MoE models—or want to start—this project is for you. We'd love to collaborate, hear your feedback, and grow this project together.

Check it out on GitHub: https://github.com/character-ai/pipelining-sft