Open Source • MIT-0 License
Reference architectures, test cases, and best practices for training large-scale models with PyTorch, Megatron-LM, NeMo, JAX, and more on AWS infrastructure.
Production-ready examples grouped by framework. Each includes Dockerfiles, Slurm scripts, and Kubernetes manifests.
🔥
Native distributed training with DDP, FSDP, TorchTitan, DeepSpeed, and more. Covers LLMs, vision, robotics, and RLHF.
FSDPDDPDeepSpeedTorchTitanPicotronvLLMTRLOpenRLHF
⚡
NVIDIA Megatron-LM and NeMo for large-scale LLM pre-training with tensor, pipeline, and expert parallelism.
Megatron-LMNeMoNeMo RLBioNeMo
🧬
Google JAX with PaxML for distributed training leveraging XLA compilation and automatic parallelism.
PaxMLXLATPU/GPU
🧠
NeuronX Distributed for training on AWS Trainium & Inferentia chips with optimized compilers.
NeuronXOptimum NeuronTrainium
🤖
Embodied AI training with NVIDIA Isaac Lab, OpenVLA, V-JEPA2, and vision-language-action models.
Isaac LabOpenVLAV-JEPA 2nanoVLM
🎯
RLHF, DPO, PPO, and scalable RL frameworks for LLM alignment and post-training.
TRLvERLSLIMEPPODPO
🧪
Knowledge distillation, compression, and model adaptation techniques for production.
DistillationCompressionTransfer Learning
CloudFormation templates and deployment guides for every AWS compute platform.
Three steps to launch your first distributed training job.
1
Launch a cluster using our CloudFormation templates for HyperPod, ParallelCluster, or EKS.
2
Use our Dockerfiles to build a training container with your framework of choice.
3
Submit your job with Slurm or Kubernetes using our ready-made launch scripts.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。