惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

A
About on SuperTechFans
Y
Y Combinator Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Microsoft Security Blog
Microsoft Security Blog
aimingoo的专栏
aimingoo的专栏
I
InfoQ
C
Check Point Blog
IT之家
IT之家
MyScale Blog
MyScale Blog
Apple Machine Learning Research
Apple Machine Learning Research
Vercel News
Vercel News
Last Week in AI
Last Week in AI
GbyAI
GbyAI
P
Proofpoint News Feed
量子位
Stack Overflow Blog
Stack Overflow Blog
Microsoft Azure Blog
Microsoft Azure Blog
月光博客
月光博客
阮一峰的网络日志
阮一峰的网络日志
人人都是产品经理
人人都是产品经理
B
Blog
T
The Blog of Author Tim Ferriss
H
Help Net Security
云风的 BLOG
云风的 BLOG

博客园 - centosboy

Thinkpad T490卡顿排查 xinference初探 diff的安装与使用 vllm部署 zabbix监控mongodb /root/.dbshell无权限解决办法 开机启动后应用limit没有生效 关于运维工作的一点感悟 将CDLINUX装入U盘 Autossh打洞 hive分区表实践 hashlib模块 rm 防误删脚本 linux 通过inode删除文件 那些年踩过的坑 htpdate代替ntpdate同步时间 Openfire搭建聊天系统 zookeeper运维 centos ping添加丢包记录 Linux tee命令输出显示在屏幕和保存到日志文件中
ubuntu docker运行大模型
centosboy · 2025-12-09 · via 博客园 - centosboy

安装docker

安装依赖
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release

添加官方 Docker 源

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo
"deb [arch=$(dpkg --print-architecture)
signed-by=/etc/apt/keyrings/docker.gpg]
https://download.docker.com/linux/ubuntu
$(lsb_release -cs) stable" |
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

安装 Docker Engine

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

启动测试

sudo systemctl enable docker
sudo systemctl start docker
sudo docker run hello-world

安装 NVIDIA Container Toolkit(让 Docker 支持 GPU)
浏览器打开地址后下载GPG密钥 https://nvidia.github.io/libnvidia-container/gpgkey

登陆github:https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.18.0
下载toolkit包
tar -xzvf nvidia-container-toolkit_1.18.0_deb_amd64.tar.gz
sudo dpkg -i ./*.deb

nvidia-ctk --version

配置 Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

测试 GPU 容器运行

sudo docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

docker info | grep -i runtime

应看到: Runtimes: nvidia runc

cat /etc/docker/daemon.json
{
"registry-mirrors": [
"https://docker.m.daocloud.io",
"https://mirror.baidubce.com",
"https://ccr.ccs.tencentyun.com",
"https://hub-mirror.c.163.com",
"https://registry.cn-hangzhou.aliyuncs.com"
],
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"args": []
}
}
}

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html