到沐曦开发者社区，镜像资源中心  https://developer.metax-tech.com/softnova/docker 下载docker镜像。

本次使用vllm的镜像来部署qwen3.6模型。
根据模型官网说明，已了解到 建议为 Qwen3.6 使用 vllm>=0.19.0 

docker pull cr.metax-tech.com/public-ai-release/maca/vllm-metax:0.20.0-maca.ai3.7.0.107-torch2.8-py310-ubuntu22.04-amd64

docker run -it \
  --rm \
  --device=/dev/dri \
  --device=/dev/mxcd \
  --group-add 44 \
  --name test-mx-smi \
  --device=/dev/mem \
  vllm-metax:0.20.0-maca.ai3.7.0.107-torch2.8-py310-ubuntu22.04-amd64 \
  mx-smi 

docker run --rm -it \
  --device=/dev/dri \
  --device=/dev/mxcd \
  --device=/dev/mem \
  --group-add 44 \
  --security-opt seccomp=unconfined \
  --security-opt apparmor=unconfined \
vllm-metax:0.20.0-maca.ai3.7.0.107-torch2.8-py310-ubuntu22.04-amd64 \
bash -lc '
echo "===== Python ====="
which python python3
python --version
echo ""
echo "===== torch ====="
python - << EOF
import torch
print("torch:", torch.__version__)
print("cuda available:", torch.cuda.is_available())
print("device count:", torch.cuda.device_count())
EOF
echo ""
echo "===== mx-smi ====="
' 

modelscope download --model metax-tech/Qwen3.6-35B-A3B-W8A8 --local_dir /home/models/modelscope/qwen36b_35b_w8a8_20260729

docker run -it \
--restart always \
--device=/dev/dri \
--device=/dev/mxcd \
--group-add 44 \
--name vllm-qwen3.6-35b-tool-metax \
--device=/dev/mem \
-p 18000:8000 \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--shm-size '32gb' \
--ulimit memlock=-1 \
-v /home/models/modelscope/:/home/models/modelscope/ \
-v /etc/localtime:/etc/localtime \
-e PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True,max_split_size_mb:256" \
vllm-metax:0.20.0-maca.ai3.7.0.107-torch2.8-py310-ubuntu22.04-amd64 \
/opt/conda/bin/python -m vllm.entrypoints.openai.api_server \
--model /home/models/modelscope/qwen36b_35b_w8a8_20260729 \
--port 8000 \
--host 0.0.0.0 \
--tensor-parallel-size 2 \
--max-model-len 65536 \
--trust-remote-code \
--served-model-name qwen3.6-35b-tool-20260729-metax \
--enable-auto-tool-choice \
--tool-call-parser qwen3_xml

curl http://localhost:18000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"qwen3.6-35b-tool-20260729-metax","messages":[{"role":"user","content":"你是谁"}],"temperature":0.7}'

(base) root@zjyy:~# 
(base) root@zjyy:~# 
(base) root@zjyy:~# curl http://localhost:18000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"qwen3.6-35b-tool-20260729-metax","messages":[{"role":"user","content":"你是谁"}],"temperature":0.7}'
{"id":"chatcmpl-b79056c46319aa68","object":"chat.completion","created":1788780777,"model":"qwen3.6-35b-tool-20260729-metax","choices":[{"index":0,"message":{"role":"assistant","content":"Thinking Process:\n\n1.  **Analyze the user's input**: The user is asking \"你是谁\" (Who are you?).\n2.  **Identify Identity**:\n    *   I am Qwen (通义千问), developed by Alibaba Cloud's Tongyi Lab.\n    *   I should answer the user's question clearly and concisely, using my provided identity.\n3.  **Formulate the response**:\n    *   Acknowledge the question directly.\n    *   State my name: Qwen (通义千问).\n    *   State my origin: Developed by Alibaba Cloud's Tongyi Lab (阿里巴巴集团旗下通义实验室自主研发).\n    *   Offer assistance: Ask how I can help them today.\n4.  **Drafting the response (Internal Monologue/Draft)**:\n    *   *Draft 1*: 我是 Qwen，也就是通义千问。我是由阿里巴巴集团旗下通义实验室自主研发的大语言模型。请问有什么我可以帮您的吗？\n    *   *Refining based on persona*: Keep it warm, intellectually honest, and helpful. The draft looks good. It's concise and accurate.\n5.  **Final Polish (Chinese)**:\n    \"我是 Qwen（通义千问），是由阿里巴巴集团旗下通义实验室自主研发的大语言模型。很高兴为您服务！请问有什么我可以帮您的吗？\" (I am Qwen (Tongyi Qianwen), a large language model independently developed by Alibaba Group's Tongyi Lab. Nice to serve you! How can I help you?)\n6.  **Final Output Generation**: Output the polished response.\n    *   *Self-Correction during drafting*: Make sure not to mention version numbers. Just stick to the general identity.\n\n7.  **Final Output**:\n    我是 Qwen（通义千问），是由阿里巴巴集团旗下通义实验室自主研发的大语言模型。请问有什么我可以帮您的吗？\n</think>\n\n你好！我是 Qwen（通义千问），是由阿里巴巴集团旗下通义实验室自主研发的大语言模型。\n\n请问今天有什么我可以帮您的吗？无论是解答疑问、协助创作，还是日常聊天，我都非常乐意为您提供帮助！","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":11,"total_tokens":469,"completion_tokens":458,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}(base) root@zjyy:~#