model.json
"version": 2,
"context_length": 2048,
"model_name": "qwen2.5-1.5b-instruct",
"model_lang": ["en", "zh"],
"model_ability": ["chat"],
"model_family": "qwen2.5",
"chat_template": "{% for message in messages %}{{ message['role'] }}: {{ message['content'] }}\n{% endfor %}assistant:",
"model_specs": [
{
"model_format": "pytorch",
"model_file_name_template": "model.safetensors",
"model_size_in_billions": 2,
"quantization": "none",
"model_hub": "huggingface",
"model_uri": "file:///data/models/qwen2.5-1.5B/qwen2.5-1.5B-instruct"
}
],
"is_builtin": false
}
注册模型
xinference launch --model-name qwen-2.5-1.5 --model-engine vllm
运行模型
xinference launch --model-name qwen-2.5-1.5b-instruct --model-engine transformers
验证:
curl -X 'POST' 'http://localhost:9997/v1/chat/completions' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{
"model": "qwen2.5-1.5b-instruct",
"messages": [
{
"role": "user",
"content": "hello"
}
]
}'
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。