September 2025 Update: We no longer offer HUGS model deployment containers.To easily deploy optimized Hugging Face model in your infra, check out the Dell Enterprise Hub and the Hugging Face Collection in Azure AI Foundry.
Today, we are thrilled to announce the launch of Hugging Face Generative AI Services a.k.a. HUGS: optimized, zero-configuration inference microservices designed to simplify and accelerate the development of AI applications with open models. Built on open-source Hugging Face technologies such as Text Generation Inference and Transformers, HUGS provides the best solution to efficiently build and scale Generative AI Applications in your own infrastructure. HUGS is optimized to run open models on a variety of hardware accelerators, including NVIDIA GPUs, AMD GPUs, and soon AWS Inferentia and Google TPUs.
HUGS simplifies the optimized deployment of open models in your own infrastructure and on a wide variety of hardware. One key challenge developers and organizations face is the engineering complexity of optimizing inference workloads for LLMs on a particular GPU or AI accelerator. With HUGS, we enable maximum throughput deployments for the most popular open LLMs with zero configuration required. Each deployment configuration offered by HUGS is fully tested and maintained to work out of the box.
HUGS model deployments provide an OpenAI compatible API for a drop-in replacement of existing Generative AI applications built on top of model provider APIs. Just point your code to the HUGS deployment to power your applications with open models hosted in your own infrastructure.
HUGS offers an easy way to build AI applications with open models hosted in your own infrastructure, with the following benefits:
We provided early access to HUGS to select Enterprise Hub customers:
HUGS is a huge timesaver to deploy locally ready-to-work models with good performances - before HUGS it would take us a week, now we can be done in less than 1 hour. For customers with sovereign AI requirements it's a game changer! - Henri Jouhaud, CTO at Polyconseil
We tried HUGS to deploy Gemma 2 on GCP using a L4 GPU - we didn't have to fiddle with libraries, versions and parameters, it just worked out of the box. HUGS gives us confidence we can scale our internal usage of open models! - Ghislain Putois, Research Engineer at Orange
Using HUGS is straightforward. Here's how you can get started:
Note: You will need access to the appropriate subscription or marketplace offering depending on your chosen deployment method.
HUGS is available through several channels:
For specific deployment instructions for each platform, please refer to the relevant documentation linked above.
HUGS offers on-demand pricing based on the uptime of each container, except for deployments on DigitalOcean.
HUGS is based on Text Generation Inference (TGI), offering a seamless inference experience. For detailed instructions and examples, refer to the Run Inference on HUGS guide. HUGS leverages the OpenAI-compatible Messages API, allowing you to use familiar tools and libraries like cURL, the huggingface_hub SDK, and the openai SDK for sending requests.
from huggingface_hub import InferenceClient
ENDPOINT_URL="REPLACE" # replace with your deployed url or IP
client = InferenceClient(base_url=ENDPOINT_URL, api_key="-")
chat_completion = client.chat.completions.create(
messages=[
{"role":"user","content":"What is Deep Learning?"},
],
temperature=0.7,
top_p=0.95,
max_tokens=128,
)
HUGS supports a growing ecosystem of open models and hardware platforms. Refer to our Supported Models and Supported Hardware pages for the most up-to-date information.
We launch today with 13 popular open LLMs:
For a detailed view of supported Models x Hardware, check out the documentation.
HUGS makes it easy to harness the power of open models, with zero-configuration optimized inference in your own infra. With HUGS, you can take control of your AI applications and easily transition proof of concept applications built with closed models to open models you host yourself.
Get started today and deploy HUGS on AWS, Google Cloud or DigitalOcean!
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。