In two short years since the advent of diffusion-based image generators, AI image models have achieved near-photographic quality. How do these models compare? Are the open-source alternatives on par with their proprietary counterparts?
The Artificial Analysis Text to Image Leaderboard aims to answer these questions with human preference based rankings. The ELO score is informed by over 45,000 human image preferences collected in the Artificial Analysis Image Arena. The leaderboard features the leading open-source and proprietary image models : the latest versions of Midjourney, OpenAI's DALL·E, Stable Diffusion, Playground and more.
Check-out the leaderboard here: https://huggingface.co/spaces/ArtificialAnalysis/Text-to-Image-Leaderboard
You can also take part in the Text to Image Arena, and get your personalized model ranking after 30 votes!
Comparing the quality of image models has traditionally been even more challenging than evaluations in other AI modalities such as language models, in large part due to the inherent variability in people’s preferences for how images should look. Early objective metrics have given way to expensive human preference studies as image models approach very high accuracy. Our Image Arena represents a crowdsourcing approach to gathering human preference data at scale, enabling comparison between key models for the first time.
We calculate an ELO score for each model via a regression of all preferences, similar to Chatbot Arena. Participants are presented with a prompt and two images, and are asked select the image that best reflects the prompt. To ensure the evaluation reflects a wide-range of use-cases we generate >700 images for each model. Prompts span diverse styles and categories including human portraits, groups of people, animals, nature, art and more.
To see the leaderboard, check out the space on Hugging Face here: https://huggingface.co/spaces/ArtificialAnalysis/Text-to-Image-Leaderboard
To participate in the ranking and contribute your preferences, select the ‘Image Arena’ tab and choose the image which you believe best represents the prompt. After 30 images, select the ‘Personal Leaderboard’ tab to see your own personalized ranking of image models based on your selections.
For updates, please follow us on Twitter and LinkedIn. (We also compare the speed and pricing of Text to Image model API endpoints on our website at https://artificialanalysis.ai/text-to-image).
We welcome all feedback! We're available via message on Twitter, as well as on **our website** via our contact form.
The Artificial Analysis Text to Image leaderboard is not the only quality image ranking or crowdsourced preference initiative. We built our leaderboard to focus on covering both proprietary and open source models to give a full picture of how leading Text to Image models compare.
Check out the following for other great initiatives:
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。