Etsy’s marketplace is defined by the creativity and craftsmanship of our sellers and the hundreds of millions of highly diverse products they offer. You can find silversmiths who cold-forge recycled sterling silver, weavers who dye raw fleece with indigo and black walnut, and ceramicists who throw stoneware on a kick wheel.
These details define each product and often determine whether it matches a buyer’s taste, style, and interests. Sometimes buyers know exactly what they want, searching for “hand-thrown ceramic mug” or “vegetable-tanned leather wallet.” Other times, especially in recommendations, that intent is implicit. In those cases, we need to surface products that align with a buyer’s preferences even when those details are not explicitly stated.
To do that well, our search and recommendation systems need to understand what sets each listing apart beyond just the product category or listing titles, capturing the details that signal what an object is, how it was made, and whom it might appeal to. The quality of these representations directly limits how precisely our models can learn relationships between buyer taste and product details, and in turn surface relevant recommendations and search results. If the representation misses what makes a product special, the model will too.
Sellers provide rich product information through listing titles, images, descriptions, tags, variations, and attributes, but the most distinguishing details are often buried in long, noisy, and inconsistently expressed data. For example, details about how a product is made might be buried in a long description alongside shipping information, return policies, and sizing charts. A product’s visual style might be clear in its images, but not all of our systems can process images. At scale, ingesting and reasoning over all this raw data for production models is difficult due to tight latency constraints.
Parallel work at Etsy addresses some of these challenges by extracting parts of this raw signal into a structured format where we define attributes and infer them across all of the products in our inventory. While structured data works well for analysis and hand-crafting ML features, the diversity of Etsy’s one-of-a-kind inventory makes defining a fixed schema to capture all product intricacies difficult at scale.
Unstructured data can give us greater flexibility to capture more of this diverse information in a way that downstream ML models can learn from. One way to represent listings with less structural constraint is with free-form natural language summaries. Language models can leverage their world knowledge to distill unique product details into concise, expressive summaries that are easy to plug into downstream models.
However, a key challenge with unstructured output is defining what a “good” solution looks like, as what's important varies widely across products and contexts. Even within the same category, the details that matter most can differ substantially between listings. For one piece of artwork, buyers might respond most strongly to the composition or color palette, while for another, it may be the brushwork or framing style that determines which buyers it is most relevant to. Which details matter most is often reflected in how buyers engage with the listing given their search intent or prior interactions.
Because these distinctions depend heavily on the specific product, expressing what information is important through a prompt or a schema would require knowing upfront what details our complex search and recommendation systems actually need. Instead, we can build richer product representations by learning which information matters directly from contextual buyer engagement signal.
Reinforcement learning (RL) provides a flexible way to shape product representations around the listing details most aligned with buyer engagement. Prior work, including a recent paper from Walmart, has shown that downstream models can be used directly as reward signal to teach an LLM how to represent and summarize the information that is important for downstream machine learning tasks.
Inspired by this work, we aimed to identify and surface the details that distinguish a listing from other similar options a buyer might consider. For example, we want our models to understand why a buyer who searched for “Sculptural Stoneware Pottery” chose to interact with one listing (engaged), but not another (non-engaged).
This perspective aligns naturally with ideas from contrastive learning. In traditional representation learning, contrastive methods train models to produce embeddings by pulling similar examples closer together and pushing dissimilar ones apart, implicitly learning which features matter most for distinguishing between them. We apply the same logic here, but instead of training a model to learn a latent embedding space, we use contrastive signal as a reward to fine-tune a language model to generate natural language summaries that emphasize distinguishing product details. These internal summaries are used to help our search and recommendation models surface products that better match buyer taste and intent.
To do this, we fine-tune Qwen3-VL-8B, an open-source LLM, using search interaction data structured as triplets: a query, an engaged listing, and a non-engaged listing. Engaged listings are those a buyer clicked, purchased, or favorited for a given query, while non-engaged listings are a mix of listings that appeared in the same search results but were not interacted with, along with randomly sampled listings. Because this data is grounded in real buyer behavior, it provides a natural signal for which details influenced a buyer’s decision in a given context.
For each triplet, the model generates several candidate summaries for both listings. These summaries are passed through a frozen neural search retrieval model, a two-tower dense retriever used for candidate generation in search, which scores their similarity to the query. The reward signal is defined as the margin between the query–engaged similarity and the query–non-engaged similarity. Summaries that increase this separation receive higher reward.
We then use Group Relative Policy Optimization (GRPO) to update the summary generation model based on the relative quality of its candidate generations, as illustrated in the figure below. Because the reward depends on the margin between paired engaged and non-engaged summaries, rather than on summaries independently, we generate and evaluate these candidates jointly within the same batch during RL training.
Throughout training, this contrastive objective nudges the model to produce summaries that push engaged listings closer to the query in the search model’s embedding space while pushing non-engaged listings further away. The margin structure also discourages reward hacking by construction: a generic, keyword-stuffed summary cannot simultaneously raise the engaged listing’s similarity and lower the non-engaged’s, so it is not rewarded for exploiting the retriever’s embedding space without grounding in the listing. The figure below illustrates how these embedding space dynamics unfold for candidate summaries generated from a given training triplet over the course of RL fine-tuning.
Over enough steps, the model learns to surface the nuanced product details that distinguish between listings for the same broad type of product, but differ in the specific details that make one more relevant to a buyer than another.
Critically, since the model is trained to distinguish between alternatives rather than distill raw listing data, it can draw on sources our search systems currently don’t directly ingest, such as images. To understand how this reward shapes the information the model surfaces in practice, we can trace how the generated summary for a single listing evolves over the course of training.
This listing's distinctive product information is spread across its image, nested attributes, and a long description that also contains care instructions and shop information. Without reinforcement learning, the base model produces a summary that largely repeats the listing title and vaguely describes some additional features. Over the course of training, the model learns to extract specific product details from the raw data while ignoring irrelevant noise. By the end, the fine-tuned model produces a summary that goes well beyond the title and surfaces the details that actually set the listing apart. More example product summaries generated by the RL-tuned model are highlighted in the table below.
Looking at examples of generated summaries and how they progress through training can help us assess if the model is surfacing the right details, but we need concrete measures of quality to guide model tuning and understand if the generated summaries are moving the needle in terms of downstream system performance.
To get signal into the quality and accuracy of the generated summaries, we partnered with Etsy’s inventory specialist to review the generated summaries alongside the raw listing data. Beyond general high quality human review, she brings specialized knowledge of what makes products in our marketplace distinctive – a key component for aligning LLM outputs to what actually matters. For each summary, she assessed whether it was grounded in the listing information, whether it surfaced concrete, distinguishing details not present in the title, and whether these product details were the most important additional information to surface.
Across all evaluated summaries, the model was consistently pulling out specific, accurate product details beyond the title. The reward signal was also working as intended, keeping generations grounded in seller-provided information and pushing the model to prioritize the details most relevant for search and recommendation models.
Beyond qualitative evaluation, we also want to measure whether these generated summaries are surfacing the right information to help downstream ML better understand the intricacies of the products on Etsy. To test this, we measured the impact of generated summaries on our Semantic Relevance model, which classifies query-listing pairs as relevant, partially relevant, or irrelevant and is used in search filtering and evaluating our search systems. This provides a good evaluation task because the omission of seemingly small product details can completely shift whether a product is semantically relevant to a specific buyer query. We generated summaries for the listings in a human-labeled evaluation set and compared model performance using description n-grams versus the RL fine-tuned product summaries as input. The table below provides a breakdown of the results.
We found that including the generated summaries as input in the semantic relevance model improved macro F1 score by 8.7% relative to using description n-grams, a simpler method that captures keywords in listing descriptions. Beyond the improvement for this specific evaluation task, this gives us evidence that despite being optimized explicitly for our neural search retrieval model, the summaries surface information that leads to gains for an entirely different model with a different objective. This generalization suggests the summaries are meaningfully capturing important product details overall and not just overfitting to our reward signal.
This work was built as a way to highlight the creativity and craft of our sellers and the extremely diverse products they create. Rather than trying to rigidly define what makes each product distinct, we learn it directly from how our systems observe buyer behavior. The result is a way to surface what makes each individual product special in a form that is expressive, consistent, and easy for downstream systems to learn from.
In the near term, we are planning online experiments integrating these summaries into production ML systems. Beyond that, we are excited about the potential feedback loop this framework opens up: better summaries give downstream models richer signal to learn from, and better models produce stronger reward signal to further fine-tune expressive summaries.
This work is part of an ongoing, collaborative effort to bring more of the creativity and craft behind every listing into how our systems represent and surface products. We would like to thank our collaborators across the Inventory ML and Search ML teams.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。