As enterprises and individuals better understand the power of machine learning tools, they often find themselves blocked by the same thing: a lack of high-quality training data. Anyone familiar with Snorkel AI knows that manual labeling presents scalability and data privacy challenges. As a researcher focused on data labeling solutions, I’m excited to introduce Alfred, an open-source tool for combining foundation models with weak supervision.
I recently presented my project to Snorkel AI researchers. You can watch the entire talk (embedded below), but I have summarized the main points here.
Alfred is a python library designed to streamline the use of foundation models (FMs) to make it easier for data scientists and researchers to use them to label data at scale.
Traditional data annotation methods often involve time-consuming and costly manual processes. Using FMs to label data directly can create inaccurate results. Alfred aims to address these limitations.
To do this, Alfred allows users to define labeling rules in plain language as prompts to FMs, such as Llama 3.1, GPT-4, or CLIP. Then it denoises and synergizes the models’ responses through weak supervision.
Alfred was initially designed with academic researchers in mind, offering a tool to streamline data science projects in academic settings. While its current iteration may not fully address the complex needs of enterprise data teams, Alfred incorporates fundamental principles similar to those found in the Snorkel Flow AI data development platform, which is enterprise-ready.
Alfred enables users—even relatively non-technical ones—to multiply their labeling impact. By transforming rigid code-based labeling functions into flexible natural language prompts, Alfred allows non-experts to leverage the expertise of pre-trained foundation models, making the annotation process more accessible.
Here’s what that process looks like:
With Alfred, users layer heuristics on top of each other. They can write labeling prompts that address different aspects of their target records. They can even use different models. From our empirical studies, we’ve found that using multiple models for the same heuristics improves accuracy by offsetting individual model biases.
In addition to providing an easy-to-use wrapper, my colleagues at Brown and I have built several useful features into Alfred, such as:
Alfred aims to integrate more automation in future updates, utilizing language models to further streamline template creation and other stages of the workflow.
In my presentation, I gave two examples of how Alfred can help in the real world.
In the first, we looked at an application to classify the sentiment of financial newss headlines. Very quickly, we noticed headlines that mentioned price targets going up or down. This could serve as a good heuristic. Traditionally, a user might use a regular expression to capture this signal, but regex would miss headlines that used synonyms or even where the writer ordered their words differently.
With Alfred, we can skip the regex and prompt a large language model (LLM). We can create a template that presents the headline to the LLM and asks it which sentiment category the headline falls into—neutral, positive, or negative.
In the second, we demonstrated image Data Annotation. For tasks involving image data, Alfred supports contrastive models like CLIP and autoregressive models like BLIP to simplify attribute-based classification. Here, again, our templates help us efficiently present the information to the model and process a response.
One of the significant challenges in data annotation is managing the complexity and computational demands of foundation models. Alfred addresses these through:
Our Alfred package represents a significant leap forward in open-source programmatic weak supervision. It offers a flexible, efficient, and user-friendly system that enhances data annotation and model training. We believe that Alfred will serve as a cornerstone for future machine learning applications—especially for academic projects— driving innovation and simplifying complex data workflows.
Follow Snorkel AI on LinkedIn, Twitter, and YouTube to be the first to see new posts and videos!
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。