惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
Security @ Cisco Blogs
H
Hacker News: Front Page
P
Privacy International News Feed
N
News and Events Feed by Topic
T
Threatpost
Simon Willison's Weblog
Simon Willison's Weblog
S
Schneier on Security
K
Kaspersky official blog
S
Secure Thoughts
V2EX - 技术
V2EX - 技术
Security Latest
Security Latest
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
www.infosecurity-magazine.com
www.infosecurity-magazine.com
C
CERT Recently Published Vulnerability Notes
L
Lohrmann on Cybersecurity
Jina AI
Jina AI
P
Proofpoint News Feed
AI
AI
雷峰网
雷峰网
T
Tailwind CSS Blog
Engineering at Meta
Engineering at Meta
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Recent Commits to openclaw:main
Recent Commits to openclaw:main
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
博客园 - 叶小钗
Webroot Blog
Webroot Blog
Apple Machine Learning Research
Apple Machine Learning Research
SecWiki News
SecWiki News
罗磊的独立博客
N
Netflix TechBlog - Medium
Martin Fowler
Martin Fowler
Google DeepMind News
Google DeepMind News
Cyberwarzone
Cyberwarzone
MongoDB | Blog
MongoDB | Blog
博客园 - Franky
Schneier on Security
Schneier on Security
The GitHub Blog
The GitHub Blog
S
Security Affairs
Blog — PlanetScale
Blog — PlanetScale
Last Week in AI
Last Week in AI
P
Proofpoint News Feed
月光博客
月光博客
D
Docker
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
S
Securelist
W
WeLiveSecurity
T
Troy Hunt's Blog
A
Arctic Wolf
博客园 - 司徒正美

Snorkel AI

Building AI-Native Systems for Federal Infrastructure: A Conversation with Rezaur Rahman Code World Models and AutoHarness for LLM Agents Benchtalks #1: Alex Shaw (Terminal-Bench, Harbor) – Building the Benchmark Factory Building FinQA: An Open RL Environment for Financial Reasoning Agents How Tool Discipline Let a 4B Model Outsmart a 235B Giant on Financial Tasks Coding agents don’t need to be perfect, they need to recover Closing the Evaluation Gap in Agentic AI SlopCodeBench: Measuring Code Erosion as Agents Iterate Introducing the Snorkel Agentic Coding Benchmark 2026: The year of environments Part V: Future Direction and Emerging Trends in Rubric-Based AI Evaluation The self-critique paradox: Why AI verification fails where it’s needed most Chat With the Terminal-Bench Team | Snorkel AI Intelligence per watt: A new metric for AI’s future Terminal-Bench 2.0: Raising the bar for AI agent evaluation Snorkeling in RL environments Introducing SnorkelSpatial: A Benchmark for LLM Spatial Reasoning Scaling Trust: Rubrics in Snorkel's Quality Process Evaluating Multi-Agent Systems in Enterprise Tool Use Evaluating Coding Agents with Terminal-Bench 2.0 Parsing isn’t neutral: why evaluation choices matter The science of rubric design The right tool for the job: An A-Z of rubrics Data quality and rubrics: how to build trust in your models Building the benchmark: inside our agentic insurance underwriting dataset Evaluating AI agents for insurance underwriting LLM observability: key practices, tools, and challenges Anthropic Claude + AWS: revolutionizing pharma data analytics with Snorkel AI Data-centric development of an enterprise AI agent with Snorkel Building the data development platform for specialized AI LLM-as-a-judge for enterprises: evaluate model alignment at scale Why GenAI evaluation requires SME-in-the-loop for validation and trust Research spotlight: is long chain-of-thought structure all that matters when it comes to LLM reasoning distillation? Why enterprise GenAI evaluation requires fine-grained metrics to be insightful What is specialized GenAI evaluation, and why is it so critical to enterprise AI? LLM alignment techniques: 4 post-training approaches Research spotlight: Is intent analysis the key to unlocking more accurate LLM question answering? Why enterprises should embrace LLM distillation Retrieval-augmented generation (RAG) failure modes and how to fix them What is large language model (LLM) alignment? Databricks + Snorkel Flow: integrated, streamlined AI development How LLM evaluation drives better models in Snorkel Flow Unlock proprietary data with Snorkel Flow and Amazon SageMaker LLM evaluation in enterprise applications: a new era in ML Snorkel AI joins the AWS ISV Accelerate Program and launches Snorkel Flow Availability in AWS Marketplace AI data development: a guide for data science projects SnorkelCon 2024: Inaugural Snorkel AI user conference gathers leaders from 30+ Fortune 500 companies Snorkel Flow 2024.R3: Supercharge your AI development with enhanced data-centric workflows Explore the new GenAI Evaluation Suite: Snorkel 2024.R3 New NLP features in Snorkel Flow 2024.R3 Enterprise data compliance and security review: Snorkel Flow 2024.R3 How a global financial services company built a specialized AI copilot accurate enough for production Task Me Anything: innovating multimodal model benchmarks Alfred: Data labeling with foundation models and weak supervision RAG: LLM performance boost with retrieval-augmented generation New GenAI features, data annotation: Snorkel Flow 2024.R2 How data slices transform enterprise LLM evaluation Meta’s Llama 3.1 405B is the new Mr. Miyagi, now what? Meta’s new Llama 3.1 models are here! Are you ready for it? Data-centric AI with Snorkel and MinIO Weak supervision for non-categorical applications + superalignment Snorkel AI signs strategic collaboration agreement with AWS to help enterprises cross the demo-to-production chasm AI alignment made simple: innovative solutions for businesses How does the Snorkel Flow label model work? Vision language models: how LLMs boost image classification Long context models in the enterprise: benchmarks and beyond How to build production-grade RAG retrieval with Snorkel Flow How Bonito helps fine-tune specialized LLMs faster than ever Walking safely before building flying saucer seatbelts: introducing Enterprise Alignment Role-based access controls in Snorkel Flow secure enterprise data Accelerating AI development in manufacturing with Snorkel Flow and AWS SageMaker How ROBOSHOT boosts zero-shot foundation model performance Discover what’s new in Snorkel Flow: Flexible data and LLM connectivity, secure data controls, and more! Faster than ever document intelligence with new Snorkel Flow FM-first workflow The art of data development for Enterprise LLMs Crossing the demo-to-production chasm with Snorkel Custom How Snorkel topped the AlpacaEval leaderboard (and why we're not there anymore) CRFM's HELM and enterprise LLM evaluation beyond accuracy How we achieved 89% accuracy on contract question answering Five sessions not to miss at Google Cloud Next 24 Content filtering breakthrough: Snorkel client reaches 96% recall in 3 days Here's how Snorkel Flow + Google AI built an enterprise-ready model in a day Snorkel teams with Microsoft to showcase new AI research at NVIDIA GTC How Skill-it! enables faster, better LLM training Fine-tuned representation models boost LLM systems. Here's how Enterprise GenAI to surge in 2024: survey results Large language model training: how three training phases shape LLMs LoRA: Low-Rank Adaptation for LLMs LLM distillation demystified: a complete guide Enterprises must shift their focus from models to data in AI development Insurance’s GenAI revolution: a business perspective Scaling human preferences in AI: Snorkel's programmatic approach Building better enterprise AI: incorporating expert feedback in system development “Fall in love with your data”—Snorkel AI’s Enterprise LLM Summit Why QBE Ventures invested in Snorkel AI New benchmark results demonstrate value of Snorkel AI approach to LLM alignment Retrieval augmented generation (RAG): a conversation with its creator Snorkel Flow 2023.R4: enhanced UI + PDF and Databricks tools How Snorkel Flow users can register custom models to Databricks Stanford professor discusses exciting advances in foundation model evaluation
Call center AI for customer experience management: a case study
kedia · 2024-08-14 · via Snorkel AI

In an effort to protect revenue, even the largest global systemically important financial institutions (G-SIFIs) are investing in their systems and teams to offer exceptional and seamless customer service across all channels. In my role at Snorkel AI, I recently had the pleasure of working on a project with one of the largest U.S. financial institutions to revolutionize how their call centers, perhaps the most information-rich channel, are informing customer experience management.

I recently spoke about this project with Matt Casey, Snorkel AI’s data science content lead. You can watch our conversation (embedded below), but I’ve also summarized the main points here.

Why call center analytics is hard—and why the bank wanted it

Call centers form the heart of customer service for many large organizations. Understanding this, leaders in many industries have a history of applying AI to customer call center engagements to improve the experience through predictive call routing, post-call sentiment analysis, and more.

While these efforts have had a tangible impact, corporations have struggled to apply these tools to the high-value application of real-time customer experience analytics. Where these systems exist, they rely on manual human labeling and focus on a narrow set of experience types, leaving them expensive to maintain and sluggish to adapt during times of dynamic change—precisely when organizational agility is most valuable.

Expanding the breadth of experiences their AI automation system could identify would enable companies to spot emerging trends as early as possible. This could enable leadership teams to launch initiatives to drive down expenses by understanding the root cause of customer behavior. They could also prioritize product and service efforts to enhance customer experience.

However, building an AI-powered system to uncover customer intent requires high-quality labeled data—a lot of it. To obtain this labeled data, employees must manually review each conversation, imposing a linear economic cost curve on the effort. Typically, shrewd assessments of the economic realities dictate that coverage can only be maintained for a portion of customer intents, leaving blind spots for emerging trends to go unnoticed. Furthermore, time-to-value of the AI-powered intent system is lengthy, giving it a profile more akin to a capital investment than a nimble experiment. Worst of all, an AI-powered intent system built on data that took nine months to refine can deteriorate rapidly as the competitive environment changes.

Despite these challenges, large enterprises regularly invest in AI-powered customer intent systems because they are powerful customer experience management tools.

The solution: call center AI fueled by Snorkel Flow

What if those economic constraints were elevated? Could you deliver a leap in customer experience if you weren’t forced to live with blind spots? What would the ROI profile of your AI-powered intent system look like if you could benefit from economies of scale when refining your data? How much easier would it be to secure funding if it took weeks instead of months to refine your data?

For this G-SIFI, the prospect was too good to pass up. Here is how Snorkel Flow’s core capabilities created a new economic reality for customer experience management.  

Scalability and agility

With Snorkel Flow, we deployed programmatic labeling functions across hundreds of intents. These labeling functions ranged from simple keyword-based functions to sophisticated sentiment analysis and other model-based approaches. Snorkel Flow intelligently combines and denoises these signals into a more powerful signal.

Snorkel Flow’s labeling function approach also allows users flexibility with their labeling schema. Subject matter experts often discover a mismatch between a project’s schema and its data over the course of an initiative. With traditional manual labeling, this kind of discovery means restarting the labeling process from zero or accepting the mismatch.

In Snorkel Flow, users can simply add, remove, or modify a handful of labeling functions and update their entire data set in as little as a few hours.

This flexibility is also helpful when the AI-powered intent system needs updating. As business needs, competitive atmosphere, or broader external factors change, Snorkel Flow users can return to the platform and modify a subset of labeling functions as needed.

Collaboration with SMEs

Successful machine learning projects hinge on effective collaboration between data scientists and domain experts.

In this case, the institution’s operational teams had years of experience and deep insights into what their customers hoped to accomplish when they called into a call center. Through Snorkel Flow, we empowered them to not only label transcripts but to explain the reasoning behind their labels. Data scientists then turn these explanations into labeling functions that Snorkel Flow applies to hundreds or thousands of records, breaking out from the linear economic cost curve of prior approaches.

In many instances, Snorkel Flow’s user interface allows analytically savvy SMEs to directly contribute their knowledge without needing deep technical expertise, further accelerating time-to-value.

Implementation call center AI and results

Let’s break down the implementation process and the significant outcomes that followed.

Identifying customer intents

Using Snorkel Flow, we created a system to automatically classify customer engagements into intents. This system encompassed a variety of labeling functions:

  • Keywords and substrings: Simple yet effective functions based on specific terms frequently associated with certain intents.
  • Embeddings: More complex functions that group similar customer engagements using NLP techniques
  • Sentiment analysis: Using pre-trained models like the SpaCy package to gauge customer sentiment and enhance categorization.

Importantly, these signals are not used directly by the final AI-powered intent system. Instead, these signals are indirectly represented by the labels, which are used to train the model that the bank deployed.

Addressing organizational agility

Non-programmatic labeling approaches struggle to keep up with changing contexts. For instance, customer concerns during the pandemic were starkly different from pre-pandemic times. While that’s an extreme example, enterprises constantly cope with emerging trends—from extreme weather to regulatory changes to new identity-theft attack vectors.

With Snorkel Flow, our client’s AI-powered intent system could adapt much more quickly to new trends. Not only does this allow for efficient maintenance, it also enables teams to easily peer back in time to assess how far back a trend emerged and conduct scenario planning for customer experience initiatives.

Deployment and Future Expansions

We recently helped deploy the first version of the enhanced AI-powered customer intent system. The system is already improving the operational efficiency of the intent system and routing freshly minted intent data to product and services teams who will leverage it to reduce call center expenses while improving customer experience.

With Snorkel Flow’s continuous updates and new features, the institution is prepared to iterate on this initial version, refining and expanding its capabilities. Furthermore, the insights gained from call data will not only improve call center operations but also inform broader business strategies.

The success of the call center initiative represents step one in a multi-year plan. Other teams within the bank are exploring similar methodologies for various customer interaction points. Having a consistent and scalable solution across different customer touchpoints ensures cohesive and comprehensive insights. This holistic view of customer intent promises to provide the financial institution with a powerful new capability to provide a frictionless and more personalized customer experience.

Snorkel AI: Enabling better banking through better data

This case study showcases machine learning’s potential to simultaneously transform operational efficiency while also delivering a big leap in customer experience. By leveraging Snorkel Flow, this global systemically important financial institution successfully addressed the scalability, time-to-market, and obsolescence challenges that customer intent identification initiatives face.

As enterprises continue to embrace digital transformation, the ability to quickly adapt to changing environments while maintaining high service standards will be a key differentiator. At Snorkel AI, we are proud to contribute to our clients’ successes by providing innovative solutions to help them realize the full benefit of AI.

Ready to accelerate AI development?

Deploy production AI and ML applications 10-100x faster with Snorkel’s experts, using our proprietary technology.

Request a demo