Sam Mugel, Ph.D., is the CTO of Multiverse Computing, a leader in developing efficient value-driven AI & quantum solutions for businesses.
getty
The promise of large language models in enterprise settings has proven difficult to deliver on. According to the RAND Corporation’s analysis, more than 80% of AI initiatives never reach meaningful production deployment.
While this research and most post-mortems point to integration as the cause, that diagnosis understates the problem. Integration failures are usually symptoms of a more fundamental issue where the model was never designed to understand the environment it was deployed in.
General-purpose models are trained on the public internet, encyclopedias, books and code repositories. This gives them remarkable breadth.
However, as Roberta Cozza, vice president analyst at Gartner's Technology and Service Providers, explained in a recent interview: “A generic AI that doesn't speak to the specific challenges, processes and content that an enterprise has is not really helping."
In other words, general-purpose models have no exposure to the proprietary knowledge that defines how most businesses actually operate: internal APIs, legacy system logic, industry-specific terminology developed over decades, compliance edge cases and the tacit decision rules embedded in operational workflows that have never been documented anywhere a public model could find them.
The most common response is fine-tuning or retrieval-augmented generation (RAG). Both approaches layer domain-specific information on top of a general model, improving surface performance without changing how the model reasons.
When pushed into unusual edge cases, these strategies often revert to general behavior when multi-step reasoning across proprietary concepts or queries requiring institutional knowledge. Performance plateaus because the foundation hasn't changed.
A domain-specific language model (DSLM) is trained from the ground up on the structures, terminology and logic of a specific domain.
The model internalizes domain knowledge during training rather than as a retrieval layer at inference time, learning to reason within the domain instead of mapping general patterns onto it. Because it only needs to understand a specific domain, it is inherently smaller, faster and less compute-intensive.
A general model operating in a specialized domain produces outputs that are linguistically fluent but semantically imprecise. It might generate a SQL query with correct syntax that misreads the schema or summarize a regulatory clause in a natural language that subtly misrepresents the compliance requirement.
By contrast, a domain-specific model fails within the logic of the domain. Meaning, a wrong answer that reflects domain understanding rather than a plausible-sounding approximation. One failure mode is harder to catch. The other is easier to diagnose and correct.
In finance, Bloomberg’s 50-billion parameter model trained on more than 360 billion tokens from four decades of its own news, filings and market analyses outperformed general models across financial natural language processing tasks. The gap was most pronounced on complex, multi-step tasks like conversational financial question answering, where general models are most likely to approximate rather than reason.
Similarly, in healthcare, Google's Med-PaLM 2 scored 86.5% on the MedQA benchmark, and physicians evaluating its responses to over 1,000 consumer medical questions preferred its answers on eight of nine clinical axes. Compared directly against GPT-4, Med-PaLM 2 was rated significantly safer, with a lower likelihood of harm and no detectable bias across patient subgroups.
Benchmarks matter, but the more instructive picture is what changes operationally.
In document-heavy workflows, general models require constant human validation because their outputs can’t be trusted to reflect institutional knowledge. Teams spend significant time catching plausible-sounding errors that arise in downstream processes.
That dynamic shifts after deploying a domain-specific language model. In one clinical trial inference system my company worked on, teams that previously reviewed every model-generated query or summary for compliance accuracy were able to shift to exception-based review, intervening only when the model flagged uncertainty or when outputs fell outside defined parameters.
That’s because a model that understands the domain fails less often. When it does fail, it fails in ways that are easier to catch and correct. This reliability is what will allow organizations to increase autonomy over time rather than keep humans in the loop for every step.
Because of the nuances of working with domain-specific data, DSLMs are not a turnkey solution. Based on my experience deploying DSLMs, here are considerations that can shape the success of an implementation:
• Start with a data audit. A domain-specific model is only as good as the data it trains on. Assess whether you have sufficient proprietary data that’s labeled correctly, governed properly and representative of the edge cases that matter before any training begins.
• Define the domain boundary tightly. The performance advantages of DSLMs come from their narrowness. Organizations that scope too broadly lose the precision advantage they were seeking.
• Evaluate on native criteria. Standard benchmarks measure general language performance and won’t surface the failure modes that matter in specialized deployments. Build evaluation sets from your own operational data, such as queries, edge cases and compliance scenarios.
• Plan for maintenance from day one. Regulations change, terminology shifts and internal processes evolve. A model trained on a static snapshot will degrade without continuous data curation and periodic retraining. Treat model maintenance as an operational function.
For CISOs evaluating AI, the key question is whether a model truly understands the environment it operates in. Organizations that embed proprietary workflows, institutional knowledge and domain expertise into model design build on a fundamentally different foundation that shows its value in complex decisions, edge cases, regulated outputs and autonomous operations.
General-purpose models are a strong starting point, but as use cases become more specialized, regulated and mission-critical, the gap between models that approximate a domain and those built specifically for it will keep widening.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。