4 Min Read
Businesses often start AI measurement in the wrong place, asking:
How many employees are using ChatGPT?
How many prompts were written?
How many licenses are active?
Those are easy to count but poor at revealing any real change.
A better starting question is: What jobs are we "hiring" frontier AI technologies to do within our organization?
I use the phrase metrics of value because it's familiar language. But what I'm really interested in are metrics of meaning -- ways of making sense of the impact and experiences that breakthrough technologies enable.
AI doesn't just add a tool; it changes how work gets done. And when work changes at the task level, value can emerge in places our dashboards don't show.
We must stop treating job titles and end-to-end workflows as the only taxonomies that matter. We need to become proficient at decomposing tasks. Research suggests the vast majority of jobs, when decomposed, contain somewhere between 15 and 25 major tasks. Once you can see the tasks, a fundamental question becomes unavoidable: To what degree can each task be responsibly, safely and accurately automated, augmented or agentically assisted?
Related:Will the music stop for AI's funding dance?
If you are restructuring the anatomy of work, task by task, you are changing something in the DNA of the workflow. So why would we expect the same legacy metrics and KPIs we used before to remain fully accurate?
As organizations implement and integrate AI, they may be generating new metrics of meaning that they haven't previously recognized or even named. Human perception is narrow. We often reuse old measurement logic because it's familiar to us. But major technology shifts can force us to invent new ways of describing reality. AI will be no different.
The following are five metrics I find practical because they map directly to how AI changes decisions, creativity, innovation and learning.
Decision velocity. How is AI shortening the decision cycle and improving responsiveness? In some knowledge-work settings -- software engineering is a common example -- there is evidence that AI can reduce task time by 30% to 40% in at least some cases. If tasks are completed faster, what happens to the speed of decision-making across a team or function?
Decision quality uplift (and better questions). If I show my reasoning to a machine, the machine can help me fine-tune my approach by identifying gaps, challenging assumptions and suggesting refinements. There's also a flip side: You improve not only the answer, but also the question. Better questions lead to better outcomes.
A human empowerment index. Generative AI can elevate human creativity: It helps people generate options, draft, iterate and explore. But it can also make people worse in some areas if they outsource too much thinking. So, track empowerment over time: Is AI expanding human capability or constraining it?
Innovation yield. Organizations generate lots of ideas, but few become real features or improvements. If AI agents are now part of ideation, what is the force multiplier? What is the improvement in yield, the percentage of ideas that make it into reality? In my teaching at MIT Sloan School of Management, I've been leveraging research suggesting humans who ideate with AI agents can outperform humans ideating alone, and that a team using a single shared AI agent can avoid the overhead of aggregating multiple agent outputs.
Learning loop efficiency. Learning loops are critical to workforce development. Apprenticeships worked because one human helped another learn over time. AI introduces the possibility of new learning loops between machines and humans, and between humans themselves when a machine is present. The question then becomes: Does AI improve the speed and quality of learning on the job?
Related:The hidden high cost of training AI on AI
Related:Red Hat CIO Marco Bill: Resource control is key for AI sovereignty
Some measures will always matter. Employee and customer satisfaction are like taking a temperature. But alongside those, we need to stay alert for the new metrics. It's dangerous to live in a world where you have impacts you can't measure. Metrics shape behavior. The metrics we choose for AI will shape what leaders optimize, what teams build and what enterprises become.
So if you want to measure AI transformation, don't start with "How many people used the tool?" Start with the work itself: decompose it, see which tasks have changed, and then ask what new meaning is being created. The value is often there. The question is whether we've built the units of measurement to see it.
Visiting lecturer, MIT Sloan
Paul McDonagh-Smith is a visiting senior lecturer in IT at the MIT Sloan School of Management. In his research and teaching, Paul creates key intersection points between technology and business. He specializes in translating computer and data science into measurable business value that evolves organizational capability, transformation and strategy.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。