Home Business Most work doesn’t need the Ferrari of AI, Wharton and Accenture find,...

Most work doesn’t need the Ferrari of AI, Wharton and Accenture find, but companies may be paying for it anyway

0
2

When companies unleash AI agents, they like to reach for the shiniest, newest, and most powerful model and use it for every job task. A new joint report from the Wharton School and Accenture published Monday suggests that might be overkill.

The landmark new report, What Does Agentic AI Really Cost?, studied more than 18,000 tasks and work activities across 923 occupations. More than 90% of tasks in the framework that AI agents could take over are routine enough for the less expensive models. Moreover, in each of the 19 industries studied, only 6% to 11% of the work AI agents could do is complex enough to typically need a bleeding edge “frontier” model, according to the research findings. Meaning, complex tasks that need frontier models are a small portion of the workflow. 

Yet, Eric Bradlow, vice dean of AI and analytics at Wharton and a coauthor of the report, said he doubts most companies are making that distinction.  

“I don’t even think most companies are thinking about deploying differential models for different parts of the workflow,” Bradlow told Fortune. “I think they’re taking possibly their most sophisticated model and applying it to all of the workflow.”

That may not deliver the best return, he said, “because certain tasks, a large fraction of tasks, just do not require the more sophisticated models.”

Chasing Shiny Objects

Bradlow said people often seek out the latest—and usually marketed as the greatest—model. 

“Most people tend to want to have, I’ll call it the shiniest object,” he said. “But for most tasks, actually, you don’t.”

To reach their findings, the researchers classified which tasks an AI agent could undertake, weighing how much human interaction and physical activity each task involved. That left 11,339 tasks. They scored each on five measures of complexity including how many interdependent steps it involves; whether legal, regulatory, or audit requirements apply; how many computer systems it affects; whether it needs sensitive data; and, how much data it needs.

James Crowley, Accenture’s Americas industry and enterprise lead and a coauthor, said complexity isn’t only about raw-computing difficulty. A task can be complex because of “the workflow navigation, or the number of systems you touch, or the regulations,” he said. 

For instance, in life sciences, tasks that scored as complex included developing quality-control systems for production and drafting legal documents like patent applications. Tasks that ranked lower included enrolling people in programs and drafting copy. 

“It’s not an argument against frontier models,” noted Crowley. “They remain essential for complex, specialized, high-stakes work.”

In the capital markets, tech, and aerospace, the report found that higher-volume tasks tend to be more complex and a better fit for frontier models more often. Crowley pointed to aerospace and defense companies trying to raise production rates, which requires close tracking of certifications, quality, and the availability of huge amounts of parts across complicated supplier networks. That’s “an extremely complex task, super high value,” he said. 

Bradlow pointed to research and development in health care, where AI can help detect promising mixes of molecules. Those tasks are complex and expensive, said Bradlow, but the return for a company could lead to billions in revenue. 

AI Isn’t Free

The choice of model is an important one because AI compute is among the largest costs in most of the report’s industry estimates and one of the main reasons cost can differ dramatically between industries. For a company with 100,000 employees running 35 agents, estimated five-year costs range from about $155 million in health care to about $520 million in insurance. 

And while AI models are getting cheaper, total spending is on its way up, said Bradlow. 

“If I ask it to do something from a year ago, it’s cheaper to do it today,” he said. “But I’m asking it to do 100 different things I wasn’t asking it to do a year ago. So my total cost is going up” even as “my per-task cost is going down.”

Crowley and Bradlow emphasized that companies should track what AI costs alongside the value it generates, in what the report calls an AI P&L, or profit and loss statement. 

“The minute you start thinking of a P&L, it changes,” said Bradlow. It can also do away with the thinking that using an LLM isn’t an expense. People think, “I’ll just use this LLM because it’s quote-unquote free,” he explained. “No, it’s not. It’s not free.”

The report recommends deciding what model capability each agent needs at the design stage, before it is built. Dynamic model routing, in which tasks are sent to different models as the work runs, is difficult to track in live operations and few enterprises can run it at scale, wrote the authors. The report is the third in a series published jointly by Wharton and Accenture. 

Crowley said switching between models is technically doable. The challenge, however, is that “the model choice often happens too late.”

“People tend to use the model that they already have access to rather than what’s best suited to the task,” said Crowley. 

Bradlow expects that to change. He envisions AI systems that can take in a request, break it into parts, and send each part to the right model on their own. 

“We’re not 100% there yet,” he said. “It’s coming, and it’s coming sooner than later” because “the customer’s going to demand greater ROI, and one way to do it is to lower costs by using less sophisticated models.”

Disclaimer : This story is auto aggregated by a computer programme and has not been created or edited by DOWNTHENEWS. Publisher: fortune.com