As more organizations successfully adopt artificial intelligence (AI), many are discovering that success comes with a growing challenge: managing costs. A recent survey of 700 engineering leaders and practitioners from organizations with more than 1,000 employees found that 26 percent of AI spending is wasted, often with little or no measurable return. For managed service providers (MSPs), that creates a significant opportunity to help customers gain better visibility, control costs, and maximize the value of their AI investments.
AI spending is rising faster than oversight
A survey published this week by Harness, a provider of a DevOps platform, found that organizations spending $1 million per month on AI waste roughly $260,000 monthly.
Nearly three-quarters of respondents said their organization experienced an unexpected AI cost spike or bill during the past year. One-third (33 percent) reported being caught off guard more than once.
Tracing the source of these cost increases is often difficult. Eighty percent of respondents said it takes a full day or longer to identify the cause of an AI cost spike, while 32 percent need as much as a week.
Measuring AI value remains a challenge
Only 26 percent of respondents said their organization has a robust method for measuring the business value generated by AI investments. More than half (56 percent) admitted AI spending forecasts are based on guesswork rather than data.
On average, fewer than half (45 percent) said they understand the cost of the AI features they build. Meanwhile, 57 percent of engineers said their organization actively encourages “tokenmaxxing” without fully understanding the business value it delivers.
Not surprisingly, only 21 percent described their AI spending management practices as fully mature.
Governance gaps create inefficiencies
The survey also revealed significant governance challenges. While 73 percent of organizations have AI cost policies in place, only 47 percent actively enforce them.
In addition, 52 percent of respondents said no single person owns AI costs within their organization. Responsibility is often split among engineering, FinOps, finance, and IT teams, making accountability difficult.
Without clear ownership and consistent oversight, controlling AI spending becomes much harder.
Why MSPs are well positioned to help
These findings will sound familiar to MSPs that have built cloud management practices. A significant amount of cloud infrastructure is often underutilized or unnecessarily allocated.
The difference is that AI infrastructure is considerably more expensive. As a result, the financial impact of inefficiencies can be much greater.
Organizations are already redirecting budgets from other areas to fund AI initiatives. That shift is likely to increase demand for services that optimize AI spending, much like organizations previously sought help managing cloud costs. In fact, interest in AI cost optimization may eventually surpass demand for traditional cloud infrastructure optimization.
The next evolution of infrastructure management
AI adoption remains in its early stages, but organizations continue to embed AI into business and IT processes at a rapid pace. The question is no longer whether optimization challenges will emerge, but how quickly they will appear.
Most organizations cannot fund every AI initiative indefinitely. Many will need to prioritize the applications that deliver the greatest business value. At the same time, they will face growing pressure to ensure expensive AI infrastructure is fully utilized rather than sitting idle between workload spikes.
For MSPs, the opportunity is clear. Most already have the tools, processes, and expertise needed to improve infrastructure efficiency. The challenge now is applying those capabilities to AI workloads, which often consume resources in less predictable ways than traditional applications.
Photo: Natee Meepian / Shutterstock
This post originally appeared on Smarter MSP.