Engineering leaders are sure AI is driving value, but struggle to quantify its impact

By Lauren Lang, Uplevel | Reviewed by Irfan Ahmad

Imagine a tech company starts shipping software features 30% faster, creating an uptick in adoption and revenue. Leadership celebrates the win, crediting engineering’s AI tool adoption for the gains.

But how do they know if AI was actually responsible, especially if it was adopted at the same time as new review processes, a reorganization, or other factors?

Many organizations today aren’t bothering to check. According to Uplevel’s new survey of 428 engineering and technology leaders, “Would You Recognize AI ROI If You Saw It?”, 96% of engineering leaders are confident the value they see is attributable to AI (rating confidence a 3 or 4, on a 4-point scale). But only 40% use controlled comparisons — looking at teams using AI vs. teams that aren’t — to isolate AI’s impact as the cause.

How leaders try to isolate Al's impact
Image Source: Uplevel’s 2026 “Would You Recognize AI ROI If You Saw It?” report

The report’s data showed that Vice Presidents (VPs) are far more likely than other leaders to attribute value they see to AI. More than 3/4 (76%) are very sure or extremely sure (rating confidence a 4, on a 4-point scale) AI is responsible.

Given that many VPs sign the purchase orders for AI tools and are held accountable for what their organizations deliver, they’re often under the most pressure to show results.

Reported Al value and attribution confidence by seniority
Image Source: Uplevel’s 2026 “Would You Recognize AI ROI If You Saw It?” report

When it comes to actually measuring AI performance, the survey revealed leaders haven’t changed their methods much. Nearly three-quarters (74%) report that they have extended their pre-AI measurement approaches to AI initiatives. Just 4% are building new metrics specifically for AI.

This poses significant problems, as the old ways of measuring engineering performance and productivity don’t translate cleanly to an AI-first software development lifecycle. Many organizations still track output metrics like story points and pull requests (PRs) closed, but AI can artificially inflate these metrics overnight. The extra code generated also creates new bottlenecks in review and deployment, and presents significant security risks.

Speed alone does not necessarily translate to value delivered. As AI redefines productivity and blurs the lines between product and engineering, the organizations positioned to make the most of it will measure value further downstream, including delivery metrics like lead time and change failure rate and business outcomes such as adoption and revenue.

Most organizations already have enough data to get started, even with a little “back of the napkin” math. AI usage still varies widely between teams. Comparing teams of AI power users with teams that use it lightly for similar work gets close enough to a controlled comparison. From there, follow the work downstream. If AI-heavy teams open twice the PRs while deployments stay flat, the gains aren’t reaching customers yet.

Realistically, AI is probably creating value in most organizations. But when a CFO sees it as a surging line item in an engineering budget, attribution matters. Leaders putting their 2027 tooling proposals together should be able to both explain where AI is creating value and quantify how much. The purpose of an attribution report is to inform future investment decisions. When it’s accurate, your case for next year’s tooling spend is much easier to make. But when the report credits AI for gains it didn’t drive, engineering organizations could end up spending millions with little to show for it.


About author: Lauren Lang is Director of Marketing and Strategic Operations at Uplevel. With extensive experience in SaaS and AI/ML technology marketing, Lauren is passionate about helping tech leaders create and sustain healthy, productive teams.

Read next:

• Advertisers That Refreshed Creative Weekly Saw 23% Higher Black Friday Returns on AppLovin

• Mermaid diagram searches hit record level after AI incorporation

Previous Post Next Post