Evals and accuracy

Companies are cutting manual testing, not testing

Amol Patil, Founder · 7 August 2026 · 8 min read

Explore with AI

ChatGPTPerplexityClaudeGoogleGrok

The short version

• EPAM told investors on 6 August that North American clients are moving budget away from task-based services such as manual testing, faster than replacement work is arriving.
• We read that as a change in what companies buy, not a decision to test less. Hours buy a judgment about one build. An automated suite produces evidence on every change.
• Move the money to the suite and its evidence: tests that run on every change, a record of each run, and owners who keep the tests working.
• Keep people for exploratory testing and judgment, and pay suppliers for tests that stay in your repository, not for hours logged.

We think the budget leaving manual QA, the testing people do by hand, marks a change in how companies buy testing, not a decision to test less. It matters because a team that cuts tester hours without paying for automated suites, and for the evidence those suites leave on every change, has cut testing itself.

What EPAM told investors

EPAM, an engineering services firm with about 62,850 staff, reported its second quarter on 6 August.

On the earnings call, chief executive Balazs Fejes said that since July, North American clients have been moving budget away from task-based services such as manual testing, user experience work and front-end JavaScript, toward AI-led modernization. He said the move is outpacing the new work that replaces it, and that clients plan to pay for what EPAM calls AI-native services mainly out of what AI saves them.

Fejes added that some of the spend leaving EPAM goes to tokens, the units AI models are billed in, and to hardware and infrastructure that clients buy directly. That is the risk we would watch.

A testing budget that turns into tokens buys more code and less evidence that the code works. The measure we argue for is cost per successful outcome, and a change counts as a success only after something has checked it.

He also reminded analysts that most of EPAM's business runs on time-and-material contracts, billed by the hour or day and renewed regularly. For a QA lead, that is the detail to note, because an hours contract is the easiest line to cut at renewal. Fejes summed up the wider shift as "Coding gets automated, engineering doesn't." Our corollary is that testing by hand gets automated, and testing does not go away.

Hours buy a judgment, a suite buys evidence

A manual test cycle produces a person's judgment about one build at one moment. When the next change lands, that judgment is out of date and has to be bought again. An automated suite produces evidence on every change: which version was checked, which tests ran, what passed and what failed. The record stays readable after the people who wrote the tests have moved on.

Google faced the same choice in 2005. In the testing chapter of Software Engineering at Google, Adam Bender describes the Google Web Server team at a point when more than 80% of its production pushes carried user-affecting bugs that had to be rolled back.

The team then required tests with every code change, run continuously. Within a year, emergency pushes fell by half, even though the project saw a record number of changes every quarter. The fix was a rule for every change, owned by the engineers making it.

Fewer checks by hand need more tests

When AI writes more of the code, people stop checking every change by hand. "AI writes most of my code now," Claire Vo said on 5 August on How I AI, a Lenny's Newsletter podcast, and her queue of pull requests, the proposed code changes waiting for review, outgrew her. Her view is that the answer is not to review all of them.

We agree, on one condition. Whatever a person stops checking by hand, a test has to check on every change. Without a suite, the person or model reviewing a change has only the changed lines to go on.

Platforms now sell the gate itself. GitHub's Code Quality, generally available since 20 July, shows test coverage, the share of code the tests exercise, on each pull request from a team's existing test reports. It can also enforce a coverage threshold before merge.

GitHub lists it at $10 per active committer per month, plus usage charges for its AI features and compute. The gate reads test reports. It cannot write the tests, and the tests are the part worth paying for.

What to buy instead of hours

Build the suite from your own failures, not from a generic checklist. Run cheap checks on every change and heavier ones less often, the verification loop we described in the agent is a compiler.

Where people still belong

We disagree with treating manual testing as a cost to delete. People find the failures nobody wrote a test for, judge whether a flow makes sense to a customer, and decide what done means. A suite checks only what someone thought to write down, so pay for that human work as judgment, not as volume.

Our human-in-the-loop approach uses the same split: automated checks take the volume, people review what the checks cannot judge, and every correction becomes a new test.

To size the gap, take your last ten production bugs and ask, for each, whether a test in your suite would have caught it before release. The number of no answers is the size of the suite you still need to buy.

Sources

• EPAM Systems, EPAM Reports Results for Second Quarter 2026, Form 8-K exhibit 99.1, 6 August 2026
• The Motley Fool, EPAM Q2 2026 earnings call transcript, 6 August 2026
• Lenny's Newsletter, Claire Vo, Build an AI code review bot in 30 minutes with Vercel Eve, How I AI, 5 August 2026
• GitHub Changelog, GitHub Code Quality is now generally available, 20 July 2026
• Adam Bender, Testing Overview, chapter 11 of Software Engineering at Google, O'Reilly, March 2020