Book a call

Evals and accuracy

Let AI approve pull requests only under a written risk policy

Amol Patil, Founder · 3 September 2026 · 8 min read

Explore with AI
ChatGPTPerplexityClaudeGoogleGrok
  • GitHub admins can now let Copilot approve pull requests, and those approvals count toward a repository's required approvals. The setting is off by default.
  • Switching it on is a policy decision. Write down which changes the AI may approve alone, which only after named tests pass, and which always need a person.
  • Enforce the policy in the tools, through Copilot's allowed file paths, code owners and required checks, and move agent-written changes up a tier.
  • Log every AI approval, audit a sample each week, and widen the AI's scope only on evidence from those audits.

We think an AI approval should count only inside a written risk policy that names the changes it may approve, the changes that need a person and the tests that must pass first. An approval is the record that auditors and incident reviews ask for, so the rules for who may give one belong in writing before anyone flips the switch.

On 1 September 2026 GitHub announced that Copilot code review can approve pull requests, in public preview on its Pro, Pro+, Max, Business and Enterprise plans.

What the new setting does

Every Copilot review now carries an approval assessment, a note on whether Copilot thinks the pull request is ready. That note alone does not count toward merge requirements. Once an admin enables approvals, Copilot can submit an approval that counts toward the repository's required-approvals rule. New commits dismiss it, as they would a person's approval.

Admins control the setting at enterprise, organization and repository level, and repository admins can choose which file paths Copilot is allowed to approve. That path list is where a written policy becomes enforceable, so we would write the policy first and set the paths from it.

The changelog does not say whether Copilot may approve a pull request that Copilot wrote. When GitHub launched its coding agent in May 2025, it set a default that the developer who asks the agent for a change cannot approve the resulting pull request, so required reviews still involve someone else. A policy for AI approvals needs an equally explicit rule for changes an agent wrote.

Why teams will switch it on

Review capacity has not kept up with agents. Intercom's engineering blog explained in April why it lets an AI reviewer approve some pull requests. Under agent volume, it argues, reviewers either let the queue back up or start rubber-stamping, glancing at a diff and clicking approve. Its reviewer refuses large pull requests, any engineer can ask for a human review, and the engineer who ships a change stays accountable for it.

Intercom's AI approvals, in its own numbers

Intercom's figures about its own system, published 21 April 2026, not an independent measurement.

Smaller teams feel the same pressure. On Lenny's Newsletter's How I AI in August, solo founder Ryan Carson described a skill that closes the loop on 40 pull requests a day without a QA team reviewing them.

We read both as the same signal. AI approval is coming to most teams, and the open question is which rules it runs under.

Three tiers, written down

We would begin with three tiers and adjust the lists to each repository. Rewind, which published its design in June, keeps a short list of paths that always need a human, covering authentication, authorization, secrets, encryption and anything that changes who may approve future changes.

Three tiers for AI approval

AI may approve aloneAI approves after tests passA person must approve
Typical changesDocs, comments, copy text and test-only changesApplication code covered by existing tests, and dependency patch updatesAuthentication, permissions, secrets, encryption, payments, data migrations, CI workflows and the policy file
Checks before the approval countsBuild and lint passThe full test suite passes, including tests that exercise the changed linesAll middle tier checks, then review by a named owner
SizeSmall diffs onlyUnder a line limit the team setsNo limit, though large changes should be split
If an agent wrote the changeTreat it as the middle tierTreat it as needing a personA person approves
How the tools enforce itCopilot's allowed file pathsRequired status checks before mergeCode owners, with code owner review required
Evidence keptThe AI review and its approvalThe AI review, its approval and the test resultsThe AI review, the person's approval and the test results

  • AI may approve alone

    Typical changes
    Docs, comments, copy text and test-only changes
    Checks before the approval counts
    Build and lint pass
    Size
    Small diffs only
    If an agent wrote the change
    Treat it as the middle tier
    How the tools enforce it
    Copilot's allowed file paths
    Evidence kept
    The AI review and its approval

  • AI approves after tests pass

    Typical changes
    Application code covered by existing tests, and dependency patch updates
    Checks before the approval counts
    The full test suite passes, including tests that exercise the changed lines
    Size
    Under a line limit the team sets
    If an agent wrote the change
    Treat it as needing a person
    How the tools enforce it
    Required status checks before merge
    Evidence kept
    The AI review, its approval and the test results

  • A person must approve

    Typical changes
    Authentication, permissions, secrets, encryption, payments, data migrations, CI workflows and the policy file
    Checks before the approval counts
    All middle tier checks, then review by a named owner
    Size
    No limit, though large changes should be split
    If an agent wrote the change
    A person approves
    How the tools enforce it
    Code owners, with code owner review required
    Evidence kept
    The AI review, the person's approval and the test results

Our starting policy. The lists are examples; set them for each repository.

Agent-written changes move up a tier because, without that rule, a change can reach production with no person having read it.

Tests come before the approval

An approval is an opinion about a change, and a passing test is evidence about it. The same difference makes a test suite a better buy than tester hours. So the tests should run first, and the AI should review only what passed.

Claire Vo's review bot, built on Vercel Eve and shown on How I AI in August, reads each pull request only after the checks pass. It scores risk on six dimensions, one of them the verification gap, and pings her in Slack when a person is needed. In the episode roundup, pull requests under 24 points are cleared, those above 64 go to a person, and the bot never merges anything.

Some files need more than tests and an AI opinion. Wiz reported in August that a GitHub Actions workflow change in a public Snowflake repository let anyone run commands on the build runner by opening an issue with a crafted title. Wiz says Copilot, as a co-author, checked the change as all clear, and the automated security scan did not flag it.

Wiz's agent found and exploited the flaw through Snowflake's disclosure program five days after merge, and Snowflake fixed it the day it was reported. That is why CI workflow files sit in the person tier.

Keep score on the AI reviewer

Audit the AI's approvals the way you would audit a new reviewer's. Log every approval with its review, its test results and the merge. Intercom says every AI-approved pull request is labelled, logged and queryable, and it brought its auditors in before scaling.

In her episode, Vo argues that auto-approved pull requests can be SOC 2 compliant if the process is auditable, queryable and written into the risk policy. This is not legal advice. Check the approach with your auditor before you rely on it.

Then sample. Have an engineer read a share of AI-approved pull requests each week and record whether each approval was right. Track reverts and incidents linked to AI approvals. Widen the AI's tier when those numbers hold and narrow it when they slip.

This is the verification loop from the agent as a compiler, applied to code review. Measure the reviewer on your own pull requests, not on a vendor's benchmark.

Write the page before you flip the switch

Before anyone enables Copilot approvals, write one page. It lists the three tiers with their file paths, the checks each tier needs, the rule for agent-written changes, the log you keep and the numbers that widen or narrow the AI's tier. Then set Copilot's allowed paths to the first tier only, and put the third tier's paths in a CODEOWNERS file with code owner review required, so a named owner must review them before merge.

Our agentic AI development method puts the same pieces around any agent, a code reviewer included: evals, guardrails, logs and a human escalation path.

Sources

· views
Amol PatilFounderFounded extendfuture in 2019. Has shipped computer vision, voice and agentic systems into production across ten industries. Amol Patil on LinkedIn

Working on something in this territory?

Tell us what you are trying to win. We answer within one business day, from the people who build.

Start a projectBook a call