AI SAST on GitHub: AI Scan Skips the Pull Requests Most Open Source Projects Merge

GitHub's AI SAST tool, AI Scan for pull requests, went generally available on 15 October. It is an AI-based scanner for code CodeQL does not analyse: the docs name PHP, Shell/Bash, Terraform HCL, Dockerfiles, JSP and Blazor as examples. Four days on, the reason to look is the GA docs revision, which dropped the preview's Copilot licence requirement and opened the feature to public organization repositories without an active-committer licence. Usage is still billed in AI Credits, public repositories included, but it now looks like an easy reviewer to add to an open source PHP or shell project.

It is not, because of one run condition: AI Scan skips pull requests from forks and pull requests opened by Dependabot. On a public repository, fork pull requests are most of the outside code a project accepts.

The claim, stated fairly

The changelog says AI Scan complements CodeQL by finding vulnerabilities in pull request changes for languages CodeQL does not cover, with no separate tool or build configuration. That much holds. It runs on every new commit to a qualifying pull request, even when CodeQL analysis fails, and sorts findings into nine categories, from string injection to unpinned third-party actions.

Three boundaries are documented and undisputed:

• Advisory only. Findings do not fail the code scanning check, and rulesets cannot use them to block a merge.
• Pull requests only. It never scans the full repository, so code merged before it was turned on is never reviewed.
• GitHub.com only. The feature's versioning file in the github/docs repository lists only the free-pro-team and Enterprise Cloud builds, not Enterprise Server.

Where the AI SAST claim flips: who opened the pull request

The fork and Dependabot exclusion was not on the July preview page. The github/docs history shows it arrived on 16 September, in the revision that unlinked AI Scan from CodeQL default setup. It is documented scope, not a launch-day gap.

To see what it means in practice, I pulled merged pull requests from eight public repositories whose main language is one AI Scan targets. For each, I took the 300 most recently updated closed pull requests from the GitHub API and kept those merged between 19 September and 19 October. A pull request counts as in scope only if its head branch lives in the base repository and Dependabot did not open it.

That is 198 in scope out of 1,050, and nextcloud/server supplies 179 of them. Leave Nextcloud out and AI Scan could have reviewed 19 of 777 merged pull requests, about 2.4 percent. Four of the eight projects would have had nothing scanned at all.

Nextcloud is the boundary condition. Its contributors push branches to the main repository, so most of its traffic qualifies. The variable is not language or size but contribution model: fork-based projects get almost no coverage, and branch-based ones get most of it.

I do not know whether any of these projects has AI Scan on; this measures eligibility, not findings. Nor does it measure detection quality: the docs read for this post give no precision figures and warn of false positives.

What survives inside a company

For private and internal repositories, the question is the one Nextcloud answers. If your team pushes branches instead of forking, the exclusion rarely applies, so check rather than assume. AI Scan there needs GitHub Code Security or GitHub Advanced Security, and Code Security AI Credits usage starts when a qualifying pull request is scanned. Credits included with a Copilot entitlement are used first, and metered usage follows. Two details in the enablement section matter:

• The enterprise policy defaults to Not allowed. Once an organization owner or security manager enables it, though, it turns on immediately for every eligible repository with code scanning. A repository admin can only opt out, and qualifying pull requests can spend credits before they do.
• If a hard budget leaves no credits, AI Scan produces no findings until the budget rises or a new billing cycle starts. A scanner that goes quiet when money runs out cannot be a gate.

This is the familiar last-mile problem from why enterprise AI doesn't ship: the model is the easy part, and the work is deciding who sees what, under which policy, at what cost.

What you'd need to see before relying on it

Treat AI Scan as an extra reviewer on your own team's branches, not as the SAST control for code you do not write. If GitHub extends AI Scan to fork pull requests, or lets rulesets require its results, the coverage argument in this post no longer holds. Either change would show up in the Limitations list of the docs page above.

Until then, measure your own exposure before you budget for it. This prints merged, fork and Dependabot counts over the same window, from the 100 most recently updated closed pull requests. The remainder is in scope:

gh api 'repos/OWNER/REPO/pulls?state=closed&per_page=100&sort=updated&direction=desc' | jq --arg since 2026-09-19 --arg bot 'dependabot[bot]' '[.[] | select(.merged_at != null and .merged_at >= $since)] | {merged: length, fork: map(select(.head.repo.full_name != .base.repo.full_name)) | length, dependabot: map(select(.user.login == $bot)) | length}'

If forks account for more than half of merged pull requests, keep the blocking scanner you run today for PHP, shell or HCL, because AI Scan will not see that traffic. If forks and Dependabot together account for less than a tenth, AI Scan covers most of your traffic, and the open question is credit spend.

Then check the switch before your organization flips it for you:

gh api orgs/ORG/code-scanning/ai-scan
gh api repos/OWNER/REPO/code-scanning/ai-scan
gh api -X PATCH repos/OWNER/REPO/code-scanning/ai-scan -f pr_scan=disabled

The first two return pr_scan as enabled or disabled. The third is the repository opt-out, worth running on high-traffic repositories until a Code Security AI Credits budget exists.