Claude Code, Copilot and Cursor ship pull requests that look perfect and still smuggle in packages that do not exist, leaked keys, and tests that test nothing. Codecop posts one verdict per PR, with evidence anyone can click and verify.
Advisory by default: Codecop never blocks a merge unless you turn enforcement on. No code is cloned, nothing is executed.
(commit dbc82b4)| Check | Result | Evidence |
|---|---|---|
| deps-exist | 1 finding | left-pod does not exist on npm - package.json:6 (registry 404) |
| secrets | 1 finding | GitHub token ghp_...1VgB - settings.py:2 (rule github-pat) |
| test-theater | 2 findings | test asserts nothing - user.test.js:8 expect(x).toEqual(x) - user.test.js:16 |
AI-generated code fails in specific, checkable ways that human review misses precisely because everything looks right. Reviewers approve the style; the failure hides in facts no human memorizes - like whether a package actually exists.
Agents invent plausible package names. smart-input-validator-pro reads like npm; it is not on npm. Best case your build breaks. Worst case someone publishes malware under that exact name and the next agent installs it - researchers call it slopsquatting.
Keys pasted into diffs for "just a quick test". Codecop scans only the lines a PR adds, so removing an old leak is never punished - and every match is redacted to ghp_...1VgB before it is stored anywhere.
Green tests that verify nothing: bodies with no assertions, tautologies like expect(x).toEqual(x), and a sneaky .skip added in the same PR that changes the source it was covering. Coverage went up; safety did not.
Codecop is deliberately not a chatbot reviewer. It will not leave ten comments about naming. It answers a single question - is this safe to merge - and shows its work.
Two clicks, zero config. Codecop starts reviewing the next pull request on the repos you choose. A codecop.yml is optional, never required.
Deterministic checks run against the exact diff: registries are asked if packages exist, gitleaks hunts credentials, parsers read your tests. Verdicts land in seconds, not minutes.
One sticky comment with verdict, evidence and file:line, updated in place on every push. A machine-readable JSON copy lets the agent that wrote the PR fix its own mistakes.
A real pull request written by Claude Code: a package that is not on npm, a GitHub token in a settings file, four tests that test nothing. Codecop answered HIGH with six findings in five seconds.
Not a mockup. Open the pull request and click through every finding yourself.
A gate that cries wolf gets uninstalled in a week. Every heuristic in Codecop errs toward silence, and an eval corpus gates every release in CI - a build that dips below the bar does not ship.
Honest framing: the first two are corpus numbers - 68 cases of seeded failures, clean diffs from real open-source PRs, and 22 traps engineered to fool us (private registries, fixture keys, snapshot tests, import/name mismatches like import cv2). The third is a read-only sweep of 57 public agent and OSS pull requests. The corpus and gates are in the open: precision under 90% fails our build. Read the full precision report, including the one false alarm we produced and how we fixed it.
The boring guarantees, up front:
Per-developer pricing punishes teams whose agents open fifty PRs a day. Codecop charges per repository, flat, unlimited pull requests.