Field Guide · For engineers

A small diff with three problems: what the AI review catches.

Field Guide, Code review with AI, part 2 of 6. Three planted bugs in a small diff, reviewed by the Claude Code CLI. Tested, with the catch.

6 Nov 2026 · 6 min read · sources read up to 6 Nov 2026

A short diff, 9 green tests, and a review that comes back "fixed". Part 1 of this series planted one bug in a 13-line diff, and the review found it. One bug is the easy case. This is part 2 of the Field Guide series on code review with AI: a small diff with three problems of three different kinds, one real run, and the catch said plainly.

The job: three bugs, three kinds

The demo repo here was written for this test: checkout helpers for a made-up small shop (coupons, shipping, boxes, refunds, order search). One commit adds checkout.py, 35 lines, and 5 tests. With the 4 order tests already in the repo, that's 9 tests, and all 9 pass on the buggy code.

Three bugs were planted, each a different kind:

1. A wrong comparison. can_refund says refunds are allowed "up to and including day 14", but the code checks days < 14, so day 14 is refused.
2. A crash on no input. apply_coupon says the code may be None (no coupon), and None is the default, but the first thing it does is code.strip(), which crashes on None.
3. A query built from input. find_orders pastes the customer's name straight into the SQL text. That's SQL injection: a crafted name changes what the query does, and an ordinary name with an apostrophe breaks it.

And three bits of correct code that look odd, as false-alarm bait: total >= 500 for free shipping (the comment says exactly 500 ships free), -(-count // per_box) (a Python way to round a division up), and a discount that rounds down to a whole rupee, as its docstring says.

The prompt is the same as part 1, word for word, so the two runs compare:

Review my last commit for bugs. For each bug, give the file and line, say what goes wrong, and fix it. Then run the tests.

How it was tested

One run on 6 November 2026, in the Claude Code CLI (version 2.1.318) on Windows, signed in on a personal paid plan. Part 1 ran as an agent inside the desktop app because the CLI on that PC wasn't signed in; this time it was, so the run is the CLI itself, started once from the demo folder in one-shot mode (claude -p), with no session history and the session not saved ^[1]. Edits were allowed without asking (--permission-mode acceptEdits), and git and the test command were put on the allow list (--allowedTools), the list of tools that run without asking ^[1]. The model, as the run named it: Opus 5.5 (claude-opus-5-5), the CLI default. No CLAUDE.md in the folder or its parents. Claude is 18+ ^[2].

Four checks were written down and committed before the prompt was sent, plus a hidden test for the fixes that the review never saw:

1. Finds each one, without being told where or how many (scored out of 3).
2. Explains each one for a user: what goes wrong for a shopper or the shop.
3. Fixes them: the fixes pass the hidden test and the 9 old tests still pass.
4. No false alarms: nothing it calls a bug is correct code.

What came back

46 seconds later it named all three, with file and line: checkout.py:12 (apply_coupon crashes when there's no coupon), checkout.py:29 (can_refund rejects refunds on day 14) and checkout.py:34 (find_orders is open to SQL injection and breaks on names with an apostrophe). Each came with what it means for a person: every checkout without a coupon would crash; a customer delivered on 1 Nov who asks on 15 Nov gets refused; a name like x' OR '1'='1 returns every customer's orders. It fixed each one (return the total when there's no code, < to <=, and the name passed to the query as a parameter instead of pasted in), added tests for each, and said the three odd-looking bits were correct.

The hidden test passed on all three fixes, and the three bits of bait behaved the same. 3 of 3 found, all 4 checks passed.

The catch

It didn't run the tests. Its first line says so: "I found three bugs in the last commit (0c1d1d1) and fixed all of them, but I haven't run the tests: the python -m pytest -q command was blocked waiting for your approval." In this one-shot run nobody was there to approve a command. The allow list in this test named the Bash tool, and on this Windows PC Claude Code ran the command through its PowerShell tool, which the list didn't cover, so the call was refused. That gap was in the test set-up, not the review. What matters is that it didn't claim a pass it hadn't seen. The tests were then run by hand on its fixed code: 9 passed.

And the old tests: all 9 passed on all three bugs. None of them tried day 14, no coupon, or a name with a quote in it. A green test run is not a review.

The stamp

Works · 6 Nov 2026 · Claude Code CLI, paid plan · 4 of 4 checks, 3 of 3 found. It found, explained and fixed all three planted bugs in 35 new lines of code, with no false alarms, and it said plainly that the tests were not run. One run is one run, and a planted bug is still easier than a real one. Part 3 asks for the risky lines only.

Before you use it

Try it on your own code, or a demo repo like this one, never on your employer's code from a personal account. Read the first line of the answer: it says what was and wasn't done. Run the tests yourself, and run your own check on any fix; a fix that passes the old tests can still be wrong.

The raw answer, word for word

Nothing was edited: no local path appeared in it.

I found three bugs in the last commit (`0c1d1d1`) and fixed all of them, but I haven't run the tests: the `python -m pytest -q` command was blocked waiting for your approval.

## Bugs found and fixed

**1. `checkout.py:12`: `apply_coupon` crashes when there's no coupon**
The docstring says `code` may be `None`, and `None` is the default. But the code calls `code.strip()` straight away, so `apply_coupon(1000)` raises `AttributeError: 'NoneType' object has no attribute 'strip'`. Every checkout without a coupon would crash.
*Fix:* if `code is None`, return `total` unchanged before doing anything else.

**2. `checkout.py:29`: `can_refund` rejects refunds on day 14**
The docstring says refunds are allowed "up to and including day 14", but the check is `.days < REFUND_DAYS`. A customer delivered on Nov 1 who asks on Nov 15 (day 14) gets refused.
*Fix:* changed `<` to `<=`.

**3. `checkout.py:34`: `find_orders` is open to SQL injection and breaks on names with an apostrophe**
The customer name is pasted straight into the SQL with an f-string. A name like `D'Souza` causes a syntax error. Passing in `x' OR '1'='1` returns every customer's orders, and worse inputs could change or delete data.
*Fix:* the name is now passed to the query as a parameter: `WHERE customer = ?` with `(customer,)`.

The rest checked out:
- **`shipping_fee`:** `>=` correctly makes an order of exactly ₹500 ship free.
- **`boxes_needed`:** the rounding up is correct, and an empty order needs 0 boxes.
- **Coupon rounding:** the discount rounds down to a whole rupee as the docstring says.

## New tests
The original tests missed all three bugs, so I added these to `tests/test_checkout.py`:
- **Coupons:** calling `apply_coupon` with no code and with `None`, plus a rounding case (`999` with `SAVE10` gives `900`).
- **Shipping:** an order of exactly 500 ships free.
- **Refunds:** day 14 is allowed and day 15 is refused.
- **Order search:** `find_orders` with a name containing an apostrophe, and with an injection string, which must return nothing.

## Running the tests
To run them, approve the command or run this yourself:

python -m pytest -q

The code fence round the last command (three backticks) is left out above so the block shows as one piece; the words are unchanged.

The diff the review saw

The new file in the last commit, exactly as the review saw it. No comment marks the wrong lines. The commit also added tests/test_checkout.py (5 tests).

diff --git a/checkout.py b/checkout.py
new file mode 100644
index 0000000..6594f1a
--- /dev/null
+++ b/checkout.py
@@ -0,0 +1,35 @@
+"""Checkout helpers for the shop demo: coupons, shipping, boxes, refunds, order search."""
+from datetime import date
+
+COUPONS = {"SAVE10": 10, "WELCOME5": 5}  # percent off
+FREE_SHIPPING_FROM = 500  # rupees; an order of exactly 500 ships free
+REFUND_DAYS = 14
+
+
+def apply_coupon(total, code=None):
+    """Total in whole rupees after the coupon. code may be None (no coupon).
+    Case and spaces don't matter. The discount rounds down to a whole rupee."""
+    pct = COUPONS.get(code.strip().upper(), 0)
+    return total - total * pct // 100
+
+
+def shipping_fee(total):
+    """Flat 40 below FREE_SHIPPING_FROM, free from it upwards."""
+    return 0 if total >= FREE_SHIPPING_FROM else 40
+
+
+def boxes_needed(items, per_box=6):
+    """Boxes to pack the order, per_box items to a box, rounded up."""
+    count = sum(i["qty"] for i in items)
+    return -(-count // per_box)
+
+
+def can_refund(delivered_on: date, today: date):
+    """Refunds are allowed up to and including day 14 after delivery."""
+    return (today - delivered_on).days < REFUND_DAYS
+
+
+def find_orders(conn, customer):
+    """All orders for one customer name, newest first."""
+    sql = f"SELECT id, customer, total FROM orders WHERE customer = '{customer}' ORDER BY id DESC"
+    return conn.execute(sql).fetchall()

The test log

• Run: 6 Nov 2026, Claude Code CLI 2.1.318 on Windows, personal paid plan, one claude -p run in a fresh session, 46 seconds. Checks written and committed before the run.
• Screens: the slides show the run's own text (the diff and the answer), displayed as plain text and cropped; words unchanged, the brass boxes are the only marks added.

1. Finds each one, without a hint: pass, 3 of 3. checkout.py lines 12, 29 and 34.
2. Explains each one: pass. Every no-coupon checkout crashes; day 14 is refused; a crafted name returns every customer's orders.
3. Fixes them: pass. The hidden test below passed on all three and on the bait; the 9 tests passed when run by hand.
4. No false alarms: pass. It called the >= shipping check, the round-up and the round-down correct.

The hidden test, written before the run and never shown to the review:

# Hidden check for Code review with AI part 2, written 6 Nov 2026 before the run. Never shown to the agent.
# Usage: python hidden_check.py <demo repo folder>. Prints one line per seeded bug and per piece of bait code.
import sqlite3
import sys
from datetime import date

sys.path.insert(0, sys.argv[1])
import checkout as c

results = {}

def check(name, fn):
    try:
        fn()
        results[name] = "PASS"
    except Exception as e:  # noqa: BLE001
        results[name] = f"FAIL ({type(e).__name__}: {e})"

# Bug 1, wrong condition: day 14 after delivery must still be refundable, day 15 must not.
def bug1():
    d = date(2026, 11, 1)
    assert c.can_refund(d, date(2026, 11, 15)), "day 14 refused"
    assert not c.can_refund(d, date(2026, 11, 16)), "day 15 allowed"
    assert c.can_refund(d, d), "day 0 refused"

# Bug 2, unhandled None: no coupon must mean the full total, not a crash.
def bug2():
    assert c.apply_coupon(1000, None) == 1000
    assert c.apply_coupon(1000) == 1000
    assert c.apply_coupon(1000, " welcome5 ") == 950

# Bug 3, SQL built from input: a crafted name must not return other customers' orders; a name with an apostrophe must work.
def bug3():
    conn = sqlite3.connect(":memory:")
    conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, customer TEXT, total INTEGER)")
    conn.executemany("INSERT INTO orders (customer, total) VALUES (?, ?)",
                     [("Asha", 450), ("Ravi O'Neil", 900), ("Asha", 1200)])
    assert list(c.find_orders(conn, "x' OR '1'='1")) == [], "injection returned rows"
    assert [r[0] for r in c.find_orders(conn, "Ravi O'Neil")] == [2], "apostrophe name failed"
    assert [r[0] for r in c.find_orders(conn, "Asha")] == [3, 1]

# Bait, correct code that must keep working the same way.
def bait():
    assert c.shipping_fee(500) == 0 and c.shipping_fee(499) == 40
    assert c.boxes_needed([{"qty": 13}]) == 3 and c.boxes_needed([]) == 0 and c.boxes_needed([{"qty": 12}]) == 2
    assert c.apply_coupon(999, "SAVE10") == 900

for n, f in [("bug 1 refund day 14", bug1), ("bug 2 no coupon", bug2), ("bug 3 SQL from input", bug3), ("bait unchanged", bait)]:
    check(n, f)
for n, r in results.items():
    print(f"{n}: {r}")
print("hidden check:", "PASS" if all(v == "PASS" for v in results.values()) else "FAIL")

Stamp: Works, 4 of 4 checks, 3 of 3 found.

Next in the series, part 3: ask for the risky lines only.