The quiet failure: when your agent summarises a block page

Ask an agent what a page says and you usually get an answer. Whether the agent read the page is a separate question, and the answer doesn't tell you.

This post is about the gap between those two: the ways a page read fails, which of them you can see, and what to check so the one you can't see stops reaching your answers.

Three ways a read fails

Refused. The tool or the site says no, out loud. On 7 November 2026 we asked Claude Code's built-in fetch for a realestate.com.au listings page:

The server returned HTTP 429 Too Many Requests.
The response body was not retrieved.

Annoying, but honest. The agent knows it has nothing, and so do you.

Empty. The site returns a shell: a few kilobytes of markup that only becomes a page when a browser runs its scripts. The status is 200. The tool hands over the shell. Depending on the agent, it either says the page was empty or fills the gap from what it already believes.

Wrong body. The site serves its bot check instead of the page - "Just a moment...", "Checking your browser", "Prove your humanity" - and still answers 200. Illustrative, but you have seen this one:

fetch("https://forum.example/thread/4471")   → 200 OK

<title>Just a moment...</title>
Checking your browser before accessing forum.example.
Please enable JavaScript and cookies to continue.

Nothing in that exchange looks like an error. The tool succeeded. The agent summarises what it was given, or decides the summary is thin and adds what it remembers about the topic. Either way, the answer cites a source it never read.

Why the quiet one is the dangerous one

A refused read changes what the agent does next: it tries another source, or tells you it couldn't. A wrong-body read changes nothing. It flows into the answer with the same confidence as a real one, and the only trace is a citation that, if anyone clicks it, shows a page the agent never saw.

The failure isn't the block. Sites are entitled to refuse automated traffic. The failure is the block being passed on as content.

What to check, whatever tool you use

Most of this is cheap and doesn't need a new product:

• Treat the status code as a hint, not a verdict. A 403, 429 or 503 is a failure. A 200 is a claim to check.
• Look for challenge markers on short pages. "Just a moment", "Checking your browser", "Prove your humanity", "Enable JavaScript and cookies". A 20 KB article that quotes one of those phrases is fine; a 2 KB page that is mostly one of them is a block page.
• Compare size to expectation. A listings page that comes back as a few hundred bytes was not read.
• Make failure a result. Return "couldn't read this, because…" to the agent as a normal outcome, with the cause, so it can pick another source or say so. An empty string is the worst possible error message.
• Keep a record. URL, outcome, time. When an answer is questioned later, you want to know whether the source was read.

Tools that do this for you

• FetchGate is a free, deterministic gate at the fetch boundary, available as an MCP server. Its README puts the idea in one line: "A 200 is not a read." It refuses to hand the model content from a page that wasn't really read.
• Agent search APIs differ on this. In our 7 November run, Tavily's extract endpoint answered "Failed to fetch url" for two protected pages: honest, though it doesn't say why.
• Levirge Search is ours, so take this as disclosure. It retries a refused public page in a full browser, detects challenge and error pages rather than serving them, and when it still can't read a page it says so with the cause. On the same listings page as above, it returned "Showing 1 – 25 of 941 properties". On a Reddit page that served its challenge, it returned blocked content … Prove your humanity and [reason=blocked retryable=false] instead of the page.

Where the line is

Reading a public page in a real browser is what a person does. Getting past a login, a paywall or a site's refusal of bulk traffic is something else, and no fetch tool should do it for you. The goal isn't to read everything. It's to know, every time, whether you did.

More on this: When your AI agent can't read the page, with two captured runs.