Ask an agent what a page says and you usually get an answer. Whether the
agent read the page is a separate question, and the answer doesn't tell you.

This post is about the gap between those two: the ways a page read fails,
which of them you can see, and what to check so the one you can't see stops
reaching your answers.

## Three ways a read fails

**Refused.** The tool or the site says no, out loud. On 7 November 2026 we
asked Claude Code's built-in fetch for a realestate.com.au listings page:

```
The server returned HTTP 429 Too Many Requests.
The response body was not retrieved.
```

Annoying, but honest. The agent knows it has nothing, and so do you.

**Empty.** The site returns a shell: a few kilobytes of markup that only
becomes a page when a browser runs its scripts. The status is 200. The tool
hands over the shell. Depending on the agent, it either says the page was
empty or fills the gap from what it already believes.

**Wrong body.** The site serves its bot check instead of the page - "Just a
moment...", "Checking your browser", "Prove your humanity" - and still answers
200. Illustrative, but you have seen this one:

```
fetch("https://forum.example/thread/4471")   → 200 OK

<title>Just a moment...</title>
Checking your browser before accessing forum.example.
Please enable JavaScript and cookies to continue.
```

Nothing in that exchange looks like an error. The tool succeeded. The agent
summarises what it was given, or decides the summary is thin and adds what it
remembers about the topic. Either way, the answer cites a source it never
read.

## Why the quiet one is the dangerous one

A refused read changes what the agent does next: it tries another source, or
tells you it couldn't. A wrong-body read changes nothing. It flows into the
answer with the same confidence as a real one, and the only trace is a
citation that, if anyone clicks it, shows a page the agent never saw.

The failure isn't the block. Sites are entitled to refuse automated traffic.
The failure is the block being passed on as content.

## What to check, whatever tool you use

Most of this is cheap and doesn't need a new product:

- **Treat the status code as a hint, not a verdict.** A 403, 429 or 503 is a
  failure. A 200 is a claim to check.
- **Look for challenge markers on short pages.** "Just a moment", "Checking
  your browser", "Prove your humanity", "Enable JavaScript and cookies". A
  20 KB article that quotes one of those phrases is fine; a 2 KB page that
  is mostly one of them is a block page.
- **Compare size to expectation.** A listings page that comes back as a few
  hundred bytes was not read.
- **Make failure a result.** Return "couldn't read this, because…" to the
  agent as a normal outcome, with the cause, so it can pick another source or
  say so. An empty string is the worst possible error message.
- **Keep a record.** URL, outcome, time. When an answer is questioned later,
  you want to know whether the source was read.

## Tools that do this for you

- **[FetchGate](https://github.com/MoAz06/FetchGate)** is a free,
  deterministic gate at the fetch boundary, available as an MCP server. Its
  README puts the idea in one line: "A 200 is not a read." It refuses to hand
  the model content from a page that wasn't really read.
- **Agent search APIs** differ on this. In our
  [7 November run](/search/vs-tavily), Tavily's extract endpoint answered
  "Failed to fetch url" for two protected pages: honest, though it doesn't say
  why.
- **Levirge Search** is ours, so take this as disclosure. It retries a refused
  public page in a full browser, detects challenge and error pages rather than
  serving them, and when it still can't read a page it says so with the cause.
  On the same listings page as above, it returned "Showing 1 – 25 of 941
  properties". On a Reddit page that served its challenge, it returned
  `blocked content … Prove your humanity` and `[reason=blocked retryable=false]`
  instead of the page.

## Where the line is

Reading a public page in a real browser is what a person does. Getting past a
login, a paywall or a site's refusal of bulk traffic is something else, and
no fetch tool should do it for you. The goal isn't to read everything. It's
to know, every time, whether you did.

More on this:
[When your AI agent can't read the page](/search/ai-agent-cant-read-the-page),
with two captured runs.