Fetching

Use your agent’s browser to reach the page state you want, then capture that page for analog view. When you need Analog to acquire a URL, analog browse uses its built-in browser, runs the page’s JavaScript, and saves what it captures. Both routes let you search directly or request an overview when context would help.

If you have not chosen between a structured view and local Markdown, start with analog assess. This guide begins after you have chosen how to use the page. Examples that return records assume you have connected an account.

On this page

• Choose a fetch path
• Capture from your browser’s current page
• Capture more of a page
• Check what was captured
• Pagination reference
• Fetching boundaries

Choose a fetch path

Supply a browser capture to analog view to use the page you already have. Ordinary HTML is also accepted with its actual --url; neither input is refetched. The capture recipe below retains observed visibility as well.

The built-in browser handles analog browse and Python’s analog(url). Use it for pages that render with JavaScript or place content behind interactive controls. It starts a fresh session without using your personal browser profile, cookies, or saved credentials.

Use HttpFetcher in Python when the initial HTTP response already contains everything you need. It does not run JavaScript or click controls. The HTTP example uses the quotes site’s static page at /, where the quotes are already in the HTML, rather than its /js/ page.

For a single page you have already captured, pass its content as html= and leave pages at its default of 1. Analog uses that HTML without fetching the page. A custom Fetcher can also supply content from an environment you control.

The Python API reference lists the fetcher interfaces. If you provide signed-in page content through a file, html=, or a custom fetcher, read the boundary in Privacy.

Render a page with JavaScript

analog browse https://quotes.toscrape.com/js/

No fetcher setting is needed. See the acquisition example for the receipt, and the optional overview for this page’s records and coverage notes.

The output shows the complete response from result.preview(); fetch progress is reported separately. The live page may change.

Fetch the static page over HTTP

from analog import HttpFetcher, analog

with HttpFetcher() as fetcher:
    result = analog("https://quotes.toscrape.com/", fetcher=fetcher)

print(result.preview())

Output

pagination: this looks like page 1 of a paginated collection (its links reach page 2) — these records cover this page only.

2 sections extracted.

section[0]  10 records · 5 fields
  fields: tags_2:text[], text:text, text_2:text, about_url:url, tags:url[]

section[1]  navigation  2 links (footer) · 4 fields
  fields: text:text, url:url, group:text, depth:text

page outline:
  not extracted: "Top Ten tags" (unknown, 10 items) — read as page structure, not records

(A single-subject record was also weighed and withheld — Analog couldn't verify enough of the page's facts for a record we'd trust.)

The preview shows quote records, navigation, and coverage notes in the same format as a browser-fetched result. The context manager closes the HTTP connection pool after the fetch.

This small HTML string stands in for content you have already captured. mode="local" converts it to Markdown on your machine, without an account or backend call:

Use HTML you already have

from analog import analog

html = '<h1>Notes</h1><p><a href="/docs/">Read the docs</a></p>'
result = analog("https://getanalog.io/", html=html, mode="local")
print(result.markdown)

Output

# Notes

[Read the docs](/docs/)

Omit mode="local" when you want structured records from the supplied HTML. That path sends the content and its URL to Analog and requires a connected account.

Capture from your browser’s current page

An agent can navigate and interact using its harness’s browser, then supply that page to Analog. This route needs a supported read-only browser evaluator and a JavaScript runtime with Node filesystem access. The resulting file must be accessible to the machine running the Analog CLI. Use your harness’s own browser instructions to select the page and obtain its evaluator.

analog browser capture-script writes a self-contained helper. It does not connect to the browser. The helper writes the page’s ordinary DOM content, actual URL, and visibility observations to a new capture JSON file, then returns only its path, URL, byte count, and SHA-256 receipt. It never scrolls, clicks, or changes the page. Keep the capture content out of the conversation; return only the receipt from your browser tool.

analog view validates the capture and applies its visibility observations locally before extraction. A validation failure stops before extraction or saving; capture again or supply ordinary HTML with --url. Capture JSON carries its own URL, and an explicit conflicting --url is rejected.

Open shadow content and frame documents are not included; observed hosts and frames produce qualifications that persist with the saved result. Closed shadow content, canvas pixels, CSS-generated content, and control state absent from HTML are not included. The file describes the captured state, not content that further interactions might reveal.

Choose a destination that does not exist:

Prepare the reusable helper

analog browser capture-script --output ./analog_capture.mjs

The command prints its absolute path. The helper needs no sibling files or npm packages. Reuse it for later pages; generate a new copy after upgrading the SDK when you want the updated helper.

This example starts after the Codex GUI’s supported browser tooling has supplied tab for the page you want. It runs inside that tool’s JavaScript runtime, not in a shell or the page console. Replace the example paths with the helper’s returned absolute path and a new capture destination:

Capture in Codex’s browser tool

const { capturePage } = await import("/absolute/path/analog_capture.mjs");
const receipt = await capturePage({
  evaluate: expression => tab.playwright.evaluate(expression),
  path: "/absolute/path/about.capture.json",
});
nodeRepl.write(receipt);

The evaluator binding belongs to the harness. This example demonstrates the Codex GUI route; availability and evaluator bindings differ in other harnesses.

Pass the capture’s real path and an unused saved name:

Extract, save, and search

analog view ./about.capture.json --save-as about --find Frankie
analog view about --find Marcus --find Frankie

The first command sends the prepared HTML and its URL to Analog for extraction, saves the result, and returns matching evidence. Matches go to stdout; the receipt and capture qualifications go to stderr. The second command searches that saved result locally. Use --mode local on the file command for local Markdown without sending the HTML or URL. See Privacy when supplying signed-in content.

Capture more of a page

Run once with the defaults. If the preview reports content the browser could have opened or followed, rerun with the matching option.

Analog looks for a show-everything control by default. Loading more items and opening each item’s details are separate choices: --load-all is off because a large page may need many button presses; --expand-all is off because opening every item’s details changes what the page presents. Both operations are bounded, and expansion is paced politely.

Recognition includes some German and Arabic load-more buttons, Korean next-page controls, and Portuguese continuation links. English-labelled controls remain usable on non-English pages. Coverage varies by the page’s controls; the same interaction options apply without a language setting.

The control examples use <url> for a page where you have observed that control. The quotes example demonstrates following a next-page link. Combine options only when the page calls for more than one kind of interaction.

If the page opens on a subset you want to retain, leave its “All” or “Clear filters” control untouched:

Keep the page’s initial filter

analog browse <url> --no-reveal-all

In Python, use reveal_all=False.

When a “Load more” or “Show more” button reveals more items:

Load additional items

analog browse <url> --load-all

In Python, use load_all=True.

When items contain disclosure toggles or accordion rows:

Open each item’s details

analog browse <url> --expand-all

In Python, use expand_all=True.

The quotes page has a next-page link. Request up to two pages and save the combined result:

Include the next page of quotes

analog browse https://quotes.toscrape.com/js/ --pages 2

In Python, use pages=2. The capture below shows how to read this run’s coverage; the pagination reference explains the limits.

Configure a supplied browser directly

The Python arguments above configure the default browser. If you pass fetcher=Browser(...), set the controls on that Browser instance instead. An HTTP fetcher or supplied HTML cannot perform browser interactions.

Check what was captured

After the two-page quotes fetch above, reopen its saved result. The response starts with fetched with: pages=2, then reports that the fetch stopped at the requested cap while more pages remain.

Here, section[0] contains 20 quotes. The sweep’s 24-record total also includes four navigation records, and its duplicate count describes records repeated across the fetched pages. source_page_url records where each retained record came from.

Run this immediately after the two-page fetch, or replace latest with that result’s saved handle:

This is the complete response after fetching the quotes page with --pages 2. Your handle and the live page content may differ.

Read the saved coverage report

analog view latest

Output

fetched with: pages=2

pagination: swept 2 pages following the site's own next links (the requested cap) — the collection continues past the sweep; 24 records merged across the sweep (4 cross-page duplicates dropped). Every retained record carries its source page URL in the source_page_url field.

3 sections extracted.

section[0]  20 records · 4 fields
  fields: tags:text[], text:text, text_2:text, source_page_url:url

section[1]  navigation  2 links (header) · 5 fields
  fields: text:text, url:url, group:text, depth:text, source_page_url:url

section[2]  navigation  2 links (footer) · 5 fields
  fields: text:text, url:url, group:text, depth:text, source_page_url:url

(A single-subject record was also weighed and withheld — Analog couldn't verify enough of the page's facts for a record we'd trust.)

If the capture still looks wrong, rerun your URL and its chosen options with --headed to watch a visible browser window:

Watch a fetch in the browser

analog browse <url> --headed

The Troubleshooting guide’s Markdown check helps distinguish content the browser never captured from content that was captured but not returned as records.

Read control and coverage notes

The result reports interactive controls the browser found and whether it used them. An unused load-more or expansion control appears under browse_actions with the option that enables it.

A one-page fetch also reports when the page appears to continue and states that the current records cover only this page. When the page publishes a total, Analog places that number beside the count it captured rather than presenting a partial result as the whole collection.

Pagination reference

Pagination sweeps are experimental and accept between 1 and 50 pages. The default is 1, which follows nothing. The requested count includes the first page or batch.

Links to another page

With --pages N, Analog follows the next link each page renders. It does not construct page URLs, stays on the page’s own host, and pauses politely between fetches. The result reports the pages covered, why the sweep stopped, and how many duplicate records were dropped.

When a structured result spans multiple pages, each retained record also carries the URL of the page that supplied it. page_sweep.source_page_field names that field, normally source_page_url; if the page already uses that name, Analog chooses a collision-safe suffix.

Structured and auto modes make one extraction request per page. Local mode makes no extraction requests and combines the pages’ Markdown.

Numbered batches on one page

Some sites replace the visible batch without changing the URL. The built-in browser can drive those numbered controls with the same --pages N option and merge the batches into one result.

page_sweep.mechanism states whether the sweep followed links or used in-page controls. For in-page controls, page_sweep.batches_captured reports the number of batches. A structured result spanning multiple batches also carries source-page URLs on its records. When the address stays the same, those values repeat; when a control changes the address, they retain the address of the batch that supplied the record.

Merging pages and batches

Compatible sections are combined. When a page lacks a field found on another page, its records receive a null value for that field and the sweep note names it. Sections that cannot be combined stay separate, and the note says which ones.

When the page states its own total in pagination text or a count heading, the result preserves both the total and where it appeared as page_sweep.stated_total and page_sweep.stated_total_source. The sweep summary places that number beside the merged record count so partial coverage remains visible.

Fetching boundaries

The built-in browser has a stable identity and respects robots.txt, including feed-verification requests made by analog assess. Analog does not use proxies, fingerprint spoofing, or CAPTCHA solving. A site declining automated visitors is a result Analog reports, not a block it tries to evade.

Analog clicks controls available to every visitor, follows links the page renders, and reports what it chose not to do. Private-network and local addresses are refused by default; Security documents that boundary.