Website Crawler
Fast HTTP pull of public HTML with no browser session. Ideal when the first response already contains the content you need.
Credit cost: 1 credit per call
Authentification
ApiKeyAuth x-api-key<token>
Your API key. Include it in the x-api-key header for every request.
In: header
Paramètres de requête
query*string
Public URL to retrieve with the Crawler tier (plain HTTP, no JavaScript). Include http:// or https://.
Example:
https://books.toscrape.com/Pick a tier:
- Static or SSR HTML: Website Crawler (this endpoint).
- Needs JavaScript: Website Rendering.
- Bot wall on an approved domain: Website WebUnlocker.
Guide: fetch tier comparison.
allow_redirects?boolean
Whether BrowserSolver should chase HTTP redirects for this fetch.
true: follow the redirect chain (default)false: stop on the first status and return that response
Tip: set false when you need the intermediate 301/302 hop itself.
Default
"true"return_page_source?boolean
Choose between processed markup and the exact bytes from the origin.
false: return BrowserSolver's processed HTML (default)true: return the untouched source from the web server
Tip: enable true for debugging parsers against the raw document.
Default
"false"Corps de la réponse
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/v1/website/crawler?query=string""<!DOCTYPE html><html lang=\"en\"><head><meta charset=\"utf-8\"><title>Welcome to Python.org</title><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\"><link rel=\"stylesheet\" href=\"/static/stylesheets/style.css\"></head><body><div id=\"touchnav-wrapper\"><header class=\"main-header\"><h1>Python</h1><nav><a href=\"/downloads/\">Downloads</a><a href=\"/doc/\">Documentation</a></nav></header><main><section class=\"launch-shell\"><h2>Get Started</h2><p>Python is a programming language that lets you work quickly.</p></section></main></div></body></html>"{ "error": "Bad Request"}{ "error": "Invalid API Key"}{ "error": "Payment required"}{ "error": "Rate limit exceeded for the API key: quota monthly"}{ "error": "Internal Error"}