Searches, extracts, maps and crawls the live web and runs cited research agents, for AI agents that need real-time web data, via REST API, SDKs, a CLI, and a hosted MCP server.
- Access
- APIMCPCLI
- Auth
- API Key
- Skills
- 5
Skills
Web Search
Runs real-time web search across multiple engines with configurable depth and focus, returning structured results.
Page Extract
Extracts clean, structured content from any URL, synchronously or as async and batch tasks with status polling.
Site Map
Discovers the URLs on a website by following links and reading sitemaps, ready for targeted extraction.
Site Crawl
Crawls multiple pages of a site with path filtering, subdomain control, and progress tracking.
Web Search Agents
Creates and runs research agents that complete a web research task and return results with per-claim citations.
Related Agents
context.dev
API · MCP · CLIScrapes, crawls and searches the web for agents through a REST API, SDKs, CLI and hosted MCP server, returning…
Oxylabs MCP
MCPLets AI agents scrape any URL with JavaScript rendering and geo-targeting, pull Google and Amazon results, and crawl…
ScrapeGraphAI
API · MCP · CLIScrapes websites from a natural-language description of the data you want, building the extraction graph automatically…
Firecrawl
API · MCP · CLITurn websites into LLM-ready data — crawl, scrape, and convert web pages to clean markdown for AI consumption.