# Web Scrapper MCP Server > **Web Scrapper MCP Server** is a hosted, multitenant Model Context Protocol (MCP) server run by **MewCP** (https://mewcp.com), giving AI agents managed access to Web Scrapper. > > MewCP takes care of all MCP infrastructure for you — credential storage, OAuth flows, > token refresh, and production-grade auto-scaling — so your AI agents can connect to > third-party services and run freely without you managing any MCP server yourself. > > To connect your agent to this server you need a MewCP account and two values: > - MEWCP_KEY — your personal API key (dashboard → Developer) > - CREDENTIAL_ID — the stored credential MewCP securely injects per request > > Server page: https://mewcp.com/mcp/web-scraper > MewCP docs: https://docs.mewcp.com > Full catalog: https://mewcp.com/llms.txt ## About Extract structured data from websites and public web pages. Fetch content, parse information and transform online data into formats that can be consumed by applications, agents and workflows. ## How to connect Server Page URL: https://mewcp.com/mcp/web-scraper Gateway URL: https://gateway.mewcp.com/web-scraper/mcp Every request to this server requires two headers: Authorization: Bearer — your MewCP API key (dashboard → Developer) x-mewcp-credential-id: — the stored credential ID for this service All connection snippets and ready-to-use code examples are available on the server page and in this document below. --- ## Server documentation **Fetch and parse any web page through AI.** A Model Context Protocol (MCP) server that fetches web pages and extracts content using CSS selectors or returns full page HTML, title, and metadata. ## Overview The Web Scraper MCP Server provides simple, no-auth web content extraction: - Fetch any public web page and return its full HTML, title, and meta description - Target specific elements on a page using CSS selectors - Works with any public HTTP/HTTPS URL — no API key or credentials required Perfect for: - AI assistants that need to read live content from any public web page - Extracting specific elements like articles, tables, or product listings - Quick page lookups without setting up a full scraping pipeline ## Tools
scrape — Scrape content from a web page Fetches a web page and returns either the full page content or elements matching a CSS selector. **Inputs:** ``` - `url` (string, required) — Fully qualified HTTP/HTTPS URL to fetch - `selector` (string, optional) — CSS selector to target specific elements on the page ``` **Output (without selector):** ```json { "url": "https://example.com", "title": "Example Domain", "description": "Meta description of the page", "html": "..." } ``` **Output (with selector):** ```json { "url": "https://example.com", "data": [ "

Example Domain

", "

This domain is for use in illustrative examples.

" ] } ```
## API Parameters Reference
CSS Selector Examples Use standard CSS selectors to target elements on the page: ``` h1 — All

headings article p — All

inside

tags .product-title — Elements with class "product-title" #main-content — Element with ID "main-content" table tr — All table rows meta[name="author"] — Meta tags with name="author" ``` When `selector` is provided, the tool returns a list of matching element HTML strings. When omitted, the full raw page HTML is returned.

URL Requirements - Must be a fully qualified URL including the scheme: `https://example.com`, not `example.com` - Must be a publicly accessible HTTP/HTTPS endpoint - Pages behind login walls or paywalls will not return protected content - Request timeout is 10 seconds — very slow pages may fail
## Troubleshooting
Missing or Invalid Headers - **Cause:** API key not provided in request headers or incorrect format - **Solution:** 1. Verify `Authorization: Bearer YOUR_API_KEY` and `X-Mewcp-Credential-Id: CREDENTIAL-ID` headers are present 2. Check the credential is active in your MewCP account
Insufficient Credits - **Cause:** API calls have exceeded your request limits - **Solution:** 1. Check credit usage in your Curious Layer dashboard 2. Upgrade to a paid plan or add credits for higher limits 3. Contact support for credit adjustments
Malformed Request Payload - **Cause:** JSON payload is invalid or missing required fields - **Solution:** 1. Validate JSON syntax before sending 2. Ensure the `url` parameter is a fully qualified HTTP/HTTPS URL 3. Check the `selector` value is valid CSS syntax
Server Not Found - **Cause:** Incorrect server name in the API endpoint - **Solution:** 1. Verify endpoint format: `{server-name}/mcp/{tool-name}` 2. Use correct server name from documentation 3. Check available servers in your Curious Layer account
Page Fetch Error - **Cause:** The target URL returned an error or timed out - **Solution:** 1. Verify the URL is publicly accessible and returns a valid HTTP response 2. Check the URL scheme is `http://` or `https://` — other protocols are not supported 3. Pages behind authentication, firewalls, or bot protection may be blocked 4. Try a simpler URL to confirm the server itself is working
Selector Returns Empty List - **Cause:** The CSS selector did not match any elements on the page - **Solution:** 1. Inspect the page HTML in a browser's dev tools to verify the selector 2. Some pages render content dynamically via JavaScript — this server fetches static HTML only 3. Try a broader selector (e.g., `body` or `div`) to confirm the page was fetched correctly
--- ### Resources - **[CSS Selectors Reference](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_selectors)** — MDN CSS selector syntax - **[BeautifulSoup Docs](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)** — Underlying HTML parsing library - **[FastMCP Docs](https://gofastmcp.com/v2/getting-started/welcome)** — FastMCP specification - **[FastMCP Credentials](https://pypi.org/project/fastmcp-credentials/)** — FastMCP Credentials package for credential handling --- ## Connection snippets ### Python (fastmcp) ```python import asyncio from fastmcp import Client from fastmcp.client.transports import StreamableHttpTransport SERVER_URL = "https://gateway.mewcp.com/web-scraper/mcp" MEWCP_KEY = "YOUR_MEWCP_KEY" CREDENTIAL_ID = "YOUR_CREDENTIAL_ID" transport = StreamableHttpTransport( url=SERVER_URL, headers={ "Authorization": f"Bearer {MEWCP_KEY}", "x-mewcp-credential-id": CREDENTIAL_ID, } ) async def main(): client = Client(transport) async with client: await client.ping() tools = await client.list_tools() resources = await client.list_resources() prompts = await client.list_prompts() # Change the tool name and arguments with actual tool and arguments available in server result = await client.call_tool("example_tool", {"param": "value"}) print(result) asyncio.run(main()) ``` ### TypeScript (MCP SDK) ```typescript import { Client } from "@modelcontextprotocol/sdk/client/index.js"; import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"; const SERVER_URL = "https://gateway.mewcp.com/web-scraper/mcp"; const MEWCP_KEY = "YOUR_MEWCP_KEY"; const CREDENTIAL_ID = "YOUR_CREDENTIAL_ID"; const transport = new StreamableHTTPClientTransport(new URL(SERVER_URL), { requestInit: { headers: { Authorization: `Bearer ${MEWCP_KEY}`, "x-mewcp-credential-id": CREDENTIAL_ID, }, }, }); const client = new Client({ name: "mewcp-client", version: "1.0.0", }); await client.connect(transport); const tools = await client.listTools(); console.log("Available tools:", tools.tools.map(t => t.name)); // Change the tool name and arguments to a tool available on your server const result = await client.callTool({ name: "example_tool", arguments: { param: "value" }, }); console.log("Tool result:", result); ``` ### VS Code (settings.json) ```json { "servers": { "mewcp-web-scraper": { "type": "http", "url": "https://gateway.mewcp.com/web-scraper/mcp", "headers": { "Authorization": "Bearer YOUR_MEWCP_KEY", "x-mewcp-credential-id": "YOUR_CREDENTIAL_ID" } } } } ``` ### Cursor (mcp.json) ```json { "mcpServers": { "web-scraper": { "url": "https://gateway.mewcp.com/web-scraper/mcp", "headers": { "Authorization": "Bearer YOUR_MEWCP_KEY", "x-mewcp-credential-id": "YOUR_CREDENTIAL_ID" } } } } ``` ### Claude Desktop (claude_desktop_config.json) ```json "mcpServers": { "web-scraper": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://gateway.mewcp.com/web-scraper/mcp", "--transport", "http-only", "--header", "Authorization: Bearer YOUR_MEWCP_KEY", "--header", "x-mewcp-credential-id: YOUR_CREDENTIAL_ID" ] } } ```