Source · repo
Scrapling: web scraping framework with an MCP server and agent skill
Summary
Scrapling is a BSD-3-Clause Python framework for web scraping, covering plain HTTP
requests, stealth browser fetching, parsing, and spider-based crawls. It ships an MCP
server with 17 tools, so Claude Code, Claude Desktop, or Cursor can fetch and drive pages
through it, and an agent skill that teaches coding agents its current API. It is rising
in GitHub trends (about 6,000 new stars in the week to 2026-11-11, 101,000 in total, with
1,825 commits). date_published is the date the README was read; the repo shows no
dated releases.
Key points
- Install:
pip install "scrapling[ai]", thenscrapling installto fetch the browsers. Docker images:pyd4vinci/scraplingorghcr.io/d4vinci/scrapling. - Add to Claude Code:
claude mcp add ScraplingServer "$(which scrapling-mcp)"(thescrapling-mcpcommand exists from v0.4.13; older versions usescrapling mcp). In Claude Desktop, add"ScraplingServer": {"command": "scrapling-mcp"}undermcpServers. - Tools: one-shot
make_requestandbrowser_fetch_once; sessions (browser_open,open_request_session,list_sessions,close_session); saved login state (browser_save_state,browser_load_state);browser_screenshot,browser_observe,browser_extract,browser_actions,browser_evaluate; and network history (browser_network_requests,browser_network_request) for reading the API responses a page loads in the background. - Transports: stdio by default;
scrapling-mcp --httpserves Streamable HTTP on 127.0.0.1:8000 and requires a bearer token (SCRAPLING_MCP_AUTH_TOKEN). Put a TLS proxy in front before exposing it beyond localhost. - Prompt-injection hygiene: with
main_content_onlyon (the default), hidden elements,aria-hiddennodes, templates, HTML comments, and zero-width characters are stripped before content reaches the model. Keep it on. - Agent skill: a separate installable skill (docs: scrapling.readthedocs.io, "AI" section) so generated scraping code matches the current API.
- Good for: logged-in scraping, sites that block simple fetchers, extracting data from a page's XHR traffic, and turning a one-off agent fetch into a reusable spider.
Why it matters
It gives a coding agent a browser-grade fetcher with saved sessions and network inspection, which built-in web fetch tools lack, and it strips common hidden-text injection tricks by default. Check the target site's terms before scraping.