Adaptive Python web scraping framework with stealth browser fetching, spiders and an MCP server.
Scrapling is a free, open-source (BSD 3-Clause) Python web scraping framework that combines fast HTTP fetching, stealth browser automation, adaptive parsing and concurrent crawling in a single library. It targets the reality of the modern web: JavaScript-heavy pages, anti-bot systems such as Cloudflare, and sites that change their markup often enough to break ordinary scrapers.
The library offers several fetchers for different situations. The plain Fetcher handles quick HTTP requests for static pages, while the StealthyFetcher drives a hardened browser that renders JavaScript and, according to the project documentation, is built to pass common anti-bot checks without extra plugins. A full spider framework sits on top, with reusable sessions, proxy support, automatic retries, pause and resume for long crawls, and an AutoThrottle that adjusts request rates automatically.
Scrapling's parser is built for resilience: it remembers where elements were found and can relocate them after a site redesign, so scrapers survive frontend changes that would break fixed selectors. Results export directly to JSON, JSONL, CSV or XML, and a built-in MCP server lets AI agents use Scrapling as a tool, including extracting pages as clean Markdown for AI pipelines.
Scrapling requires Python 3.10 or later and installs from PyPI with pip install scrapling. Version 0.5.1 is the current release. The project is developed by Karim Shoair (D4Vinci) and has gathered around 97,000 stars on GitHub, with documentation at scrapling.readthedocs.io. Note that the stealth fetcher downloads browser dependencies on first use, and anti-bot evasion is an ongoing arms race, so keeping the library updated matters.
Renders JavaScript through a hardened browser; per the documentation, it is built to pass common anti-bot checks such as Cloudflare, without extra plugins.
The parser remembers where elements were and relocates them after site redesigns, so scrapers survive markup changes.
Concurrent crawlers with sessions, proxy support, automatic retries, pause/resume and AutoThrottle.
Built-in MCP server lets AI agents scrape directly, plus terminal fetching for one-off jobs.
Run pip install scrapling (Python 3.10 or later required). The stealth fetcher downloads its browser dependencies on first use.
No. It is a Python library with a command-line interface for quick fetches, and an MCP server so AI agents can use it as a tool.
Results can be exported to JSON, JSONL, CSV and XML, and pages can be extracted as clean Markdown for AI pipelines.