Why Not Just Use Requests

Because the site blocks you. Or serves you a Cloudflare challenge page. Or fingerprints your TLS stack and returns garbage. Or redesigns their HTML and all your selectors break overnight.

Scrapling handles all of this. 35,000 stars. Three fetcher tiers depending on how hostile the target is:

• Fetcher (HTTP level): fastest, uses httpx with browser-grade TLS fingerprinting. Good for APIs and sites without bot detection.
• StealthyFetcher (real browser): spins up a Playwright browser with anti-detection patches. Handles JavaScript rendering, Cloudflare Turnstile, and most bot checks.
• PlayWrightFetcher (full control): same browser engine but gives you direct Playwright API access for complex flows.

Pick the lightest tier that works. Escalate only when you need to. Most sites fold to Fetcher with the right TLS config.

The Selector Trick

The bit that sold me: adaptive selectors. You write a selector once and Scrapling generates multiple fallback strategies (text matching, attribute similarity, structural position). When the site changes their class names or restructures the DOM, your scraper keeps working because it falls back to a selector that still matches.

That's the difference between a scraper you maintain weekly and one you maintain monthly.

For Agents

I use it as a fallback in my /research skill when markdown.new or Jina Reader can't get through. Cloudflare blocks them, anti-bot walls go up, paywall gates slam shut. Scrapling's StealthyFetcher punches through most of it.

pip install scrapling

from scrapling import StealthyFetcher
page = StealthyFetcher.fetch("https://blocked-site.com")
print(page.css("article").text)

For a deeper comparison with every other scraping and browser tool, see the Browser Tools series.