Crawl4AI

Crawl4AI turns pages into LLM-ready Markdown. Configure it to attach to a Driver session instead of launching its own browser.

pip install crawl4ai requests

Python

import asyncio, os, requests
from crawl4ai import AsyncWebCrawler, BrowserConfig

API = "https://api.driver.dev"
headers = {"Authorization": f"Bearer {os.environ['DRIVER_API_KEY']}"}


async def main():
    r = requests.post(f"{API}/v1/browser/session", headers=headers, json={"country": "US"})
    r.raise_for_status()
    session = r.json()
    try:
        config = BrowserConfig(browser_mode="custom", cdp_url=session["cdpUrl"], use_managed_browser=True)
        async with AsyncWebCrawler(config=config) as crawler:
            result = await crawler.arun(url="https://www.nbcnews.com/business")
            print(result.markdown)
    finally:
        requests.delete(
            f"{API}/v1/browser/session", headers=headers, params={"sessionId": session["sessionId"]}
        )


asyncio.run(main())

Notes

browser_mode="custom" with cdp_url attaches Crawl4AI to the browser you give it. use_managed_browser=True makes it drive that browser over CDP instead of through a Playwright launch. Leaving the async with block closes Crawl4AI’s connection only; the session runs until you stop it.
Crawl4AI is Playwright underneath and injects its own scripts, so it carries the stock-Playwright fingerprint. Use it on sites that don’t challenge automation.
Pin the crawl4ai version you tested with. Its post-install step downloads a browser you won’t use. Harmless.
Crawl several URLs in one session. Creating the session is the expensive step.

Start

Browser sessions

Session options

Frameworks

Account

Notes

​Notes

Notes