#crawler
2 items
Repo
D4Vinci/Scrapling
Scrape websites with a Python framework that adapts to page changes, bypasses anti-bot systems and scales from a single request to full crawls.
Repo
unclecode/crawl4ai
Crawl websites and turn them into clean Markdown that you can feed to LLMs, AI agents, RAG systems or data pipelines.