Crawl4AI is an open-source, LLM-friendly web crawler that became the #1 trending GitHub repository. It provides asynchronous parallel crawling, structured data extraction, and markdown conversion optimized for feeding content into LLMs and RAG pipelines.
ScrapeGraphAI is an open-source web scraping library that uses LLMs to automatically extract structured data from websites. Instead of writing CSS selectors or XPath queries, developers describe what data they want in natural language. Supports multiple LLM providers and handles dynamic JavaScript-rendered pages.
Respan lets you trace LLM and agent calls across any model or framework, A/B test prompts on production traffic, and route requests across 500+ models through one gateway.