A TypeScript library that uses LLMs to extract structured data from webpages based on Zod or JSON Schema. It requires writing TypeScript code and configuring a large language model provider to function.
Project overview
A TypeScript library that uses LLMs to extract structured data from webpages based on Zod or JSON Schema. It requires writing TypeScript code and configuring a large language model provider to function.
Project type
Workflow · Data Processing
Use cases
Data Analysis · Automation
Deployment
Refer to project documentation
License
MIT
Best for
Developers working in TypeScript who need to extract structured data from webpages using large language models.
Key capabilities
Extracts structured data from any webpage using LLMs based on user-defined schemas.
Allows defining extraction schemas using Zod or JSON Schema for full type-safety.
Supports 6 formatting modes including html, raw_html, markdown, text, image, and custom content.
Streams partial objects during the extraction process.
Generates a re-usable Playwright script to scrape contents according to a defined schema.
Limitations and risks
Requires an external LLM provider API or a local LLM instance, meaning the system cannot operate independently of these services.
Using external LLM providers may incur API costs.
Requires writing TypeScript code and initializing an LLM provider.
Getting started
Install dependencies from npm (npm i zod playwright llm-scraper), initialize an LLM provider such as OpenAI, Anthropic, Google, Groq, or Ollama, and create a new scraper instance using the configured provider.
Evidence and sources
GitHub project description: Turn any webpage into structured data using LLMs
README: LLM Scraper is a TypeScript library that allows you to extract structured data from **any** webpage using LLMs.
README: Schemas defined with Zod or JSON Schema - Full type-safety with TypeScript - Based on Playwright framework - Streaming objects - [Code-generation](#code-generation) - Supports 6 f…
README: Replace your `run` function with `stream` to get a partial object stream.
README: Using the `generate` function you can generate re-usable playwright script that scrapes the contents according to a schema.