This project streamlines the ingestion and pre-processing of unstructured data for large language models. It provides modular functions and an MCP server to convert images and text documents into structured outputs.
Project overview
The system automatically detects file types and routes them to specific partitioning functions, and an MCP server gives agents the ability to process over 60 file types.
Project type
MCP · Document Processing · Data Processing
Use cases
Documents & Office · Knowledge Q&A
Deployment
Python environment
License
Apache-2.0
Best for
Developers and data teams who need to automate the ingestion and partitioning of images and text documents for LLM workflows.
Operators using MCP-compatible agent frameworks who need to turn files into structured data.
Key capabilities
Modular functions ingest and pre-process images and text documents, automatically detecting file type and routing it to the appropriate file-specific partitioning function.
A server that gives agents the ability to turn 60+ file types into structured data by parsing, enriching, chunking, and embedding files.
Limitations and risks
Interaction with the library requires coding.
Setup requires installing specific system dependencies like libmagic-dev and poppler-utils based on document type.
GPU requirements, data boundary specifics, and cost dependencies are not documented.
Getting started
Install the Python SDK via pip. Note that setup requires installing specific system dependencies like libmagic-dev and poppler-utils depending on the document type. Run a partition function on a test document and print the string representation of the output.
Alternatives and comparisons
Converts PDF documents to Markdown and extracts text and images. It operates as a library and is self-hostable.
Converts Excel workbooks into structured JSON including tables, shapes, and charts.
Converts PDF, image, DOCX, PPTX, and XLSX inputs into machine-readable formats such as Markdown and JSON.
GitHub project description: Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models.
README: The `unstructured` library provides open-source components for ingesting and pre-processing images and text documents, such as PDFs, HTML, Word docs, and many more.
README: Unstructured Transform brings production-grade document processing to your agents as an MCP server. It gives them the ability to turn 60+ file types into structured data that's re…
README: Transform works with virtually any MCP-compatible host or agent framework — Claude Code, Cursor, Codex CLI and more.
README: Ready to move your data processing pipeline to production, and take advantage of advanced features? Check out Unstructured Pipelines. In addition to better processing performance,…