This project is a lightweight implementation of a deep research agent designed to iteratively refine research direction on any topic. It combines web search, web scraping, and large language models to generate detailed markdown reports.
Project overview
It aims to provide a simple deep research agent implementation under 500 lines of code that is easy to understand and build upon.
Project type
AI Agent · RAG
Use cases
Knowledge Q&A · Search & Research
Deployment
Refer to project documentation
License
MIT
Best for
Researchers or developers seeking an easy-to-understand, under 500 LoC implementation of a deep research agent to build upon.
Users requiring comprehensive markdown reports with findings and sources from parallel and iterative web searches.
Key capabilities
Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings.
Uses LLMs to generate targeted search queries based on research goals and previous findings.
Provides configurable parameters to control how wide (breadth) and deep (depth) the research goes.
Generates follow-up questions to better understand research needs.
Produces detailed markdown reports with findings and sources.
Handles multiple searches and result processing in parallel for efficiency.
Limitations and risks
The free version of Firecrawl may cause rate limit errors, requiring reduced concurrency which slows down execution.
Operation requires the Firecrawl API for web search and the OpenAI API for the o3 mini model by default. OpenAI is required unless a local LLM is configured.
External services are required for this application, meaning data is sent outside the local environment during inference and research operations.
Getting started
Setup difficulty is medium. It requires installing Node.js dependencies, obtaining external API keys, and setting up environment variables in a `.env.local` file. Docker is supported for easier environment setup.
Clone the repository, install dependencies via `npm install`, configure environment variables, run via `npm start`, and answer prompts in the CLI to generate a markdown report.
Evidence and sources
README: An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models. The goal of this repo is…
README: - **Iterative Research**: Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings - **Intelligent Query Generation…
README: - Node.js environment - API keys for: - Firecrawl API (for web search and content extraction) - OpenAI API (for o3 mini model)
README: 1. Clone the repository 2. Install dependencies: ```bash npm install ``` 3. Set up environment variables in a `.env.local` file: ```bash FIRECRAWL_KEY="your_firecrawl_key"
README: Run the research assistant: ```bash npm start ``` You'll be prompted to: 1. Enter your research query 2. Specify research breadth (recommended: 3-10, default: 4) 3. Specify resear…