open_clip is an open source implementation of CLIP that supports training and evaluating various multimodal architectures. It provides a PyTorch-based library installable from PyPI for researchers and engineers building image-text models.
Project overview
open_clip delivers a rare combination of open source CLIP training and evaluation capabilities alongside experimental variable-resolution image processing through NaFlex CLIP, making it a practical choice for teams working on multimodal architectures.
Project type
Model Runtime · Image & Vision
Deployment
Refer to project documentation
License
License pending
Best for
AI engineers who need an open source CLIP implementation for training and evaluating multimodal models.
Researchers exploring variable-resolution and aspect-ratio image processing strategies for CLIP.
Key capabilities
An open source implementation of CLIP provided as a PyTorch library.
Variable-resolution and aspect-ratio image towers with token-budget batching. This feature is experimental.
A post-refactor training stack organized around TrainingTask wrappers, dict-based batches, and FSDP2 support.
Limitations and risks
Downstream integrations should review main branch changes before upgrading because the training stack scope has grown.
The main branch uses a post-refactor training stack by default, which may break compatibility for scripts expecting the older API.
Getting started
Install the library via PyPI as open_clip_torch. For stable API usage, pin to a specific PyPI release or branch rather than tracking the main branch.
Evidence and sources
GitHub project description: An open source implementation of CLIP.
README: main now uses the post-refactor training stack by default. Training is organized around TrainingTask wrappers, dict-based batches, FSDP2 support, NaFlex image/audio pipelines, and…