turbo-fieldfare is a Swift + Metal runtime that runs Gemma 4 26B-A4B inference on Apple Silicon Macs with limited RAM by streaming only the needed model experts from SSD, keeping memory usage to about 2 GB. It includes a native Mac application for model installation, configuration, and text generation.
Project overview
By streaming only the experts required per token from SSD rather than loading an entire model into memory, turbo-fieldfare enables large mixture-of-experts model inference on machines with as little as 8 GB of RAM—a custom runtime approach distinct from wrapping MLX or llama.cpp.
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Apple Silicon Mac owners with limited RAM who want to run Gemma 4 26B-A4B text inference locally.
Developers interested in a purpose-built Swift + Metal runtime for SSD-backed mixture-of-experts streaming.
Key capabilities
Streams only the experts needed for each token from SSD and keeps the shared core and KV cache in memory to run models in approximately 2 GB of RAM.
A SwiftUI/AppKit Mac app for model installation, configuration, and text generation.
A CLI for instruction chat and raw completion using an existing model installation.
An experimental local server supporting Chat Completions, streaming, and function tools.
Build iPhone and iPad apps to measure inference speed and memory use on mobile hardware.
Limitations and risks
An Apple Silicon Mac is required; older macOS and Metal versions are not supported.
Currently scoped to text-only inference from the pinned Gemma 4 26B-A4B instruction checkpoint.
Images, audio, and video are not supported.
The OpenAI-compatible server must be kept on loopback as it has no remote authentication or TLS.
The model can still repeat itself or give incorrect answers, so important results must be checked.
Getting started
Setup difficulty is medium, requiring a build from source via Swift Package Manager. Clone the repository, run swift build -c release, execute .build/release/TurboFieldfareMac, click Download to fetch the model, then click Load Model and Generate. Coding is optional for use.