Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Why CEREBRO kept it
Efficient Qwen inference on resource-constrained devices.
The text below is an automated extraction of the article at https://github.com/leonickson1/Swiftlet, stored verbatim in the public cerebro-vault repository. Copyright remains with the original publisher (github.com).
Run 35B and 80B Qwen models on ordinary Apple devices, including iPhones. Swiftlet is a Swift + Metal runtime for the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family. It keeps only the small dense core of a model resident in memory and streams the routed Mixture-of-Experts weights from storage on demand. The result: | Model | Disk | Peak RAM | Decode speed (M5 Mac) | |---|---|---|---| | Qwen3.6-35B-A3B, 4-bit | 18 GB | 2.6 GB | 7 to 11 tok/s | | Qwen3-Next-80B-A3B, 4-bit | 42 GB | 4.3 GB | 4.5 to 5 tok/s | The 35B also runs on an iPhone 17 in about 2.5 GB of RAM, at about 1 tok/s today.
Community take
Prefill becomes the bottleneck—disk-swapping methods deliver 10 tokens/hour and degrade hardware.
Backlinks
Appeared in 1 briefing