About this video
How big an AI can a 16 GB Mac run? I climbed from 8 billion to 125 billion parameters.
8B: Ollama loads the whole model (6.2 GB). https://ollama.com
26B: TurboFieldfare streams Gemma 4 from the SSD (needs macOS 26; their M1 Air benchmark). https://github.com/drumih/turbo-fieldfare
35B: Swiftlet, an 18 GB model in 3.2 GB of memory on my M1 Air, about 2-3 tokens/s. https://github.com/leonickson1/Swiftlet
125B: Slotstream, run by someone on a 16 GB M1 Mac mini at 1.76 tokens/s (105 GB model). https://github.com/carloslfu/slotstream
All the links and commands: pranavinnovates.com/giant
Honest notes: the 26B and 125B numbers are other people's runs. Swiftlet is slow on an M1: fine for short answers, long
prompts take minutes. Music: Cipher by Kevin MacLeod (incompetech.com), CC BY 3.0.
What the video says
This eighteen-gig AI is running on my sixteen-gig laptop. It's using about three. How big can a sixteen-gig Mac go? Let's climb. Eight billion parameters, the usual way: Ollama loads the whole thing. Six gigs of memory. Twenty-six billion: TurboFieldfare. Someone's M1 Air like mine gets about seven tokens a second. It needs macOS 26, so I couldn't test it. Thirty-five billion: Swiftlet. Eighteen gigs on disk. It keeps a tiny core in memory and pulls only the pieces each step needs from the SSD. Peak memory: three point two gigs. And a hundred and twenty-five billion: Slotstream. Someone ran it on a sixteen-gig M1 Mac mini, under two tokens a second. It's a hundred-and-five-gig download, so I didn't. The catch? Swiftlet's slow on my Air: two to three tokens a second. Fine for a haiku, not a novel. So: a hundred and twenty-five billion, on sixteen gigs. Slow, but real. Comment GIANT for all four links. How big would you go?