Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU
A tech enthusiast successfully ran a modern 26-billion-parameter AI model on 13-year-old server hardware without a GPU. The project highlights the engineering challenges of optimizing software for older CPU architectures lacking modern instruction sets.
There’s a server in my basement that has no business running a modern language model. It’s a repurposed HP StoreVirtual storage box, roughly thirteen years old, two Ivy Bridge Xeons, no GPU. It was built to hold disks, not do math. As of this week it runs Google’s Gemma 4 , a 26-billion-parameter open-weights mixture-of-experts model, at about five tokens per second. Reading speed.
Get the full story
Sign up for Headlinne to unlock AI insights, political bias analysis, and your personalized news feed.
Create free accountAlready have an account? Sign in