Local AI at home · 4 of 6 posts
Making AI Work on Linux
Recycled office PCs, a mini PC and a graphics card: built and benchmarked, not yet in daily use, and the work of putting local AI to real jobs on this site.
- Part 1 · 5 min read
What I built, and what I actually use it for
An honest inventory of the home AI machines, what runs on them today, what is defined but idle, and the fact that most of my AI work is still done in the cloud.
- Part 2 · 7 min read
Memory bandwidth decides the speed
Five office PCs, a mini PC and a graphics card suggest that, for dense models, memory bandwidth rather than drive speed or core count sets how fast a model talks, with one mixture-of-experts model as the exception.
- Part 3 · 5 min read
Six small boxes that turned out to be good at arithmetic
A price-plan model I run reproduced cell for cell on spare office PCs and a mini PC, which makes them fit for batch arithmetic, though none of them beats the fastest machine for a single job.
- Part 4 · 8 min read
An accelerator only helps if it brings its own memory
A graphics card with its own memory runs models fast until one no longer fits, while the extra processors in my mini PC share the CPU's memory and add little I could find.
- Part 5 Coming
Should I Buy a Bigger Box, and the Test a Local Model Must Pass
The jobs a local model will be asked to do on this site, the pass lines it has to meet, and whether that justifies a bigger machine.
- Part 6 Coming
The Cost Accounting
What the hardware, the electricity and the cloud alternative actually cost, once the measurements exist.