memory bandwidth
- Part 2 ยท 7 min read
Memory bandwidth decides the speed
Five office PCs, a mini PC and a graphics card suggest that, for dense models, memory bandwidth rather than drive speed or core count sets how fast a model talks, with one mixture-of-experts model as the exception.