Tags

benchmarks

• Part 2 · 7 min read

  Memory bandwidth decides the speed

  Five office PCs, a mini PC and a graphics card suggest that, for dense models, memory bandwidth rather than drive speed or core count sets how fast a model talks, with one mixture-of-experts model as the exception.

• Part 4 · 8 min read

  An accelerator only helps if it brings its own memory

  A graphics card with its own memory runs models fast until one no longer fits, while the extra processors in my mini PC share the CPU's memory and add little I could find.