Six small boxes that turned out to be good at arithmetic

• 707707gate-file values, all equal

Every cell of the result tables, and all 707 values in the gate file, came out equal. That was the whole test: take a model I run on the desktop, run it again on spare machines, and see whether the numbers agree. The gate file is the record of checks the model writes out beside its answers, from its settings to each pass-or-fail flag and the figures behind every failure, and every value in it has to come out the same every time. It came out the same on a cheap refurbished office PC and on a mini PC, to the last digit.

This is the one place so far where the small boxes have passed a real test on real work. It has nothing to do with language models. They turned out to be good at arithmetic, and I only found out because I needed some done.

The job

The model compares electricity price plans for my household. For each plan it works out how best to run the battery against that plan’s prices. That is an optimisation problem: choosing a great many numbers, subject to rules such as “the battery cannot store more than its capacity”, to get the lowest bill. It is all arithmetic, it uses the processor flat out, and it needs no graphics card.

It also splits neatly. Each plan is independent of the others, so the work divides by plan, and the pieces can run on different machines at the same time. I have five office PCs and a mini PC sitting mostly idle, and a desktop I would rather keep free for other things. So I wrote a small dispatcher to spread the pieces across the spare machines. A “worker” below means one piece of the job running on one processor thread.

How it is put together

The design is four lines long.

1. One script, run from the desktop over SSH (a standard way to run commands on another computer), decides which box gets which piece.
2. It ships the committed version of the code and the data it needs, so every box runs exactly the same thing and not whatever happens to be lying about.
3. Jobs run at low priority, so a box with another job to do first, as most of them have, gives way to it.
4. Each box has one pinned Python environment, a private copy of the language and its libraries at fixed versions, so no two boxes can disagree about which version of anything they are using.

The pinning matters more than it sounds. If two machines use different library versions, tiny differences in how they add up decimals can, in principle, make an optimiser pick a different answer. The whole exercise is only worth doing if the answer does not depend on which machine did the sum. Pinning does not settle it on its own: the libraries pick different low-level routines for different processors, and the desktop, the office PCs and the mini PC are three different chips from two makers, so the last digit could in principle still differ.

The test

I tested it in two ways.

First, the model’s own 130 tests passed on both kinds of box. They check the model’s logic, not its speed, but they showed that the new environments behaved.

Second, and more to the point, I re-ran a full job I already had an answer for, the one for the 2024 price year, on an office PC and again on the mini PC, and compared the results with the copy I had already saved from the desktop. I compared them cell by cell, with no tolerance at all: not “close enough”, but equal. Every cell of the result tables matched on both boxes, and all 707 values in the gate file came out equal.

The files themselves are not byte-identical, and the reason is dull. The model runs its pieces in parallel and appends each piece’s rows as it finishes. On the desktop one plan finished first; on the cluster a different one did. So the rows, and some of the columns, sit in a different order. The content is the same. The comparison therefore matches rows by what they describe (which plan, which day, which half hour) and ignores their position.

The times, and where it loses

Here is the same job on three machines, one run each.

The mini PC, with six workers, finished in 6 m 06 s. That beat the desktop. The desktop, with two, took 7 m 29 s, and an office PC with six workers took 12 m 20 s. So the mini PC is the fastest of the three, and the desktop, with a third of the workers, still beats the office PC.

That is the honest part of the result: the old office PCs are the slowest machine in the lineup for a single job. They are cheap, they are idle, and they do not beat my best machine.

The other honest part is why adding workers does not help beyond a point. The job is bounded by its longest single task, one plan run with the extra battery tactics switched on, and one worker runs that task from start to finish. It took about 737 seconds on an office PC against about 393 on the desktop. More workers would not shorten it, because the task is not cut into smaller pieces. Once every other piece has finished, the job is waiting on that one.

So the gain from the spare boxes is throughput, not speed. If I want the answer for four price years, or a sweep across many plans, I can run them side by side on machines that were otherwise idle, and the desktop stays free for whatever else I am doing with it. A single job gets no faster. Many jobs get done in the time of one.

Two traps

One of them cost me time, and both are cheap to avoid.

The first is Sunday morning. The boxes install updates automatically, and if a kernel update is waiting they reboot in a short window early on Sunday morning. A job that spans that window can be killed mid-run. The dispatcher notices a job that has vanished and gives it to another box, but a long job is better started after it.

The second is how to find a run again. Every run is saved under a name, and the sensible-looking shortcut, picking the most recent one, will pick whichever run saved last, which is not necessarily yours. It cost me a restart during the build. Pass the run’s recorded name, and check it before you act on it.

What this does and does not show

The language-model side of these machines is a different story, and a slower one. I have built it and measured it, but I do not use it every day yet. This arithmetic job is the one task where the test is already done. I have not yet given the cluster a new job of its own, but I trust it with one, because the numbers matched to the last cell.

The job was entirely processor-bound and divided by plan, and for that kind of job even slow, idle office PCs turned out to be the right tool.