The Stack

• eighteenerrors in a draft that two AI review panels passed
• $157.47earned from exports over the 130 days
• 62per cent of that from 205 half-hours

The first draft of my first blog post had been through two rounds of AI review and scored above my own threshold on every axis. Then I read it, and found eighteen things wrong. Its sums of what my solar exports had earned were on the wrong side of zero.

People keep asking me what I’m “using”. They mean: which chatbot. The answer is longer than that, and the chatbot isn’t the interesting part. What I have is a small personal stack, in which an AI model is one component among several. Not the centre. A component. The value comes from how the pieces connect and what state they share. This is a home project, one house and one operator, and none of it is advice for how anyone else should set themselves up.

The four layers

Layer 1: the physical system. Solar array, battery, inverter (the box that turns the panels’ and battery’s direct current into household power), hot water heat pump, heating and cooling, a weather station, a smart meter. This is the thing being optimised. Nothing else matters if it isn’t instrumented properly.

Layer 2: data and control. Home Assistant, the home automation software, runs on its own small computer, separate from my cluster of other machines. It reads everything the inverter and sensors emit and, where I’ve told it to, sends commands back to the inverter, the hot water unit and the heating. Its own database is short-lived, so everything is also copied to a time-series store (a database built for readings over time) on one of the cluster machines. That is where I go for anything I want to analyse months later.

The on-site sensors are only half the picture. The house also pulls in outside data. A solar forecasting service models the next two days of generation, and I store it next to the inverter readings, so I can compare what the forecast said yesterday with what happened today, half-hour by half-hour. It looked as if the forecast over-predicted wildly. Mostly it didn’t: on sunny days I had been comparing it with my battery-curtailed output, not what the panels would have produced if the battery hadn’t been full. I was throttling the inverter and blaming the messenger. On cloudy days it does still run high, by a margin three days of data can’t pin down. The national weather bureau’s forecast for my location lets the heating start earlier on a cold day, while the sun is still doing the work.

And I have my own weather station on the roof: irradiance, outdoor temperature, humidity, rain, wind, lightning. It is the ground truth the forecasts get checked against. A third-party model is only as good as your ability to check it against your own instruments. Without my own irradiance reading I couldn’t tell a forecast error from a curtailment artefact.

Photo to come

Layer 3: the workbench. A folder on my computer of plain files: drafts, analyses, runbooks, a project knowledge base, and a plain-text list of open work. The knowledge base is the most valuable artefact in the stack, and the one I’d rebuild first if I lost everything else.

Layer 4: the co-pilot. An AI assistant with access to the workbench and to the house’s automation system, and no authority to change anything physical without my say-so. It drafts, reviews, analyses, writes code, and keeps the list of open work up to date. It holds no state between sessions except through the workbench, so everything it “remembers” is plain text I can read, correct or delete.

The hard line

Configuration never goes to the live system without my explicit yes. “Ready to build” is not a yes. An earlier paragraph in the same conversation is not a yes. A confident “shall I deploy?” from the assistant is not a yes. The only yes is me typing go, with the draft open in front of me.

And the hard line moved from plumbing to rule. In April the assistant had no way to change the house; I copied each change across by hand. Now it does, and what stops it is mostly a rule: nothing goes live until I type go. A rule is a weaker guarantee than a key it can’t reach, and I know it. It’s the one place the stack is less safe than I described in April, and I’d rather say so than leave the old paragraph standing.

The cluster

I also run a small cluster of second-hand computers. It mostly teaches me things: benchmarking five identical machines showed that inference speed is bounded by memory bandwidth, not storage, and the faster drives I had just fitted to three of them made no difference.

A worked example: the feed-in tariff rerun

The first draft leaned on feed-in tariffs, the rate a retailer pays for the solar you export. The worst of the eighteen were sign errors. My data store records the feed-in rate with an accounting sign, where a negative number means money coming in. The knowledge base had it the other way round, and the draft, and the reviewers, all trusted the knowledge base. That’s a Layer 3 failure: no sensors involved, just one wrong line of plain text read confidently by everything downstream.

So I went to the data. A script matched four months of invoices against the half-hourly data store, and once the sign was negated the two agreed to the cent, every billing period. The invoices also showed the feed-in rate was the wholesale price only, with no network charges added, which the first draft also had wrong. With the sign settled, the data store says that over 130 days (28 November to 6 April) the household exported 2,069.81 kWh and earned $157.47.

Then I asked for a by-hour-of-day profile of the feed-in rate, and the chart told a story the draft had missed. The headline was not “solar exports earn a reliable few cents a unit”. Through the middle of the day the rate sagged towards zero and often went negative, when exporting earns nothing or costs money. The money was in the evening. Of the $157.47, $97.48, which is 62 per cent, came from just 205 half-hours when the rate was above 10 cents. The average was close to meaningless. The shape of the distribution was the story.

Chart to come

Every affected draft was then rerun from a brief that folded in all eighteen corrections, and every step left a plain-text trace I can read back.

The stack did not catch the problem. The reviewers I had built shared the knowledge base the error lived in, and passed it. What caught it was me reading the finished draft and not believing the numbers. The stack is good at executing on a question and checking an answer against data. It is not yet good, in my hands at least, at noticing that the question was set up wrongly in the first place.

And then there is me

The stack works because of the person in the middle of it. Every piece arrived because a specific question wouldn’t yield without it: the weather station because I couldn’t trust the forecasts without ground truth, the long-memory store because Home Assistant’s own database was too short to show seasonal patterns, the structured critique habit because I caught myself accepting first drafts that were politely wrong.

The AI sits inside that loop, not around it. The assistant is good at executing a well-framed question and surprisingly bad at deciding which question is worth asking. When I get the brief wrong, it confidently produces something plausible and wrong, and I have to catch it. None of the boxes do anything on their own. The stack works because someone in the middle knows what they are trying to find out, writes down what they thought yesterday so they can argue with it tomorrow, and treats every output, from the inverter, the forecast or the assistant, as a claim to be checked rather than a fact to be accepted. Take that person out and the boxes are just boxes.

What I’d change starting from scratch

Build the workbench folder and the list of open work before buying a single sensor: the most valuable piece of the stack cost the least. Sort out the long-term data store on day one, with the invoice reconciliation built in, because half the analytical pain has been retrofitting trust into data I was already using. And resist buying compute. If the goal had been “optimise my energy bill and write about it”, none of the cluster was on the critical path. Productive indulgence, but indulgence. Everything else could be swapped without much pain, because the interesting state lives in plain text in a folder.