The Automation Stack (and the Sunday it cost me fifty cents)

• 8.7 kWh8.7 kWhexported into negative prices, about 50 cents
• 45 → 5 minworst-case exposure before and after the fix
• 6 of 6heat pumps switched straight back off by a two-command race

The bit nobody writes about

Most solar-and-battery write-ups stop at the install photo. Panels on the roof, a shiny cabinet on the wall, a dashboard with a green arrow pointing the right way. Job done.

It isn’t done. The system is a pile of expensive metal until something decides what it should do minute by minute. Charging? Discharging? Holding? Curtailing? Pre-heating the hot water tank because the sun is about to come out? That is all software, and in my house it lives in Home Assistant, written by me, on weekends, with the lights occasionally flickering.

This post is about that layer, and the two times it bit me. The second cost about fifty cents on a Sunday morning when I was making toast and not paying attention.

Why Home Assistant and not the app

Every inverter ships with an app. The app is fine. It will keep your battery charged, export when there’s a surplus and show you a pleasant green flow diagram. If that’s what you want, you’ve already won.

What the app can’t do is decide using tomorrow’s weather, today’s wholesale price, the temperature of the hot water tank, the state of the heat pumps and whether the cooker is on. Those live in different systems built by different companies with no interest in talking to each other.

Home Assistant is the room where they meet. It is ordinary open-source software whose only real talent is sitting in the middle of a household and knowing things about it. Once the inverter, the price feed, the hot water unit, the heat pumps and the weather forecast are all talking to it, you can write rules. That’s where a pile of expensive metal turns into a system.

What the rules look like

The rules split into two stacks.

The first runs the battery and the grid connection. Every half hour it decides whether the house should import, export, hold, or curtail exports because the wholesale price has gone negative and exporting would cost me money. It is built from a few helper flags and four automations that read the price, the battery charge, the solar output and the time of day, then nudge the inverter into the right mode.

The second runs the hot water tank: keep it hot enough to be useful, heat when energy is cheap or free, and never let it go cold overnight. That last one took a while to find. The upper temperature probe stays warm while the lower half of the tank fills with cold mains water. A small watchdog rule watches both probes and forces a top-up when the bottom one drops while the top one is lying. Stratification, and exactly as undignified to debug as it sounds.

Neither stack is clever. Each rule is a dozen lines at most: a trigger (“when the price drops below zero”), a condition (“and the curtailment flag is off”) and an action (“close the export gate, set the flag, tell me”). The cleverness, such as it is, lives in how the rules combine. A baseline runs once a minute and resets everything to a known-safe state. Price-driven rules override it when the market does something interesting. A morning rule tops the battery up from the grid in the cheapest hour of the night, but only when tomorrow’s forecast is poor enough to justify it. Most of it is bookkeeping.

It works because Home Assistant is fast and stubborn. It evaluates rules in tens of milliseconds, remembers state and doesn’t get bored. A human watching a price feed eighteen hours a day would get bored, make toast and miss a half hour. More on the toast shortly.

Failure one: the day the heat pumps did the splits

Six small heat pumps do the work of the old ducted gas system, one per room, all controlled by Home Assistant. I wrote “quick action” buttons for the dashboard: off, heat everything to 21, cool everything to 24, sleep mode. The first time I pressed “heat everything to 21”, all six units came on, all six immediately turned themselves off, and the dashboard sat there looking innocent.

The script sent two commands to each unit: “set mode to heat”, then “set temperature to 21”. The two raced. The temperature command arrived before the mode change had finished, so the unit received “set temperature to 21, current mode off”, set the target and stayed off. Six times, beautifully consistent, completely useless.

The fix was one line: combine mode and temperature into a single call, so the unit can’t get stuck in the in-between state. Every button then worked first time. The bug had nothing to do with the heat pumps and everything to do with my assumption that two commands sent in order would arrive in order.

The lesson: when two things must happen together, ask the system to do them together. Don’t stitch them yourself unless you’re willing to own the race condition.

Failure two: the Sunday I cost myself fifty cents

Sunday morning in April. Wholesale prices had gone strongly negative. The export-curtailment automation, which closes the export gate when the price goes below zero, was supposed to be in charge. Don’t push electricity onto a grid that is paying you to stop.

I was making toast. Something looked off in the energy widget on my phone. I dismissed it. The toast was more interesting.

About forty-five minutes later I came back to the desk. The dashboard read −6 cents per kilowatt-hour and the export meter was happily clocking over. I was paying the grid to take my power: 8.7 kWh at −4 to −8 cents, about fifty cents in all.

Fifty cents is less than a cup of tea. The question is why the rule that exists to prevent this didn’t fire. I had written it, tested it, and watched it work on earlier negative days. The rule was fine. The trigger was not.

What the trigger actually does

“When the price drops below zero, close the gate” really means “when the price was above zero and is now below it, fire”. It is a change trigger, not a state trigger. The value has to cross the threshold while the system is watching.

Earlier that morning I had restarted the automation server. When it came back it restored every sensor’s last known value from a database, including a wholesale price that was already slightly negative. From the rule’s point of view the price hadn’t crossed zero. It had simply been below zero since the system woke up. No transition, no event. The rule sat there, healthy, watching for a crossing that had already happened in the dark.

Meanwhile the baseline rule had cheerfully reset the export gate to “open, 15 kilowatts allowed”, as it does. On a normal day the curtailment rule would re-close the gate at once. This time it was waiting for an event that wasn’t coming.

The negative-price window ran for the better part of an hour. I made toast.

The fix, in two layers

Layer one runs at the end of the baseline reset. It pauses thirty seconds for the sensors to settle, then asks: is the price below zero right now, however it got there? If so, it closes the gate without waiting for a transition. That catches the restart case.

Layer two is a periodic check on the curtailment rule itself. Besides firing on a transition, it now fires every five minutes, and acts only if the gate is open when it should be closed. The cost is nothing, and even if every other layer fails the worst case is a five-minute exposure instead of forty-five.

Always both. The baseline catches the restart; the periodic check catches drift, races and futures I haven’t thought of.

It looks obvious in retrospect. Of course you re-evaluate state after a restart. None of that was obvious before the toast.

What this is actually a lesson in

Fifty cents isn’t the point. Had the window run the full afternoon, it would have cost three or four dollars. The stakes are small at domestic scale, which is why the lessons are cheap.

The shape of the lesson is worth keeping. The code is right, the logic is right, the hand test worked, and the system still does the wrong thing because the world entered the room in a state your mental model didn’t allow for. Restart-after-event is one case; the heat pump ordering is another. Both are assumptions about when things happen, not what happens.

So when a rule controls something that costs money or comfort, ask what happens if the system reboots while the rule’s preconditions are already met. If the answer is “nothing”, you have my bug. Re-evaluate state on startup, and run a slow periodic check that doesn’t depend on transitions.

One more rule in my stack still has this gap: the opportunistic-charge rule that tops the battery up on a cheap morning hour. A missed cheap hour is a missed opportunity, not an active loss, but it’s the same bug, and it isn’t fixed yet.

Where the stack goes next

The rules are tuned for summer abundance: battery full by mid-morning, solar doing the household’s work, and the question is where to put the surplus. Winter inverts it. Solar drops, the heat pumps dominate, and the question becomes where to get energy at the lowest price. Thresholds, windows and priorities will need re-tuning, possibly re-architecting. That’s a project for autumn.

The stack is small, boring and mostly bookkeeping. The fifty-cent Sunday is why I trust it now. A system I’ve seen fail, fixed, and watched survive the same condition a second time is one I can build on.