CH NEO-ZÜRICH EDITION
WEATHER · CLEAR 24°C
BLEND OF THE DAY · 07/ROGUE
EST. 2027
THE AEC CYBER MORNING NEWS

PAZ Kaffi

DESIGN · DEMOLITION · CAFFEINE · DISPATCH
EDITION 0618 · 18 June 2026
BROADCAST 04:42 CET
2,400 BROADSHEETS PRINTED
READ TIME · 47 MIN
The Building's On-Premise Brain Hallucinates Its Own API
General
FRAME · 06:55
10-06-2026

The Building's On-Premise Brain Hallucinates Its Own API

PowerCodeBench shows on-premise LLMs hallucinate pandapower APIs — demand-guided docs lift accuracy 32–56 points at 41% token cost. Building-mind angle.

The new PowerCodeBench paper from June 2026 is, on its face, about power-grid code. Read it as a building paper. Its single finding is that open-weight large language models running on-premise do not fail at reasoning about a network — they fail at knowing the API of the library that touches the network. Hallucinated function names, misused parameters, mishandled result tables in pandapower. A 2,000-task frozen release, ten open-weight LLMs from 1.5B to 480B parameters, and a “boundary-aware intervention” that lifts every model above 7B by 32 to 56 accuracy points while using 41% of the prompt-token cost.

←TODAY: PowerCodeBench shows on-premise LLMs hallucinate pandapower APIs; demand-guided documentation injection lifts accuracy 32–56 points at 41% of the prompt cost. →3012: Every Zurich 3012 building runs its own brain in its own basement because the grid latency budget will not survive a cloud round-trip. Fulcrum: The building’s mind has to be small enough to live downstairs and humble enough to know which page of the docs it has not read.

Why the topology matters

A smart building today depends on a chain of single points: a cloud LLM endpoint, a vendor portal, a SaaS BMS, a weather feed, a utility’s demand-response API. Old habit at this desk is to draw that chain on paper and circle every box you do not own. The PowerCodeBench result is interesting because it cuts one of those circles. Utilities and energy-research labs require on-premise serving for confidentiality, regulatory, reproducibility, and cost reasons; the same four constraints push smart-building operators toward local inference. The L0–L3 documentation-depth boundary the paper probes is exactly the boundary a building-OS hits when it tries to write its own control script against a library it has only half-read.

The mechanism is not magic. As PAZ’s concept panel on attention in transformers makes clear, every token sees every other token in one O(n²) matrix multiplication — which is why pasting the entire pandapower documentation works, and also why it is wasteful. The intervention estimates the query’s API demand, injects only the relevant docstrings proactively, and routes a reactive correction when the model still trips. The targeted prompts preserve the full-context accuracy ceiling at 41% of the cost. Llama-3.1-405B and Qwen3-Coder-480B led the panel; 70B–120B open-weights matched the mid-tier commercial APIs without leaving the building. Fabrizio Ferri Benedetti’s “local first” thesis, which we have been tracking, finally has the accuracy numbers it was missing.

Building-sense: A building running an on-premise model with demand-guided intervention would behave less like an oracle and more like a graduate engineer with a binder. It would refuse a control move it could not justify against a specific pandapower function and ask the librarian — the documentation index — for the page it has not read. The hallucination rate drops because the building’s mind has been taught to admit what it does not know.

Atelier: In a PAZ atelier this maps cleanly onto the Grasshopper↔Archicad bridge. The same boundary errors that plague pandapower haunt every IFC-touching script we write: hallucinated ifcopenshell attribute names, wrong property-set keys, silent failures on a result table. Borrow the L0–L3 idea. Ship every studio Python kernel with an offline IFC4 spec index and a thin proactive-documentation hook in the prompt. The model writes less; the script runs.

Hack: This Hack teaches you to probe your local LLM’s API knowledge boundary in five lines. The DOMAIN is AI / ML; the medium is Python against any OpenAI-compatible local endpoint (Ollama, vLLM, llama.cpp). Ask the model to name the parameters of a function whose real signature you already have, then diff.

import inspect, ollama, ifcopenshell
truth = set(inspect.signature(ifcopenshell.open).parameters)
reply = ollama.chat(model="qwen3-coder:30b", messages=[
    {"role":"user","content":"Name every parameter of ifcopenshell.open(). Comma-separated, no prose."}])
guess = {s.strip() for s in reply["message"]["content"].split(",")}
print("L0 boundary:", truth - guess)

Run it against three local models, log the deltas, and you have your first L0 probe. The next step is to inject only the missing docstring into the next prompt — demand-guided intervention, in a Friday afternoon.

The trade-off is plain. Local-first inference asks you to own a GPU, a model registry, and a documentation index — three things the cloud quietly owned for you. The PAZ position is that this is a feature: a building that owns its mind owns its failure modes. Draw your real dependency graph this week. Not the architecture diagram — the dependency graph. Circle the third single point you did not know you had, and start pulling it inside the wall.

FILED FROM
CO-SIGNERS
PAZ Academy
CONFIDENCE
HIGH
REPRINTS
© PAZ - PARAMETRIC ACADEMY ZURICH · ALL RIGHTS RESERVED

SOURCE ·

⚑ REPORT AN ERROR · SUBMIT A CORRECTION
◂ BACK TO FRONT PAGE · PAZ KAFFI

© 2026 PAZ Academy.