Two preprints landed over the summer and between them they gave our office a number for something we had been arguing about without evidence for two years: what an LLM agent can actually do inside an Industry Foundation Classes model.

The first is BIM-Edit (arXiv 2606.20146, June 2026). Nithyanantham et al., working at Universität Rostock’s Institute for Visual and Analytic Computing with TU Clausthal, built 324 editing tasks across 11 realistic building models and 36 synthetic scenes, in three instruction categories: direct, spatial, topological. The agent gets one tool, execute_ifc_code, which runs the Python it writes itself inside an IfcOpenShell sandbox with the model preloaded. Twenty tool calls per task, 120 seconds per call. Scoring averages geometry, semantics and topology — and a task counts as solved only when every metric reaches at least 98 %.

Read the two lines of the results table in the right order. Average score: Gemini 3.0 Flash 49.48 %, Qwen 3.6 Plus 47.63 %, Claude Sonnet 4.6 45.31 %, GPT-5.4 Pro 43.94 %. Solved, under the 98 % bar: no model fully solves more than 3.4 % of the 324 tasks. Those are not the same measurement, and the gap between them is the whole point. The Rostock team put it in a sentence an office should tape to the coordination screen: results that look right in a viewer "remain invalid as structured engineering artifacts", and there is "a substantial gap between current LLM capabilities and the requirements of structured engineering design workflows."

The second is IFCMemoryBench (arXiv 2607.26072, 13 July 2026), where Changyu Du and André Borrmann of TU München, with Alexander Vosseler and Filippo Mazza of the Nemetschek Group, asked a different question: can an agent remember the project while querying the model? They measured 143 multi-session tasks across 19 projects, built on 4,016 prior sessions, each task 25–40 chat exchanges with project facts scattered through the conversation, then a probe question answerable only by combining what was said earlier with a live IFC query. Under deployment-realistic ingestion the best memory system answers 32.4 %; a graph memory 21.0 %; no memory at all, 0.0 %. Stuff every prior user message into the prompt and you reach 83.2 %, at a per-query cost Du and Borrmann themselves call prohibitive. Their conclusion: reliable BIM agents need "domain-aware memory representations linking conversations, project knowledge, and structured model entities". Nemetschek is Graphisoft’s parent, and it co-published a 32.4 % rather than a launch slide. That deserves saying out loud.

←TODAY: both benchmarks are live preprints, the RIBA AI Report is two months old, and the KBOB Faktenblatt is the clause sitting in this autumn’s public contracts.
→3012: the practices still standing are the ones that wrote down which operations they delegated, on which model state, and who signed.
Fulcrum: a benchmark number is worthless as a verdict on the tools and priceless as a specification for the contract.

The profession is already at 74 %

The RIBA AI Report 2026, released 22 July, found 74 % of UK practices now use AI on most projects, up from 59 % in 2025; 75 % report productivity gains, 57 % a positive return, and 61 % say it is now harder for early-career professionals to acquire essential skills. RIBA President Chris Williamson framed it plainly: "Professional oversight of design remains essential if tomorrow’s buildings are going to be safe and sustainable." It is a UK survey and the release states no sample size — a read on the profession’s mood, not on Zurich.

Put 74 % next to 3.4 % and the picture resolves. The profession is working at the edges of the model — texts, minutes, drafts — and the benchmarks say the inside is not yet somewhere to send an agent unsupervised. That is not a contradiction. It is a boundary, and a boundary is something a practice can write down.

Three doors, and which ones we opened

An office does not decide "AI yes or no". It decides which door into the model an agent may use — and that belongs in the BIM-Abwicklungsplan, one page, next to the LOIN table.

Door 1 — Read. Query and report: quantities, property audits, every element without a classification. Failure mode is a wrong answer a human checks. IFCMemoryBench adds the operating condition: at 32.4 %, the prompt must carry the project facts. Never assume the agent remembers them.

Door 2 — Write-to-side. Artefacts next to the model: schedules, Ausschreibungstexte, minutes, a Grasshopper definition or Python snippet a human runs, IDS or LOIN drafts. The model is untouched; the artefact is reviewed and filed. This is where the 74 % live and where the ROI is.

Door 3 — Write-into. Editing the IFC. Closed for now, except on a sandbox copy with an automated validity check and a named reviewer — not out of fear, but because a wrong edit can look completely convincing in the viewer while its relations are broken underneath.

Our own logbook, briefly: door 2 opened in 2023, for Ausschreibungstexte and minutes. Door 1 in 2025, for property audits, with the brief pasted into the prompt every single time. Door 3 is a sandbox copy, a diff, and a partner who signs the diff. The uncomfortable part is that door 2 paid for itself inside a quarter while door 1 took a year of discipline — the cheap door and the useful door were not the same door.

The Swiss clause underneath it

The KBOB Faktenblatt "Umgang mit Daten bei BIM" (1 September 2022) sets out, for public clients, Eigentum und Datenhoheit, Datennutzung, Datensicherheit, Datenschutz and Haftung — each with a definition, a worked example and suggested contract wording. It says nothing about AI; it predates all of this. That is exactly why it matters. If an agent writes into a model whose data sovereignty and liability are already contractually defined, the party the clause names is the office. Professional-indemnity cover has no column for "the agent did it". One Swiss thread worth a line: Apertus, released 2 September 2025 by ETH Zurich, EPFL and CSCS under Apache 2.0, with v1.5 in July 2026 — not an IFC editor and never benchmarked as one, but an on-premise option for the low-risk edges where data may not leave the building.

Atelier: For a team living with AI this week, the honest read of these two papers is that your agent policy is a model-access policy, not a tool-selection policy. The trade-off is plain: door 2 buys real hours at the cost of a review burden nobody budgeted, and door 3 buys very little at the cost of artefacts that fail silently. The Monday move: take one page of the BAP and write the three doors into it — which door each agent use is allowed through, on which model state, who reviews, and what gets logged.

Hack: Accept an agent’s IFC edit on the diff, never on the screenshot — count element and relation types before and after, and treat any change the instruction did not ask for as a rejection. Illustrative, runs with IfcOpenShell.

import ifcopenshell
before, after = ifcopenshell.open("model_before.ifc"), ifcopenshell.open("model_after.ifc")
count = lambda m: {t: len(m.by_type(t)) for t in ("IfcWall", "IfcDoor", "IfcRelContainedInSpatialStructure", "IfcRelDefinesByProperties")}
b, a = count(before), count(after)
print({t: a[t] - b[t] for t in b})   # non-zero and unasked-for = rejection

The relation counts are the ones to watch: BIM-Edit weights topology relations at 0.7 for exactly this reason — geometry can be right while the graph that makes it a building is quietly wrong. PAZ has covered that seam before: an open format guarantees the bytes are readable, not that the meaning survived. An agent does not change that contract; it just writes into it faster.

PAZ Takeaway: The pattern worth borrowing from PAZ Academy is the read-and-report shape behind the PAZ Grasshopper↔Archicad Library and PAZGPT: an agent that queries a Graphisoft-native or IFC model and hands back a checked schedule or property audit, with the project facts supplied in the prompt and the model never written to. Door 1 and door 2 as a way of working rather than a plugin purchase — the version of Archicad automation that survives the numbers above.

Two asks before the week starts. Write the three doors into your BAP, filed next to the LOIN table and the KBOB data-sovereignty clause. And until domain-aware memory exists, put the project facts in the prompt, not in the agent’s memory.