The missing channel in embodied AI was never vision. Egocentric datasets — Ego4D, Project Aria — already give models millions of hours of hands moving through the world. What they never carried was force. A camera watches a hand close on a plywood offcut and cannot tell you whether the grip is 2 newtons or 20. That gap is exactly what a new arXiv paper, TouchAnything (arXiv:2605.13083, cs.RO), sets out to close: infer contact, pressure and force directly from RGB video, so tactile supervision scales without bolting a tactile sensor onto every wrist.

Read the release as two artifacts, not one — the topology matters. EgoTouch is the dataset: 208 manipulation tasks, 1,891 episodes, indoor and outdoor, with three synchronized RGB streams (one head-mounted egocentric view, two wrist-mounted), bimanual 3D hand pose, and continuous pressure maps read off wearable tactile sensors during capture. TouchAnything is the model that eats it: egocentric view as the primary input, wrist views fused optionally at inference. The sensors are worn only while building the corpus; the shipped model wants nothing but cameras.

System. This is the same architectural bet the field keeps making — infer an expensive modality from a cheap one. The mechanism underneath is the attention operator PAZ has diagrammed before (see our concept panel on Attention in engineering): a set of image tokens attends across views, letting a wrist glimpse reweight what the head-cam saw. That is why fusing the wrist streams helps at all — the reported gains are up to 5.0% relative Contact IoU and 6.1% relative Volumetric IoU over egocentric-only. Note the honest limit in the topology: those are relative numbers. The paper gives no absolute baseline, no sensor resolution, no license confirmed live. A cartographer marks that as an unresolved edge, not a dead end.

←TODAY: In 2026 you can teach a gripper contact force from ordinary video, no gel sensor on every finger. →3012: Trade craft — formwork, panel handling, the wrist memory of a good Bauleiter — becomes a queryable corpus. Fulcrum: The cheap modality only teaches the expensive one because someone paid, once, to record both in register.

Street. Construction work is contact-rich in ways factory pick-and-place never was: placing, fastening, finishing, feeling when a fixing has seated. The hardware tax on teaching a site robot that has been the ceiling. If force can be learned from a headset most crews could wear anyway, the barrier drops from instrumented cells to a camera and a body. Chris Paxton, flagging the dataset as “another large, open egocentric pretraining” resource, points at the real shift: the bottleneck is moving from sensors to annotated human video at scale.

Atelier: For a Büro watching embodied AI approach the site, the practical read is that your archive of on-site footage is quietly becoming training data — which means it now carries GDPR and EU AI Act weight the moment it captures bystanders or private space. Monday move: write a one-page capture-consent and retention policy for any wearable-camera recording on your projects, before a research partner asks for the footage.

Hack: Read a tactile prediction the way the paper scores it — Contact IoU is just the overlap between where you predicted contact and where the pressure map actually lit up. Thresholding turns a soft pressure field into a contact mask, then it is a set operation. This is the metric that decides whether a wrist view earned its place in the fusion.

import numpy as np
def contact_iou(pred, gt, thr=0.1):
    p, g = pred > thr, gt > thr
    return (p & g).sum() / (p | g).sum()

Run it on your own paired frames and you feel where the model is honest and where it hallucinates grip. The transferable AEC pattern is the same shape as the whole paper: infer structural stress or a hidden defect from an ordinary photograph, and validate it against the one instrumented capture you actually paid for. Draw that dependency the way I would draw any pipeline — the cheap sensor is only trustworthy as far back as the expensive one that trained it. Go find the paired dataset your inference secretly depends on, and put a name on it.

Source: arXiv