keyword

probabilistic functional 3D scene graphs

A probabilistic functional 3D scene graph is a structured representation of a three-dimensional physical environment that models the operational and interactive relationships among objects and parts while explicitly quantifying uncertainty over those connections. Unlike conventional 3D scene graphs that primarily record spatial arrangements and static geometry, functional graphs capture actionable affordances, operational dependencies, and usage roles across entities in a scene. By incorporating probabilistic modeling, this representation evaluates the likelihood of relations holistically across the environment, leveraging semantic priors and geometric constraints to resolve ambiguities and assign calibrated confidence scores to predicted interactions.

1 item

FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

Zhengyu Fu, René Zurbrügg, Kaixian Qu, Marc Pollefeys, Marco Hutter, Hermann Blum, Zuria Bauer

OrganizationsETH ZurichMicrosoftUniversity of Bonn & Lamarr Institute

Why you should read this

Proposes a factor-graph reasoning framework that combines geometric constraints with language model priors to construct probabilistic functional 3D scene graphs with accurately calibrated relation predictions from RGB-D images.

Recent work in 3D scene understanding is moving beyond purely spatial analysis toward functional scene understanding. However, existing methods often consider functional relationships between object pairs in isolation, failing to capture the scene-wide interdependence that humans use to resolve ambiguity. We introduce FunFact, a framework for constructing probabilistic open-vocabulary functional 3D scene graphs from posed RGB-D images. FunFact first builds an object- and part-centric 3D map and uses foundation models to propose semantically plausible functional relations. These candidates are converted into factor graph variables and constrained by both LLM-derived common-sense priors and geometric priors. This formulation enables joint probabilistic inference over all functional edges and their marginals, yielding substantially better calibrated confidence scores. To benchmark this setting, we introduce FunThor, a synthetic dataset based on AI2-THOR with part-level geometry and rule-based functional annotations. Experiments on SceneFun3D, FunGraph3D, and FunThor show that FunFact improves node and relation discovery recall and significantly reduces calibration error for ambiguous relations, highlighting the benefits of holistic probabilistic modeling for functional scene understanding. See our project page at this https URL

Source

https://arxiv.org/pdf/2604.03696.pdf

Added

2026-09-30