3D Semantic Parsing of Large-Scale Indoor Spaces

Iro ArmeniOzan SenerAmir ZamirHelen JiangI. BrilakisMartin FischerS. Savarese

article2016CVPR2,116 citations

Modern 3D scanning technology can readily capture the physical geometry of entire buildings, producing massive point clouds comprising hundreds of millions of data points across thousands of square meters. However, turning these vast raw geometric datasets into structured, semantically meaningful models remains a major bottleneck for facility management, workspace analytics, and robotics. Existing computer vision techniques generally focus on single-view depth images or small-scale room scenes, failing to efficiently process whole-building scans and struggling to handle occlusions caused by interior clutter.

The article demonstrates an automated, hierarchical framework designed to parse entire building point clouds into disjoint functional spaces—such as rooms and hallways—and subsequently detect both structural and movable indoor elements using global geometric priors. Rather than framing building element recognition as a conventional point-by-point segmentation task, the authors approach it as an object detection problem operating within a normalized 3D space.

The evaluated approach processes raw 3D scans in two main stages. First, it identifies space-dividing walls by detecting the empty void space bounded between wall surfaces rather than attempting to fit surface planes to cluttered points; these detected boundaries segment the floor plan into individual rooms that are normalized into a standardized coordinate system. Second, sliding-window classifiers trained on geometric and appearance features propose candidate building elements, which are then globally optimized for contextual consistency using a graphical model. To evaluate this framework, the authors benchmarked their method across a newly collected dataset covering over 6,020 square meters across five indoor areas from three buildings, totaling more than 215 million points across twelve semantic element categories.

The experimental findings show that the proposed framework substantially outperforms standard baselines. In floor-plan space parsing, the void-detection method achieved an average Adjusted Rand Index score of 0.77, compared to 0.20 for standard plane-fitting algorithms and 0.31 for line-fitting baselines. For element parsing, the model achieved a mean average precision of 49.93%, surpassing competing 3D object detection methods that reached 36.11% and 39.21%. Performance was highest on major structural elements, achieving 91.77% average precision on columns, 88.70% on floors, and 72.86% on walls, though furniture recognition remained lower at an average of 25.51% due to wide visual and geometric variety across buildings. Ablation tests revealed that normalized global geometric position contributed far more to recognition accuracy than color or fine local geometry.

These results indicate that automated building analysis is most effectively executed directly on comprehensive 3D point clouds rather than on individual 2D or depth images. By prioritizing void spaces and global architectural regularity over local surface textures, organizations can reliably extract floor plans and core building structures without manual intervention. This capability lowers the cost and turnaround time for generating digital building models, supporting applications such as structural assessments, automated floor plan updates, and lighting simulations.

Organizations seeking to implement large-scale spatial modeling should adopt hierarchical, detection-oriented 3D workflows for structural asset logging while continuing to treat movable furniture classification as an area requiring supplementary data or localized models. Future development should focus on incorporating fine-grained features, such as edge detection, to improve precise bounding box localization and expanding geometric templates to naturally support non-rectangular and curved architectural layouts.

The primary limitations of this work involve its foundational assumption of roughly planar, rectangular room geometry aligned with primary axes, which reduces effectiveness in circular or irregularly shaped spaces. Additionally, while the model reliably determines the presence and approximate count of interior objects, fine-grained localization boundaries occasionally exhibit gaps. Nevertheless, confidence in the framework's structural parsing capabilities remains high across standard office and educational environments.

3d-scanning building model classifiers Spaces Parsing Point

Armeni et al (2016).pdf

Paper: SUN RGB-D: A RGB-D scene understanding benchmark suite, Shuran Song et al. (2015). This benchmark established standardized 3D bounding box annotations and RGB-D indoor scene metrics upon which subsequent large-scale parsing frameworks build.
Paper: Indoor Segmentation and Support Inference from RGBD Images, N. Silberman et al. (2012). It introduced early paradigms for inferring planar structures, physical support, and semantic spatial partitions from aligned RGB-D indoor data.
Paper: 3D ShapeNets: A deep representation for volumetric shapes, Zhirong Wu et al. (2014). It pioneered 3D volumetric geometric representations for indoor shape recognition and spatial occupancy modeling.
Paper: Fully convolutional networks for semantic segmentation, Jonathan Long et al. (2015). It provides the foundational deep learning framework for dense semantic parsing that motivated the extension of spatial segmentation into 3D environments.
Paper: A volumetric method for building complex models from range images, B. Curless et al. (1996). It established the seminal volumetric signed-distance field and space-carving principles used to process raw 3D scans and identify void spaces.

Paper: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, Charles R. Qi et al. (2017). It introduced direct deep learning on unordered 3D point sets, heavily advancing indoor semantic segmentation and setting new baselines on datasets like S3DIS.
Paper: ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes, Angela Dai et al. (2017). It scales up real-world indoor 3D scene datasets with rich semantic annotations and CAD alignments, building upon large-scale indoor scanning benchmarks.
Paper: Matterport3D: Learning from RGB-D Data in Indoor Environments, Angel Chang et al. (2017). It advances building-scale 3D understanding by introducing whole-building RGB-D scans with global alignments and instance-level semantic labels.
Paper: RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds, Qingyong Hu et al. (2019). It proposes an efficient neural architecture specifically designed to scale semantic segmentation to massive, millions-of-points 3D scenes such as S3DIS.
Paper: Frustum PointNets for 3D Object Detection from RGB-D Data, Charles R. Qi et al. (2018). It builds on 3D bounding-box detection from RGB-D scans by proposing 3D viewing frustums directly evaluated on PointNet variants.
Paper: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks, Christopher Choy et al. (2019). It introduces high-dimensional sparse tensor convolutions, advancing indoor 3D semantic segmentation benchmarks including the Stanford 3D Indoor Spaces dataset.
Paper: Deep Learning for 3D Point Clouds: A Survey, Yulan Guo et al. (2019). This survey provides a comprehensive overview of the deep learning paradigms that emerged for 3D point cloud segmentation and detection following large-scale benchmarks.
Paper: PCT: Point cloud transformer, Meng-Hao Guo et al. (2020). It extends point cloud feature learning by applying transformer architectures to large-scale indoor semantic segmentation on S3DIS.

1. Introduction
2. Related Work
3. Parsing Point Cloud into Disjoint Spaces
3.1. Detection incorporating void spaces
3.1.1 Detecting the peak-gap-peak pattern
3.1.2 Merging
3.2. Canonical Coordinate System Among Spaces
4. Parsing Disjoint Spaces into Elements
4.1. Enforcing Contextual Consistency using CRF
4.2. Updating the Disjoint Space Parsing Results
5. Experiments
5.1. Dataset
5.2. Parsing into Disjoint Spaces
5.3. Parsing into Semantic Elements
5.4. Comparison with Conventional RGB-D
6. Conclusion
References

Knowls

Knowl 1 — Peak-Gap-Peak Filter Bank Formulation for Void Space Detection
equation
To identify space dividers (walls) separating neighboring rooms in large-scale 3D point clouds without relying on planar surface fitting, wall detection is formulated as finding an empty void bounded by two parallel point-density margins (the outer shells of the wall surfaces). Given a 1D density histogram $H(s)$ of point coordinates along an axis $s \in \{x, y, z\}$ aligned with the building's structure, a parametric filter bank $g_{\delta, c}(s)$ is defined as:

$g_{\delta, c}(s) = \frac{1}{2c} \Pi_{\frac{\delta}{2} + c}(s) - \frac{1}{2c} \Pi_{\frac{\delta}{2}}(s)$

where $\Pi_k(s) = \mathbf{1}[|s| \le k]$ is a symmetric rectangular boxcar function of half-width $k$ , $\mathbf{1}[\cdot]$ is the indicator function, $\delta > 0$ represents the stop-width (the void/gap width between the two wall surfaces), and $c > 0$ represents the pass-width (the thickness of the outer peak bands).

Convolving $H(s)$ with $g_{\delta, c}(s)$ produces a 3D response surface $R(s, c, \delta) = (H * g_{\delta, c})(s)$ . Candidate wall positions along axis $s$ are obtained by max-pooling across the scale parameters $c$ and $\delta$ :

$\bar{R}(s) = \max_{c, \delta} R(s, c, \delta)$

followed by non-maximum suppression (NMS) on $\bar{R}(s)$ to extract discrete wall centerlines.
Knowl 2 — Graph-Based Slicing and Merging Algorithm for Disjoint Space Parsing
algorithm
The algorithm parses an unsegmented large-scale building point cloud into disjoint semantic rooms by detecting space dividers along 1D axes, generating an over-segmented cell decomposition, and merging cells across non-divider boundaries.
Input: Point cloud $P \subset \mathbb{R}^3$ , filter parameter ranges $C$ and $\Delta$
Output: Set of disjoint spaces $\mathcal{S} = \{S_1, S_2, \dots, S_M\}$
Align principal axes of $P$ to Cartesian frame $(x, y, z)$ via Principal Component Analysis
for each axis $a \in \{x, y, z\}$ do
    Compute 1D coordinate density histogram $H_a(s)$
    for each $c \in C$ and $\delta \in \Delta$ do
        Compute convolution $R_a(s, c, \delta) = (H_a * g_{\delta, c})(s)$
    Pool responses across scale parameters: $\bar{R}_a(s) = \max_{c \in C, \delta \in \Delta} R_a(s, c, \delta)$
    Extract 1D candidate divider planes along axis $a$ via Non-Maximum Suppression on $\bar{R}_a(s)$
Slice $P$ along all detected divider planes across all three axes to form over-segmented 3D volumetric cells $\mathcal{V} = \{v_1, v_2, \dots, v_K\}$
Construct adjacency graph $G = (\mathcal{V}, \mathcal{E})$ , where $(v_i, v_j) \in \mathcal{E}$ if cells $v_i$ and $v_j$ share a spatial boundary
for each edge $(v_i, v_j) \in \mathcal{E}$ do
    Form local point sub-cloud $P_{ij} = v_i \cup v_j$
    Test for peak-gap-peak void signature in $P_{ij}$ along the boundary normal
    if a space divider signature is confirmed between $v_i$ and $v_j$ then
        Remove edge: $\mathcal{E} \leftarrow \mathcal{E} \setminus \{(v_i, v_j)\}
Compute the connected components of graph $G = (\mathcal{V}, \mathcal{E})$
for each connected component $\mathcal{C}_m \subseteq \mathcal{V}$ do
     $S_m \leftarrow \bigcup_{v \in \mathcal{C}_m} v$
return $\mathcal{S} = \{S_1, S_2, \dots, S_M\}$
Because 1D divider detection scales linearly with building floor area, the initial slicing creates an over-segmented grid in $O(N)$ time. The subsequent local neighbor testing resolves over-segmentation across irregularly shaped or non-continuous spaces via graph connectivity.
Knowl 3 — Canonical Space Normalization and Voxelized 3D Sliding Window Representation
model/method
To enable cross-building generalization for semantic element detection, each parsed disjoint space is transformed into a standardized canonical reference frame:
1. Z-axis: Aligned with the gravitational axis.
2. X-axis: Aligned along the primary room entrance wall, making the y-axis strictly perpendicular to the entrance.
3. Scaling: Point coordinates are normalized to the unit cube $[0, 1]^3$ .
A 3D detection window within this unit cube is parameterized by its anchor position $P = (P^x, P^y, P^z) \in [0, 1]^3$ (bottom-left corner), its normalized 3D bounding size $S = (S^x, S^y, S^z) \in [0, 1]^3$ , and a 3D binary occupancy grid $B \in \{0, 1\}^{K \times K \times K}$ divided into $K^3$ equal sub-voxels. Each candidate window is represented by a concatenated feature vector $\boldsymbol{\Phi}$ composed of:
- Global Features: Normalized window position $P$ (3 dimensions) and size $S$ (3 dimensions).
- Local Features per Voxel $l \in [K \times K \times K]$ : Sub-voxel occupancy indicator $B_l \in \{0, 1\}$ (1 dimension); point ratio $d_l = \frac{N_l}{N_{\text{total}}}$ where $N_l$ is point count in voxel $l$ and $N_{\text{total}}$ is the total points in the window (1 dimension); average RGB color $(C^r_l, C^g_l, C^b_l)$ (3 dimensions); average surface normal $(n^x_l, n^y_l, n^z_l)$ (3 dimensions); and surface curvature $\kappa_l$ (1 dimension). Non-occupied voxels receive feature vectors of all zeros.
Candidate window shapes are learned per class by taking ground truth bounding boxes and occupancy patterns from training data, clustering them via Affinity Propagation using intersection-over-union and occupancy distance, and taking the cluster mean size and mode occupancy pattern as dictionary shape atoms.
Knowl 4 — CRF Formulation for Contextual Semantic Element Selection
equation
To exploit spatial co-occurrence constraints between candidate 3D element proposals, proposal selection is framed as maximum a posteriori inference over a Conditional Random Field (CRF). Let $V$ denote the set of detection proposals generated via sliding window SVMs, where proposal $v \in V$ has class label $e_v$ , SVM confidence score $l_v$ , position $P_v$ , size $S_v$ , and occupancy grid $B_v$ . An undirected graph $G = (V, E)$ connects each node $v$ to its $k_e$ nearest spatial proposals belonging to each class $e$ .

Each proposal is assigned a binary decision variable $y_v \in \{0, 1\}$ , where $y_v = 1$ indicates that proposal $v$ is selected as a final semantic element. Pairwise edge features between proposals $u$ and $v$ are defined as:

$\boldsymbol{\Phi}_{(u,v)} = \left[ B_u, B_v, S_u, S_v, |P_u - P_v| \right]$

The log-linear energy maximization problem is:

$\arg\max_{\mathbf{y} \in \{0,1\}^{|V|}} \sum_{v \in V} w_0 l_v y_v + \sum_{(u,v) \in E} y_u y_v \left( \mathbf{w}_{e_u, e_v} \cdot \boldsymbol{\Phi}_{(u,v)} \right)$

Linearizing the quadratic interaction terms via auxiliary variables $y_{uv} = y_u y_v$ yields the following Integer Linear Program (ILP):

$\arg\max_{\mathbf{y}, \mathbf{y}_E} \sum_{v \in V} w_0 l_v y_v + \sum_{(u,v) \in E} y_{uv} \left( \mathbf{w}_{e_u, e_v} \cdot \boldsymbol{\Phi}_{(u,v)} \right)$

$\text{subject to} \quad y_{uv} \le y_u, \quad y_{uv} \le y_v, \quad \forall (u,v) \in E$

$y_u + y_v \le y_{uv} + 1, \quad \forall (u,v) \in E$

$y_v \in \{0, 1\} \quad \forall v \in V, \quad y_{uv} \in [0, 1] \quad \forall (u,v) \in E$

where $w_0$ is a scalar unary weight, $\mathbf{w}_{e_u, e_v}$ is the class-pair weight vector learned via Structured Support Vector Machines (S-SVM), and inference is solved via off-the-shelf Mixed Integer Programming (MIP) / Linear Programming (LP) solvers.
Knowl 5 — Disjoint Space Parsing Refinement via Detected Wall Semantics
model/method
Narrow architectural corridors and bottlenecks can exhibit 1D density histogram dips that mimic peak-gap-peak space dividers, leading to false partitioning of single continuous spaces (such as hallways). To resolve this, a bidirectional feedback loop updates the space segmentation using the output of the 3D semantic element detector.

After candidate proposals are filtered via CRF inference, the spatial bounding boxes corresponding to the detected wall class are mapped back into the global point cloud coordinate frame. An adjacency graph over the parsed spaces is constructed, where an edge exists between neighboring spaces that share a partitioned boundary. For each adjacent pair, the boundary interface is examined for the presence of a detected semantic wall. If no detected wall bounding box intersects the boundary zone, the divider is identified as a false positive, the separating edge in the adjacency graph is retained, and connected components are computed over the graph. Falsely partitioned spaces connected across wall-free boundaries are merged into a single coherent space.
Knowl 6 — The Stanford Large-Scale 3D Indoor Spaces Dataset
experimental setup
A benchmark dataset was collected comprising 3D colored point clouds from 5 large-scale indoor areas across 3 different buildings, covering approximately 1,900, 450, 1,700, 870, and 1,100 square meters (total area of 6,020 m $^2$ ) and over 215 million points. Data was captured using Matterport 3D scanning technology without manual scan registration.

Ground truth annotations were created in 3D for 12 semantic classes categorized into:
- Structural Elements (7 classes): ceiling, floor, wall, beam, column, window, door.
- Furniture and Items (5 classes): table, chair, sofa, bookcase, board.
To facilitate comparison with conventional 2D/RGB-D methods, 300 registered RGB-D images were semi-automatically aligned to the point clouds, with 3D semantic labels projected to 2D image coordinates to ensure consistency across modalities.

Knowl 7 — Evaluation of Disjoint Space Parsing via Adjusted Rand Index

data/table

The performance of point cloud space parsing (floor plan generation) was evaluated across four building areas using the unsupervised clustering metric Adjusted Rand Index (ARI), which measures the agreement between predicted spatial cluster memberships and ground truth room assignments while adjusting for chance.

Building	Ours (final)	Ours (over-segm)	RANSAC based [27]	2D Hough based [38]
(1)	0.94	0.59	0.29	0.27
(2)	0.82	0.76	0.30	0.31
(3)	0.69	0.44	0.14	0.37
(4)	0.66	0.42	0.15	0.30
mean	0.77	0.55	0.20	0.31

The final proposed method achieved a mean ARI of 0.77, outperforming the RANSAC plane-fitting baseline (0.20 ARI) and the 2D Hough transform line-fitting baseline (0.31 ARI). Even before graph-based merging and element refinement, the intermediate over-segmented slices of the void-based approach achieved an ARI of 0.55, demonstrating the superior discriminative capability of the peak-gap-peak void signature over surface-detection heuristics in cluttered indoor scenes.

Knowl 8 — 3D Semantic Element Detection Performance and Feature Ablation Analysis

data/table

Semantic element detection was evaluated using mean Average Precision (mAP) following the Pascal VOC criterion (true positive requires 3D bounding box Intersection over Union $\text{IoU} > 0.5$ against ground truth). Evaluations were conducted via k-fold cross-validation where each building serves as an unseen test fold.

	Structural Elements								Furniture						Overall
Method / Feature Ablation	ceil	floor	wall	beam	col	win	door	mean	table	chair	sofa	book	board	mean	mAP
Ours (full)	71.61	88.70	72.86	66.67	91.77	25.92	54.11	67.38	46.02	16.15	6.78	54.71	3.91	25.51	49.93
Ours (no global geom.)	48.93	83.76	65.25	62.86	83.15	22.55	41.08	57.27	37.57	11.80	4.57	45.49	3.06	20.35	41.87
Ours (no local geom.)	50.74	80.48	65.59	68.53	85.08	21.17	45.39	58.73	39.87	11.43	4.91	57.76	3.73	23.78	44.19
Ours (no color)	48.05	80.95	67.78	68.02	87.41	25.32	44.31	59.73	50.56	11.83	6.32	52.33	4.76	25.30	45.41
mBOW [6]	–	–	–	–	–	–	–	–	–	–	–	–	–	–	36.11
Vote3D [37]	–	–	–	–	–	–	–	–	–	–	–	–	–	–	39.21

The full pipeline achieved 49.93% overall mAP, outperforming mBOW (36.11%) and Vote3D (39.21%). Removing global geometry features caused the largest performance degradation (dropping mAP to 41.87%), confirming that canonical reference coordinates provide the primary predictive signal for structural elements (67.38% mean AP). Performance on furniture (25.51% mean AP) is lower due to higher intra-class variation and weaker spatial positioning regularities.

Knowl 9 — 2D Semantic Segmentation Comparison via 3D Parsing Projection

data/table

To evaluate whether semantic understanding is more effectively achieved in full 3D point cloud space versus 2D single-view images, point-level semantic parsing predictions were projected onto 300 registered RGB-D test images and evaluated using 2D mean Intersection-over-Union (mIoU) across shared semantic classes with the NYU-RGBD benchmark.

Projecting 3D bounding-box element detections into image pixel masks achieved 38.5% mIoU, significantly outperforming native RGB-D semantic segmentation (20.9% mIoU). This indicates that operating directly in large-scale global 3D coordinate frames provides structural context and occlusion robustness that single-view 2.5D representations lack.

Knowl 10 — Planar-Wall and Manhattan-Alignment Assumption for Space Parsing
limitation
The 1D histogram template-matching method for space divider detection assumes that building layouts adhere roughly to Manhattan frame geometries with planar walls aligned to orthogonal axes. Consequently, the filter bank does not natively handle circular, curved, oval, or arbitrarily angled walls that deviate from the building's principal coordinate axes. While extending the filter bank to 2D convolution kernels parameterized over curvature could theoretically handle non-planar walls, such configurations represent a minor fraction of typical building layouts.

Coverage note — None was omitted; all primary contributed methods (void-based filtering, slicing and merging algorithm, canonical space normalization, CRF proposal selection, wall-based update loop), the dataset, experimental benchmarks, and limitations are fully covered.

References

1.Matterport 3d models of interior spaces. http://matterport.com/. Accessed: 2015-06-01.
2.Microsoft kinect. https://www.microsoft.com/en-us/kinectforwindows/. Accessed: 2015-06-01.
3.Accurate Localization of 3D Objects from RGB-D Data using Segmentation Hypotheses, 2013.
4.M. Alzantot and M. Youssef. Crowdinside: automatic construction of indoor floorplans. In Proceedings of the 20th International Conference on Advances in Geographic Information Systems, pages 99–108. ACM, 2012.
5.I. Armeni, S. Ozan, Z. Amir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. Supplementary material for: 3d semantic parsing for large-scale indoor spaces. http://buildingparser.stanford.edu/images/supp_mat.pdf. Accessed: 2016-04-09.
6.J. Behley, V. Steinhage, and A. Cremers. Laser-based segment classification using a mixture of bag-of-words. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, pages 4195–4200. IEEE, 2013.
7.L. Bo, K. Lai, X. Ren, and D. Fox. Object recognition with hierarchical kernel descriptors. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1729–1736. IEEE, 2011.
8.M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010.
9.R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871–1874, 2008.
10.B. J. Frey and D. Dueck. Clustering by passing messages between data points. Science, 315:972–976, 2007.
11.A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, page 0278364913491297, 2013.
12.B. Ghanem, A. Thabet, J. Carlos Niebles, and F. Caba Heilbron. Robust manhattan frame estimation from a single rgb-d image. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.
13.S. Gupta, R. Girshick, P. Arbeláez, and J. Malik. Learning rich features from rgb-d images for object detection and segmentation. In Computer Vision–ECCV 2014, pages 345–360. Springer, 2014.
14.A. Hermans, G. Floros, and B. Leibe. Dense 3d semantic mapping of indoor scenes from rgb-d images. In Robotics and Automation (ICRA), 2014 IEEE International Conference on, pages 2631–2638. IEEE, 2014.
15.D. Koller and N. Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
16.H. Koppula and A. Saxena. Anticipating human activities using object affordances for reactive robotic response. 2013.
17.H. S. Koppula, R. Gupta, and A. Saxena. Learning human activities and object affordances from rgb-d videos. The International Journal of Robotics Research, 32(8):951–970, 2013.
18.C. Liu, A. Schwing, K. Kundu, R. Urtasun, and S. Fidler. Rent3d: Floor-plan priors for monocular layout estimation. In CVPR, 2015.
19.T. Malisiewicz, A. Gupta, A. Efros, et al. Ensemble of exemplar-svms for object detection and beyond. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 89–96. IEEE, 2011.
20.C. Mura, O. Mattausch, A. J. Villanueva, E. Gobbetti, and R. Pajarola. Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts. Computers & Graphics, 44:20–32, 2014.
21.L. Nan, K. Xie, and A. Sharf. A search-classify approach for cluttered indoor scene understanding. ACM Transactions on Graphics (TOG), 31(6):137, 2012.
22.A. Nüchter and J. Hertzberg. Towards semantic maps for mobile robots. Robotics and Autonomous Systems, 56(11):915–926, 2008.
23.S. Ochmann, R. Vock, R. Wessel, M. Tamke, and R. Klein. Automatic generation of structural building descriptions from 3d point cloud scans. In GRAPP 2014 - International Conference on Computer Graphics Theory and Applications. SCITEPRESS, Jan. 2014.
24.J. Papon, A. Abramov, M. Schoeler, and F. Wörgötter. Voxel cloud connectivity segmentation-supervoxels for point clouds. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 2027–2034. IEEE, 2013.
25.W. M. Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, 66(336):846–850, 1971.
26.X. Ren, L. Bo, and D. Fox. Rgb-(d) scene labeling: Features and algorithms. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2759–2766. IEEE, 2012.
27.R. Schnabel, R. Wahl, and R. Klein. Efficient ransac for point-cloud shape detection. In Computer graphics forum, volume 26, pages 214–226. Wiley Online Library, 2007.
28.O. Sener and A. Saxena. rcrf: Recursive belief estimation over crfs in rgb-d activity videos. In Proceedings of Robotics: Science and Systems, Rome, Italy, July 2015.
29.T. Shao, W. Xu, K. Zhou, J. Wang, D. Li, and B. Guo. An interactive approach to semantic modeling of indoor scenes with an rgbd camera. ACM Transactions on Graphics (TOG), 31(6):136, 2012.
30.N. Silberman and R. Fergus. Indoor scene segmentation using a structured light sensor. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pages 601–608. IEEE, 2011.
31.N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012, pages 746–760. Springer, 2012.
32.S. Song and J. Xiao. Sliding shapes for 3d object detection in rgb-d images. In European Conference on Computer Vision, volume 2, page 6, 2014.
33.P. Steadman. Why are most buildings rectangular? Architectural Research Quarterly, 10(02):119–130, 2006.
34.J. Straub, G. Rosman, O. Freifeld, J. J. Leonard, and J. W. Fisher. A mixture of manhattan frames: Beyond the manhattan world. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 3770–3777. IEEE, 2014.
35.I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun. Support vector machine learning for interdependent and structured output spaces. In Proceedings of the twenty-first international conference on Machine learning, page 104. ACM, 2004.
36.E. Turner and A. Zakhor. Floor plan generation and room labeling of indoor environments from laser range data, 2014.
37.D. Z. Wang and I. Posner. Voting for voting in online point cloud object detection. In Proceedings of Robotics: Science and Systems, Rome, Italy, July 2015.
38.J. Xiao and Y. Furukawa. Reconstructing the worlds museums. In Computer Vision–ECCV 2012, pages 668–681. Springer, 2012.
39.J. Xiao, A. Owens, and A. Torralba. Sun3d: A database of big spaces reconstructed using sfm and object labels. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 1625–1632. IEEE, 2013.
40.X. Xiong, A. Adan, B. Akinci, and D. Huber. Automatic creation of semantically rich 3d building models from laser scanner data. Automation in Construction, 31:325–337, 2013.
41.Y. Zhang, S. Song, P. Tan, and J. Xiao. Panocontext: A whole-room 3d context model for panoramic scene understanding. In Computer Vision–ECCV 2014, pages 668–686. Springer, 2014.

Citation

MLA

Armeni, I., et al. “3D Semantic Parsing of Large-Scale Indoor Spaces”. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1534–43, https://doi.org/10.1109/CVPR.2016.170.

APA

Armeni, I., Sener, O., Zamir, A. R., Jiang, H., Brilakis, I., Fischer, M., & Savarese, S. (2016). 3D Semantic Parsing of Large-Scale Indoor Spaces. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1534–1543. https://doi.org/10.1109/CVPR.2016.170

Chicago

Armeni, I., O. Sener, A. R. Zamir, et al. 2016. “3D Semantic Parsing of Large-Scale Indoor Spaces”. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1534–43. https://doi.org/10.1109/CVPR.2016.170.

Harvard

Armeni, I. et al. (2016) “3D Semantic Parsing of Large-Scale Indoor Spaces”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 1534–1543. Available at: https://doi.org/10.1109/CVPR.2016.170.

Vancouver

1. Armeni I, Sener O, Zamir AR, Jiang H, Brilakis I, Fischer M, Savarese S (2016) 3D Semantic Parsing of Large-Scale Indoor Spaces. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1534–1543

BibTeX

@inproceedings{Armeni_2016, title={3D Semantic Parsing of Large-Scale Indoor Spaces}, url={http://dx.doi.org/10.1109/CVPR.2016.170}, DOI={10.1109/cvpr.2016.170}, booktitle={2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, publisher={IEEE}, author={Armeni, Iro and Sener, Ozan and Zamir, Amir R. and Jiang, Helen and Brilakis, Ioannis and Fischer, Martin and Savarese, Silvio}, year={2016}, month=June, pages={1534–1543} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE

3D Semantic Parsing of Large-Scale Indoor Spaces

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Peak-Gap-Peak Filter Bank Formulation for Void Space Detection

Knowl 2 — Graph-Based Slicing and Merging Algorithm for Disjoint Space Parsing

Knowl 3 — Canonical Space Normalization and Voxelized 3D Sliding Window Representation

Knowl 4 — CRF Formulation for Contextual Semantic Element Selection

Knowl 5 — Disjoint Space Parsing Refinement via Detected Wall Semantics

Knowl 6 — The Stanford Large-Scale 3D Indoor Spaces Dataset

Knowl 7 — Evaluation of Disjoint Space Parsing via Adjusted Rand Index

Knowl 8 — 3D Semantic Element Detection Performance and Feature Ablation Analysis

Knowl 9 — 2D Semantic Segmentation Comparison via 3D Parsing Projection

Knowl 10 — Planar-Wall and Manhattan-Alignment Assumption for Space Parsing

References

Citation

BibTeX

Access the Paper

Method	mIoU (%)
Native RGB-D Segmentation (Bo et al. [7])	20.9
Ours (3D Detection Projected to 2D)	38.5