Back

Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

Lihe YangBingyi KangZilong HuangXiao-Gang XuJiashi FengHengshuang Zhao

article2024CVPR2,144 citationsHonorable Mention

Presents Depth Anything, a foundation model for monocular depth estimation trained on 62 million automatically annotated images using semantic priors and optimization perturbations, setting new state-of-the-art zero-shot benchmarks across diverse image domains.

Listen

Monocular depth estimation—the task of predicting three-dimensional depth from a single two-dimensional image—is crucial for emerging applications in autonomous driving, robotics, and augmented reality. Historically, developing reliable AI models for this task has been hindered by the high cost and difficulty of acquiring sensor-labeled depth data, which severely limits how well these models generalize to unseen real-world environments.

The article introduces and evaluates Depth Anything, a robust foundation model for single-image depth estimation designed to process arbitrary images across diverse conditions. The primary objective is to demonstrate that massive scaling using cheap, unlabeled monocular images can overcome previous generalization bottlenecks.

To achieve this, the authors built an automated data engine that combined 1.5 million labeled images from six public datasets with approximately 62 million unlabeled images from eight diverse open-source collections. An initial teacher model trained on labeled data assigned estimated depth labels to the unlabeled images. A student model was then trained on this combined dataset using two core strategies: injecting heavy color and spatial perturbations (such as image region mixing) into the unlabeled images to force robust representation learning, and enforcing an auxiliary feature-alignment constraint with a pre-trained image encoder to preserve rich semantic scene understanding.

The evaluation yielded several key findings. First, in zero-shot relative depth estimation across six benchmark datasets, Depth Anything substantially outperformed previous leading models such as MiDaS v3.1; notably, the smallest version (24.8 million parameters) outperformed MiDaS models that were more than ten times larger on multiple datasets. Second, when fine-tuned on standard benchmarks like KITTI and NYUv2, the model established new state-of-the-art accuracy in metric depth estimation. Third, when repurposed for high-level semantic segmentation on the Cityscapes and ADE20K benchmarks, the pre-trained encoder surpassed existing models trained on ImageNet-21K. Finally, qualitative assessments confirmed that the model significantly improves downstream applications, including depth-conditioned image generation and video editing.

These findings demonstrate that high-cost sensor data collection can be largely bypassed by intelligently leveraging unlabeled visual data. For organizations developing perception systems, this approach lowers data acquisition expenses, decreases deployment risks in unstructured open-world settings, and offers a unified vision encoder capable of handling both mid-level geometry and high-level semantic tasks simultaneously.

Organizations in robotics, autonomous navigation, and generative media should adopt Depth Anything as a strong baseline or pre-trained backbone for perception and image generation pipelines. Engineering teams should select model sizes based on computational constraints, as the small-parameter variants provide high accuracy suitable for resource-limited edge devices.

The reported results carry high confidence across standard public benchmarks and varied photographic environments. However, practitioners should note that the current training resolution is capped at standard dimensions, and the architecture has so far scaled up to large vision transformers. Future improvements will require training on higher-resolution imagery and scaling to giant model sizes to further reduce error in highly detailed scenes.

ByteDanceThe Chinese University of Hong KongUniversity of Hong KongZhejiang Universitydata augmentationdepth estimationencodersDepthModelImages
arXiv: 2401.10891LiheYoung/Depth-Anything
Cover for Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

Abstract

This work presents Depth Anything, a highly practical solution for robust monocular depth estimation. Without pursuing novel technical modules, we aim to build a simple yet powerful foundation model dealing with any images under any circumstances. To this end, we scale up the dataset by designing a data engine to collect and automatically annotate large-scale unlabeled data (~62M), which significantly enlarges the data coverage and thus is able to reduce the generalization error. We investigate two simple yet effective strategies that make data scaling-up promising. First, a more challenging optimization target is created by leveraging data augmentation tools. It compels the model to actively seek extra visual knowledge and acquire robust representations. Second, an auxiliary supervision is developed to enforce the model to inherit rich semantic priors from pre-trained encoders. We evaluate its zero-shot capabilities extensively, including six public datasets and randomly captured photos. It demonstrates impressive generalization ability. Further, through fine-tuning it with metric depth information from NYUv2 and KITTI, new SOTAs are set. Our better depth model also results in a better depth-conditioned ControlNet. Our models are released at this https URL.

Topics

data augmentationdepth estimationencodersengineoptimizationsemanticsoptimization problemsintegrated datamarkov random fields

Keywords

DepthModelImagesUnlabeledDepth EstimationUnlabeled ImagesEncoderDataDepth AnythingSemantic

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Depth Anything
  • 3.1 Learning Labeled Images
  • 3.2 Unleashing the Power of Unlabeled Images
  • 3.3 Semantic-Assisted Perception
  • 4 Experiment
  • 4.1 Implementation Details
  • 4.2 Zero-Shot Relative Depth Estimation
  • 4.3 Fine-tuned to Metric Depth Estimation
  • 4.3.1 In-Domain Metric Depth Estimation
  • 4.3.2 Zero-Shot Metric Depth Estimation
  • 4.4 Fine-tuned to Semantic Segmentation
  • 4.5 Ablation Studies
  • 4.6 Qualitative Results
  • 5 Conclusion
  • 6 More Implementation Details
  • 7 More Ablation Studies
  • 8 Limitations and Future Works
  • 9 More Qualitative Results
  • References

Knowls

  1. Knowl 1 — Semi-Supervised Self-Training Framework for Monocular Depth Estimation

    model/method

    Depth Anything employs a two-stage semi-supervised training pipeline leveraging both labeled and large-scale unlabeled images:

    1. Teacher Training on Labeled Data: An initial depth estimation teacher model TT, initialized with a DINOv2 pre-trained Vision Transformer (ViT) encoder and a Dense Prediction Transformer (DPT) decoder, is trained on a collection of labeled datasets Dl={(xi,di)}i=1M\mathcal{D}^l = \{(x_i, d_i)\}_{i=1}^M totaling approximately 1.5 million images (from BlendedMVS, DIML, HRWSI, IRS, MegaDepth, and TartanAir). A pre-trained semantic segmentation network is used to identify sky regions and set their disparity to 0 (representing infinite distance).

    2. Dense Pseudo-Label Generation: The trained teacher TT produces dense disparity maps for an unlabeled dataset of approximately 62 million images Du={ui}i=1N\mathcal{D}^u = \{u_i\}_{i=1}^N gathered from eight large-scale public datasets (BDD100K, Google Landmarks, ImageNet-21K, LSUN, Objects365, Open Images V7, Places365, and SA-1B), generating the pseudo-labeled dataset D^u={(ui,T(ui))}i=1N\hat{\mathcal{D}}^u = \{(u_i, T(u_i))\}_{i=1}^N.

    3. Student Training: A student model SS is re-initialized (with DINOv2 weights for its encoder rather than initialized from TT) and trained on the combined set Dl∪D^u\mathcal{D}^l \cup \hat{\mathcal{D}}^u. The student is trained with a labeled-to-unlabeled sample ratio of 1:2 per batch, minimizing an objective combining labeled loss Ll\mathcal{L}_l, unlabeled perturbed loss Lu\mathcal{L}_u, and an auxiliary semantic feature alignment loss Lfeat\mathcal{L}_{feat}.

  2. Knowl 2 — Affine-Invariant Disparity Loss for Relative Depth Estimation

    equation

    To jointly train monocular depth estimation models on heterogeneous datasets with unknown and inconsistent metric scales and depth shifts, depth values are first converted to disparity d=1/td = 1/t and normalized to [0,1][0, 1] per image. The prediction d∗d^* and ground truth dd across HWHW pixels are aligned to zero translation and unit scale via an affine-invariant mean absolute error loss:

    Ll=1HW∑i=1HWρ(di∗,di)=1HW∑i=1HW∣d^i∗−d^i∣\mathcal{L}_l = \frac{1}{HW}\sum_{i=1}^{HW} \rho(d^*_i, d_i) = \frac{1}{HW}\sum_{i=1}^{HW} |\hat{d}^*_i - \hat{d}_i|

    where d^i∗\hat{d}^*_i and d^i\hat{d}_i are the scaled and shifted versions of prediction pixel di∗d^*_i and ground truth pixel did_i, defined by:

    d^i=di−t(d)s(d),d^i∗=di∗−t(d∗)s(d∗)\hat{d}_i = \frac{d_i - t(d)}{s(d)}, \quad \hat{d}^*_i = \frac{d^*_i - t(d^*)}{s(d^*)}

    The translation parameter t(d)∈Rt(d) \in \mathbb{R} and scale parameter s(d)∈R+s(d) \in \mathbb{R}^+ for any disparity map d∈RHWd \in \mathbb{R}^{HW} are determined using the median and mean absolute deviation:

    t(d)=median(d),s(d)=1HW∑i=1HW∣di−t(d)∣t(d) = \text{median}(d), \quad s(d) = \frac{1}{HW}\sum_{i=1}^{HW} |d_i - t(d)|

  3. Knowl 3 — Perturbed Unlabeled Learning with Color Distortions and Spatial CutMix

    model/method

    Direct self-training on pseudo-labeled images fails when labeled data and pre-training representations are already strong, because teacher and student share identical architectures and tend to make similar predictions. To compel the student model SS to learn robust representations and extra visual cues from unlabeled data, strong perturbations are applied to the unlabeled images fed into the student, while the teacher TT generates pseudo-labels from unperturbed images.

    Perturbations include strong color distortions (color jittering and Gaussian blurring) and spatial CutMix applied with a 50% probability. For CutMix, a pair of unlabeled images uau_a and ubu_b is blended using a binary rectangular mask M∈{0,1}H×WM \in \{0, 1\}^{H \times W}:

    uab=ua⊙M+ub⊙(1−M)u_{ab} = u_a \odot M + u_b \odot (1 - M)

    The student SS predicts depth on the composite perturbed image uabu_{ab}, and the loss is computed separately against the teacher's unperturbed predictions T(ua)T(u_a) and T(ub)T(u_b) across the valid regions:

    LuM=ρ(S(uab)⊙M, T(ua)⊙M)\mathcal{L}^M_u = \rho\big(S(u_{ab}) \odot M,\, T(u_a) \odot M\big)

    Lu1−M=ρ(S(uab)⊙(1−M), T(ub)⊙(1−M))\mathcal{L}^{1-M}_u = \rho\big(S(u_{ab}) \odot (1 - M),\, T(u_b) \odot (1 - M)\big)

    where ρ\rho denotes the affine-invariant mean absolute error. The total unlabeled loss Lu\mathcal{L}_u is aggregated via spatial area weighting:

    Lu=∑MHWLuM+∑(1−M)HWLu1−M\mathcal{L}_u = \frac{\sum M}{HW} \mathcal{L}^M_u + \frac{\sum (1 - M)}{HW} \mathcal{L}^{1-M}_u

  4. Knowl 4 — Semantic Prior Preservation via Cosine Feature Alignment with Tolerance Margin

    model/method

    To equip the depth encoder with high-level scene semantics without the information loss caused by decoding into a discrete class space, Depth Anything transfers continuous semantic features from a frozen pre-trained DINOv2 encoder to the online student depth model SS.

    Semantic encoders map different parts of an object (e.g., the front and rear of a car) to similar feature representations, whereas depth estimation requires distinguishing different depths within the same object part. To prevent enforcing excessive semantic invariance where depth varies, a cosine similarity tolerance margin α\alpha is enforced: if the cosine similarity between the student feature fif_i and the frozen DINOv2 feature fi′f'_i at pixel ii exceeds α\alpha, that pixel is omitted from the feature alignment loss.

    The feature alignment loss is formulated over the HWHW spatial positions as:

    Lfeat=1−1HW∑i=1HWcos⁡(fi,fi′)⋅I(cos⁡(fi,fi′)≤α)\mathcal{L}_{feat} = 1 - \frac{1}{HW} \sum_{i=1}^{HW} \cos(f_i, f'_i) \cdot \mathbb{I}\big(\cos(f_i, f'_i) \le \alpha\big)

    where cos⁡(u,v)=u⊤v∥u∥2∥v∥2\cos(u, v) = \frac{u^\top v}{\|u\|_2 \|v\|_2}, fif_i is the feature from the student encoder without projection layers, fi′f'_i is the corresponding feature from the frozen DINOv2 encoder, and the margin is set to α=0.85\alpha = 0.85. This auxiliary loss is applied exclusively to unlabeled training samples.

  5. Knowl 5 — Depth Anything Implementation Details and Training Protocol

    experimental setup

    Depth Anything uses DINOv2 Vision Transformer encoders (ViT-S with 24.8M parameters, ViT-B with 97.5M parameters, and ViT-L with 335.3M parameters) paired with the Dense Prediction Transformer (DPT) decoder.

    • Input Preprocessing: During training, images are resized so the shorter side is 518 pixels while maintaining aspect ratio, followed by random 518×518 cropping. During inference, inputs are kept at original aspect ratios with dimensions adjusted to multiples of the patch size 14, and predictions are interpolated to the original resolution.
    • Data Augmentation: Labeled images receive only horizontal flipping. Unlabeled images receive color jittering, Gaussian blur, and CutMix (50% probability).
    • Optimizer & Schedules: Training utilizes the AdamW optimizer with a linear learning rate decay. The base learning rate for the pre-trained encoder is 5×10−65 \times 10^{-6}, while the randomly initialized decoder uses a 10×10\times larger learning rate (5×10−55 \times 10^{-5}).
    • Training Regimes: Stage 1 trains teacher TT on 1.5M labeled images for 20 epochs. Stage 2 trains student SS jointly on labeled and unlabeled data for one full pass over the 62M unlabeled images, using a batch sampling ratio of 1 labeled to 2 unlabeled samples.
    • Total Loss: The joint training loss is the uniform combination L=Ll+Lu+Lfeat\mathcal{L} = \mathcal{L}_l + \mathcal{L}_u + \mathcal{L}_{feat} with feature alignment tolerance margin α=0.85\alpha = 0.85.

  6. Knowl 6 — Zero-Shot Relative Depth Estimation Benchmarks

    data/table

    Zero-shot depth generalization of Depth Anything models (ViT-S, ViT-B, ViT-L) evaluated across six unseen public datasets: KITTI, NYUv2, Sintel, DDAD, ETH3D, and DIODE. Metrics are absolute relative error (AbsRel=1N∑∣d∗−d∣/d\text{AbsRel} = \frac{1}{N}\sum |d^* - d|/d, lower is better) and threshold accuracy (δ1=percentage of max⁡(d∗/d,d/d∗)<1.25\delta_1 = \text{percentage of } \max(d^*/d, d/d^*) < 1.25, higher is better). Predictions are affine-aligned to ground truth scale and shift prior to evaluation.

    Method Encoder KITTI NYUv2 Sintel DDAD ETH3D DIODE
    AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1
    MiDaS v3.1 ViT-L 0.127 0.850 0.048 0.980 0.587 0.699 0.251 0.766 0.139 0.867 0.075 0.942
    Depth Anything ViT-S 0.080 0.936 0.053 0.972 0.464 0.739 0.247 0.768 0.127 0.885 0.076 0.939
    Depth Anything ViT-B 0.080 0.939 0.046 0.979 0.432 0.756 0.232 0.786 0.126 0.884 0.069 0.946
    Depth Anything ViT-L 0.076 0.947 0.043 0.981 0.458 0.760 0.230 0.789 0.127 0.882 0.066 0.952

    Depth Anything ViT-L outperforms the strongest MiDaS v3.1 model (DPT-BEiT_L-512) across all six datasets. Depth Anything ViT-S (24.8M parameters, under 1/10 the size of MiDaS ViT-L) outperforms MiDaS v3.1 on Sintel, DDAD, and ETH3D. On KITTI and NYUv2, Depth Anything achieves superior zero-shot performance without having seen their training sets, whereas MiDaS v3.1 was trained directly on them.

  7. Knowl 7 — Downstream Metric Depth Estimation Fine-Tuning Performance

    data/table

    When fine-tuned for metric depth estimation using the ZoeDepth framework, Depth Anything's pre-trained ViT-L encoder establishes new state-of-the-art results for both in-domain benchmarks and zero-shot cross-dataset evaluation.

    In-domain fine-tuning comparisons:

    • NYUv2 (indoor): Depth Anything achieves δ1=0.984\delta_1 = 0.984, δ2=0.998\delta_2 = 0.998, δ3=1.000\delta_3 = 1.000, AbsRel=0.056\text{AbsRel} = 0.056, RMSE=0.206\text{RMSE} = 0.206, and log⁡10=0.024\log_{10} = 0.024, outperforming previous state-of-the-art VPD (δ1=0.964\delta_1 = 0.964, AbsRel=0.069\text{AbsRel} = 0.069, RMSE=0.254\text{RMSE} = 0.254) and ZoeDepth (δ1=0.951\delta_1 = 0.951, AbsRel=0.077\text{AbsRel} = 0.077).
    • KITTI (outdoor): Depth Anything achieves δ1=0.982\delta_1 = 0.982, δ2=0.998\delta_2 = 0.998, δ3=1.000\delta_3 = 1.000, AbsRel=0.046\text{AbsRel} = 0.046, RMSE=1.896\text{RMSE} = 1.896, and RMSE log=0.069\text{RMSE log} = 0.069, outperforming NDDepth (δ1=0.978\delta_1 = 0.978, AbsRel=0.050\text{AbsRel} = 0.050) and ZoeDepth (δ1=0.971\delta_1 = 0.971, AbsRel=0.054\text{AbsRel} = 0.054).

    Zero-shot metric depth estimation (models fine-tuned on NYUv2 evaluated on unseen indoor datasets; models fine-tuned on KITTI evaluated on unseen outdoor datasets):

    Method SUN RGB-D iBims-1 HyperSim Virtual KITTI 2 DIODE Outdoor
    AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1 AbsRel δ1\delta_1
    ZoeDepth 0.520 0.545 0.169 0.656 0.407 0.302 0.106 0.844 0.814 0.237
    Depth Anything 0.500 0.660 0.150 0.714 0.363 0.361 0.085 0.913 0.794 0.288

    Replacing the MiDaS encoder in ZoeDepth with the Depth Anything ViT-L encoder consistently improves zero-shot metric depth accuracy across all indoor and outdoor target domains.

  8. Knowl 8 — Transfer of Depth Pre-Trained Encoder to Downstream Semantic Segmentation

    data/table

    To evaluate semantic feature quality, the Depth Anything ViT-L encoder trained via large-scale MDE is fine-tuned for semantic segmentation using Mask2Former without COCO or Mapillary pre-training.

    Dataset Method / Backbone mIoU (Single-Scale) mIoU (Multi-Scale)
    Cityscapes SegFormer (MiT-B5) 82.4 84.0
    Mask2Former (Swin-L) 83.3 84.3
    OneFormer (ConvNeXt-XL) 83.6 84.6
    DDP (ConvNeXt-L) 83.2 83.9
    Depth Anything (ViT-L) 84.8 86.2
    ADE20K Segmenter (ViT-L) 51.8 -
    Mask2Former (Swin-L) 56.4 -
    ViT-Adapter (BEiT-L) 58.3 -
    Original DINOv2 (ViT-L) 58.8 -
    Depth Anything (ViT-L) 59.4 -

    The Depth Anything ViT-L encoder outperforms models pre-trained on ImageNet-21K (e.g., Swin-L at 84.3 mIoU on Cityscapes, BEiT-L at 58.3 mIoU on ADE20K) and exceeds the original DINOv2 ViT-L baseline (58.8 vs 59.4 mIoU on ADE20K, and 82.1 vs 84.8 mIoU on Cityscapes compared to MiDaS ViT-L), demonstrating that large-scale depth pre-training with feature alignment yields a versatile encoder for high-level semantic tasks.

  9. Knowl 9 — Ablation Study on Perturbations, Semantic Alignment, Margin Value, and Data Target

    empirical result

    Ablation experiments conducted on the ViT-S architecture isolate the impact of the core design components:

    1. Unlabeled Supervision and Perturbations:

      • Training with labeled loss Ll\mathcal{L}_l alone gives a mean zero-shot AbsRel of 0.180 across the six unseen test datasets (KITTI 0.085, NYUv2 0.053, Sintel 0.492, DDAD 0.245, ETH3D 0.134, DIODE 0.070).
      • Naively adding unperturbed unlabeled data with pseudo-labels (Ll+Lu\mathcal{L}_l + \mathcal{L}_u) yields a mean AbsRel of 0.180 (no gain over labeled baseline).
      • Introducing strong perturbations (color distortions and CutMix) to unlabeled images (Ll+Lu+S\mathcal{L}_l + \mathcal{L}_u + S) reduces mean AbsRel to 0.175 (KITTI 0.081, NYUv2 0.048, Sintel 0.469, DDAD 0.235, ETH3D 0.134, DIODE 0.068).
      • Adding the semantic feature alignment constraint (Ll+Lu+S+Lfeat\mathcal{L}_l + \mathcal{L}_u + S + \mathcal{L}_{feat}) further lowers mean AbsRel to 0.170 (KITTI 0.076, NYUv2 0.043, Sintel 0.458, DDAD 0.230, ETH3D 0.127, DIODE 0.066).

    2. Tolerance Margin α\alpha for Lfeat\mathcal{L}_{feat}:

      • α=1.00\alpha = 1.00 (enforcing exact cosine alignment across all tokens) degrades mean AbsRel to 0.188.
      • α=0.85\alpha = 0.85 achieves optimal mean AbsRel of 0.175.
      • α=0.70\alpha = 0.70 achieves mean AbsRel of 0.178.

    3. Application Target for Feature Alignment:

      • Applying Lfeat\mathcal{L}_{feat} to unlabeled data yields mean AbsRel 0.175.
      • Applying Lfeat\mathcal{L}_{feat} to labeled data yields mean AbsRel 0.179 (negligible change vs. 0.180 baseline without alignment), because manual depth labels are already precise and semantic alignment can interfere with high-quality ground-truth depth regression.

    4. Generalization Transfer of Individual Training Sets:

      • Single-dataset relative MDE training shows that HRWSI (containing only 20K images) achieves the strongest cross-dataset transfer (mean AbsRel 0.200, δ1=0.846\delta_1 = 0.846), outperforming datasets with over 900K images (DIML: mean AbsRel 0.226).

  10. Knowl 10 — Limitations of Depth Anything

    limitation

    The Depth Anything framework has two main identified limitations:

    1. Model Capacity Constraint: The maximum model scale evaluated is Vision Transformer Large (ViT-L, 335.3M parameters). Scaling up to ViT-Giant could enable a more capable teacher model to generate higher-accuracy pseudo-labels for distilling into smaller models.
    2. Training Resolution: The current models are trained at a fixed resolution of 518×518 pixels, which limits spatial detail and geometric precision when deployed in ultra-high-resolution real-world settings compared to training at resolutions exceeding 700×700 or 1000×1000 pixels.

Coverage note — None was omitted; qualitative visualizations of predicted depth maps and ControlNet conditional generation examples were excluded in accordance with the factual/reproducible knowl extraction requirements.

References

  1. 1.Manuel Lopez Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bulo`, Yubin Kuang, and Peter Kontschieder. Mapillary planet-scale depth dataset. In ECCV, 2020. 7, 9
  2. 2.Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In ICLR, 2022. 7
  3. 3.Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In CVPR, 2021. 2, 6
  4. 4.Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M¨uller. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv:2302.12288, 2023. 2, 6, 7, 9
  5. 5.Reiner Birkl, Diana Wofk, and Matthias M¨uller. Midas v3. 1–a model zoo for robust monocular relative depth estimation. arXiv:2307.14460, 2023. 2, 3, 5, 7, 8, 9, 10, 11
  6. 6.Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv:2108.07258, 2021. 1
  7. 7.Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012. 5, 7, 9
  8. 8.Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2. arXiv:2001.10773, 2020. 7
  9. 9.Po-Yi Chen, Alexander H Liu, Yen-Cheng Liu, and Yu-Chiang Frank Wang. Towards scene understanding: Unsupervised monocular depth estimation with semantic-aware representation. In CVPR, 2019. 2, 4
  10. 10.Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng. Single-image depth perception in the wild. In NeurIPS, 2016. 2
  11. 11.Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In ICLR, 2023. 7
  12. 12.Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR, 2022. 7, 9
  13. 13.Jaehoon Cho, Dongbo Min, Youngjung Kim, and Kwanghoon Sohn. Diml/cvl rgb-d dataset: 2m rgb-d images of natural indoor and outdoor scenes. arXiv:2110.11590, 2021. 3, 7
  14. 14.MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark. https : / / github . com / open -mmlab/mmsegmentation, 2020. 9
  15. 15.Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016. 1, 6, 9
  16. 16.Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. 7, 9
  17. 17.David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. In NeurIPS, 2014. 2
  18. 18.Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. IJRR, 2013. 1, 2, 3, 5, 6, 7, 9
  19. 19.Jean-Bastien Grill, Florian Strub, Florent Altch´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. In NeurIPS, 2020. 5
  20. 20.Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos, and Adrien Gaidon. 3d packing for self-supervised monocular depth estimation. In CVPR, 2020. 5, 7, 9
  21. 21.Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus, and Adrien Gaidon. Semantically-guided representation learning for self-supervised monocular depth. In ICLR, 2020. 2, 4
  22. 22.Vitor Guizilini, Igor Vasiljevic, Dian Chen, Rares, Ambrus, , and Adrien Gaidon. Towards zero-shot scale-aware monocular depth estimation. In ICCV, 2023. 2
  23. 23.Derek Hoiem, Alexei A Efros, and Martial Hebert. Recovering surface layout from an image. IJCV, 2007. 2
  24. 24.Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. In CVPR, 2023. 7
  25. 25.Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. Ddp: Diffusion model for dense visual prediction. In ICCV, 2023. 7
  26. 26.Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NeurIPS, 2023. 4
  27. 27.Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV, 2023. 1, 2, 3
  28. 28.Marvin Klingner, Jan-Aike Term¨ohlen, Jonas Mikolajczyk, and Tim Fingscheidt. Self-supervised monocular depth estimation: Solving the dynamic object problem by semantic guidance. In ECCV, 2020. 4
  29. 29.Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco Korner. Evaluation of cnn-based single-image depth estimation methods. In ECCVW, 2018. 7
  30. 30.Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. IJCV, 2020. 2, 3
  31. 31.Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICMLW, 2013. 2
  32. 32.Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He. Depth and surface normal estimation from monocular images using regression on deep features and hierarchical crfs. In CVPR, 2015. 2
  33. 33.Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 1, 3, 7
  34. 34.Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. arXiv:2204.00987, 2022. 2
  35. 35.Jun Hao Liew, Hanshu Yan, Jianfeng Zhang, Zhongcong Xu, and Jiashi Feng. Magicedit: High-fidelity and temporally coherent video editing. arXiv:2308.14749, 2023. 8
  36. 36.Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 1, 9
  37. 37.Ce Liu, Jenny Yuen, Antonio Torralba, Josef Sivic, and William T Freeman. Sift flow: Dense correspondence across different scenes. In ECCV, 2008. 2
  38. 38.Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv:2303.05499, 2023. 4
  39. 39.Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021. 6, 7
  40. 40.Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In CVPR, 2022. 6
  41. 41.Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022. 6, 7
  42. 42.Jia Ning, Chen Li, Zheng Zhang, Chunyu Wang, Zigang Geng, Qi Dai, Kun He, and Han Hu. All in tokens: Unifying output space of visual tasks via soft token. In ICCV, 2023. 6
  43. 43.Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. TMLR, 2023. 3, 4, 5, 8, 9
  44. 44.Vaishakh Patil, Christos Sakaridis, Alexander Liniger, and Luc Van Gool. P3depth: Monocular depth estimation with a piecewise planarity prior. In CVPR, 2022. 6
  45. 45.Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021. 1
  46. 46.Ren´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. TPAMI, 2020. 1, 2, 3, 5, 7, 9
  47. 47.Ren´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In ICCV, 2021. 5, 6
  48. 48.Alex Rasla and Michael Beyeler. The relative importance of depth cues and semantic edges for indoor mobility using simulated prosthetic vision in immersive virtual reality. In VRST, 2022. 1
  49. 49.Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In ICCV, 2021. 7
  50. 50.Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 2015. 3
  51. 51.Ashutosh Saxena, Min Sun, and Andrew Y Ng. Make3d: Learning 3d scene structure from a single still image. TPAMI, 2008. 2
  52. 52.Thomas Sch¨ops, Johannes L Sch¨onberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, 2017. 5, 7, 9
  53. 53.Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, 2019. 3
  54. 54.Shuwei Shao, Zhongcai Pei, Weihai Chen, Xingming Wu, and Zhengguo Li. Nddepth: Normal-distance assisted monocular depth estimation. In ICCV, 2023. 2, 6
  55. 55.Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012. 1, 2, 3, 5, 6, 7, 9
  56. 56.Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020. 2, 4
  57. 57.Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, 2015. 6, 7
  58. 58.Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. In ICCV, 2021. 7
  59. 59.Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv:2302.13971, 2023. 1
  60. 60.Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea F Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode: A dense indoor and outdoor depth dataset. arXiv:1908.00463, 2019. 5, 7, 9
  61. 61.Chaoyang Wang, Simon Lucey, Federico Perazzi, and Oliver Wang. Web stereo video supervision for depth prediction from dynamic scenes. In 3DV, 2019. 3
  62. 62.Qiang Wang, Shizhen Zheng, Qingsong Yan, Fei Deng, Kaiyong Zhao, and Xiaowen Chu. Irs: A large naturalistic indoor robotics stereo dataset to train deep models for disparity and surface normal estimation. In ICME, 2021. 3, 7
  63. 63.Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. In IROS, 2020. 3, 7
  64. 64.Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. In CVPR, 2019. 1
  65. 65.Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval. In CVPR, 2020. 3
  66. 66.Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, and Vivienne Sze. Fastdepth: Fast monocular depth estimation on embedded systems. In ICRA, 2019. 1
  67. 67.Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. Monocular relative depth perception with web stereo data supervision. In CVPR, 2018. 2, 3
  68. 68.Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao. Structure-guided ranking loss for single image depth prediction. In CVPR, 2020. 2, 3, 7
  69. 69.Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understanding. In ECCV, 2018. 7
  70. 70.Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. In NeurIPS, 2021. 3, 7
  71. 71.Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In ICCV, 2021. 2
  72. 72.Xiaogang Xu, Hengshuang Zhao, Vibhav Vineet, Ser-Nam Lim, and Antonio Torralba. Mtformer: Multi-task learning via transformer and cross-task reasoning. In ECCV, 2022. 4
  73. 73.I Zeki Yalniz, Herv´e J´egou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi-supervised learning for image classification. arXiv:1905.00546, 2019. 2
  74. 74.Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, and Yang Gao. St++: Make self-training work better for semi-supervised semantic segmentation. In CVPR, 2022. 4
  75. 75.Lihe Yang, Lei Qi, Litong Feng, Wayne Zhang, and Yinghuan Shi. Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. In CVPR, 2023. 2
  76. 76.Xiaodong Yang, Zhuang Ma, Zhiyu Ji, and Zhe Ren. Gedepth: Ground embedding for monocular depth estimation. In ICCV, 2023. 2, 6
  77. 77.Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020. 3, 7
  78. 78.Wei Yin, Yifan Liu, Chunhua Shen, and Youliang Yan. Enforcing geometric constraints of virtual normal for depth prediction. In ICCV, 2019. 2
  79. 79.Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In ICCV, 2023. 2
  80. 80.Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving. In ICLR, 2020. 1
  81. 81.Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365, 2015. 3
  82. 82.Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In CVPR, 2020. 2, 3
  83. 83.Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. New crfs: Neural window fully-connected crfs for monocular depth estimation. arXiv:2203.01502, 2022. 2, 6
  84. 84.Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, 2019. 4
  85. 85.Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 8
  86. 86.Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, et al. Recognize anything: A strong image tagging model. arXiv:2306.03514, 2023. 4
  87. 87.Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffusion models for visual perception. In ICCV, 2023. 6
  88. 88.Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. TPAMI, 2017. 3
  89. 89.Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017. 6, 9
  90. 90.Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoc Le. Rethinking pre-training and self-training. In NeurIPS, 2020. 2

Citation

MLA
Yang, L., et al. “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data”. arXiv, 2024, http://arxiv.org/abs/2401.10891v2.
APA
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024). Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. arXiv. http://arxiv.org/abs/2401.10891v2
Chicago
Yang, L., B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao. 2024. “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data”. arXiv. http://arxiv.org/abs/2401.10891v2.
Harvard
Yang, L. et al. (2024) “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2401.10891v2.
Vancouver
1. Yang L, Kang B, Huang Z, Xu X, Feng J, Zhao H (2024) Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. arXiv

BibTeX

@article{yang2024depth,
  title = {Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data},
  author = {Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  year = {2024},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2401.10891v2},
  eprint = {2401.10891}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE