Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation

Boah KimYujin OhJong Chul Ye

article2023ICLR95 citations

Proposes a self-supervised diffusion adversarial learning framework that isolates complex background signals to achieve accurate, single-step vessel segmentation across retinal and angiography images without manual annotations.

Listen

Accurately identifying blood vessels in medical imaging is critical for diagnosing vascular diseases and planning interventions. While machine learning tools have proven capable of automating this task, conventional approaches require massive amounts of manual, expert-annotated labels. In addition, existing unsupervised methods struggle with real-world clinical imagery, where noisy backgrounds, motion artifacts, and low contrast frequently obscure delicate vascular branches. Diffusion models have emerged as powerful generative tools, but they typically require lengthy multi-step sampling and have not been successfully deployed for label-free semantic segmentation.

The article demonstrates a novel framework called Diffusion Adversarial Representation Learning to achieve high-quality vessel segmentation without relying on manual ground-truth labels. The goal is to provide an efficient, single-step automated method that learns rich vascular representations from unlabeled medical images.

To accomplish this, the authors combined a denoising diffusion module with an adversarial generation module using switchable normalization layers. The diffusion component is trained intensively to capture non-contrast background patterns, forcing the network to treat vessel structures as outliers and highlight them within the learned latent features. Unlabeled coronary angiography images and synthetic vessel masks are processed through a cyclic adversarial framework to synthesize realistic angiography scans while simultaneously extracting segmentation masks. The system was evaluated across coronary X-ray angiograms, noisy low-dose scans, and entirely different imaging modalities such as retinal photographs.

The findings show that the proposed framework significantly outperforms existing unsupervised and self-supervised segmentation baselines. On standard coronary benchmark data, the model achieved an Intersection over Union score of 0.471 and a Dice similarity score of 0.636, surpassing competing self-supervised approaches by roughly 10% to 15%. When tested on external datasets from different clinical machines, the model maintained superior accuracy and demonstrated cross-organ generalization by leading performance on retinal vessel segmentation. Furthermore, the model exhibited exceptional resilience to image noise, maintaining practical segmentation quality under simulated low-dose imaging conditions where baseline models completely failed. The unified generator architecture also reduced overall computational complexity by roughly 26% compared to dual-network frameworks.

These results demonstrate that clinical workflows can achieve reliable, real-time vessel segmentation without costly and labor-intensive manual labeling. The method provides robust performance on low-radiation scans, potentially lowering patient radiation exposure risks while improving diagnostic speed and consistency across various anatomical imaging types.

Healthcare technology leaders and clinical deployment teams should evaluate this framework as a foundation for label-free automated vessel analysis. Recommended next steps include establishing multi-center clinical pilot studies to validate segmentation performance across diverse patient demographics and equipment vendors. Future research should focus on extending the framework to full three-dimensional vascular scans and integrating it into real-time surgical guidance systems to assess its direct impact on clinical outcomes.

Korea Advanced Institute of Science and Technology adversarial machine learning medical images neural networks Diffusion Adversarial Representation Learning self-supervised vessel segmentation medical image segmentation

arXiv: 2209.14566 boahK/DARL

Paper: Diffusion Models in Vision: A Survey, Florinel-Alin Croitoru et al. (2022). This survey provides essential background on denoising diffusion probabilistic models and their formulation in computer vision tasks, which DARL builds upon for background representation learning.
Paper: Diffusion Models: A Comprehensive Survey of Methods and Applications, Ling Yang et al. (2022). It offers foundational mathematical theory and operational mechanics of diffusion models and their integration with generative adversarial concepts.
Paper: Generative Adversarial Networks: An Overview, Antonia Creswell et al. (2017). It reviews fundamental principles of generative adversarial networks and adversarial representation learning that DARL adapts for synthesizing fake vessel images and masks.
Paper: Self-Supervised Learning: Generative or Contrastive, Xiao Liu et al. (2020). This paper establishes the taxonomy and theoretical trade-offs between generative and contrastive self-supervised learning frameworks underlying label-free representation learning.
Paper: Denoising Diffusion Restoration Models, Bahjat Kawar et al. (2022). It demonstrates how generative diffusion models can be formulated for unsupervised inverse restoration problems, informing DARL's background noise modeling approach.
Paper: Adversarially Learned Inference, Vincent Dumoulin et al. (2017). It details how to jointly pair generative modeling with inference and adversarial games to learn robust unlabelled representations.
Paper: Generative Adversarial Network in Medical Imaging: A Review, Xin Yi et al. (2018). This review surveys the application of generative adversarial architectures to medical image synthesis and segmentation, illustrating the domain-specific challenges DARL addresses.
Paper: A survey on deep learning in medical image analysis, Geert Litjens et al. (2017). It provides a broad foundation for deep learning architectures applied to clinical medical image analysis and vascular segmentation.

Paper: Segment anything in medical images, Jun Ma et al. (2023). Reading DARL prepares the reader to evaluate MedSAM, which extends promptable foundation models across multi-modal medical segmentation benchmarks including challenging vascular structures.
Paper: Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model, Yinhuai Wang et al. (2023). This work extends the paradigm of using pre-trained diffusion models for label-free, zero-shot image restoration and decomposition via null-space projections.

Cover for Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation

Abstract

Vessel segmentation in medical images is one of the important tasks in the diagnosis of vascular diseases and therapy planning. Although learning-based segmentation approaches have been extensively studied, a large amount of ground-truth labels are required in supervised methods and confusing background structures make neural networks hard to segment vessels in an unsupervised manner. To address this, here we introduce a novel diffusion adversarial representation learning (DARL) model that leverages a denoising diffusion probabilistic model with adversarial learning, and apply it to vessel segmentation. In particular, for self-supervised vessel segmentation, DARL learns the background signal using a diffusion module, which lets a generation module effectively provide vessel representations. Also, by adversarial learning based on the proposed switchable spatially-adaptive denormalization, our model estimates synthetic fake vessel images as well as vessel segmentation masks, which further makes the model capture vessel-relevant semantic information. Once the proposed model is trained, the model generates segmentation masks in a single step and can be applied to general vascular structure segmentation of coronary angiography and retinal images. Experimental results on various datasets show that our method significantly outperforms existing unsupervised and self-supervised vessel segmentation methods.

Topics

adversarial machine learning medical images neural networks probabilistic models representation learning segmentation methods semantic information diffusion probabilistic models high resolution image

Keywords

Diffusion Adversarial Representation Learning self-supervised vessel segmentation medical image segmentation adversarial representation learning angiogram generation forward diffusion retinal imaging datasets

1 Introduction
2 Backgrounds and related works
3 Diffusion adversarial representation learning
3.1 Network Training
3.1.1 Loss function
3.1.2 Image perturbation for the model input
3.2 Inference of vessel segmentation
4 Experiments
4.1 Experimental results
5 Conclusion
References
A Details of network architecture
B Details of dataset
C Additional experimental results
C.1 Study on synthetic angiogram generation
C.2 Study on hyperparameter setting
C.3 Study on angiogram perturbation in model training
C.4 Study on adversarial learning with two discriminators
C.5 Study on diffusion model in latent feature estimation
C.6 Study on model contribution
C.7 Study on image processing for the unsupervised methods
C.8 Study on training complexity
D Additional visual results of vessel segmentation
D.1 Results of ablation study
D.2 Results of our DARL model

Knowls

Knowl 1 — Diffusion Adversarial Representation Learning (DARL) Framework for Self-Supervised Vessel Segmentation
model/method
Diffusion Adversarial Representation Learning (DARL) is an end-to-end self-supervised framework designed to segment blood vessels from medical imagery (such as X-ray coronary angiography) without requiring manual pixel-level annotations. The architecture comprises a diffusion module $\epsilon_\theta$ , a unified generation module $G$ , and two discriminators $D_s$ and $D_a$ . Training is conducted on unpaired real background images $x_0^b$ (acquired prior to contrast agent injection) and real angiography images $x_0^a$ (acquired following contrast injection), together with procedurally generated vessel-like fractal masks $s^f$ .

The framework operates along two interdependent data paths:
1. Path (A) — Vessel Segmentation: Real angiography images perturbed with Gaussian noise $x_{t_a}^a$ at schedule step $t_a$ are provided to the diffusion module $\epsilon_\theta$ to produce spatial latent score features $\epsilon_\theta(x_{t_a}^a, t_a)$ . The generator $G$ receives these latent features (without semantic mask conditioning) and estimates a vessel segmentation mask $\hat{s}^v = G(\epsilon_\theta(x_{t_a}^a, t_a); 0)$ . The discriminator $D_s$ learns to differentiate between estimated segmentation masks $\hat{s}^v$ and synthetic fractal masks $s^f$ .
2. Path (B) — Angiogram Synthesis and Cycle Consistency: Real background images perturbed with Gaussian noise $x_{t_b}^b$ at schedule step $t_b$ and synthetic fractal masks $s^f$ are supplied to the system. The diffusion module estimates background latent features $\epsilon_\theta(x_{t_b}^b, t_b)$ , and $G$ uses switchable spatially-adaptive denormalization conditioned on $s^f$ to synthesize a realistic angiography image $\hat{x}^a = G(\epsilon_\theta(x_{t_b}^b, t_b); s^f)$ . The discriminator $D_a$ learns to distinguish synthetic angiograms $\hat{x}^a$ from real angiography images $x_0^a$ .
To enforce semantic consistency, the generated synthetic angiogram $\hat{x}^a$ is subsequently forwarded through Path (A) to reconstruct the original fractal layout mask via $\hat{s}^f = G(\epsilon_\theta(\hat{x}^a, 0); 0)$ , establishing a cycle-consistency reconstruction constraint.
Knowl 2 — Switchable Spatially-Adaptive Denormalization (S-SPADE)
model/method
The generation module $G$ employs Switchable Spatially-Adaptive Denormalization (S-SPADE) in its residual blocks to dynamically switch between conditional image synthesis (using semantic vessel layouts) and unconditional segmentation prediction (extracting vessel masks) within a single set of shared network parameters.

Let $v \in \mathbb{R}^{M \times C \times H \times W}$ denote the feature map tensor within a residual block, where $M$ , $C$ , $H$ , and $W$ represent batch size, channel dimension, height, and width, respectively. Given an optional semantic layout mask input $s$ , the S-SPADE operation is defined as:

$v = \begin{cases} \text{SPADE}(v, s), & \text{if mask } s \text{ is provided} \\ \text{IN}(v), & \text{otherwise} \end{cases}$

where $\text{IN}$ denotes Instance Normalization. When a synthetic fractal mask $s^f$ is supplied (Path B for synthetic angiogram generation), SPADE normalizes and modulates each activation $x_{m,c,h,w}$ at batch index $m$ , channel $c$ , and spatial location $(h, w)$ according to:

$x_{m,c,h,w} = \gamma_{c,h,w}(s^f) \left( \frac{x_{m,c,h,w} - \mu_c}{\sigma_c} \right) + \beta_{c,h,w}(s^f)$

where $\mu_c$ and $\sigma_c$ are the spatial mean and standard deviation of feature map channel $c$ , and $\gamma_{c,h,w}(s^f)$ and $\beta_{c,h,w}(s^f)$ are learned modulation parameters generated by convolutions applied to $s^f$ . When no mask is provided (Path A for vessel segmentation), SPADE is bypassed, and the layer executes standard instance normalization.
Knowl 3 — Multi-Objective Training Loss for DARL
equation
The training of the diffusion module $\epsilon_\theta$ , generator $G$ , segmentation discriminator $D_s$ , and angiogram discriminator $D_a$ is structured as a simultaneous minimax optimization problem under the Least Squares Generative Adversarial Networks (LSGAN) framework:

$\min_{\theta, G} \mathcal{L}^G(\epsilon_\theta, G, D_s, D_a), \quad \min_{D_s, D_a} \mathcal{L}^D(\epsilon_\theta, G, D_s, D_a)$

The generator loss $\mathcal{L}^G$ combines a diffusion score-matching loss $\mathcal{L}_{diff}$ , an adversarial generator loss $\mathcal{L}_{adv}^G$ , and an $L_1$ cycle-consistency loss $\mathcal{L}_{cyc}$ :

$\mathcal{L}^G(\epsilon_\theta, G, D_s, D_a) = \mathcal{L}_{diff}(\epsilon_\theta) + \alpha \mathcal{L}_{adv}^G(\epsilon_\theta, G, D_s, D_a) + \beta \mathcal{L}_{cyc}(\epsilon_\theta, G)$

with hyperparameters $\alpha = 0.2$ and $\beta = 5.0$ .

The diffusion loss is calculated exclusively on background images $x_0 = x_0^b$ :

$\mathcal{L}_{diff}(\epsilon_\theta) = \mathbb{E}_{t, x_0^b, \epsilon} \left[ \left\| \epsilon - \epsilon_\theta\left(\sqrt{\alpha_t} x_0^b + \sqrt{1 - \alpha_t} \epsilon, t\right) \right\|^2 \right]$

where $\epsilon \sim \mathcal{N}(0, I)$ , $t \sim \mathcal{U}[0, T]$ , $\alpha_t = \prod_{s=1}^t (1 - \beta_s)$ with variance schedule $\beta_s \in [0, 1]$ .

The adversarial loss for the generator is:

$\mathcal{L}_{adv}^G(\epsilon_\theta, G, D_s, D_a) = \mathbb{E}_{x^a} \left[ \left( D_s(G(\epsilon_\theta(x^a); 0)) - 1 \right)^2 \right] + \mathbb{E}_{x^b, s^f} \left[ \left( D_a(G(\epsilon_\theta(x^b); s^f)) - 1 \right)^2 \right]$

The combined discriminator loss $\mathcal{L}^D = \mathcal{L}_{adv}^{D_s} + \mathcal{L}_{adv}^{D_a}$ is:

$\mathcal{L}_{adv}^{D_s}(\epsilon_\theta, G, D_s) = \frac{1}{2} \mathbb{E}_{s^f} \left[ \left( D_s(s^f) - 1 \right)^2 \right] + \frac{1}{2} \mathbb{E}_{x^a} \left[ \left( D_s(G(\epsilon_\theta(x^a); 0)) \right)^2 \right]$

$\mathcal{L}_{adv}^{D_a}(\epsilon_\theta, G, D_a) = \frac{1}{2} \mathbb{E}_{x_0^a} \left[ \left( D_a(x_0^a) - 1 \right)^2 \right] + \frac{1}{2} \mathbb{E}_{x^b, s^f} \left[ \left( D_a(G(\epsilon_\theta(x^b); s^f)) \right)^2 \right]$

The cycle-consistency loss enforces reconstruction of the fractal layout from the synthetic angiogram:

$\mathcal{L}_{cyc}(\epsilon_\theta, G) = \mathbb{E}_{x^b, s^f} \left[ \left\| G\left(\epsilon_\theta\left(G(\epsilon_\theta(x^b); s^f)\right); 0\right) - s^f \right\|_1 \right]$
Knowl 4 — Asymmetric Diffusion Time-Step Scheduling for Background-Outlier Representation
model/method
DARL employs an asymmetric noise scheduling strategy during the forward diffusion perturbation of background and angiography images. Forward noise perturbation is applied according to:

$x_t = \sqrt{\alpha_t} x_0 + \sqrt{1 - \alpha_t} \epsilon, \quad \epsilon \sim \mathcal{N}(0, I)$

with total time steps $T = 2000$ and noise variance $\beta_t$ linearly scheduled from $10^{-6}$ to $10^{-2}$ .

For non-contrast background images $x_0^b$ in Path (B), the perturbation time step $t_b$ is sampled uniformly from the full range $[0, T]$ . As a result, the diffusion module $\epsilon_\theta$ is trained primarily on background anatomy. For angiography images $x_0^a$ in Path (A), the perturbation time step $t_a$ is sampled uniformly from a restricted range $[0, T_a]$ where $T_a = 200 \ll T$ .

Because the diffusion module is optimized to represent background signals, the vessel structures present in angiography images appear as out-of-distribution outliers when computing latent score features $\epsilon_\theta(x_{t_a}^a, t_a)$ . This causes the latent representations $\epsilon_\theta$ to isolate and emphasize vascular contours while suppressing background tissues, providing high-contrast structural cues directly to the generation module $G$ .
Knowl 5 — Single-Step Feed-Forward Inference Algorithm for Vessel Segmentation
algorithm
Unlike standard diffusion probabilistic models that rely on iterative stochastic reverse-sampling trajectories, DARL performs deterministic single-step feed-forward inference to produce vessel segmentation masks.
Input: Angiography image $x_0^a \in \mathbb{R}^{H \times W}$ , trained diffusion network $\epsilon_\theta$ , trained generator $G$ , binarization threshold $\tau \in [0, 1]$
Output: Binary vessel segmentation mask $\hat{M}^v \in \{0, 1\}^{H \times W}$
1. Set evaluation time step $t_a = 0$
2. Extract spatial latent feature representation from the unperturbed image:
    $z = \epsilon_\theta(x_0^a, 0)$
3. Compute continuous vessel segmentation map using the generation module with S-SPADE disabled:
    $\hat{s}^v = G(z; 0)$
4. Binarize the output map:
   for each pixel coordinate $(h, w)$ do:
       if $\hat{s}^v(h, w) \ge \tau$ then:
            $\hat{M}^v(h, w) = 1$
       else:
            $\hat{M}^v(h, w) = 0$
5. return $\hat{M}^v$
Setting $t_a = 0$ during inference matches the clean test image distribution and yields sharp vascular boundaries in a single neural network forward pass.

Knowl 6 — Quantitative Evaluation of Self-Supervised Vessel Segmentation

data/table

Quantitative comparison of DARL against unsupervised methods (Spatial-Guided Clustering [SGC], Redrawing, Deep Spectral [DS]) and self-supervised methods (STEGO, Deep Adversarial [DA], Self-Supervised Vessel Segmentation [SSVS]). All models were trained solely on the unlabeled XCAD dataset (1,621 frames) without ground-truth masks. Testing was conducted on internal XCAD (114 test images), external coronary angiography datasets (134 XCA, 30 XCA), and cross-modality retinal datasets (DRIVE, STARE).

Data	Metric	Unsupervised			Self-supervised			Ours
		SGC	Redrawing	DS	STEGO	DA	SSVS
XCAD	IoU	$0.060 \pm 0.034$	$0.059 \pm 0.032$	$0.366 \pm 0.105$	$0.146 \pm 0.070$	$0.375 \pm 0.066$	$0.410 \pm 0.087$	$\mathbf{0.471 \pm 0.076}$
	Dice	$0.111 \pm 0.060$	$0.109 \pm 0.056$	$0.526 \pm 0.131$	$0.249 \pm 0.103$	$0.542 \pm 0.073$	$0.575 \pm 0.091$	$\mathbf{0.636 \pm 0.072}$
	Precision	$0.062 \pm 0.034$	$0.139 \pm 0.081$	$0.469 \pm 0.127$	$0.152 \pm 0.077$	$0.557 \pm 0.115$	$0.590 \pm 0.119$	$\mathbf{0.701 \pm 0.115}$
External test: Coronary angiography
134 XCA	IoU	$0.045 \pm 0.035$	$0.056 \pm 0.018$	$0.256 \pm 0.110$	$0.134 \pm 0.081$	$0.190 \pm 0.155$	$0.318 \pm 0.128$	$\mathbf{0.426 \pm 0.059}$
	Dice	$0.085 \pm 0.063$	$0.105 \pm 0.033$	$0.394 \pm 0.159$	$0.228 \pm 0.109$	$0.291 \pm 0.217$	$0.468 \pm 0.156$	$\mathbf{0.595 \pm 0.058}$
	Precision	$0.047 \pm 0.036$	$0.058 \pm 0.019$	$0.280 \pm 0.123$	$0.136 \pm 0.088$	$0.506 \pm 0.201$	$0.592 \pm 0.125$	$\mathbf{0.781 \pm 0.118}$
30 XCA	IoU	$0.083 \pm 0.039$	$0.048 \pm 0.022$	$0.339 \pm 0.086$	$0.191 \pm 0.072$	$0.298 \pm 0.109$	$0.324 \pm 0.146$	$\mathbf{0.427 \pm 0.184}$
	Dice	$0.150 \pm 0.064$	$0.091 \pm 0.040$	$0.499 \pm 0.113$	$0.314 \pm 0.100$	$0.447 \pm 0.148$	$0.468 \pm 0.193$	$\mathbf{0.572 \pm 0.205}$
	Precision	$0.090 \pm 0.041$	$0.144 \pm 0.074$	$0.525 \pm 0.130$	$0.200 \pm 0.081$	$0.612 \pm 0.174$	$0.613 \pm 0.212$	$\mathbf{0.729 \pm 0.152}$
Cross-modality test: Retinal imaging
DRIVE	IoU	$0.063 \pm 0.055$	$0.057 \pm 0.033$	$0.217 \pm 0.143$	$0.152 \pm 0.073$	$0.245 \pm 0.090$	$0.314 \pm 0.101$	$\mathbf{0.372 \pm 0.148}$
	Dice	$0.115 \pm 0.093$	$0.105 \pm 0.059$	$0.333 \pm 0.201$	$0.257 \pm 0.106$	$0.386 \pm 0.117$	$0.469 \pm 0.119$	$\mathbf{0.525 \pm 0.161}$
	Precision	$0.069 \pm 0.061$	$0.199 \pm 0.155$	$0.243 \pm 0.175$	$0.169 \pm 0.100$	$0.503 \pm 0.218$	$0.549 \pm 0.216$	$\mathbf{0.617 \pm 0.271}$
STARE	IoU	$0.055 \pm 0.045$	$0.074 \pm 0.048$	$0.180 \pm 0.141$	$0.125 \pm 0.076$	$0.237 \pm 0.122$	$0.311 \pm 0.148$	$\mathbf{0.368 \pm 0.191}$
	Dice	$0.101 \pm 0.077$	$0.134 \pm 0.080$	$0.281 \pm 0.201$	$0.216 \pm 0.109$	$0.367 \pm 0.167$	$0.454 \pm 0.185$	$\mathbf{0.508 \pm 0.216}$
	Precision	$0.058 \pm 0.047$	$0.227 \pm 0.157$	$0.205 \pm 0.172$	$0.135 \pm 0.092$	$0.427 \pm 0.233$	$0.490 \pm 0.230$	$\mathbf{0.537 \pm 0.280}$

DARL achieves the highest performance across all datasets. On internal XCAD, DARL outperforms SSVS by $0.061$ in IoU and $0.111$ in Precision. When deployed without fine-tuning on external angiography (134 XCA and 30 XCA) and retinal fundus datasets (DRIVE and STARE), DARL maintains superior IoU, Dice, and Precision, demonstrating cross-device and cross-organ transferability.

Knowl 7 — Vessel Segmentation Robustness to Simulated Gaussian Noise

data/table

To evaluate model robustness under simulated low-dose radiation conditions, Gaussian noise with standard deviation $\sigma \in \{10, 25, 50\}$ was added to XCAD test angiograms. The table below presents quantitative segmentation metrics across noise levels.

$\sigma$	Metric	Unsupervised			Self-supervised			Ours
		SGC	Redrawing	DS	STEGO	DA	SSVS
10	IoU	$0.066 \pm 0.033$	$0.052 \pm 0.031$	$0.331 \pm 0.104$	$0.144 \pm 0.073$	$0.353 \pm 0.065$	$0.258 \pm 0.079$	$\mathbf{0.451 \pm 0.080}$
	Dice	$0.122 \pm 0.059$	$0.096 \pm 0.053$	$0.487 \pm 0.133$	$0.245 \pm 0.107$	$0.519 \pm 0.073$	$0.404 \pm 0.099$	$\mathbf{0.617 \pm 0.076}$
	Precision	$0.069 \pm 0.035$	$0.126 \pm 0.077$	$0.480 \pm 0.135$	$0.157 \pm 0.091$	$0.481 \pm 0.104$	$0.477 \pm 0.117$	$\mathbf{0.710 \pm 0.115}$
25	IoU	$0.069 \pm 0.035$	$0.036 \pm 0.021$	$0.232 \pm 0.094$	$0.118 \pm 0.064$	$0.247 \pm 0.072$	$0.059 \pm 0.033$	$\mathbf{0.389 \pm 0.088}$
	Dice	$0.128 \pm 0.061$	$0.069 \pm 0.039$	$0.366 \pm 0.132$	$0.206 \pm 0.095$	$0.391 \pm 0.092$	$0.109 \pm 0.058$	$\mathbf{0.554 \pm 0.092}$
	Precision	$0.072 \pm 0.036$	$0.095 \pm 0.058$	$0.446 \pm 0.159$	$0.144 \pm 0.115$	$0.371 \pm 0.106$	$0.149 \pm 0.082$	$\mathbf{0.727 \pm 0.119}$
50	IoU	$0.070 \pm 0.025$	$0.020 \pm 0.012$	$0.077 \pm 0.065$	$0.060 \pm 0.050$	$0.102 \pm 0.056$	$0.021 \pm 0.013$	$\mathbf{0.269 \pm 0.081}$
	Dice	$0.130 \pm 0.045$	$0.040 \pm 0.022$	$0.136 \pm 0.109$	$0.108 \pm 0.088$	$0.180 \pm 0.091$	$0.041 \pm 0.025$	$\mathbf{0.417 \pm 0.101}$
	Precision	$0.072 \pm 0.026$	$0.061 \pm 0.038$	$0.221 \pm 0.168$	$0.076 \pm 0.067$	$0.169 \pm 0.094$	$0.060 \pm 0.038$	$\mathbf{0.716 \pm 0.147}$

At $\sigma = 50$ , baseline methods experience severe performance collapse (e.g., SSVS IoU drops from $0.410$ to $0.021$ ; Deep Spectral drops from $0.366$ to $0.077$ ). DARL maintains an IoU of $0.269$ , Dice of $0.417$ , and Precision of $0.716$ , demonstrating resilience to high-frequency noise corruption due to its diffusion perturbation training.

Knowl 8 — Ablation Analysis of DARL Architecture and Loss Components

data/table

An ablation study on the XCAD dataset quantifies the individual impact of the diffusion module, Switchable SPADE (S-SPADE) layers, cyclic consistency loss $\mathcal{L}_{cyc}$ , and the reconstruction loss norm ( $L_1$ versus Cross-Entropy).

Method	Module		Loss function			Metric
	Diffusion	Generation	$\mathcal{L}_{diff}$	$\mathcal{L}_{adv}$	$\mathcal{L}_{cyc}$	IoU	Dice	Precision
Ours	✓	✓	✓	✓	✓	$\mathbf{0.471 \pm 0.076}$	$\mathbf{0.636 \pm 0.072}$	$\mathbf{0.701 \pm 0.115}$
(a)		✓		✓	✓	$0.449 \pm 0.077$	$0.616 \pm 0.074$	$0.646 \pm 0.106$
(b)		w/o S-SPADE		✓	✓	$0.439 \pm 0.080$	$0.606 \pm 0.080$	$0.620 \pm 0.111$
(c)	✓	✓	✓	✓		$0.322 \pm 0.055$	$0.485 \pm 0.064$	$0.580 \pm 0.112$
(d)	✓	✓	✓	✓	$L_1 \to \text{CE}$	$0.346 \pm 0.084$	$0.508 \pm 0.094$	$0.672 \pm 0.147$

Key findings:

Removing the diffusion module and $\mathcal{L}_{diff}$ (a) reduces IoU by $0.022$ and Precision by $0.055$ , demonstrating that diffusion score features provide informative vessel cues.
Removing S-SPADE and splitting generation and segmentation into two separate networks (b) lowers IoU to $0.439$ and Precision to $0.620$ , verifying that parameter-shared S-SPADE layers create synergistic representations across tasks.
Removing the cyclic reconstruction loss $\mathcal{L}_{cyc}$ (c) causes a severe performance drop (IoU $0.322$ , Dice $0.485$ ), showing that pseudo-label cycle consistency is necessary for vessel localization.
Substituting $L_1$ loss with Cross-Entropy (CE) loss in the cycle path (d) results in an IoU of $0.346$ , confirming the advantage of framing mask reconstruction as continuous image generation.

Knowl 9 — Evaluation Across Supervised, Semi-Supervised, and Background-Free Settings

data/table

DARL was evaluated across few-label supervised (trained on 12 labeled validation pairs), semi-supervised pseudo-labeling, and background-free self-supervised regimes on coronary and retinal datasets.

Data	Metric	Supervised (few-label)		Semi-supervised (pseudo-pair)		Self-supervised
		(A) path	(A)+(B) path	Data from (A)	Data from (B)	Ours	Ours w/o BG
XCAD	IoU	$0.423 \pm 0.079$	$0.548 \pm 0.072$	$0.478 \pm 0.078$	$0.445 \pm 0.072$	$0.471 \pm 0.076$	$0.479 \pm 0.080$
	Dice	$0.590 \pm 0.081$	$0.705 \pm 0.062$	$0.643 \pm 0.073$	$0.613 \pm 0.071$	$0.636 \pm 0.072$	$0.644 \pm 0.076$
	Precision	$0.637 \pm 0.134$	$0.746 \pm 0.088$	$0.703 \pm 0.115$	$0.645 \pm 0.117$	$0.701 \pm 0.115$	$0.700 \pm 0.125$
External test: Coronary angiography
134 XCA	IoU	$0.273 \pm 0.161$	$0.323 \pm 0.195$	$0.388 \pm 0.187$	$0.378 \pm 0.179$	$0.426 \pm 0.059$	$0.377 \pm 0.187$
	Dice	$0.404 \pm 0.201$	$0.454 \pm 0.237$	$0.531 \pm 0.211$	$0.522 \pm 0.205$	$0.595 \pm 0.058$	$0.518 \pm 0.215$
	Precision	$0.439 \pm 0.215$	$0.453 \pm 0.236$	$0.477 \pm 0.213$	$0.458 \pm 0.198$	$0.781 \pm 0.118$	$0.469 \pm 0.208$
30 XCA	IoU	$0.365 \pm 0.042$	$0.484 \pm 0.066$	$0.429 \pm 0.064$	$0.425 \pm 0.080$	$0.427 \pm 0.184$	$0.429 \pm 0.070$
	Dice	$0.534 \pm 0.045$	$0.649 \pm 0.061$	$0.598 \pm 0.064$	$0.592 \pm 0.082$	$0.572 \pm 0.205$	$0.597 \pm 0.070$
	Precision	$0.717 \pm 0.121$	$0.829 \pm 0.083$	$0.772 \pm 0.124$	$0.737 \pm 0.133$	$0.729 \pm 0.152$	$0.762 \pm 0.124$
Cross-modality test: Retinal imaging
DRIVE	IoU	$0.344 \pm 0.134$	$0.388 \pm 0.141$	$0.365 \pm 0.148$	$0.341 \pm 0.147$	$0.372 \pm 0.148$	$0.392 \pm 0.135$
	Dice	$0.497 \pm 0.153$	$0.544 \pm 0.153$	$0.518 \pm 0.159$	$0.490 \pm 0.163$	$0.525 \pm 0.161$	$0.550 \pm 0.144$
	Precision	$0.617 \pm 0.249$	$0.754 \pm 0.213$	$0.585 \pm 0.271$	$0.504 \pm 0.265$	$0.617 \pm 0.271$	$0.659 \pm 0.247$
STARE	IoU	$0.319 \pm 0.168$	$0.332 \pm 0.195$	$0.375 \pm 0.187$	$0.354 \pm 0.183$	$0.368 \pm 0.191$	$0.393 \pm 0.183$
	Dice	$0.458 \pm 0.205$	$0.464 \pm 0.235$	$0.517 \pm 0.210$	$0.495 \pm 0.208$	$0.508 \pm 0.216$	$0.538 \pm 0.205$
	Precision	$0.529 \pm 0.267$	$0.591 \pm 0.295$	$0.528 \pm 0.269$	$0.481 \pm 0.254$	$0.537 \pm 0.280$	$0.566 \pm 0.257$

Key takeaways:

Self-supervised DARL ( $0.636$ Dice on XCAD) outperforms a few-label supervised baseline trained on Path (A) alone with 12 labeled pairs ( $0.590$ Dice).
Using DARL predictions as pseudo-labels in semi-supervised training achieves $0.478$ IoU on XCAD, demonstrating that DARL outputs are reliable supervision targets.
In settings without pre-injection non-contrast background images ( $x^b$ ), training DARL by replacing $x^b$ with angiography images $x^a$ ("Ours w/o BG") achieves equivalent or slightly improved segmentation performance ( $0.479$ vs. $0.471$ IoU on XCAD; $0.392$ vs. $0.372$ IoU on DRIVE).

Knowl 10 — Synthesis Quality (FID) and Computational Complexity (GFLOPS)

data/table

The visual realism of synthetic angiograms and vessel masks was evaluated using Fréchet Inception Distance (FID; lower is better) on 1,621 generated samples against Deep Adversarial (DA), Self-Supervised Vessel Segmentation (SSVS), and OASIS. Computational complexity was evaluated by total training GFLOPS.

Method	(A) path	(B) path	Cycle path	Discriminator	Total (GFLOPS)
DA	121.80	121.80	$121.80 \times 2$	$6.24 \times 2$	499.68
SSVS	121.80	121.80	$121.80 \times 2$	$6.24 \times 2$	499.68
Ours	90.66	173.51	$90.66 \times 1$	$6.24 \times 2$	367.31

DARL produces lower FID scores for both vessel masks ( $93.49$ ) and synthetic angiograms ( $177.59$ ) than comparative methods, indicating higher fidelity to real image distributions. Furthermore, by using a single unified generator with S-SPADE rather than two bidirectional networks as in CycleGAN frameworks, DARL computes the cycle consistency path in a single forward evaluation ( $90.66$ GFLOPS vs. $121.80 \times 2$ ), reducing total training complexity from $499.68$ to $367.31$ GFLOPS (a $26.5\%$ reduction).

Coverage note — None was omitted; all key contributions—including model architecture, loss functions, noise scheduling, inference algorithm, multi-dataset benchmarking, noise robustness, ablation studies, alternate learning regimes, and computational complexity—are fully captured.

References

1.Euijoon Ahn, Dagan Feng, and Jinman Kim. A spatial guided self-supervised clustering network for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 379–388. Springer, 2021.
2.Tomer Amit, Eliya Nachmani, Tal Shaharbany, and Lior Wolf. Segdiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390, 2021.
3.Dmitry Baranchuk, Andrey Voynov, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Label-efficient semantic segmentation with diffusion models. In International Conference on Learning Representations, 2021.
4.Fernando Cervantes-Sanchez, Ivan Cruz-Aceves, Arturo Hernandez-Aguirre, Martha Alicia Hernandez-Gonzalez, and Sergio Eduardo Solorio-Meza. Automatic segmentation of coronary arteries in x-ray angiograms using multiscale analysis and artificial neural networks. Applied Sciences, 9(24), 2019. ISSN 2076-3417. doi: 10.3390/app9245507. URL https://www.mdpi.com/2076-3417/9/24/5507.
5.Mickaël Chen, Thierry Artières, and Ludovic Denoyer. Unsupervised object segmentation by redrawing. Advances in neural information processing systems, 32, 2019.
6.Yang Chen, Luyao Shi, Qianjing Feng, Jian Yang, Huazhong Shu, Limin Luo, Jean-Louis Coatrieux, and Wufan Chen. Artifact suppressed dictionary learning for low-dose ct image processing. IEEE transactions on medical imaging, 33(12):2271–2292, 2014.
7.Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12413–12422, 2022.
8.Weijian Cong, Jian Yang, Danni Ai, Yang Chen, Yue Liu, and Yongtian Wang. Quantitative analysis of deformable model-based 3-d reconstruction of coronary artery from multiple angiograms. IEEE Transactions on Biomedical Engineering, 62(8):2079–2090, 2015.
9.Jingfan Fan, Jian Yang, Yachen Wang, Siyuan Yang, Danni Ai, Yong Huang, Hong Song, Aimin Hao, and Yongtian Wang. Multichannel fully convolutional network for coronary artery segmentation in x-ray angiograms. Ieee Access, 6:44635–44643, 2018.
10.Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. Unsupervised semantic segmentation by distilling feature correspondences. arXiv preprint arXiv:2203.08414, 2022.
11.Dongdong Hao, Song Ding, Linwei Qiu, Yisong Lv, Baowei Fei, Yueqi Zhu, and Binjie Qin. Sequential vessel segmentation via deep channel attention network. Neural Networks, 128:172–187, 2020. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2020.05.005. URL https://www.sciencedirect.com/science/article/pii/S0893608020301672.
12.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
13.Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
14.Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
15.Adam Hoover and Michael Goldbaum. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE transactions on medical imaging, 22(8):951–958, 2003.
16.Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134, 2017.
17.Boah Kim, Inhwa Han, and Jong Chul Ye. Diffusemorph: Unsupervised deformable image registration along continuous trajectory using diffusion models. arXiv preprint arXiv:2112.05149, 2021.
18.Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
19.Max WK Law and Albert CS Chung. Efficient implementation for spherical flux computation and its application to vascular segmentation. IEEE transactions on image processing, 18(3):596–612, 2009.
20.Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11461–11471, 2022.
21.Yuxin Ma, Yang Hua, Hanming Deng, Tao Song, Hao Wang, Zhengui Xue, Heng Cao, Ruhui Ma, and Haibing Guan. Self-supervised vessel segmentation via adversarial learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7536–7545, 2021.
22.Faisal Mahmood, Daniel Borders, Richard J Chen, Gregory N McKay, Kevan J Salimian, Alexander Baras, and Nicholas J Durr. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE transactions on medical imaging, 39(11):3257–3267, 2019.
23.Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2794–2802, 2017.
24.Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8364–8375, 2022.
25.Ebrahim Nasr-Esfahani, Shadrokh Samavi, Nader Karimi, SM Reza Soroushmehr, Kevin Ward, Mohammad H Jafari, Banafsheh Felfeliyan, B Nallamothu, and Kayvan Najarian. Vessel extraction in x-ray angiograms using deep learning. In 2016 38th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp. 643–646. IEEE, 2016.
26.Yujin Oh and Jong Chul Ye. CXR Segmentation by AdaIN-based Domain Adaptation and Knowledge Distillation. arXiv preprint arXiv:2104.05892, 2021.
27.Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2337–2346, 2019.
28.Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc., 2019. URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
29.Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer, 2015.
30.Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. arXiv preprint arXiv:2104.07636, 2021.
31.Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1–10, 2022.
32.Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256–2265. PMLR, 2015.
33.Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
34.Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
35.Joes Staal, Michael D Abramoff, Meindert Niemeijer, Max A Viergever, and Bram Van Ginneken. Ridge-based vessel segmentation in color images of the retina. IEEE transactions on medical imaging, 23(4):501–509, 2004.
36.Vadim Sushko, Edgar Schönfeld, Dan Zhang, Juergen Gall, Bernt Schiele, and Anna Khoreva. You only need adversarial supervision for semantic image synthesis. arXiv preprint arXiv:2012.04781, 2020.
37.Maryam Taghizadeh Dehkordi, Ali Mohamad Doost Hoseini, Saeed Sadri, and Hamid Soltanian-zadeh. Local feature fitting active contour for segmenting vessels in angiograms. IET Computer Vision, 8(3):161–170, 2014.
38.Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6924–6932, 2017.
39.Shaoyan Xia, Haogang Zhu, Xiaoli Liu, Ming Gong, Xiaoyong Huang, Lei Xu, Hongjia Zhang, and Jialong Guo. Vessel segmentation of x-ray coronary angiographic image sequence. IEEE Transactions on Biomedical Engineering, 67(5):1338–1348, 2019.
40.Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223–2232, 2017.

Citation

MLA

Kim, B., et al. “Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation”. arXiv, 2022, http://arxiv.org/abs/2209.14566v2.

APA

Kim, B., Oh, Y., & Ye, J. C. (2022). Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation. arXiv. http://arxiv.org/abs/2209.14566v2

Chicago

Kim, B., Y. Oh, and J. C. Ye. 2022. “Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation”. arXiv. http://arxiv.org/abs/2209.14566v2.

Harvard

Kim, B., Oh, Y. and Ye, J.C. (2022) “Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2209.14566v2.

Vancouver

1. Kim B, Oh Y, Ye JC (2022) Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation. arXiv

BibTeX

@article{kim2022diffusion,
  title = {Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation},
  author = {Kim, Boah and Oh, Yujin and Ye, Jong Chul},
  year = {2022},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2209.14566v2},
  eprint = {2209.14566}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

FID Metric $\downarrow$	DA	SSVS	OASIS	Ours
Vessel mask FID	123.20	149.00	N/A	93.49
Angiogram FID	261.64	191.68	307.50	177.59

Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Diffusion Adversarial Representation Learning (DARL) Framework for Self-Supervised Vessel Segmentation

Knowl 2 — Switchable Spatially-Adaptive Denormalization (S-SPADE)

Knowl 3 — Multi-Objective Training Loss for DARL

Knowl 4 — Asymmetric Diffusion Time-Step Scheduling for Background-Outlier Representation

Knowl 5 — Single-Step Feed-Forward Inference Algorithm for Vessel Segmentation

Knowl 6 — Quantitative Evaluation of Self-Supervised Vessel Segmentation

Knowl 7 — Vessel Segmentation Robustness to Simulated Gaussian Noise

Knowl 8 — Ablation Analysis of DARL Architecture and Loss Components

Knowl 9 — Evaluation Across Supervised, Semi-Supervised, and Background-Free Settings

Knowl 10 — Synthesis Quality (FID) and Computational Complexity (GFLOPS)

References

Citation

BibTeX

Source Code

Access the Paper