GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Maayan Frid-AdarIdit DiamantEyal KlangMichal AmitaiJacob GoldbergerHayit Greenspan

article2018Neurocomputing1,920 citations

Demonstrates that augmenting limited CT datasets with GAN-generated synthetic lesion images significantly improves convolutional neural network accuracy in classifying liver cysts, metastases, and hemangiomas.

Listen

Medical imaging research faces a persistent bottleneck due to the limited availability of large, annotated clinical datasets. Expert radiologists must manually annotate images, making dataset collection expensive, slow, and restricted in size. At the same time, computer-aided diagnostic tools powered by deep learning require vast amounts of data to achieve reliable accuracy. While standard data transformations like image rotations or shifts help prevent model errors, they add minimal new visual information. Consequently, automated systems often plateau in diagnostic performance, particularly when differentiating subtle or overlapping medical conditions such as benign versus malignant liver lesions.

To address this challenge, the article evaluated whether deep generative models—specifically Generative Adversarial Networks (GANs)—could synthesize realistic computed tomography (CT) images of liver lesions and whether adding these synthetic samples into training workflows improves the diagnostic classification performance of Convolutional Neural Networks (CNNs).

The evaluation used a clinical dataset of 182 two-dimensional CT scans of liver lesions from 2009 to 2014, consisting of 53 cysts, 64 metastases, and 65 hemangiomas. Researchers cropped the lesion regions and developed a specialized classification network. They first trained the network using only traditional image manipulations to identify performance limits. Next, they trained generative networks to learn the visual data distributions of the lesion categories and generate high-quality synthetic lesion images. The team then augmented the baseline training data with varying amounts of synthetic examples using cross-validation to assess improvements. Finally, two expert radiologists evaluated the realism and diagnostic clarity of both real and generated samples.

The findings demonstrate substantial improvements across several key dimensions. First, generative synthesis broke through the performance ceiling of standard augmentation: baseline classification accuracy reached a plateau at 78.6% (78.6% sensitivity and 88.4% specificity), but incorporating synthetic images increased total accuracy to 85.7% (85.7% sensitivity and 92.4% specificity), representing an absolute gain of roughly 7%. Second, performance notably improved on the clinically difficult task of distinguishing malignant metastases from benign hemangiomas, where sensitivity rose from 68.7% to 81.2% and from 72.3% to 78.5%, respectively. Third, class-specific generative models (Deep Convolutional GANs) outperformed multi-class conditional architectures (Auxiliary Classifier GANs), which reached only 81.3% sensitivity and 90.0% specificity. Fourth, the synthetic-enhanced network surpassed established non-deep-learning benchmarks. Finally, the synthesized images proved visually authentic: expert radiologists correctly identified real versus synthetic images in only about 59% to 63% of trials—near random guessing—and achieved identical diagnostic accuracy on both real and synthetic images.

These results show that synthetic data augmentation effectively expands the diversity of small medical imaging datasets without requiring costly manual labeling campaigns. In clinical practice, misidentifying a malignant metastasis as a benign hemangioma poses severe safety risks and delays critical cancer treatments. By reducing diagnostic confusion and lifting classification accuracy, generative augmentation offers a viable path toward creating more reliable automated decision-support systems for clinical radiology.

Based on these findings, medical technology teams should adopt synthetic generative augmentation when training diagnostic models on constrained datasets. When implementing this method, teams should prefer training dedicated generative models per disease category rather than a single multi-class model, as separate models yielded higher downstream accuracy. Further development is recommended to extend this methodology from two-dimensional image crops to full three-dimensional volumetric scans and to evaluate the approach across other imaging modalities and organ systems.

Readers should interpret these results within certain study boundaries. The findings rely on a relatively small single-institution dataset of 182 two-dimensional lesion crops rather than complete three-dimensional anatomical volumes used in standard radiology workflows. Additionally, training separate generative models for each disease category increases computational complexity. Nonetheless, the experimental methodology and expert clinician evaluations provide high confidence that synthetic data augmentation reliably boosts automated medical image classification.

Bar-Ilan University Sheba Medical Center Tel Aviv University classification computer vision convolutional neural networks Lesion Classi Cation Liver Lesion

arXiv: 1803.01229

Paper: Generative Adversarial Networks, Ian J. Goodfellow et al. (2014). Introduces the core theoretical framework and min-max optimization of generative adversarial networks upon which the synthetic lesion synthesis is built.
Paper: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Alec Radford et al. (2016). Establishes deep convolutional GAN architectures and training guidelines critical for generating high-quality image patches and representations.
Paper: Conditional Generative Adversarial Nets, Mehdi Mirza et al. (2014). Presents class-conditional GAN formulation necessary for synthesizing class-specific lesion categories such as cysts, metastases, and hemangiomas.
Paper: Improved Techniques for Training GANs, Tim Salimans et al. (2016). Provides essential stabilization and feature-matching techniques to improve GAN sample diversity and training stability.
Paper: The Effectiveness of Data Augmentation in Image Classification using Deep Learning, Luis Perez et al. (2017). Evaluates generative and learned data augmentation strategies for improving image classification generalization on small datasets.
Paper: A survey on deep learning in medical image analysis, Geert Litjens et al. (2017). Surveys foundational deep learning architectures and small-data challenges across medical image analysis domains.
Paper: Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning, Hoo-Chang Shin et al. (2016). Analyzes CNN training characteristics, transfer learning, and dataset size effects on CT-based lesion and pathology detection.

Paper: Training Generative Adversarial Networks with Limited Data, Tero Karras et al. (2020). Introduces adaptive discriminator augmentation to train high-quality GANs directly in extremely limited data regimes without dataset leakage.
Paper: Self-Attention Generative Adversarial Networks, Han Zhang et al. (2019). Extends generative modeling capabilities using self-attention mechanisms to enforce global anatomical coherence across synthetic samples.
Paper: Albumentations: fast and flexible image augmentations, Alexander Buslaev et al. (2018). Provides a comprehensive and fast software framework for standardizing complex image augmentation pipelines alongside synthetic generators.

Cover for GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Abstract

Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classification. Our novel method is demonstrated on a limited dataset of computed tomography (CT) images of 182 liver lesions (53 cysts, 64 metastases and 65 hemangiomas). We first exploit GAN architectures for synthesizing high quality liver lesion ROIs. Then we present a novel scheme for liver lesion classification using CNN. Finally, we train the CNN using classic data augmentation and our synthetic data augmentation and compare performance. In addition, we explore the quality of our synthesized examples using visualization and expert assessment. The classification performance using only classic data augmentation yielded 78.6% sensitivity and 88.4% specificity. By adding the synthetic data augmentation the results increased to 85.7% sensitivity and 92.4% specificity. We believe that this approach to synthetic data augmentation can generalize to other medical classification applications and thus support radiologists' efforts to improve diagnosis.

I Introduction
II Liver Lesion Classification
II-A Data
II-B CNN Architecture
III Generating Synthetic Liver Lesions
III-A Classic Data Augmentation
III-B Generative Adversarial Networks for Lesion Synthesis
III-C Conditional Lesion Synthesis
IV Experiments and Results
IV-A Dataset Evaluation and Implementation Details
IV-B Evaluation of the Synthetic Data Augmentation
IV-B1 Classical data augmentation
IV-B2 Synthetic data augmentation
IV-C Visualization using t-SNE
IV-D Expert Assessment of Synthetic Data
IV-E Comparison with Other Classification Methods
V Discussion And Conclusions
References

Knowls

Knowl 1 — Two-Stage GAN-Based Synthetic Data Augmentation Framework for Small Medical Datasets
model/method
A two-stage data augmentation pipeline improves classification performance on small medical imaging datasets where classical geometric augmentation hits a performance plateau:
1. Classical Augmentation Baseline: Standard affine transformations (rotation, translation, flipping, scaling) are applied to the region-of-interest (ROI) training images. The classification network is evaluated over increasing augmentation volumes until performance saturates at an optimal sample size $D_{\text{aug}}^{\text{optimal}}$ .
2. Generative Modeling: Deep convolutional generative adversarial networks (DCGANs) are trained on the augmented dataset $D_{\text{aug}}^{\text{optimal}}$ separately for each lesion class to capture intra-class data distributions and visual variability.
3. Combined Training: A synthetic dataset $D_{\text{synth}}$ of generated ROIs is sampled with balanced class distribution and combined with $D_{\text{aug}}^{\text{optimal}}$ to form an expanded training set $D_{\text{aug}}^{\text{optimal}} + D_{\text{synth}}$ . The classification CNN is then retrained on this composite dataset.

Knowl 2 — Classification Performance Across Augmentation Schemes and Lesion Categories

data/table

Using 3-fold cross-validation on 182 CT liver lesions (53 cysts, 64 metastases, 65 hemangiomas), training a classification CNN with combined classical and DCGAN-generated synthetic data (CNN-AUG-GAN) yields substantial improvements over classical augmentation alone (CNN-AUG) and the baseline dictionary learning method (BoVW-MI):

Lesion Class	CNN-AUG-GAN		CNN-AUG		BoVW-MI
	Sensitivity	Specificity	Sensitivity	Specificity	Sensitivity	Specificity
Cysts	100.0%	97.7%	98.1%	98.4%	96.3%	96.9%
Metastases	81.2%	89.0%	68.7%	83.9%	75.0%	82.2%
Hemangiomas	78.5%	91.4%	72.3%	84.6%	66.1%	87.2%
Weighted Average	85.7%	92.4%	78.6%	88.4%	78.0%	88.3%

Without any data augmentation, the CNN achieves only $57.0\%$ total accuracy due to overfitting on small sample sizes. Adding classical augmentations improves accuracy up to a saturation ceiling of $78.6\%$ at 5,000 augmented samples per fold ( $D_{\text{aug}}^6$ ). Incorporating 3,000 synthetic samples per fold (1,000 per class) on top of the 5,000 classically augmented samples increases overall accuracy to $85.7\%$ (an absolute gain of $7.1\%$ ).

Knowl 3 — DCGAN Architecture and Hyperparameters for Liver Lesion ROI Synthesis
model/method
A Deep Convolutional Generative Adversarial Network (DCGAN) is configured to synthesize single-channel $64 \times 64$ focal liver lesion regions of interest (ROIs) for each lesion class independently:
- Generator:
  - Input: 100-dimensional noise vector $z \sim \mathcal{U}(-1, 1)^{100}$ .
  - Dense fully connected layer reshaped to $4 \times 4 \times 1024$ .
  - Four fractionally-strided convolutional layers with $5 \times 5$ kernel sizes to upsample features sequentially to $8 \times 8 \times 512$ , $16 \times 16 \times 256$ , $32 \times 32 \times 128$ , and $64 \times 64 \times 1$ .
  - Batch normalization applied to all intermediate layers (omitted at the output layer).
  - Activations: ReLU on hidden layers; $\tanh$ on the output layer.
- Discriminator:
  - Input: $64 \times 64 \times 1$ image scaled to $[-1, 1]$ .
  - Four strided convolutional layers with $5 \times 5$ filters (stride replacing spatial pooling).
  - Batch normalization on intermediate convolutional layers (omitted at input and output layers).
  - Activations: Leaky ReLU with leak slope $\alpha = 0.2$ on hidden layers; Sigmoid on the single-scalar output.
- Training Parameters:
  - Optimizer: Adam with learning rate $\eta = 0.0002$ , $\beta_1 = 0.5$ , $\beta_2 = 0.999$ .
  - Weight initialization: Zero-centered Gaussian $\mathcal{N}(0, 0.02^2)$ .
  - Mini-batch size: $m = 64$ real images and $m = 64$ noise vectors.
  - Training duration: 70 epochs per class.
Knowl 4 — CNN Architecture and Training Procedure for Focal Liver Lesion Classification
model/method
The 3-class liver lesion classification network processes fixed-size $64 \times 64 \times 1$ CT lesion ROIs rescaled to $[0, 1]$ :
- Network Architecture:
  - Layer 1--2: Convolution ( $3 \times 3$ , 32 filters, ReLU) $\to$ Max-Pooling.
  - Layer 3--4: Convolution ( $3 \times 3$ , 64 filters, ReLU) $\to$ Max-Pooling.
  - Layer 5--6: Convolution ( $3 \times 3$ , 128 filters, ReLU) $\to$ Max-Pooling.
  - Fully Connected Layer: 256 units with ReLU activation and Dropout probability $p = 0.5$ .
  - Output Layer: Dense 3 units with Softmax activation corresponding to Cyst, Metastasis, and Hemangioma.
  - Total parameters: approximately 1.3 million.
- Training Procedure:
  - Input preprocessing: Channel-wise mean subtraction of the training set.
  - Optimizer: Stochastic Gradient Descent (SGD) with Nesterov momentum updates.
  - Hyperparameters: Learning rate $\eta = 0.001$ , batch size 64, trained for 150 epochs.

Knowl 5 — Expert Radiologist Assessment of Synthesized Liver Lesion Realism and Class Distinctiveness

empirical result

Two independent expert radiologists evaluated a randomized pool of 302 lesion ROIs (182 real CT lesions and 120 DCGAN-synthesized lesions) under two blinded testing tasks: classifying lesion category (cyst, metastasis, or hemangioma) and determining whether each ROI was real or synthetic.

The expert radiologists achieved near-chance detection accuracy (approximately $60\%$ ) when attempting to discriminate synthetic ROIs from real ones. Furthermore, for both experts, diagnostic classification accuracy was virtually identical between real and synthetic subsets ( $78.0\%$ vs $77.5\%$ for Expert 1; $69.2\%$ vs $69.2\%$ for Expert 2), confirming that GAN-generated ROIs capture anatomically plausible and clinically distinct pathological features.

Knowl 6 — Performance Comparison Between Generative Augmentation Models (DCGAN vs ACGAN)

data/table

In evaluating generative modeling architectures for data augmentation in liver lesion classification, training class-specific DCGANs outperformed a single class-conditioned Auxiliary Classifier GAN (ACGAN):

While synthetic augmentation using ACGAN improves upon classical augmentation alone ( $78.6\%$ sensitivity, $88.4\%$ specificity), joint multi-class conditioning within ACGAN degrades sample quality relative to class-specific DCGANs. Directly using the trained ACGAN auxiliary classifier discriminator on the test set yields $79.1\%$ sensitivity and $88.8\%$ specificity, which is approximately $2\%$ lower than retraining a dedicated CNN classifier using the ACGAN-generated samples.

Knowl 7 — Domain-Preserving Classical Affine Augmentation for CT Lesion ROIs
algorithm
To expand small lesion datasets without distorting diagnostic pathology shape (e.g., avoiding shear transformations), each centered lesion ROI is augmented via a sequence of bounded affine transformations:
Input: Lesion ROI $I \in \mathbb{R}^{H \times W}$ , Lesion diameter $d$ in pixels
Hyperparameters: $N_{rot} = 30$ , $N_{flip} = 3$ , $N_{trans} = 7$ , $N_{scale} = 5$
Output: Set of augmented ROIs $\mathcal{S}$ of uniform size $64 \times 64$
$\mathcal{S} \leftarrow \emptyset$
$p \leftarrow \min(4, 0.1 \times d)$
for $i = 1$ to $N_{rot}$ do
    Sample angle $\theta \sim \mathcal{U}(0^{\circ}, 180^{\circ})$
     $I_{rot} \leftarrow \text{Rotate}(I, \theta)$
     $\mathcal{S} \leftarrow \mathcal{S} \cup \{ \text{Resize}(I_{rot}, 64 \times 64) \}$
    for $j = 1$ to $N_{flip}$ do
         $I_{flip} \leftarrow \text{Flip}(I_{rot}, \text{axis} \in \{\text{horizontal}, \text{vertical}\})$
         $\mathcal{S} \leftarrow \mathcal{S} \cup \{ \text{Resize}(I_{flip}, 64 \times 64) \}$
    for $k = 1$ to $N_{trans}$ do
        Sample translation offsets $[\Delta x, \Delta y] \sim \mathcal{U}(-p, p)^2$
         $I_{trans} \leftarrow \text{Translate}(I_{rot}, \Delta x, \Delta y)$
         $\mathcal{S} \leftarrow \mathcal{S} \cup \{ \text{Resize}(I_{trans}, 64 \times 64) \}$
    for $m = 1$ to $N_{scale}$ do
        Sample margin scale factor $s \sim \mathcal{U}(0.1 \times d, 0.4 \times d)$
         $I_{scale} \leftarrow \text{AdjustSurroundingParenchyma}(I_{rot}, s)$
         $\mathcal{S} \leftarrow \mathcal{S} \cup \{ \text{Resize}(I_{scale}, 64 \times 64) \}$
return $\mathcal{S}$
Each lesion ROI yields $N = N_{\text{rot}} \times (1 + N_{\text{flip}} + N_{\text{trans}} + N_{\text{scale}}) = 30 \times (1 + 3 + 7 + 5) = 480$ augmented images, which are resized to $64 \times 64$ pixels using bicubic interpolation.
Knowl 8 — Auxiliary Classifier GAN (ACGAN) Loss Formulation for Multi-Class Lesion Synthesis
equation
In the Auxiliary Classifier GAN (ACGAN) framework for multi-class lesion synthesis, the discriminator $D$ outputs both a real/fake probability distribution $P(S \mid X)$ over source state $S \in \{\text{real}, \text{fake}\}$ and a class probability distribution $P(C \mid X)$ over lesion category $C \in \{\text{Cyst}, \text{Metastasis}, \text{Hemangioma}\}$ .

Given real lesion-label pairs $(X_{\text{real}}, c)$ and synthetic images $X_{\text{fake}} = G(c, z)$ generated from class condition $c$ and noise $z \sim p_z$ , the objective is partitioned into source log-likelihood $L_s$ and classification log-likelihood $L_c$ :

$L_s = \mathbb{E}_{X_{\text{real}} \sim p_{\text{data}}}\left[\log P(S = \text{real} \mid X_{\text{real}})\right] + \mathbb{E}_{z \sim p_z, c \sim p_c}\left[\log P(S = \text{fake} \mid G(c, z))\right]$

$L_c = \mathbb{E}_{X_{\text{real}}, c \sim p_{\text{data}}}\left[\log P(C = c \mid X_{\text{real}})\right] + \mathbb{E}_{z \sim p_z, c \sim p_c}\left[\log P(C = c \mid G(c, z))\right]$

During joint adversarial optimization:
- The discriminator parameters are trained to maximize $L_s + L_c$ .
- The generator parameters are trained to maximize $L_c - L_s$ .
Knowl 9 — t-SNE Feature Space Separation with Synthetic Augmentation
empirical result
t-Distributed Stochastic Neighbor Embedding (t-SNE) dimensionality reduction applied to the penultimate dense feature representations (256 dimensions) of the lesion classification CNN reveals the mechanism behind synthetic augmentation performance gains:
1. Cyst Representations: Cysts form a tightly localized, well-separated cluster in both classically augmented (CNN-AUG) and synthetically augmented (CNN-AUG-GAN) feature spaces, corresponding to near-perfect cyst sensitivity ( $\ge 98.1\%$ ).
2. Metastasis vs Hemangioma Separation: In the CNN-AUG feature space, metastases and hemangiomas exhibit significant cluster overlap due to shared density characteristics (such as calcifications and vascular enhancement). In the CNN-AUG-GAN feature space, the inter-class overlap is substantially reduced, providing distinct cluster boundaries and explaining the sensitivity gains for metastases ( $68.7\% \to 81.2\%$ ) and hemangiomas ( $72.3\% \to 78.5\%$ ).
Knowl 10 — Methodological and Structural Limitations of the GAN-Based Lesion Augmentation Approach
limitation
The proposed GAN-based synthetic medical image augmentation methodology has several documented limitations:
1. 2-D vs 3-D Context: Image generation and classification operate solely on 2-D single-slice CT ROIs ( $64 \times 64$ ), omitting the 3-D volumetric context and sequential multi-slice anatomical continuity utilized by clinical radiologists.
2. Multi-Model Complexity: High-quality generation required training separate DCGAN architectures for each individual lesion category; attempts to unify generation via a single conditional ACGAN resulted in decreased classification performance ( $81.3\%$ sensitivity vs $85.7\%$ for DCGAN).
3. Unsupervised Data & Loss Regularization: The generative pipeline relies solely on labeled lesion crops without incorporating unlabeled medical imaging data, and does not include pixel-wise or perceptual regularization terms ( $L_1$ or $L_2$ losses) in the GAN objective.

Coverage note — None was omitted; all contributed models, dataset details, algorithms, equations, experimental comparisons, radiologist evaluation results, and limitations are fully covered.

References

1.H. R. Roth, L. Lu, J. Liu, J. Yao, A. Seff, K. Cherry, L. Kim, and R. M. Summers, “Improving computer-aided detection using convolutional neural networks and random view aggregation,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1170–1181, May 2016.
2.G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis,” arXiv preprint arXiv:1702.05747, 2017.
3.H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1153–1159, May 2016.
4.N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway, and J. Liang, “Convolutional neural networks for medical image analysis: Full training or fine tuning?” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1299–1312, May 2016.
5.J. Shi, S. Zhou, X. Liu, Q. Zhang, M. Lu, and T. Wang, “Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset,” Neurocomputing, vol. 194, pp. 87–94, 2016.
6.A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
7.I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
8.A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.
9.E. L. Denton, S. Chintala, R. Fergus et al., “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, 2015, pp. 1486–1494.
10.M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014.
11.A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” arXiv preprint arXiv:1610.09585, 2016.
12.P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” arXiv preprint arXiv:1611.07004, 2016.
13.R. Yeh, C. Chen, T. Y. Lim, M. Hasegawa-Johnson, and M. N. Do, “Semantic image inpainting with perceptual and contextual losses,” arXiv preprint arXiv:1607.07539, 2016.
14.P. Costa, A. Galdran, M. I. Meyer, M. D. Abramoff, M. Niemeijer, A. M. Mendonça, and A. Campilho, “Towards adversarial retinal image synthesis,” arXiv preprint arXiv:1701.08974, 2017.
15.W. Dai, J. Doyle, X. Liang, H. Zhang, N. Dong, Y. Li, and E. P. Xing, “Scan: Structure correcting adversarial network for chest x-rays organ segmentation,” arXiv preprint arXiv:1703.08770, 2017.
16.Y. Xue, T. Xu, H. Zhang, R. Long, and X. Huang, “Segan: Adversarial network with multi-scale $l\_1$ loss for medical image segmentation,” arXiv preprint arXiv:1706.01805, 2017.
17.D. Nie, R. Trullo, C. Petitjean, S. Ruan, and D. Shen, “Medical image synthesis with context-aware generative adversarial networks,” arXiv preprint arXiv:1612.05362, 2016.
18.T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs, “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” in International Conference on Information Processing in Medical Imaging. Springer, 2017, pp. 146–157.
19.V. Alex, M. S. KP, S. S. Chennamsetty, and G. Krishnamurthi, “Generative adversarial networks for brain lesion detection,” in SPIE Medical Imaging. International Society for Optics and Photonics, 2017, pp. 101 330G–101 330G.
20.A. Ben-Cohen, E. Klang, S. P. Raskin, M. M. Amitai, and H. Greenspan, “Virtual pet images from ct data using deep convolutional networks: Initial results,” arXiv preprint arXiv:1707.09585, 2017.
21.National cancer institute. [Online]. Available: https://www.cancer.gov/types/metastatic-cancer
22.J. Ferlay, I. Soerjomataram, R. Dikshit, S. Eser, C. Mathers, M. Rebelo, D. M. Parkin, D. Forman, and F. Bray, “Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012,” International journal of cancer, vol. 136, no. 5, 2015.
23.T. Murakami, Y. Imai, M. Okada, T. Hyodo, W.-J. Lee, M.-J. Kim, T. Kim, and B. I. Choi, “Ultrasonography, computed tomography and magnetic resonance imaging of hepatocellular carcinoma: toward improved treatment decisions,” Oncology, vol. 81, no. Suppl. 1, pp. 86–99, 2011.
24.M. Gletsos, S. G. Mougiakakou, G. K. Matsopoulos, K. S. Nikita, A. S. Nikita, and D. Kelekis, “A computer-aided diagnostic system to characterize ct focal liver lesions: design and optimization of a neural network classifier,” IEEE Transactions on Information Technology in Biomedicine, vol. 7, no. 3, pp. 153–162, Sept 2003.
25.A. Adcock, D. Rubin, and G. Carlsson, “Classification of hepatic lesions using the matching metric,” Computer vision and image understanding, vol. 121, pp. 36–42, 2014.
26.C.-C. Chang, H.-H. Chen, Y.-C. Chang, M.-Y. Yang, C.-M. Lo, W.-C. Ko, Y.-F. Lee, K.-L. Liu, and R.-F. Chang, “Computer-aided diagnosis of liver tumors on computed tomography images,” Computer Methods and Programs in Biomedicine, vol. 145, pp. 45–51, 2017.
27.M. Bilello, S. B. Gokturk, T. Desser, S. Napel, R. B. Jeffrey, and C. F. Beaulieu, “Automatic detection and classification of hypodense hepatic lesions on contrast-enhanced venous-phase ct,” Medical physics, vol. 31, no. 9, pp. 2584–2593, 2004.
28.S. G. Mougiakakou, I. K. Valavanis, A. Nikita, and K. S. Nikita, “Differential diagnosis of ct focal liver lesions using texture features, feature selection and ensemble driven classifiers,” Artificial Intelligence in Medicine, vol. 41, no. 1, pp. 25–37, 2007.
29.I. Diamant, A. Hoogi, C. F. Beaulieu, M. Safdari, E. Klang, M. Amitai, H. Greenspan, and D. L. Rubin, “Improved patch-based automated liver lesion classification by separate analysis of the interior and boundary regions,” IEEE Journal of Biomedical and Health Informatics, vol. 20, no. 6, pp. 1585–1594, Nov 2016.
30.I. Diamant, E. Klang, M. Amitai, E. Konen, J. Goldberger, and H. Greenspan, “Task-driven dictionary learning based on mutual information for medical image classification,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 6, pp. 1380–1392, June 2017.
31.S. A. Napel, C. F. Beaulieu, C. Rodriguez, J. Cui, J. Xu, A. Gupta, D. Korenblum, H. Greenspan, Y. Ma, and D. L. Rubin, “Automated retrieval of ct images of liver lesions on the basis of image similarity: Method and preliminary results,” Radiology, vol. 256, no. 1, pp. 243–252, 2010.
32.H. C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers, “Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285–1298, May 2016.
33.A. A. A. Setio, F. Ciompi, G. Litjens, P. Gerke, C. Jacobs, S. J. van Riel, M. M. W. Wille, M. Naqibullah, C. I. Snchez, and B. van Ginneken, “Pulmonary nodule detection in ct images: False positive reduction using multi-view convolutional networks,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1160–1169, May 2016.
34.N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting.” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
35.Y. Nesterov, “A method for unconstrained convex minimization problem with the rate of convergence o (1/k2),” in Doklady an SSSR, vol. 269, no. 3, 1983, pp. 543–547.
36.T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” in Advances in Neural Information Processing Systems, 2016, pp. 2234–2242.
37.S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International Conference on Machine Learning, 2015, pp. 448–456.
38.D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
39.F. Chollet et al. (2015) Keras. [Online]. Available: https://github.com/fchollet/keras
40.M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin et al., “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint arXiv:1603.04467, 2016.
41.L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of Machine Learning Research, vol. 9, no. Nov, pp. 2579–2605, 2008.

Citation

MLA

Frid-Adar, M., et al. “GAN-based Synthetic Medical Image Augmentation for Increased CNN Performance in Liver Lesion Classification”. Neurocomputing, vol. 321, 2018, pp. 321–31, https://doi.org/10.1016/j.neucom.2018.09.013.

APA

Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018). GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 321, 321–331. https://doi.org/10.1016/j.neucom.2018.09.013

Chicago

Frid-Adar, M., I. Diamant, E. Klang, M. Amitai, J. Goldberger, and H. Greenspan. 2018. “GAN-based Synthetic Medical Image Augmentation for Increased CNN Performance in Liver Lesion Classification”. Neurocomputing 321: 321–31. https://doi.org/10.1016/j.neucom.2018.09.013.

Harvard

Frid-Adar, M. et al. (2018) “GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification”, Neurocomputing, 321, pp. 321–331. Available at: https://doi.org/10.1016/j.neucom.2018.09.013.

Vancouver

1. Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321:321–331

BibTeX

@article{Frid_Adar_2018, title={GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification}, volume={321}, ISSN={0925-2312}, url={http://dx.doi.org/10.1016/j.neucom.2018.09.013}, DOI={10.1016/j.neucom.2018.09.013}, journal={Neurocomputing}, publisher={Elsevier BV}, author={Frid-Adar, Maayan and Diamant, Idit and Klang, Eyal and Amitai, Michal and Goldberger, Jacob and Greenspan, Hayit}, year={2018}, month=Dec, pages={321–331} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Evaluator	Classification Accuracy			Real vs Fake Detection
	Real ROIs	Synthetic ROIs	Total Score	Total Score
Expert 1	78.0%	77.5%	235/302 (77.8%)	189/302 (62.5%)
Expert 2	69.2%	69.2%	209/302 (69.2%)	177/302 (58.6%)

GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Two-Stage GAN-Based Synthetic Data Augmentation Framework for Small Medical Datasets

Knowl 2 — Classification Performance Across Augmentation Schemes and Lesion Categories

Knowl 3 — DCGAN Architecture and Hyperparameters for Liver Lesion ROI Synthesis

Knowl 4 — CNN Architecture and Training Procedure for Focal Liver Lesion Classification

Knowl 5 — Expert Radiologist Assessment of Synthesized Liver Lesion Realism and Class Distinctiveness

Knowl 6 — Performance Comparison Between Generative Augmentation Models (DCGAN vs ACGAN)

Knowl 7 — Domain-Preserving Classical Affine Augmentation for CT Lesion ROIs

Knowl 8 — Auxiliary Classifier GAN (ACGAN) Loss Formulation for Multi-Class Lesion Synthesis

Knowl 9 — t-SNE Feature Space Separation with Synthetic Augmentation

Knowl 10 — Methodological and Structural Limitations of the GAN-Based Lesion Augmentation Approach

References

Citation

BibTeX

Access the Paper

Method	Sensitivity	Specificity
CNN-AUG-GAN (DCGAN)	85.7%	92.4%
CNN-AUG-GAN (ACGAN)	81.3%	90.0%
ACGAN Discriminator (Direct)	79.1%	88.8%