Deep One-Class Classification

Lukas RuffNico GörnitzLucas DeeckeShoaib Ahmed SiddiquiRobert A. VandermeulenAlexander BinderEmmanuel MüllerMarius Kloft

article2018ICML2,796 citations

Introduces Deep Support Vector Data Description (Deep SVDD), a direct one-class classification objective that trains neural networks to enclose normal representations within a minimal-volume hypersphere while establishing architectural principles to prevent hypersphere collapse.

Listen

The article addresses the challenge of anomaly detection in high-dimensional data, a task critical for applications such as cybersecurity intrusion detection, fraud monitoring, and medical diagnosis. As datasets grow in scale and complexity, classical kernel-based methods like the One-Class SVM and Kernel Density Estimation struggle with computational scalability and the curse of dimensionality, while existing deep learning approaches rely on indirect objectives such as reconstruction error from autoencoders or generative adversarial networks rather than optimizing directly for anomaly detection.

The article sets out to develop and evaluate a new method called Deep Support Vector Data Description (Deep SVDD) that trains a neural network end-to-end to minimize the volume of a hypersphere enclosing the representations of normal data points. This approach draws on kernel-based one-class classification ideas but adapts them to deep networks, with theoretical analysis ensuring the network avoids trivial constant mappings.

The method was tested through one-class classification experiments on the MNIST and CIFAR-10 image datasets, creating ten setups per dataset where one class represents normality, and through detection of adversarial examples generated via Boundary Attack on the GTSRB stop-sign dataset. Performance was measured using AUC against baselines including shallow OC-SVM/SVDD, KDE, Isolation Forest, deep convolutional autoencoders, and AnoGAN, with networks initialized via pre-training and optimized using Adam.

Deep SVDD achieved the highest average AUC on MNIST across all classes and delivered competitive or superior results on CIFAR-10, with the one-class variant slightly outperforming the soft-boundary version. On the adversarial detection task it again led with an AUC of 80.3 percent. The method requires no storage of training data for inference and scales linearly with dataset size via stochastic gradient descent.

These results indicate that directly optimizing a compact hypersphere objective allows neural networks to extract common factors of variation more effectively for anomaly scoring than reconstruction-based heuristics, lowering memory use and enabling faster deployment in operational monitoring systems. The ν-property proven for the soft-boundary objective further permits explicit control over the expected outlier fraction.

The article recommends initializing the hypersphere center as the mean of an initial forward pass and removing bias terms or bounded activations to prevent collapse to trivial solutions. Further validation on additional domains and larger-scale datasets would strengthen confidence before broad adoption. The main limitations are elevated variance from stochastic optimization and sensitivity to network architecture choice, suggesting caution when applying the method without architecture tuning or when training data is very limited.

German Research Center for Artificial Intelligence Hasso Plattner Institute Humboldt University of Berlin Singapore University of Technology and Design Technische Universität Berlin Technische Universität Kaiserslautern University of Edinburgh anomaly detection methods deep learning generative model deep one-class classification Deep SVDD anomaly detection

lukasruff/Deep-SVDD-PyTorch Ruff et al (2018).pdf

Paper: Support Vector Method for Novelty Detection, B. Schölkopf et al. (1999). This foundational paper introduces kernel-based one-class classification and the ν-parameter boundary formulation that Deep SVDD directly adapts into deep neural network architectures.
Paper: Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery, Thomas Schlegl et al. (2017). This paper establishes the AnoGAN framework for deep anomaly detection, which serves as a primary generative baseline and conceptual predecessor evaluated in the source paper.
Paper: Isolation-Based Anomaly Detection, Fei Tony Liu et al. (2012). This work presents Isolation Forest, providing essential context on classic anomaly isolation algorithms evaluated as key baselines against Deep SVDD.
Paper: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks, Dan Hendrycks et al. (2017). This work establishes standard baseline methodologies and metrics for evaluating out-of-distribution and anomaly detection performance in neural networks.
Paper: Extracting and composing robust features with denoising autoencoders, Pascal Vincent et al. (2008). This paper provides the underlying methodology for autoencoder-based representation learning and pre-training used to initialize Deep SVDD networks and avoid representational collapse.
Paper: Support-vector networks, Corinna Cortes et al. (1995). This classic paper details support vector optimization principles that form the mathematical underpinning of support vector data description.

Paper: Deep Anomaly Detection with Outlier Exposure, Dan Hendrycks et al. (2019). This paper extends deep anomaly detection methodology by exposing networks to auxiliary outlier datasets during training to significantly enhance out-of-distribution boundaries.
Paper: Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection, Bo Zong et al. (2018). This work presents an alternative end-to-end deep framework combining latent compression with Gaussian mixture density estimation for unsupervised anomaly detection.
Paper: Energy-based Out-of-distribution Detection, Weitang Liu et al. (2020). This study advances out-of-distribution anomaly scoring by introducing energy-based objectives that improve upon distance- and density-based anomaly boundaries.
Paper: Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift, Stephan Rabanser et al. (2019). This empirical study investigates practical methods to detect dataset shift and distribution anomalies across high-dimensional benchmarks like MNIST and CIFAR-10.

Abstract

Despite the great advances made by deep learning in many machine learning problems, there is a relative dearth of deep learning approaches for anomaly detection. Those approaches which do exist involve networks trained to perform a task other than anomaly detection, namely generative models or compression, which are in turn adapted for use in anomaly detection; they are not trained on an anomaly detection based objective. In this paper we introduce a new anomaly detection method—Deep Support Vector Data Description—, which is trained on an anomaly detection based objective. The adaptation to the deep regime necessitates that our neural network and training procedure satisfy certain properties, which we demonstrate theoretically. We show the effectiveness of our method on MNIST and CIFAR-10 image benchmark datasets as well as on the detection of adversarial examples of GTSRB stop signs.

Topics

anomaly detection methods deep learning generative model support vector vector data detection algorithm anomaly detector anomaly-detection algorithms classification models convolutional neural networks

Keywords

deep one-class classification Deep SVDD anomaly detection outlier detection representation learning kernel density estimation adversarial examples global contrast normalization

1. Introduction
2. Related Work
2.1. Kernel-based One-Class Classification
2.2. Deep Approaches to Anomaly Detection
3. Deep SVDD
3.1. The Deep SVDD Objective
3.2. Optimization of Deep SVDD
3.3. Properties of Deep SVDD
4. Experiments
4.1. Competing methods
4.2. One-class classification on MNIST and CIFAR-10
4.3. Adversarial attacks on GTSRB stop signs
5. Conclusion
Acknowledgements
References

Knowls

Knowl 1 — Soft-Boundary Deep Support Vector Data Description Objective
model/method
Soft-Boundary Deep Support Vector Data Description (Deep SVDD) is an unsupervised anomaly detection method that jointly learns neural network representations while minimizing the volume of a data-enclosing hypersphere in the output space.

Let $\mathcal{X} \subseteq \mathbb{R}^d$ be the input space and $\mathcal{F} \subseteq \mathbb{R}^p$ be the output representation space. Let $\phi(\cdot; \mathcal{W}): \mathcal{X} \to \mathcal{F}$ denote a neural network with $L \in \mathbb{N}$ hidden layers and weight parameters $\mathcal{W} = \{W^1, \dots, W^L\}$ , where $W^\ell$ denotes the weight matrix of layer $\ell \in \{1, \dots, L\}$ . Given a training dataset $\mathcal{D}_n = \{x_1, \dots, x_n\} \subset \mathcal{X}$ , a fixed hypersphere center $c \in \mathcal{F}$ , a trade-off hyperparameter $\nu \in (0, 1]$ , and a weight decay hyperparameter $\lambda > 0$ , the soft-boundary Deep SVDD objective is given by

$\min_{R, \mathcal{W}} R^2 + \frac{1}{\nu n} \sum_{i=1}^n \max\left\{0, \|\phi(x_i; \mathcal{W}) - c\|^2 - R^2\right\} + \frac{\lambda}{2} \sum_{\ell=1}^L \|W^\ell\|_F^2$

where $R > 0$ is the hypersphere radius and $\|\cdot\|_F$ denotes the Frobenius norm.

For a test point $x \in \mathcal{X}$ and trained network parameters $\mathcal{W}^*$ and radius $R^*$ , the anomaly score $s(x)$ can be defined as

$s(x) = \|\phi(x; \mathcal{W}^*) - c\|^2 - (R^*)^2$

where positive scores indicate anomalous points mapped outside the hypersphere, while negative scores indicate normal inliers falling within the hypersphere.
Knowl 2 — One-Class Deep Support Vector Data Description Objective
model/method
One-Class Deep Support Vector Data Description (One-Class Deep SVDD) is a simplified variant of Deep SVDD designed for scenarios where the training dataset is assumed to contain exclusively or predominantly normal (non-anomalous) samples.

Let $\mathcal{X} \subseteq \mathbb{R}^d$ be the input space and $\mathcal{F} \subseteq \mathbb{R}^p$ be the output space. Let $\phi(\cdot; \mathcal{W}): \mathcal{X} \to \mathcal{F}$ be a neural network parameterized by weights $\mathcal{W} = \{W^1, \dots, W^L\}$ across $L$ layers. Given training data $\mathcal{D}_n = \{x_1, \dots, x_n\} \subset \mathcal{X}$ , a fixed center $c \in \mathcal{F}$ , and a weight decay hyperparameter $\lambda > 0$ , the One-Class Deep SVDD objective minimizes the mean squared Euclidean distance of the data representations to $c$ :

$\min_{\mathcal{W}} \frac{1}{n} \sum_{i=1}^n \|\phi(x_i; \mathcal{W}) - c\|^2 + \frac{\lambda}{2} \sum_{\ell=1}^L \|W^\ell\|_F^2$

where $\|\cdot\|_F$ is the Frobenius norm.

Instead of optimizing a radius parameter $R$ , the hypersphere is contracted by minimizing the mean distance of all training sample embeddings to the center $c$ . For a test sample $x \in \mathcal{X}$ , the anomaly score is given directly by the squared distance to the center:

$s(x) = \|\phi(x; \mathcal{W}^*) - c\|^2$

where $\mathcal{W}^*$ denotes the optimal trained network weights.
Knowl 3 — Hypersphere Collapse via Learnable Center and All-Zero Weights
theoretical result
Let $\mathcal{W}_0$ denote the set of all-zero network weights, i.e., $W^\ell = 0$ for every layer $\ell \in \{1, \dots, L\}$ . For any input $x \in \mathcal{X}$ , this parameter configuration causes the neural network to output a constant value $c_0 := \phi(x; \mathcal{W}_0)$ . If the hypersphere center $c \in \mathcal{F}$ is treated as a free optimization variable or fixed to $c = c_0$ :
1. For both soft-boundary and One-Class Deep SVDD objectives, the optimal solution collapses to $\mathcal{W}^* = \mathcal{W}_0$ , $R^* = 0$ , and $c^* = c_0$ , yielding an empirical loss of zero.
2. This failure mode is termed hypersphere collapse, in which the network learns a trivial constant function mapping all inputs directly to the hypersphere center.
To prevent hypersphere collapse, the hypersphere center $c$ must be fixed before optimization begins and set such that $c \neq c_0$ (e.g., $c \neq 0$ for networks using ReLU activations). A standard practical strategy is to set $c$ to the empirical mean of the representations produced by performing an initial forward pass on a batch of training data.
Knowl 4 — Hypersphere Collapse via Neural Network Bias Terms
theoretical result
If any hidden layer $\ell \in \{1, \dots, L\}$ in the neural network $\phi(\cdot; \mathcal{W}): \mathcal{X} \to \mathcal{F}$ contains a learnable bias term $b^\ell$ , there exists a trivial global optimum for both soft-boundary and One-Class Deep SVDD with radius $R^* = 0$ and $\phi(x; \mathcal{W}^*) = c$ for all inputs $x \in \mathcal{X}$ , for any fixed center $c \in \mathcal{F}$ .

Setting the incoming weight matrix $W^\ell = 0$ produces a layer output $z^\ell(x) = \sigma^\ell(b^\ell)$ that is constant for every input $x \in \mathcal{X}$ . The bias vector $b^\ell$ and downstream layer weights can then be configured to map this constant representation directly to $c$ . Consequently, neural networks used in Deep SVDD must omit bias terms in all hidden and output units to avoid hypersphere collapse.
Knowl 5 — Hypersphere Collapse via Bounded Activation Functions
theoretical result
Consider a neural network unit with a monotonic activation function $\sigma(\cdot)$ that is upper bounded by $\sup_z \sigma(z) \neq 0$ or lower bounded by $\inf_z \sigma(z) \neq 0$ . For any finite set of inputs $\{z_1, \dots, z_n\}$ sharing at least one feature with a constant non-zero sign across all samples, the unit can be driven into saturation by scaling the corresponding weight arbitrarily large while setting all other input weights to zero.

This saturation uniformly approximates the non-zero supremum (or infimum) across all inputs, effectively emulating a constant bias term in the subsequent layer and enabling hypersphere collapse. Therefore, unbounded activation functions (or activations bounded only at zero, such as ReLU and Leaky ReLU) must be preferred over bounded activations (such as sigmoid or tanh) in Deep SVDD architectures.
Knowl 6 — Upper and Lower Outlier Fraction Bounds via the nu-Property
theoretical result
For the soft-boundary Deep SVDD objective parameterized by $\nu \in (0, 1]$ :

$\min_{R, \mathcal{W}} R^2 + \frac{1}{\nu n} \sum_{i=1}^n \max\left\{0, \|\phi(x_i; \mathcal{W}) - c\|^2 - R^2\right\} + \frac{\lambda}{2} \sum_{\ell=1}^L \|W^\ell\|_F^2$

the parameter $\nu$ satisfies the $\nu$ -property:
1. $\nu$ is an upper bound on the fraction of outliers in the training set, meaning that at the optimal radius $R^*$ , the number of outliers $n_{\text{out}} = |\{i : \|\phi(x_i; \mathcal{W}) - c\|^2 > (R^*)^2\}|$ satisfies $\frac{n_{\text{out}}}{n} \le \nu$ .
2. $\nu$ is a lower bound on the fraction of training samples that lie either on or outside the boundary of the hypersphere, i.e., $\frac{1}{n} |\{i : \|\phi(x_i; \mathcal{W}) - c\|^2 \ge (R^*)^2\}| \ge \nu$ .
Knowl 7 — Deep SVDD Optimization and Training Procedure
model/method
Training Deep SVDD models involves the following procedure:
1. Network Design: Construct a neural network without bias terms in any layer and using unbounded activation functions (e.g., Leaky ReLU with leakiness $\alpha = 0.1$ ) to prevent hypersphere collapse.
2. Pre-training: Initialize the network encoder parameters using weights pre-trained via a Deep Convolutional Autoencoder (DCAE) trained with mean squared error loss.
3. Center Fixing: Set the hypersphere center $c \in \mathcal{F}$ to the empirical mean of the output representations obtained from an initial forward pass on the training data using the pre-trained weights, and keep $c$ fixed throughout training.
4. Optimization Scheme:
  - For One-Class Deep SVDD, update network parameters $\mathcal{W}$ end-to-end via stochastic gradient descent (e.g., Adam) using backpropagation.
  - For Soft-Boundary Deep SVDD, employ alternating minimization / block coordinate descent: train network weights $\mathcal{W}$ for $k$ epochs (e.g., $k = 5$ ) with $R$ fixed, then update the radius $R$ via a line search over the current data representations $\{\phi(x_i; \mathcal{W})\}_{i=1}^n$ , repeating until convergence.

Knowl 8 — One-Class Classification Performance on MNIST and CIFAR-10 Image Benchmarks

data/table

The table below reports the average Area Under the ROC Curve (AUC in %) and standard deviations over 10 random seeds for one-class anomaly detection on MNIST and CIFAR-10. Models are trained only on images from the specified normal class ( $n \approx 6000$ for MNIST, $n = 5000$ for CIFAR-10) and evaluated on 10,000 test images where the other nine classes serve as anomalies. Pre-processing consists of global contrast normalization via the $L^1$ -norm and min-max scaling to $[0, 1]$ .

Normal Class	OC-SVM/SVDD	KDE	IF	DCAE	AnoGAN	Soft-Bound. Deep SVDD	One-Class Deep SVDD
0	98.6 $\pm$ 0.0	97.1 $\pm$ 0.0	98.0 $\pm$ 0.3	97.6 $\pm$ 0.7	96.6 $\pm$ 1.3	97.8 $\pm$ 0.7	98.0 $\pm$ 0.7
1	99.5 $\pm$ 0.0	98.9 $\pm$ 0.0	97.3 $\pm$ 0.4	98.3 $\pm$ 0.6	99.2 $\pm$ 0.6	99.6 $\pm$ 0.1	99.7 $\pm$ 0.1
2	82.5 $\pm$ 0.1	79.0 $\pm$ 0.0	88.6 $\pm$ 0.5	85.4 $\pm$ 2.4	85.0 $\pm$ 2.9	89.5 $\pm$ 1.2	91.7 $\pm$ 0.8
3	88.1 $\pm$ 0.0	86.2 $\pm$ 0.0	89.9 $\pm$ 0.4	86.7 $\pm$ 0.9	88.7 $\pm$ 2.1	90.3 $\pm$ 2.1	91.9 $\pm$ 1.5
4	94.9 $\pm$ 0.0	87.9 $\pm$ 0.0	92.7 $\pm$ 0.6	86.5 $\pm$ 2.0	89.4 $\pm$ 1.3	93.8 $\pm$ 1.5	94.9 $\pm$ 0.8
5	77.1 $\pm$ 0.0	73.8 $\pm$ 0.0	85.5 $\pm$ 0.8	78.2 $\pm$ 2.7	88.3 $\pm$ 2.9	85.8 $\pm$ 2.5	88.5 $\pm$ 0.9
6	96.5 $\pm$ 0.0	87.6 $\pm$ 0.0	95.6 $\pm$ 0.3	94.6 $\pm$ 0.5	94.7 $\pm$ 2.7	98.0 $\pm$ 0.4	98.3 $\pm$ 0.5
7	93.7 $\pm$ 0.0	91.4 $\pm$ 0.0	92.0 $\pm$ 0.4	92.3 $\pm$ 1.0	93.5 $\pm$ 1.8	92.7 $\pm$ 1.4	94.6 $\pm$ 0.9
8	88.9 $\pm$ 0.0	79.2 $\pm$ 0.0	89.9 $\pm$ 0.4	86.5 $\pm$ 1.6	84.9 $\pm$ 2.1	92.9 $\pm$ 1.4	93.9 $\pm$ 1.6
9	93.1 $\pm$ 0.0	88.2 $\pm$ 0.0	93.5 $\pm$ 0.3	90.4 $\pm$ 1.8	92.4 $\pm$ 1.1	94.9 $\pm$ 0.6	96.5 $\pm$ 0.3
Airplane	61.6 $\pm$ 0.9	61.2 $\pm$ 0.0	60.1 $\pm$ 0.7	59.1 $\pm$ 5.1	67.1 $\pm$ 2.5	61.7 $\pm$ 4.2	61.7 $\pm$ 4.1
Automobile	63.8 $\pm$ 0.6	64.0 $\pm$ 0.0	50.8 $\pm$ 0.6	57.4 $\pm$ 2.9	54.7 $\pm$ 3.4	64.8 $\pm$ 1.4	65.9 $\pm$ 2.1
Bird	50.0 $\pm$ 0.5	50.1 $\pm$ 0.0	49.2 $\pm$ 0.4	48.9 $\pm$ 2.4	52.9 $\pm$ 3.0	49.5 $\pm$ 1.4	50.8 $\pm$ 0.8
Cat	55.9 $\pm$ 1.3	56.4 $\pm$ 0.0	55.1 $\pm$ 0.4	58.4 $\pm$ 1.2	54.5 $\pm$ 1.9	56.0 $\pm$ 1.1	59.1 $\pm$ 1.4
Deer	66.0 $\pm$ 0.7	66.2 $\pm$ 0.0	49.8 $\pm$ 0.4	54.0 $\pm$ 1.3	65.1 $\pm$ 3.2	59.1 $\pm$ 1.1	60.9 $\pm$ 1.1
Dog	62.4 $\pm$ 0.8	62.4 $\pm$ 0.0	58.5 $\pm$ 0.4	62.2 $\pm$ 1.8	60.3 $\pm$ 2.6	62.1 $\pm$ 2.4	65.7 $\pm$ 2.5
Frog	74.7 $\pm$ 0.3	74.9 $\pm$ 0.0	42.9 $\pm$ 0.6	51.2 $\pm$ 5.2	58.5 $\pm$ 1.4	67.8 $\pm$ 2.4	67.7 $\pm$ 2.6
Horse	62.6 $\pm$ 0.6	62.6 $\pm$ 0.0	55.1 $\pm$ 0.7	58.6 $\pm$ 2.9	62.5 $\pm$ 0.8	65.2 $\pm$ 1.0	67.3 $\pm$ 0.9
Ship	74.9 $\pm$ 0.4	75.1 $\pm$ 0.0	74.2 $\pm$ 0.6	76.8 $\pm$ 1.4	75.8 $\pm$ 4.1	75.6 $\pm$ 1.7	75.9 $\pm$ 1.2
Truck	75.9 $\pm$ 0.3	76.0 $\pm$ 0.0	58.9 $\pm$ 0.7	67.3 $\pm$ 3.0	66.5 $\pm$ 2.8	71.0 $\pm$ 1.1	73.1 $\pm$ 1.2

Deep SVDD consistently outperforms both shallow baselines (OC-SVM/SVDD, KDE, Isolation Forest) and deep baselines (DCAE, AnoGAN) on MNIST. On CIFAR-10, One-Class Deep SVDD achieves the strongest performance across several classes, but shallow KDE/SVDD outperform deep methods on classes characterized by strong global color/spatial structures (e.g., Deer, Frog, Truck), highlighting the influence of CNN inductive biases.

Knowl 9 — Detection of Adversarial Examples on GTSRB Stop Signs

data/table

The table below presents the detection performance (AUC in % $\pm$ standard deviation over 10 seeds) on the German Traffic Sign Recognition Benchmark (GTSRB) stop signs dataset. The model is trained on $n = 780$ normal stop sign images and evaluated on a test set comprising 270 normal stop sign images and 20 adversarial examples generated via Boundary Attack.

One-Class Deep SVDD achieves the highest detection AUC (80.3%), outperforming shallow methods and DCAE. AnoGAN fails to converge on this dataset due to the small sample size ( $n = 780$ ), which is insufficient to train the underlying generative adversarial network.

Coverage note — No substantial contributed material was omitted. All objectives, architectural constraints, collapse proofs/theorems, optimization schemes, and empirical benchmark evaluations are covered.

References

1.Aggarwal, C. Outlier Analysis. Springer, 2nd edition, 2016.
2.An, J. and Cho, S. Variational Autoencoder based Anomaly Detection using Reconstruction Probability. SNU Data Mining Center, Tech. Rep., 2015.
3.Andrews, J. T. A., Morton, E. J., and Griffin, L. D. Detecting Anomalous Data Using Auto-Encoders. IJMLC, 6(1):21, 2016.
4.Aronszajn, N. Theory of reproducing kernels. Transactions of the American mathematical society, 68(3):337–404, 1950.
5.Bengio, Y., Courville, A., and Vincent, P. Representation Learning: A Review and New Perspectives. IEEE TPAMI, 35(8):1798–1828, 2013.
6.Brendel, W., Rauber, J., and Bethge, M. Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models. In ICLR, 2018.
7.Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. LOF: Identifying Density-Based Local Outliers. In SIG-MOD Record, volume 29, pp. 93–104, 2000.
8.Chandola, V., Banerjee, A., and Kumar, V. Anomaly Detection: A Survey. ACM Computing Surveys, 41(3):1–58, 2009.
9.Chen, J., Sathe, S., Aggarwal, C., and Turaga, D. Outlier Detection with Autoencoder Ensembles. In SDM, pp. 90–98, 2017.
10.Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P. Natural Language Processing (Almost) from Scratch. JMLR, 12(Aug):2493–2537, 2011.
11.Emmott, A., Das, A., Dietterich, T., Fern, A., and Wong, W.-K. Anomaly detection meta-analysis benchmarks. 2016.
12.Erfani, S. M., Rajasegarar, S., Karunasekera, S., and Leckie, C. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognition, 58:121–134, 2016.
13.Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., and Vázquez, E. Anomaly-based network intrusion detection: Techniques, systems and challenges. Computers & Security, 28(1-2):18–28, 2009.
14.Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, pp. 249–256, 2010.
15.Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative Adversarial Nets. In NIPS, pp. 2672–2680, 2014.
16.Goodfellow, I., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In ICLR, 2015.
17.Hawkins, S., He, H., Williams, G., and Baxter, R. Outlier Detection Using Replicator Neural Networks. In DaWaK, volume 2454, pp. 170–180, 2002.
18.He, K., Zhang, X., Ren, S., and Sun, J. Deep Residual Learning for Image Recognition. In CVPR, June 2016.
19.Hinton, G. E. and Salakhutdinov, R. R. Reducing the Dimensionality of Data with Neural Networks. Science, 313 (5786):504–507, 2006.
20.Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. Deep Neural Networks for Acoustic Modeling in Speech Recognition. IEEE Signal Processing Magazine, 29(6):82–97, 2012.
21.Ioffe, S. and Szegedy, C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML, pp. 448–456, 2015.
22.Kingma, D. and Ba, J. Adam: A Method for Stochastic Optimization. arXiv:1412.6980, 2014.
23.Kingma, D. P. and Welling, M. Auto-Encoding Variational Bayes. In ICLR, 2013.
24.Krizhevsky, A. and Hinton, G. E. Learning multiple layers of features from tiny images. 2009.
25.Krizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks. In NIPS, pp. 1090–1098, 2012.
26.Lavin, A. and Ahmad, S. Evaluating Real-time Anomaly Detection Algorithms — the Numenta Anomaly Benchmark. In 14th ICMLA, pp. 38–44, 2015.
27.LeCun, Y., Cortes, C., and Burges, C. MNIST handwritten digit database. AT&T Labs, 2, 2010.
28.LeCun, Y., Bengio, Y., and Hinton, G. E. Deep Learning. Nature, 521(7553):436–444, 2015.
29.Liu, F. T., Ting, K. M., and Zhou, Z.-H. Isolation Forest. In ICDM, pp. 413–422, 2008.
30.Makhzani, A. and Frey, B. K-sparse Autoencoders. arXiv:1312.5663, 2013.
31.Makhzani, A. and Frey, B. J. Winner-Take-All Autoencoders. In NIPS, pp. 2791–2799, 2015.
32.Masci, J., Meier, U., Cireșan, D., and Schmidhuber, J. Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. ICANN, pp. 52–59, 2011.
33.Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J. Unrolled Generative Adversarial Networks. In ICLR, 2017.
34.Moya, M. M., Koch, M. W., and Hostetler, L. D. One-class classifier networks for target recognition applications. In Proceedings World Congress on Neural Networks, pp. 797–801, 1993.
35.Pal, M. and Foody, G. M. Feature selection for classification of hyperspectral data by SVM. IEEE Transactions on Geoscience and Remote Sensing, 48(5):2297–2307, 2010.
36.Parzen, E. On Estimation of a Probability Density Function and Mode. The annals of mathematical statistics, 33(3): 1065–1076, 1962.
37.Phua, C., Lee, V., Smith, K., and Gayler, R. A Comprehensive Survey of Data Mining-based Fraud Detection Research. Clayton School of Information Technology, Monash University, Tech. Rep., 2005.
38.Platt, J. Sequential minimal optimization: A fast algorithm for training support vector machines. 1998.
39.Radford, A., Metz, L., and Chintala, S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv:1511.06434, 2015.
40.Rahimi, A. and Recht, B. Random features for large-scale kernel machines. In NIPS, 2007.
41.Richter, C. and Roy, N. Safe Visual Navigation via Deep Learning and Novelty Detection. In Robotics: Science and Systems Conference, 2017.
42.Sabokrou, M., Fayyaz, M., Fathy, M., et al. Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes. arXiv:1609.00866, 2016.
43.Sakurada, M. and Yairi, T. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the 2nd MLSDA Workshop, pp. 4, 2014.
44.Salem, O., Guerassimov, A., Mehaoua, A., Marcus, A., and Furht, B. Sensor Fault and Patient Anomaly Detection and Classification in Medical Wireless Sensor Networks. In ICC, pp. 4373–4378, 2013.
45.Schlegl, T., Seeböck, P., Waldstein, S. M., Schmidt-Erfurth, U., and Langs, G. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In IPMI, pp. 146–157, 2017.
46.Schmidhuber, J. Deep Learning in Neural Networks: An Overview. Neural networks, 61:85–117, 2015.
47.Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., and Williamson, R. C. Estimating the Support of a High-Dimensional Distribution. Neural computation, 13(7): 1443–1471, 2001.
48.Seeböck, P., Waldstein, S., Klimscha, S., Gerendas, B. S., Donner, R., Schlegl, T., Schmidt-Erfurth, U., and Langs, G. Identifying and Categorizing Anomalies in Retinal Imaging Data. arXiv:1612.00686, 2016.
49.Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. The German Traffic Sign Recognition Benchmark: A multi-class classification competition. In IJCNN, pp. 1453–1460, 2011.
50.Tax, D. M. J. and Duin, R. P. W. Support Vector Data Description. Machine learning, 54(1):45–66, 2004.
51.Tsybakov, A. B. On Nonparametric Estimation of Density Level Sets. The Annals of Statistics, 25(3):948–969, 1997.
52.Vempati, S., Vedaldi, A., Zisserman, A., and Jawahar, C. Generalized RBF feature maps for Efficient Detection. In 21st BMVC, pp. 1–11, 2010.
53.Vert, R. and Vert, J.-P. Consistency and Convergence Rates of One-Class SVMs and Related Algorithms. JMLR, 7 (May):817–854, 2006.
54.Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. Extracting and Composing Robust Features with Denoising Autoencoders. In ICML, pp. 1096–1103, 2008.
55.Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., and Manzagol, P.-A. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. JMLR, 11(Dec):3371–3408, 2010.
56.Xu, D., Ricci, E., Yan, Y., Song, J., and Sebe, N. Learning Deep Representations of Appearance and Motion for Anomalous Event Detection. In BMVC, pp. 8.1–8.12, 2015.

Citation

MLA

Ruff, L., et al. “Deep One-Class Classification”. International Conference on Machine Learning, vol. 80, 2018, pp. 4393–402, https://proceedings.mlr.press/v80/ruff18a.html.

APA

Ruff, L., Vandermeulen, R., Goernitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018). Deep One-Class Classification. International Conference on Machine Learning, 80, 4393–4402. https://proceedings.mlr.press/v80/ruff18a.html

Chicago

Ruff, L., R. Vandermeulen, N. Goernitz, et al. 2018. “Deep One-Class Classification”. International Conference on Machine Learning 80: 4393–4402. https://proceedings.mlr.press/v80/ruff18a.html.

Harvard

Ruff, L. et al. (2018) “Deep One-Class Classification”, International Conference on Machine Learning. PMLR, pp. 4393–4402. Available at: https://proceedings.mlr.press/v80/ruff18a.html.

Vancouver

1. Ruff L, Vandermeulen R, Goernitz N, Deecke L, Siddiqui SA, Binder A, Müller E, Kloft M (2018) Deep One-Class Classification. In: International Conference on Machine Learning. PMLR, pp 4393–4402

BibTeX

@InProceedings{pmlr-v80-ruff18a,
  title = 	 {Deep One-Class Classification},
  author =       {Ruff, Lukas and Vandermeulen, Robert and Goernitz, Nico and Deecke, Lucas and Siddiqui, Shoaib Ahmed and Binder, Alexander and M{\"u}ller, Emmanuel and Kloft, Marius},
  booktitle = 	 {Proceedings of the 35th International Conference on Machine Learning},
  pages = 	 {4393--4402},
  year = 	 {2018},
  editor = 	 {Dy, Jennifer and Krause, Andreas},
  volume = 	 {80},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {10--15 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v80/ruff18a/ruff18a.pdf},
  url = 	 {https://proceedings.mlr.press/v80/ruff18a.html},
  abstract = 	 {Despite the great advances made by deep learning in many machine learning problems, there is a relative dearth of deep learning approaches for anomaly detection. Those approaches which do exist involve networks trained to perform a task other than anomaly detection, namely generative models or compression, which are in turn adapted for use in anomaly detection; they are not trained on an anomaly detection based objective. In this paper we introduce a new anomaly detection method—Deep Support Vector Data Description—, which is trained on an anomaly detection based objective. The adaptation to the deep regime necessitates that our neural network and training procedure satisfy certain properties, which we demonstrate theoretically. We show the effectiveness of our method on MNIST and CIFAR-10 image benchmark datasets as well as on the detection of adversarial examples of GTSRB stop signs.}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

Method	AUC (%)
OC-SVM/SVDD	67.5 $\pm$ 1.2
KDE	60.5 $\pm$ 1.7
IF	73.8 $\pm$ 0.9
DCAE	79.1 $\pm$ 3.0
AnoGAN	—
Soft-Bound. Deep SVDD	77.8 $\pm$ 4.9
One-Class Deep SVDD	80.3 $\pm$ 2.8

Deep One-Class Classification

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Soft-Boundary Deep Support Vector Data Description Objective

Knowl 2 — One-Class Deep Support Vector Data Description Objective

Knowl 3 — Hypersphere Collapse via Learnable Center and All-Zero Weights

Knowl 4 — Hypersphere Collapse via Neural Network Bias Terms

Knowl 5 — Hypersphere Collapse via Bounded Activation Functions

Knowl 6 — Upper and Lower Outlier Fraction Bounds via the nu-Property

Knowl 7 — Deep SVDD Optimization and Training Procedure