ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models

Pin-Yu ChenHuan ZhangYash SharmaJinfeng YiCho-Jui Hsieh

article2017AISec@CCS2,246 citations

Proposes a zeroth-order optimization method that directly estimates gradients from output prediction scores to generate black-box adversarial attacks on deep neural networks, achieving white-box level attack success without requiring substitute models.

Listen

Deep neural networks are increasingly deployed in mission-critical applications such as autonomous driving and malware prevention, yet they remain highly vulnerable to adversarial examples—minor, imperceptible input modifications that cause systems to fail. Evaluating model security typically relies on either white-box testing, which requires full access to internal parameters, or black-box testing that trains approximate substitute models. However, substitute models suffer from severe transferability loss and do not scale to complex, real-world networks.

The article evaluates a black-box attack method called Zeroth Order Optimization (ZOO), designed to generate targeted and untargeted adversarial examples against deep learning image classifiers by observing only input images and output confidence scores, completely removing the need to train substitute models.

The researchers formulated the attack as an optimization problem using a modified loss function and estimated numerical gradients directly from target queries using finite differences. To scale the approach to large models, they employed stochastic coordinate descent combined with three acceleration techniques: attack-space dimension reduction, hierarchical attacks that scale resolution dynamically, and importance sampling to prioritize key image regions. The approach was validated across standard benchmark datasets (MNIST and CIFAR10) and a full-scale ImageNet model (Inception-v3).

The evaluation revealed several critical findings. First, the proposed method achieves attack success rates close to 100% on standard datasets (MNIST and CIFAR10), matching the effectiveness of state-of-the-art white-box methods (Carlini & Wagner) and vastly outperforming substitute-model black-box baselines, which achieved as low as 5% to 27% on targeted tasks. Second, the added noise distortions were minimal and visually imperceptible, closely matching white-box distortion levels. Third, the method successfully scaled to the complex Inception-v3 ImageNet model, yielding an 88.9% untargeted success rate within 1,500 iterations and achieving a successful targeted attack that flipped model confidence from 97% to below 1% for the correct class while boosting the target class by over 1,000 times.

These findings demonstrate that concealing model parameters and internal architectures provides false security. Practical black-box systems exposed via public query interfaces (such as cloud prediction APIs or mobile applications) face vulnerability levels comparable to fully exposed white-box models. Organizations relying on deep neural networks in safety-critical deployments must reassess their security posture and assume that access to classification outputs is sufficient for adversaries to execute precise, highly effective attacks.

Organizations should incorporate derivative-free black-box evaluations into robust adversarial testing pipelines, use black-box attack generation to inform adversarial retraining defenses, and extend robustness verification to other data modalities, such as speech, text, and time series. While the proposed approach requires more queries and computation time than direct white-box attacks, the experimental evidence firmly demonstrates that model obscurity alone fails to protect deep neural networks from imperceptible adversarial manipulation.

AI Foundations Group IBM University of California, Davis autonomous driving classification coordinate descent Attack Adversarial Black-box

arXiv: 1708.03999 huanzhang12/ZOO-Attack

Paper: Practical Black-Box Attacks against Machine Learning, Nicolas Papernot et al. (2017). Reading this foundational study on black-box attacks via substitute models provides essential context for understanding the motivation behind replacing substitutes with zeroth-order optimization.
Paper: Towards Deep Learning Models Resistant to Adversarial Attacks, A. Ma̧dry et al. (2017). Understanding projected gradient descent and optimization landscapes is crucial for grasping how zeroth-order optimization estimates gradients directly from model outputs.

Paper: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks, Francesco Croce et al. (2020). This work builds on foundational attack methodologies by introducing an ensemble of parameter-free attacks to reliably evaluate adversarial robustness across diverse architectures.

Cover for ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models

Abstract

Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern on the robustness to adversarial examples, especially for security-critical tasks such as traffic sign identification for autonomous driving. Studies have unveiled the vulnerability of a well-trained DNN by demonstrating the ability of generating barely noticeable (to both human and machines) adversarial images that lead to misclassification. Furthermore, researchers have shown that these adversarial images are highly transferable by simply training and attacking a substitute model built upon the target model, known as a black-box attack to DNNs.

Similar to the setting of training substitute models, in this paper we propose an effective black-box attack that also only has access to the input (images) and the output (confidence scores) of a targeted DNN. However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples. We use zeroth order stochastic coordinate descent along with dimension reduction, hierarchical attack and importance sampling techniques to efficiently attack black-box models. By exploiting zeroth order optimization, improved attacks to the targeted DNN can be accomplished, sparing the need for training substitute models and avoiding the loss in attack transferability. Experimental results on MNIST, CIFAR10 and ImageNet show that the proposed ZOO attack is as effective as the state-of-the-art white-box attack and significantly outperforms existing black-box attacks via substitute models.

1 Introduction
1.1 Adversarial attacks and transferability
1.2 Black-box attacks and substitute models
1.3 Defending adversarial attacks
1.4 Black-box attack using zeroth order optimization: benefits and challenges
1.5 Contributions
2 Related Work
3 ZOO: A Black-box Attack without Training Substitute Models
3.1 Notation for deep neural networks
3.2 Formulation of C&W attack
3.3 Proposed black-box attack via zeroth order stochastic coordinate descent
3.4 Attack-space dimension reduction
3.5 Hierarchical attack
3.6 Optimize the important pixels first
4 Performance Evaluation
4.1 Setup
4.2 MNIST and CIFAR10
4.3 Inception network with ImageNet
5 Conclusion and Future Work
References

Knowls

Knowl 1 — ZOO Black-Box Adversarial Optimization and Confidence-Score Loss Formulation
model/method
The Zeroth Order Optimization (ZOO) black-box attack generates an adversarial example $x \in [0, 1]^p$ from an original benign image $x_0 \in [0, 1]^p$ by querying only the target classifier's output probabilities $F(x) \in [0, 1]^K$ , where $K$ is the number of classes, $[F(x)]_k$ denotes the predicted probability of class $k$ , and $\sum_{k=1}^K [F(x)]_k = 1$ . The attack minimizes the perturbation distance while enforcing misclassification via the optimization problem:

$\min_{x \in [0, 1]^p} \|x - x_0\|_2^2 + c \cdot f(x, t)$

where $\|\cdot\|_2$ is the Euclidean norm and $c > 0$ is a regularization parameter.

To formulate the adversarial objective $f(x, t)$ purely from output probabilities $F(x)$ without access to internal logits or gradients, ZOO uses a hinge-like loss on the logarithmic scale:
- Targeted Attack (forcing classification into target class $t$ ):
$f(x, t) = \max\left\{ \max_{i \neq t} \log[F(x)]_i - \log[F(x)]_t, -\kappa \right\}$
- Untargeted Attack (forcing misclassification away from original class $t_0$ ):
$f(x) = \max\left\{ \log[F(x)]_{t_0} - \max_{i \neq t_0} \log[F(x)]_i, -\kappa \right\}$

where $\kappa \ge 0$ controls the confidence margin (default $\kappa = 0$ , with $\log 0 = -\infty$ ). The logarithmic transformation prevents the exponential dominance of single-class softmax probabilities from collapsing gradient estimates while preserving the ranking order of class confidences.

Box constraints $x \in [0, 1]^p$ are handled either by projecting pixel updates into $[0, 1]^p$ or by applying the change-of-variable $x = \frac{1 + \tanh w}{2}$ with $w \in \mathbb{R}^p$ to yield an unconstrained optimization problem over $w$ .
Knowl 2 — Zeroth-Order Gradient and Coordinate-Wise Hessian Estimation
equation
For an objective function $f: \mathbb{R}^p \to \mathbb{R}$ that incorporates both the classification loss and perturbation regularization, ZOO computes derivative-free directional derivatives along coordinate directions using finite differences. For coordinate index $i \in \{1, \dots, p\}$ and standard basis vector $e_i \in \mathbb{R}^p$ , the first-order partial derivative estimate $\hat{g}_i$ and coordinate-wise Hessian estimate $\hat{h}_i$ are given by:

$\hat{g}_i \triangleq \frac{\partial f(x)}{\partial x_i} \approx \frac{f(x + h e_i) - f(x - h e_i)}{2h}$

$\hat{h}_i \triangleq \frac{\partial^2 f(x)}{\partial x_i^2} \approx \frac{f(x + h e_i) - 2f(x) + f(x - h e_i)}{h^2}$

where $h > 0$ is a small constant step size ( $h = 0.0001$ ).

The symmetric difference quotient provides a truncation error of order $\mathcal{O}(h^2)$ . Evaluating $\hat{g}_i$ for a single coordinate requires two objective evaluations, $f(x + h e_i)$ and $f(x - h e_i)$ . Computing $\hat{h}_i$ requires no additional forward evaluations beyond evaluating the central point $f(x)$ once, which is shared across all coordinate updates.
Knowl 3 — ZOO-ADAM Algorithm
algorithm
ZOO-ADAM optimizes the black-box adversarial objective using zeroth-order stochastic coordinate descent combined with coordinate-wise adaptive moment estimation (ADAM). Each iteration randomly samples a coordinate (or a batch of coordinates $B$ , such as $B = 128$ ), estimates the coordinate gradient via two forward function queries, updates per-coordinate running first and second moments, and applies a bias-corrected step.
Input: Initial image $x \in [0, 1]^p$ , step size $\eta$ , parameters $\beta_1 = 0.9$ , $\beta_2 = 0.999$ , $\epsilon = 10^{-8}$ , finite difference step $h = 0.0001$
Output: Adversarial image $x$
Initialize moment vectors $M \leftarrow 0 \in \mathbb{R}^p$ , $v \leftarrow 0 \in \mathbb{R}^p$ , step count $T \leftarrow 0 \in \mathbb{Z}^p$
while not converged do
    Randomly pick coordinate index $i \in \{1, \dots, p\}
    Estimate gradient $\hat{g}_i \leftarrow \frac{f(x + h e_i) - f(x - h e_i)}{2h}$
     $T_i \leftarrow T_i + 1$
     $M_i \leftarrow \beta_1 M_i + (1 - \beta_1) \hat{g}_i$
     $v_i \leftarrow \beta_2 v_i + (1 - \beta_2) \hat{g}_i^2$
     $\hat{M}_i \leftarrow \frac{M_i}{1 - \beta_1^{T_i}}$
     $\hat{v}_i \leftarrow \frac{v_i}{1 - \beta_2^{T_i}}$
     $\delta^* \leftarrow -\eta \frac{\hat{M}_i}{\sqrt{\hat{v}_i} + \epsilon}$
     $x_i \leftarrow x_i + \delta^*$
end while
return $x$
In GPU batch implementations, $B$ coordinates are estimated and updated simultaneously per iteration.
Knowl 4 — ZOO-Newton Algorithm
algorithm
ZOO-Newton applies zeroth-order stochastic coordinate descent using coordinate-wise Newton updates driven by both estimated first-order gradients $\hat{g}_i$ and estimated second-order Hessians $\hat{h}_i$ . When the coordinate-wise Hessian is strictly positive ( $\hat{h}_i > 0$ ), a Newton step scaled by learning rate $\eta$ is executed; if the local curvature along coordinate $i$ is non-positive (concave or flat, $\hat{h}_i \le 0$ ), the update defaults to a first-order gradient descent step.
Input: Initial image $x \in [0, 1]^p$ , step size $\eta$ , finite difference step $h = 0.0001$
Output: Adversarial image $x$
while not converged do
    Randomly pick coordinate index $i \in \{1, \dots, p\}
    Estimate gradient $\hat{g}_i \leftarrow \frac{f(x + h e_i) - f(x - h e_i)}{2h}$
    Estimate Hessian $\hat{h}_i \leftarrow \frac{f(x + h e_i) - 2f(x) + f(x - h e_i)}{h^2}$
    if $\hat{h}_i \le 0$ then
         $\delta^* \leftarrow -\eta \hat{g}_i$
    else
         $\delta^* \leftarrow -\eta \frac{\hat{g}_i}{\hat{h}_i}$
    end if
     $x_i \leftarrow x_i + \delta^*$
end while
return $x$
Knowl 5 — Attack-Space Dimension Reduction and Hierarchical Scaling
model/method
To prevent excessive query requirements on large-dimensional inputs (e.g., $p = 299 \times 299 \times 3 = 268,203$ variables on ImageNet), the attack space is constrained to a lower-dimensional variable $y \in \mathbb{R}^m$ ( $m < p$ ) via an upscaling transformation $D: \mathbb{R}^m \to \mathbb{R}^p$ (such as bilinear interpolation or discrete cosine transform). The optimization problem becomes:

$\min_{y \in \mathbb{R}^m} \|D(y)\|_2^2 + c \cdot f(x_0 + D(y), t) \quad \text{subject to } x_0 + D(y) \in [0, 1]^p$

To balance early optimization efficiency with fine-grained perturbation capacity, a hierarchical attack scheme employs a progression of transformation operators $D_1, D_2, \dots$ mapping from dimensions $m_1 < m_2 < \dots$ to $p$ (for example, $m_1 = 32 \times 32 \times 3$ , $m_2 = 64 \times 64 \times 3$ , $m_3 = 128 \times 128 \times 3$ ). When objective progress stalls in space $m_{j-1}$ , the attack variable is upscaled to the next dimension $m_j$ via:

$y_j = D_j^{-1}(D_{j-1}(y_{j-1}))$

where $D_j^{-1}$ represents the inverse or downsampling transformation corresponding to $D_j$ .
Knowl 6 — Spatial Importance Sampling for Coordinate Updates
model/method
Rather than sampling pixel coordinates uniformly during coordinate descent on large image spaces, ZOO uses spatial importance sampling to prioritize pixels where perturbations are most active.
1. Region Division: The current adversarial perturbation image $\Delta x = x - x_0$ is partitioned into $8 \times 8$ pixel regions across all RGB channels.
2. Importance Aggregation: For each region, the maximum absolute change among its pixels is computed via $4 \times 4$ max-pooling.
3. Probability Map Generation: The pooled values are up-sampled to the current attack-space dimension $m$ and normalized across all coordinates such that their sum equals 1.
4. Dynamic Update: The probability distribution is recomputed periodically during optimization to reflect recent changes, focusing gradient queries on object-relevant regions and edges while avoiding background coordinates.
Knowl 7 — ADAM State Reset Strategy at Adversarial Feasibility
model/method
The overall attack objective consists of the adversarial classification loss $l_1 = c \cdot f(x, t)$ and the perturbation norm $l_2 = \|x - x_0\|_2^2$ . Prior to finding a valid adversarial image, $l_1 > 0$ dominates the optimization, causing $l_2$ to grow as the optimizer pushes class confidence past the target threshold. Once $l_1 = 0$ , the loss becomes non-negative and flat with respect to further confidence gains, and the optimizer must transition to minimizing $l_2$ .

Because coordinate descent over large dimensions causes the historical momentum statistics ( $M$ and $v$ ) in ADAM to become stale, the optimizer continues pushing class probabilities unnecessarily rather than reducing $l_2$ . To rectify this, ZOO resets all ADAM internal states ( $M_i \leftarrow 0$ , $v_i \leftarrow 0$ , $T_i \leftarrow 0$ for all $i$ ) immediately upon the first occurrence of $l_1 = 0$ , redirecting gradient updates toward decreasing $l_2$ distortion.

Knowl 8 — Adversarial Attack Comparison on MNIST and CIFAR-10 Datasets

data/table

ZOO was evaluated against the white-box Carlini & Wagner (C&W) attack and black-box substitute-model attacks using FGSM (perturbation $\epsilon = 0.4$ ) and C&W (transfer confidence $\kappa = 20$ ). Tests evaluated 200 untargeted attacks and 900 targeted attacks across 100 randomly sampled test images (excluding originally misclassified samples). Runtimes show average attack time per sample plus total substitute model training time (Jacobian dataset augmentation, $\lambda = 0.1$ , 5 epochs on 150 hold-out images).

MNIST
	Untargeted			Targeted
Method	Success	Avg. $L_2$	Avg. Time	Success	Avg. $L_2$	Avg. Time
White-box (CW)	100%	1.48066	0.48 min	100%	2.00661	0.53 min
Black-box (Substitute + FGSM)	40.6%	-	0.002 s (+ 6.16 min)	7.48%	-	0.002 s (+ 6.16 min)
Black-box (Substitute + CW)	33.3%	3.6111	0.76 min (+ 6.16 min)	26.74%	5.272	0.80 min (+ 6.16 min)
Proposed ZOO-ADAM	100%	1.49550	1.38 min	98.9%	1.987068	1.62 min
Proposed ZOO-Newton	100%	1.51502	2.75 min	98.9%	2.057264	2.06 min
CIFAR10
White-box (CW)	100%	0.17980	0.20 min	100%	0.37974	0.16 min
Black-box (Substitute + FGSM)	76.1%	-	0.005 s (+ 7.81 min)	11.48%	-	0.005 s (+ 7.81 min)
Black-box (Substitute + CW)	25.3%	2.9708	0.47 min (+ 7.81 min)	5.3%	5.7439	0.49 min (+ 7.81 min)
Proposed ZOO-ADAM	100%	0.19973	3.43 min	96.8%	0.39879	3.95 min
Proposed ZOO-Newton	100%	0.23554	4.41 min	97.0%	0.54226	4.40 min

ZOO achieves nearly 100% success rate with $L_2$ distortion nearly identical to the white-box C&W attack, outperforming substitute models by large margins without suffering transferability loss.

Knowl 9 — ImageNet Inception-v3 Attack Performance and Ablation of Acceleration Techniques

data/table

ZOO was evaluated on Inception-v3 ( $299 \times 299 \times 3$ , 1000 classes) on ImageNet. For 150 untargeted attacks using a fixed attack space of $32 \times 32 \times 3$ , fixed $c=10$ , step size $\eta = 0.002$ , and a 1,500 iteration limit (192,000 gradient queries total), ZOO-ADAM achieved an 88.9% success rate with average $L_2$ of 1.19916 (compared to 100% success and average $L_2 = 0.37310$ for white-box C&W). Substitute model attacks failed to scale to Inception-v3.

An ablation study evaluated a hard targeted attack (transforming a "bagel" classified with 97.0% initial confidence into a "grand piano" with 0.0006% initial confidence) across 20,000 iterations using dimension scaling ( $32 \times 32 \times 3 \to 64 \times 64 \times 3 \to 128 \times 128 \times 3$ at iterations 2,000 and 10,000):

Technique Configuration	Success?	First Valid Iteration	Final $L_2$	Final Loss
All techniques	Yes	15,227	3.42500	11.735
No Hierarchical Attack	No	-	-	62.439
No Importance Sampling	Yes	17,403	3.63486	13.216
No ADAM State Reset	Yes	15,227	3.47935	12.111

Hierarchical scaling is essential for finding targeted adversarial perturbations on high-resolution models, importance sampling accelerates discovery of the first valid adversarial example by >2,000 iterations, and resetting ADAM states lowers the final $L_2$ distortion.

Coverage note — None was omitted; all contributed algorithms, formulations, acceleration techniques, and empirical evaluation data across MNIST, CIFAR-10, and ImageNet are covered.

References

1.Marco Barreno, Blaine Nelson, Anthony D Joseph, and JD Tygar. 2010. The security of machine learning. Machine Learning 81, 2 (2010), 121–148.
2.Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, and J Doug Tygar. 2006. Can machine learning be secure?. In Proceedings of the 2006 ACM Symposium on Information, computer and communications security. ACM, 16–25.
3.Dimitri P Bertsekas. Nonlinear programming.
4.Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. 2013. Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 387–402.
5.Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. Poisoning Attacks Against Support Vector Machines. In Proceedings of the International Coference on Inter- national Conference on Machine Learning. 1467–1474.
6.John Bradshaw, Alexander G de G Matthews, and Zoubin Ghahramani. 2017. Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks. arXiv preprint arXiv:1707.02476 (2017).
7.Nicholas Carlini and David Wagner. 2017. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. arXiv preprint arXiv:1705.07263 (2017).
8.Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP). IEEE, 39–57.
9.Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. 2017. Robust Physical-World Attacks on Machine Learning Models. arXiv preprint arXiv:1707.08945 (2017).
10.Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. 2017. Detecting Adversarial Samples from Artifacts. arXiv preprint arXiv:1703.00410 (2017).
11.Saeed Ghadimi and Guanghui Lan. 2013. Stochastic first-and zeroth-order meth- ods for nonconvex stochastic programming. SIAM Journal on Optimization 23, 4 (2013), 2341–2368.
12.Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
13.Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. 2017. On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280 (2017).
14.Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 (2016).
15.Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).
16.Weiwei Hu and Ying Tan. 2017. Black-Box Attacks against RNN based Malware Detection Algorithms. arXiv preprint arXiv:1705.08131 (2017).
17.Weiwei Hu and Ying Tan. 2017. Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN. arXiv preprint arXiv:1702.05983 (2017).
18.Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2016. Safety verification of deep neural networks. arXiv preprint arXiv:1610.06940 (2016).
19.Jonghoon Jin, Aysegul Dundar, and Eugenio Culurciello. 2015. Robust convolu- tional neural networks under adversarial noise. arXiv preprint arXiv:1511.06306 (2015).
20.Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimiza- tion. arXiv preprint arXiv:1412.6980 (2014).
21.Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 (2016).
22.Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 (2016).
23.Peter D Lax and Maria Shea Terrell. 2014. Calculus with applications. Springer.
24.Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.
25.Xiangru Lian, Huan Zhang, Cho-Jui Hsieh, Yijun Huang, and Ji Liu. 2016. A comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order. In Advances in Neural Information Processing Systems. 3054–3062.
26.Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. 2016. Delving into transfer- able adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770 (2016).
27.Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards Deep Learning Models Resistant to Adversarial Attacks. arXiv preprint arXiv:1706.06083 (2017).
28.Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. 2017. On detecting adversarial perturbations. arXiv preprint arXiv:1702.04267 (2017).
29.Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pas- cal Frossard. 2016. Universal adversarial perturbations. arXiv preprint arXiv:1610.08401 (2016).
30.Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. 2017. Analysis of universal adversarial perturbations. arXiv preprint arXiv:1705.09554 (2017).
31.Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2574– 2582.
32.Yurii Nesterov et al. 2011. Random gradient-free minimization of convex func- tions. Technical Report. Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
33.Nicolas Papernot and Patrick McDaniel. 2017. Extending Defensive Distillation. arXiv preprint arXiv:1705.05264 (2017).
34.Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. 2016. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016).
35.Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Ce- lik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the ACM on Asia Conference on Computer and Commu- nications Security. ACM, 506–519.
36.Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy (EuroS&P). 372– 387.
37.Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016. Crafting adversarial input sequences for recurrent neural networks. In IEEE Military Communications Conference (MILCOM). 49–54.
38.Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy (SP). 582–597.
39.Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818–2826.
40.Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).
41.Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, and Patrick McDaniel. 2017. Ensemble Adversarial Training: Attacks and Defenses. arXiv preprint arXiv:1705.07204 (2017).
42.Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks. arXiv preprint arXiv:1704.01155 (2017).
43.Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature Squeezing Mitigates and Detects Carlini/Wagner Adversarial Examples. arXiv preprint arXiv:1705.10686 (2017).
44.Valentina Zantedeschi, Maria-Irina Nicolae, and Ambrish Rawat. 2017. Efficient Defenses Against Adversarial Attacks. arXiv preprint arXiv:1707.06728.
45.Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. 2016. Improving the robustness of deep neural networks via stability training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4480–4488.

Citation

MLA

Chen, P.-Y., et al. “ZOO”. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 2017, pp. 15–26, https://doi.org/10.1145/3128572.3140448.

APA

Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C.-J. (2017). ZOO. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 15–26. https://doi.org/10.1145/3128572.3140448

Chicago

Chen, P.-Y., H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh. 2017. “ZOO”. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 15–26. https://doi.org/10.1145/3128572.3140448.

Harvard

Chen, P.-Y. et al. (2017) “ZOO”, Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security. ACM, pp. 15–26. Available at: https://doi.org/10.1145/3128572.3140448.

Vancouver

1. Chen P-Y, Zhang H, Sharma Y, Yi J, Hsieh C-J (2017) ZOO. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security. ACM, pp 15–26

BibTeX

@inproceedings{Chen_2017, series={CCS ’17}, title={ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models}, url={http://dx.doi.org/10.1145/3128572.3140448}, DOI={10.1145/3128572.3140448}, booktitle={Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security}, publisher={ACM}, author={Chen, Pin-Yu and Zhang, Huan and Sharma, Yash and Yi, Jinfeng and Hsieh, Cho-Jui}, year={2017}, month=Nov, pages={15–26}, collection={CCS ’17} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — ZOO Black-Box Adversarial Optimization and Confidence-Score Loss Formulation

Knowl 2 — Zeroth-Order Gradient and Coordinate-Wise Hessian Estimation

Knowl 3 — ZOO-ADAM Algorithm

Knowl 4 — ZOO-Newton Algorithm

Knowl 5 — Attack-Space Dimension Reduction and Hierarchical Scaling

Knowl 6 — Spatial Importance Sampling for Coordinate Updates