Stealing Machine Learning Models via Prediction APIs

Florian TramèrFan ZhangAri JuelsMichael K. ReiterThomas Ristenpart

article2016USENIX Security Symposium2,203 citations

Demonstrates how adversaries can duplicate proprietary machine learning models with near-perfect fidelity simply by querying public prediction APIs, exposing critical intellectual property risks for commercial platforms like Amazon Machine Learning and BigML.

Listen

Cloud-based predictive analytics services have expanded rapidly, allowing organizations to train machine learning models on proprietary data and monetize access by charging external users on a pay-per-query basis. This creates a critical conflict between business confidentiality and public accessibility. Confidential models often embody substantial commercial value, intellectual property, or sensitive training data, and their exposure can enable evasion of security systems like fraud or spam filters. The article evaluates this vulnerability by systematically demonstrating and measuring "model extraction attacks," where an external user steals the exact parameters or logic of a black-box model using standard prediction application programming interfaces (APIs).

The researchers assessed popular machine learning model types—including logistic regression, multilayer neural networks, support vector machines, and decision trees—across local benchmarks and live commercial platforms, specifically Amazon Machine Learning and BigML. To execute the extraction, the team developed two core techniques: non-adaptive equation-solving attacks for probability-generating models and adaptive path-finding algorithms for decision trees. The evaluation tested whether full models could be duplicated using public API features such as high-precision confidence values and support for incomplete feature queries.

The findings demonstrate that extracting models from current cloud services is highly effective, fast, and inexpensive. For linear and logistic regression models, an attacker can extract full model parameters with 100% fidelity using only a small set of queries (such as d+1 queries for a d-dimensional feature space), costing mere cents on commercial platforms. Multilayer neural networks were duplicated with greater than 99.9% accuracy using a modest query budget. For decision trees, leveraging confidence scores as pseudo-identifiers enabled complete path reconstruction, requiring fewer than 4,000 queries even on complex models. Furthermore, extracted models were shown to leak sensitive training data—including facial images and underlying distribution statistics—and could facilitate white-box privacy attacks at a fraction of the cost of direct querying.

These results show that existing API designs undermine the economic models of predictive cloud services and introduce substantial compliance and security risks. Countermeasures such as omitting confidence scores and returning only basic class labels increase the attacker's required query volume by up to 100-fold; however, adaptive retraining methods still allow attackers to recreate models with over 99% accuracy. Limiting confidence values to fewer decimal places degrades tree extraction but only marginally slows the extraction of regression models.

To mitigate these extraction risks, service providers and system operators should immediately evaluate API access policies. Recommended short-term actions include removing high-precision confidence scores where feasible, restricting incomplete queries, and implementing query-rate monitoring. Future development should explore parameter-level differential privacy and ensemble model architectures to harden systems against extraction. While these findings demonstrate high confidence across standard and production workloads, the article notes that reverse-engineering non-linear text transformations or complex feature engineering pipelines remains an ongoing area for practical analysis.

Cornell University École Polytechnique Fédérale de Lausanne Jacobs Technion-Cornell Institute University of North Carolina at Chapel Hill feature vectors logistic regression logistics online model extraction attacks prediction APIs adversarial machine learning

arXiv: 1609.02943 ftramer/Steal-ML

Paper: Intriguing properties of neural networks, Christian Szegedy et al. (2014). Provides foundational insights into the vulnerability of machine learning models to small perturbations, which underpins black-box model extraction techniques.

Paper: Membership Inference Attacks Against Machine Learning Models, Reza Shokri et al. (2016). Extends the study of prediction APIs to demonstrate membership inference attacks that exploit the same query-access vulnerabilities.
Paper: Practical Black-Box Attacks against Machine Learning, Nicolas Papernot et al. (2017). Builds directly on black-box prediction API access to demonstrate practical input manipulation and model evasion attacks.

Cover for Stealing Machine Learning Models via Prediction APIs

Abstract

Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis.

The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.

Topics

feature vectors logistic regression logistics neural networks sensitive informations random forests online learning hierarchical model

Keywords

online model extraction attacks prediction APIs adversarial machine learning training-data privacy model inversion attacks cross-user model extraction equation-solving attacks black-box attacks evasion attacks

1 Introduction
2 Background
3 Model Extraction Attacks
4 Extraction with Confidence Values
4.1 Equation-Solving Attacks
4.1.1 Binary logistic regression
4.1.2 Multiclass LRs and Multilayer Perceptrons
4.1.3 Training Data Leakage for Kernel LR
4.1.4 Model Inversion Attacks on Extracted Models
4.2 Decision Tree Path-Finding Attacks
4.2.1 Extraction Algorithms
4.2.2 Attack Evaluation
5 Online Model Extraction Attacks
5.1 Case Study 1: BigML
5.2 Case Study 2: Amazon Web Services
5.3 Discussion
6 Extraction Given Class Labels Only
6.1 Linear Binary Models
6.2 Multiclass LR Models
6.3 Neural Networks
6.4 RBF Kernel SVMs
7 Extraction Countermeasures
8 Related Work
9 Conclusion
References
A Some Details on Models
B Details on Data Sets
C Analysis of the Path-Finding Algorithm
D A Note on Improper Extraction

Knowls

Knowl 1 — Threat Model and Error Metrics for Model Extraction Attacks
definition
A model extraction attack occurs when an adversary $\mathcal{A}$ interacting with a black-box machine learning model $f: \mathcal{X} \to \mathcal{Y}$ over a prediction API attempts to reconstruct an approximating model $\hat{f}: \mathcal{X} \to \mathcal{Y}$ using a sequence of $q$ prediction queries.

Queries are categorized into:
1. Direct queries: The adversary supplies arbitrary feature vectors $x \in \mathcal{X}$ and observes $f(x)$ .
2. Indirect queries: The adversary supplies points $M$ in an raw input space $\mathcal{M}$ and observes $f(\text{ex}(M))$ , where $\text{ex}: \mathcal{M} \to \mathcal{X}$ is a feature extraction function that may be unknown to $\mathcal{A}$ .
The fidelity of an extracted model $\hat{f}$ relative to the target $f$ is quantified by two primary error metrics under a distance function $d_{\mathcal{Y}}$ on outputs:
1. Test Error ( $R_{\text{test}}$ ): The average disagreement over a test dataset $D \subset \mathcal{X} \times \mathcal{Y}$ drawn from the training distribution: $R_{\text{test}}(f, \hat{f}) = \frac{1}{|D|} \sum_{(x, y) \in D} d_{\mathcal{Y}}(f(x), \hat{f}(x))$
2. Uniform Error ( $R_{\text{unif}}$ ): The average disagreement over a set $U$ of inputs sampled uniformly at random from the input domain $\mathcal{X}$ : $R_{\text{unif}}(f, \hat{f}) = \frac{1}{|U|} \sum_{x \in U} d_{\mathcal{Y}}(f(x), \hat{f}(x))$
When evaluating class label predictions ( $y \in \mathbb{Z}_c$ ), $d_{\mathcal{Y}}$ is the 0-1 indicator distance $d(y, y') = 0$ if $y = y'$ and $1$ otherwise. When evaluating continuous predicted class probability vectors $y, y' \in [0, 1]^c$ , total variation distance is used: $d_{\text{TV}}(y, y') = \frac{1}{2} \sum_{i=0}^{c-1} |y[i] - y'[i]|$ with corresponding error metrics denoted $R^{\text{TV}}_{\text{test}}(f, \hat{f})$ and $R^{\text{TV}}_{\text{unif}}(f, \hat{f})$ .
Knowl 2 — Exact Parameter Extraction of Binary Logistic Regression via Equation Solving
model/method
For a binary logistic regression model $f: \mathbb{R}^d \to [0, 1]$ parameterized by weight vector $w \in \mathbb{R}^d$ and bias $\beta \in \mathbb{R}$ , the output class probability for class 1 on input $x \in \mathbb{R}^d$ is given by the standard sigmoid function: $f_1(x) = \sigma(w \cdot x + \beta) = \frac{1}{1 + e^{-(w \cdot x + \beta)}}$

Applying the logit transformation (the inverse sigmoid $\sigma^{-1}$ ) to the confidence value $f_1(x) \in (0, 1)$ yields a linear equation in the model parameters: $w \cdot x + \beta = \sigma^{-1}(f_1(x)) = \ln\left(\frac{f_1(x)}{1 - f_1(x)}\right)$

An adversary querying $d + 1$ linearly independent inputs $x^{(1)}, x^{(2)}, \dots, x^{(d+1)} \in \mathbb{R}^d$ obtains a system of $d + 1$ linear equations with $d + 1$ unknowns ( $w_1, \dots, w_d, \beta$ ). Because the required query points can be chosen independently and uniformly at random, the entire query set can be submitted as a single non-adaptive batch request, recovering the exact parameters $w$ and $\beta$ up to floating-point numerical precision ( $R_{\text{test}} = R_{\text{unif}} = 0$ , $R^{\text{TV}} < 10^{-9}$ ).
Knowl 3 — Equation-Solving Extraction for Multiclass Logistic Regression and Multilayer Perceptrons
model/method
When a prediction API returns class probability vectors $f(x) \in [0, 1]^c$ for models with a logistic output layer, model extraction reduces to solving a system of non-linear equations over model parameters.
1. Multiclass Logistic Regression (Softmax and One-vs-Rest): A softmax model with $c$ classes maps an input $x \in \mathbb{R}^d$ to probabilities: $f_i(x) = \frac{e^{w_i \cdot x + \beta_i}}{\sum_{j=0}^{c-1} e^{w_j \cdot x + \beta_j}}, \quad i \in \{0, \dots, c-1\}$ with $k = c(d + 1)$ unknowns ( $w_i \in \mathbb{R}^d, \beta_i \in \mathbb{R}$ ). Each query $x$ yields $c$ non-linear equations. Minimizing the regularized logistic loss via quasi-Newton optimization (such as BFGS) on $k$ non-adaptive random queries recovers the exact parameters ( $R_{\text{test}} = R_{\text{unif}} = 0$ ) because the objective is strongly convex.
2. Multilayer Perceptrons (MLPs): For a feedforward neural network with a single hidden layer of $h$ hidden units and activation function $g$ (e.g., tanh or $\sigma$ ), hidden activations are $h_i(x) = g(w_i^{(1)} \cdot x + \beta_i^{(1)})$ and the output is a softmax layer over $h(x)$ . The total number of unknowns is $k = d \cdot h + h \cdot c + h + c$ . Solving the non-linear equation system using stochastic gradient descent on cross-entropy loss over non-adaptive batch queries achieves $>99.9\%$ agreement with $5 \cdot k$ to $10 \cdot k$ queries.
Knowl 4 — Decision Tree Path-Finding Extraction Algorithm
algorithm
The path-finding attack extracts the exact structure and decision thresholds of a decision tree by exploiting the leaf confidence score returned by prediction APIs as a unique pseudo-identifier for each leaf path.
Input: Leaf-identity oracle $O: \mathcal{X} \to \text{IDs}$ , precision parameter $\epsilon > 0$ , feature space $\mathcal{X} = X_1 \times \dots \times X_d$
Output: Dictionary $P$ mapping each discovered leaf ID to the set of bounding predicates on its path
$x_{init} \leftarrow (x_1, \dots, x_d) \in \mathcal{X}$ chosen at random
$Q \leftarrow \{x_{init}\}$
$P \leftarrow \{\}$
while $Q$ is not empty do
     $x \leftarrow Q\text{.pop}()$
     $id \leftarrow O(x)$
    if $id \in P$ then
        continue
    end if
     $P[id] \leftarrow \emptyset$
    for $i = 1$ to $d$ do
        if $X_i$ is continuous with bounds $[a, b]$ then
            intervals $\leftarrow \text{LineSearch}(O, x, i, \epsilon, [a, b])$
            for $(\alpha, \beta] \in intervals$ do
                if $x_i \in (\alpha, \beta]$ then
                     $P[id] \leftarrow P[id] \cup \{x_i \in (\alpha, \beta]\}$
                else
                     $x' \leftarrow x$ with feature $i$ set to $\beta$
                     $Q\text{.push}(x')$
                end if
            end for
        else
             $S, V \leftarrow \text{CategorySplit}(O, x, i, id)$
             $P[id] \leftarrow P[id] \cup \{x_i \in S\}$
            for $v \in V$ do
                 $x' \leftarrow x$ with feature $i$ set to $v$
                 $Q\text{.push}(x')$
            end for
        end if
    end for
end while
return $P$
LineSearch performs binary search along dimension $i$ with all other features fixed to identify split thresholds $t$ rounded to granularity $\epsilon$ , identifying all contiguous intervals leading to distinct leaf IDs. CategorySplit queries each categorical level $v \in X_i$ to partition the categories into those staying in leaf $id$ ( $S$ ) versus representative values leading to alternative leaves ( $V$ ).
Knowl 5 — Top-Down Decision Tree Extraction via Incomplete Queries
model/method
When a cloud prediction API supports incomplete queries (inputs where missing features are assigned a null symbol $\perp$ ), an adversary can extract decision trees top-down layer-by-layer rather than traversing paths from root to leaves.
1. Oracle Semantics: The API evaluates $x \in (X_1 \cup \{\perp\}) \times \dots \times (X_d \cup \{\perp\})$ by traversing the tree until it encounters a split on a feature $x_i = \perp$ , returning the node identifier and prediction at that internal node or leaf.
2. Traversal Strategy: Starting with the fully empty query $(\perp, \dots, \perp)$ , the adversary identifies the root node. By testing features one by one, exactly one feature causes a state change, identifying the root's splitting feature. The splitting criterion (threshold for continuous features or category subset for categorical features) is extracted using line search or categorical queries.
3. Recursive Descent: The algorithm fixes predicates satisfied along each explored branch and recurses downward on child nodes. By directly inspecting API response metadata (such as returned active field lists), this method eliminates redundant leaf searches, reducing query counts by $2\times$ to $10\times$ compared to leaf-only path finding and achieving 100% fidelity even in trees containing duplicate leaf IDs.
Knowl 6 — Query Complexity of Decision Tree Extraction
theoretical result
Let $T$ be a decision tree over $d_{\text{cat}}$ categorical features (each of arity at most $k$ ) and $d_{\text{cont}}$ continuous features with range $[0, b]$ . Let $m$ be the number of leaves in $T$ , and let $\epsilon > 0$ be the threshold precision parameter for continuous split detection.

Assuming each leaf has a unique identifier assigned by an oracle $O$ :
1. For each of the $m$ leaves, all $d$ features are inspected.
2. Finding a single continuous threshold via binary line search requires at most $\log_2(b / \epsilon)$ queries. Across all leaves, at most $m$ splits exist on any single continuous feature, requiring at most $m \log_2(b / \epsilon)$ queries per feature.
3. Evaluating a categorical feature requires $k$ queries.
The total query complexity of path-finding extraction is: $\mathcal{O}\left(m \cdot \left(d_{\text{cat}} \cdot k + d_{\text{cont}} \cdot m \cdot \log(b / \epsilon)\right)\right)$

For Boolean trees ( $d_{\text{cont}} = 0, k = 2$ ), the query complexity is $\mathcal{O}(m \cdot d)$ . In contrast, classical membership-query algorithms operating strictly on class labels (e.g., Kushilevitz-Mansour) have complexity $\text{poly}(d, 2^{\delta})$ where $\delta$ is tree depth; since $2^\delta$ can be exponential in $m$ for degenerate trees, confidence-based leaf identities provide an exponential speedup.

Knowl 7 — Empirical Performance of Extraction Attacks on Commercial MLaaS Providers

data/table

Model extraction attacks evaluated against live cloud prediction APIs on Amazon Machine Learning and BigML achieved 100% functional equivalence ( $R_{\text{test}} = R_{\text{unif}} = 0$ ) using small query budgets:

Service	Model Type	Dataset	Queries	Time (s)
Amazon	Logistic Regression	Digits	650	70
Amazon	Logistic Regression	Adult	1,485	149
BigML	Decision Tree	German Credit	1,150	631
BigML	Decision Tree	Steak Survey	4,013	2,088

The total attack execution time was dominated by network prediction latency (approximately 100 ms per query for Amazon and 500 ms per query for BigML). On Amazon, the monetary cost to steal models ranged between $\$0.03$ and $\$0.15$ at standard pricing ( $\$0.0001$ per prediction).

Knowl 8 — Reverse-Engineering Pre-Processing and Quantile Binning Feature Transformations
model/method
When a cloud ML service applies proprietary input feature transformations $\text{ex}: \mathcal{M} \to \mathcal{X}$ prior to feeding data into a classifier $f$ , direct equation solving in input space $\mathcal{M}$ is obstructed. On Amazon Machine Learning, default pipelines apply:
1. One-Hot Encoding (OHE): Categorical input spaces $\mathcal{M}_i = \mathbb{Z}_k$ map to $k$ binary indicators in $\mathcal{X}$ .
2. Quantile Binning: Continuous feature values in $\mathcal{M}_i = [a, b]$ are partitioned into $k$ data-dependent quantile bins mapped to $k$ binary indicators in $\mathcal{X}$ .
An adversary reverse-engineers this pipeline by:
- Performing binary line searches across the range of each numeric feature in input space $\mathcal{M}$ to detect points where the output probability exhibits a discontinuous derivative or step, revealing quantile bin cutoffs.
- Exploiting missing feature handling: when an input query leaves a feature omitted, Amazon sets all corresponding binary features in $\mathcal{X}$ to zero. Querying inputs with only a single active feature isolates individual weights in $\mathcal{X}$ , reducing parameter recovery to decoupled single-variable equations.
Knowl 9 — Accelerated Model Inversion and Training Data Leakage via Extracted Models
empirical result
Extracted models can be leveraged locally to recover sensitive training data faster and with fewer remote API queries than direct black-box attacks:
1. Accelerated Model Inversion: Inverting a facial recognition softmax classifier (AT&T Faces, 40 classes, $d = 10,304$ , 412,160 unknown weights) directly via black-box gradient estimation requires $\approx 20,600$ queries per target identity (totaling $\approx 800,000$ online queries taking 16 hours for all 40 faces). Extracting an equivalent model $\hat{f}$ via equation-solving required only 41,216 online queries (under 1 hour), after which white-box gradient-based model inversion was run locally, achieving visually indistinguishable facial reconstructions with $20\times$ fewer queries.
2. Direct Training Data Leakage in Kernel Logistic Regression: For Kernel Logistic Regression models with RBF kernel $K(x, x') = \exp(-\gamma \|x - x'\|^2)$ , outputs are given by $f_i(x) \propto \exp(\sum_{r=1}^s \alpha_{i,r} K(x, x_r) + \beta_i)$ , where $x_1, \dots, x_s$ are training representer points. Solving the non-linear equation system over $\{\alpha_{i,r}, \beta_i, x_r\}$ directly recovers the exact training input vectors $x_r$ (or class-average centroid images when $s$ is underestimated).
Knowl 10 — Model Extraction on Label-Only Prediction APIs via Active Retraining
model/method
When prediction APIs omit confidence scores and return only discrete class labels $\mathcal{Y} = \mathbb{Z}_c$ , equation-solving is inapplicable. Extraction can be achieved via active boundary retraining:
1. Lowd-Meek Search: For binary linear classifiers ( $w \cdot x + \beta > 0$ ), line searches locate points on the decision boundary ( $w \cdot x + \beta \approx 0$ ) to compute $w$ and $\beta$ . This requires $\approx 2,050$ queries on average ( $50\times$ more than equation-solving) and does not extend directly to multiclass or RBF kernel boundaries.
2. Adaptive Retraining Algorithm: For multiclass logistic regression, SVMs, and neural networks, active learning iteratively refines a local surrogate $\hat{f}$ $\hat{f}$ over $r$ $r$ rounds with query budget $m$ $m$ :
  - In round 1, query $m/r$ points chosen uniformly at random and train initial $\hat{f}$ .
  - In subsequent rounds $2, \dots, r$ , sample candidate points and query the oracle on the $m/r$ points closest to the decision boundary of $\hat{f}$ (points where predicted confidence is least certain).
  - Retrain $\hat{f}$ on the accumulated dataset.
For multiclass logistic regression, adaptive retraining with a query budget of $100 \cdot c(d + 1)$ (averaging $\approx 26,000$ queries) achieves over $99.9\%$ agreement ( $1 - R_{\text{test}} > 0.999$ ).
Knowl 11 — Impact of Confidence Score Truncation on Model Extraction
empirical result
Rounding class probabilities returned by prediction APIs to a fixed number of decimal places degrades equation-solving and path-finding attacks only at extreme rounding thresholds:
1. Softmax Regression: Rounding class probabilities to 4 or 5 decimal places has zero effect on extraction error ( $R_{\text{test}} \approx 0$ ). Rounding to 3 decimal places slightly elevates extraction error, while rounding to 2 decimal places increases $R_{\text{test}}$ to $\approx 10^{-2}$ at budget factor $\alpha = 10$ . However, even at 2 decimal places, truncated equation-solving outperforms active retraining on class labels alone.
2. Decision Trees: For regression trees, continuous leaf predictions are unique, rendering output rounding ineffective. For classification trees, rounding confidence scores to 3 or 4 decimal places induces identifier collisions between distinct paths, severely reducing the discovery rate of path-finding algorithms; however, robust trees (e.g., IRS Tax Patterns) retain near 100% extraction fidelity until confidences are rounded to 2 decimal places.

Coverage note — None was omitted; all major attack formulations (equation solving, path finding, label-only active retraining), reverse-engineering pipelines, data leakage results, and countermeasure analyses were incorporated into standalone knowls.

References

1.AMAZON WEB SERVICES. https://aws.amazon.com/machine-learning. Accessed Feb. 10, 2016.
2.ANDREWS, R., DIEDERICH, J., AND TICKLE, A. Survey and critique of techniques for extracting rules from trained artificial neural networks. KBS 8, 6 (1995), 373–389.
3.ANGLUIN, D. Queries and concept learning. Machine learning 2, 4 (1988), 319–342.
4.ATENIESE, G., MANCINI, L. V., SPOGNARDI, A., VILLANI, A., VITALI, D., AND FELICI, G. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. IJSN 10, 3 (2015), 137–150.
5.AT&T LABORATORIES CAMBRIDGE. The ORL database of faces. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html.
6.BARRENO, M., NELSON, B., SEARS, R., JOSEPH, A. D., AND TYGAR, J. D. Can machine learning be secure? In ASIACCS (2006), ACM, pp. 16–25.
7.BELLARE, M. A technique for upper bounding the spectral norm with applications to learning. In COLT (1992), ACM, pp. 62–70.
8.BENEDEK, G. M., AND ITAI, A. Learnability with respect to fixed distributions. TCS 86, 2 (1991), 377–389.
9.BIGGIO, B., CORONA, I., MAIORCA, D., NELSON, B., ŠRNDIĆ, N., LASKOV, P., GIACINTO, G., AND ROLI, F. Evasion attacks against machine learning at test time. In ECML PKDD. Springer, 2013, pp. 387–402.
10.BIGGIO, B., NELSON, B., AND LASKOV, P. Poisoning attacks against support vector machines. In ICML (2012).
11.BIGML. https://www.bigml.com. Accessed Feb. 10, 2016.
12.BLUM, A. L., AND LANGLEY, P. Selection of relevant features and examples in machine learning. Artificial intelligence 97, 1 (1997), 245–271.
13.BLUMER, A., EHRENFEUCHT, A., HAUSSLER, D., AND WARMUTH, M. K. Occam’s razor. Readings in machine learning (1990), 201–204.
14.BOSER, B. E., GUYON, I. M., AND VAPNIK, V. N. A training algorithm for optimal margin classifiers. In COLT (1992), ACM, pp. 144–152.
15.BSHOUTY, N. H. Exact learning boolean functions via the monotone theory. Inform. Comp. 123, 1 (1995), 146–153.
16.BUCILUĂ, C., CARUANA, R., AND NICULESCU-MIZIL, A. Model compression. In KDD (2006), ACM, pp. 535–541.
17.CHAUDHURI, K., AND MONTELEONI, C. Privacy-preserving logistic regression. In NIPS (2009), pp. 289–296.
18.COHN, D., ATLAS, L., AND LADNER, R. Improving generalization with active learning. Machine learning 15, 2 (1994), 201–221.
19.CRAVEN, M. W., AND SHAVLIK, J. W. Extracting tree-structured representations of trained networks. In NIPS (1996).
20.CYBENKO, G. Approximation by superpositions of a sigmoidal function. MCSS 2, 4 (1989), 303–314.
21.DALVI, N., DOMINGOS, P., SANGHAI, S., VERMA, D., ET AL. Adversarial classification. In KDD (2004), ACM, pp. 99–108.
22.DWORK, C. Differential privacy. In ICALP (2006), Springer.
23.FREDRIKSON, M., JHA, S., AND RISTENPART, T. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS (2015), ACM, pp. 1322–1333.
24.FREDRIKSON, M., LANTZ, E., JHA, S., LIN, S., PAGE, D., AND RISTENPART, T. Privacy in pharmacogenetics: An end-to-end case study of personalized Warfarin dosing. In USENIX Security (2014), pp. 17–32.
25.GOOGLE PREDICTION API. https://cloud.google.com/prediction. Accessed Feb. 10, 2016.
26.HICKEY, W. How Americans Like their Steak. http://fivethirtyeight.com/datalab/how-americans-like-their-steak, 2014. Accessed Feb. 10, 2016.
27.HINTON, G., VINYALS, O., AND DEAN, J. Distilling the knowledge in a neural network. arXiv:1503.02531 (2015).
28.HORNIK, K., STINCHCOMBE, M., AND WHITE, H. Multilayer feedforward networks are universal approximators. Neural networks 2, 5 (1989), 359–366.
29.HUANG, L., JOSEPH, A. D., NELSON, B., RUBINSTEIN, B. I., AND TYGAR, J. Adversarial machine learning. In AISec (2011), ACM, pp. 43–58.
30.JACKSON, J. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In FOCS (1994), IEEE, pp. 42–53.
31.JAGANNATHAN, G., PILLAIPAKKAMNATT, K., AND WRIGHT, R. N. A practical differentially private random decision tree classifier. In ICDMW (2009), IEEE, pp. 114–121.
32.KLOFT, M., AND LASKOV, P. Online anomaly detection under adversarial impact. In AISTATS (2010), pp. 405–412.
33.KUSHILEVITZ, E., AND MANSOUR, Y. Learning decision trees using the Fourier spectrum. SICOMP 22, 6 (1993), 1331–1348.
34.LI, N., QARDAJI, W., SU, D., WU, Y., AND YANG, W. Membership privacy: A unifying framework for privacy definitions. In CCS (2013), ACM.
35.LICHMAN, M. UCI machine learning repository, 2013.
36.LOWD, D., AND MEEK, C. Adversarial learning. In KDD (2005), ACM, pp. 641–647.
37.LOWD, D., AND MEEK, C. Good word attacks on statistical spam filters. In CEAS (2005).
38.MICROSOFT AZURE. https://azure.microsoft.com/services/machine-learning. Accessed Feb. 10, 2016.
39.NELSON, B., RUBINSTEIN, B. I., HUANG, L., JOSEPH, A. D., LEE, S. J., RAO, S., AND TYGAR, J. Query strategies for evading convex-inducing classifiers. JMLR 13, 1 (2012), 1293–1332.
40.NEWSOME, J., KARP, B., AND SONG, D. Paragraph: Thwarting signature learning by training maliciously. In RAID (2006), Springer, pp. 81–105.
41.NOCEDAL, J., AND WRIGHT, S. Numerical optimization. Springer Science & Business Media, 2006.
42.PEDREGOSA, F., VAROQUAUX, G., GRAMFORT, A., MICHEL, V., THIRION, B., GRISEL, O., BLONDEL, M., PRETTENHOFER, P., WEISS, R., DUBOURG, V., VANDERPLAS, J., PASSOS, A., COURNAPEAU, D., BRUCHER, M., PERROT, M., AND DUCHESNAY, E. Scikit-learn: Machine learning in Python. JMLR 12 (2011), 2825–2830.
43.PREDICTIONIO. http://prediction.io. Accessed Feb. 10, 2016.
44.RUBINSTEIN, B. I., BARTLETT, P. L., HUANG, L., AND TAFT, N. Learning in a large function space: Privacy-preserving mechanisms for SVM learning. JPC 4, 1 (2012), 4.
45.RUBINSTEIN, B. I., NELSON, B., HUANG, L., JOSEPH, A. D., LAU, S.-H., RAO, S., TAFT, N., AND TYGAR, J. Antidote: understanding and defending against poisoning of anomaly detectors. In IMC (2009), ACM, pp. 1–14.
46.SAAR-TSECHANSKY, M., AND PROVOST, F. Handling missing values when applying classification models. JMLR (2007).
47.SETTLES, B. Active learning literature survey. University of Wisconsin, Madison 52, 55-66 (1995), 11.
48.SHOKRI, R., AND SHMATIKOV, V. Privacy-preserving deep learning. In CCS (2015), ACM, pp. 1310–1321.
49.SMITH, T. W., MARSDEN, P., HOUT, M., AND KIM, J. General social surveys, 1972-2012, 2013.
50.STEVENS, D., AND LOWD, D. On the hardness of evading combinations of linear classifiers. In AISec (2013), ACM, pp. 77–86.
51.THEANO DEVELOPMENT TEAM. Theano: A Python framework for fast computation of mathematical expressions. arXiv:1605.02688 (2016).
52.TOWELL, G. G., AND SHAVLIK, J. W. Extracting refined rules from knowledge-based neural networks. Machine learning 13, 1 (1993), 71–101.
53.VALIANT, L. G. A theory of the learnable. Communications of the ACM 27, 11 (1984), 1134–1142.
54.VINTERBO, S. Differentially private projected histograms: Construction and use for prediction. In ECML-PKDD (2012).
55.ŠRNDIĆ, N., AND LASKOV, P. Practical evasion of a learning-based classifier: A case study. In Security and Privacy (SP) (2014), IEEE, pp. 197–211.
56.ZHANG, J., ZHANG, Z., XIAO, X., YANG, Y., AND WINSLETT, M. Functional mechanism: regression analysis under differential privacy. In VLDB (2012).
57.ZHU, J., AND HASTIE, T. Kernel logistic regression and the import vector machine. In NIPS (2001), pp. 1081–1088.

Citation

MLA

Tramèr, F., et al. “Stealing Machine Learning Models via Prediction APIs”. arXiv, 2016, http://arxiv.org/abs/1609.02943v2.

APA

Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016). Stealing Machine Learning Models via Prediction APIs. arXiv. http://arxiv.org/abs/1609.02943v2

Chicago

Tramèr, F., F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart. 2016. “Stealing Machine Learning Models via Prediction APIs”. arXiv. http://arxiv.org/abs/1609.02943v2.

Harvard

Tramèr, F. et al. (2016) “Stealing Machine Learning Models via Prediction APIs”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1609.02943v2.

Vancouver

1. Tramèr F, Zhang F, Juels A, Reiter MK, Ristenpart T (2016) Stealing Machine Learning Models via Prediction APIs. arXiv

BibTeX

@article{tramer2016stealing,
  title = {Stealing Machine Learning Models via Prediction APIs},
  author = {Tramèr, Florian and Zhang, Fan and Juels, Ari and Reiter, Michael K. and Ristenpart, Thomas},
  year = {2016},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1609.02943v2},
  eprint = {1609.02943}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Stealing Machine Learning Models via Prediction APIs

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Threat Model and Error Metrics for Model Extraction Attacks

Knowl 2 — Exact Parameter Extraction of Binary Logistic Regression via Equation Solving

Knowl 3 — Equation-Solving Extraction for Multiclass Logistic Regression and Multilayer Perceptrons

Knowl 4 — Decision Tree Path-Finding Extraction Algorithm

Knowl 5 — Top-Down Decision Tree Extraction via Incomplete Queries

Knowl 6 — Query Complexity of Decision Tree Extraction

Knowl 7 — Empirical Performance of Extraction Attacks on Commercial MLaaS Providers

Knowl 8 — Reverse-Engineering Pre-Processing and Quantile Binning Feature Transformations

Knowl 9 — Accelerated Model Inversion and Training Data Leakage via Extracted Models

Knowl 10 — Model Extraction on Label-Only Prediction APIs via Active Retraining

Knowl 11 — Impact of Confidence Score Truncation on Model Extraction

References

Citation

BibTeX

Source Code

Access the Paper