Back

Scalable Membership Inference Attacks via Quantile Regression

Martín BertránShuai TangAaron RothMichael KearnsJamie MorgensternSteven Wu

article2023NeurIPS95 citations

Proposes a quantile regression framework that executes highly effective black-box membership inference attacks by training only a single model without requiring any knowledge of the target architecture.

Listen

Machine learning models often leak sensitive information about the records used to train them. Membership inference attacks aim to determine whether a specific data record was part of a target model's private training dataset. Existing state-of-the-art attacks typically train dozens or hundreds of "shadow models" to simulate target model behaviors. However, this shadow-model approach requires massive compute budgets and detailed knowledge of the target model's internal architecture, making it impractical against large, proprietary commercial systems.

The article introduces a scalable, black-box membership inference attack that requires training only a single quantile regression model. The main objective is to demonstrate that directly estimating sample-dependent confidence thresholds via pinball loss minimization achieves competitive or superior inference accuracy compared to expensive shadow-model baselines without requiring any knowledge of the target model's architecture.

The authors evaluate this method across standard image classification benchmarks (CIFAR-10, CIFAR-100, CINIC-10, and ImageNet-1k) and tabular datasets (OpenML and US Census ACS data). In the image domain, the target models included various ResNet architectures, while the single attack model utilized a fixed, off-the-shelf ConvNeXt architecture querying the target via black-box confidence scores. Tabular evaluations used gradient-boosted decision trees. Performance was measured by precision and true positive rates across fixed, low false positive rate thresholds.

The findings establish that the proposed quantile regression approach achieves state-of-the-art results on complex tasks while drastically cutting computational overhead. On ImageNet-1k, the single-model attack outperformed shadow-model baselines across all error rates, achieving 97.45% precision at a 1% false positive rate and 99.64% precision at a 0.1% false positive rate. On tabular benchmarks, the attack matched the accuracy of shadow-model methods requiring 16 to 64 models. Across all experiments, minimizing pinball loss directly corresponded to maximized membership inference accuracy. On smaller image datasets with limited data (CIFAR), the method outperformed standard marginal baselines but trailed shadow-model techniques due to sample scarcity.

These results demonstrate that proprietary commercial systems face severe membership privacy risks from adversaries with modest computational resources and no internal model access. For organizations deploying machine learning models, this shifts the privacy threat model by showing that architecture obfuscation provides no defense. Crucially, the same technique enables organizations to conduct fast, low-cost privacy auditing and compliance checks on their own models prior to deployment.

Organizations should incorporate quantile regression auditing into pre-deployment security workflows to detect training data leakage. Security teams can establish reliable auditing pipelines without the massive cost of training shadow ensembles. However, decision-makers should note that the attack's effectiveness depends on the availability of a representative public data sample and scales best with large datasets. When auditing smaller datasets, standard shadow models or parameterized distributions may still provide higher precision.

Amazon Web ServicesCarnegie Mellon UniversityUniversity of PennsylvaniaUniversity of Washingtoninferenceprobabilistic inferencedenial of service attacksmembership inference attacksquantile regressionsingle-model quantile regression attack
arXiv: 2307.03694
  • Paper: Membership Inference Attacks Against Machine Learning Models, Reza Shokri et al. (2016). This seminal paper introduced membership inference attacks and the foundational shadow-model methodology that the source directly targets and aims to replace with an efficient quantile regression approach.
  • Paper: LLM Dataset Inference: Did you train on my dataset?, Pratyush Maini et al. (2024). This work critiques and builds upon standard individual-record membership inference attacks by demonstrating their failures in large language models and extending the paradigm to aggregate dataset inference.
Cover for Scalable Membership Inference Attacks via Quantile Regression

Abstract

Membership inference attacks are designed to determine, using black box access to trained models, whether a particular example was used in training or not. Membership inference can be formalized as a hypothesis testing problem. The most effective existing attacks estimate the distribution of some test statistic (usually the model’s confidence on the true label) on points that were (and were not) used in training by training many shadow models—i.e. models of the same architecture as the model being attacked, trained on a random subsample of data. While effective, these attacks are extremely computationally expensive, especially when the model under attack is large.

We introduce a new class of attacks based on performing quantile regression on the distribution of confidence scores induced by the model under attack on points that are not used in training. We show that our method is competitive with state-of-the-art shadow model attacks, while requiring substantially less compute because our attack requires training only a single model. Moreover, unlike shadow model attacks, our proposed attack does not require any knowledge of the architecture of the model under attack and is therefore truly “black-box”. We show the efficacy of this approach in an extensive series of experiments on various datasets and model architectures. Our code is available at github.com/amazon-science/quantile-mia.

Topics

inferenceprobabilistic inferencedenial of service attacksarchitecture designsregression modelsocial sciences

Keywords

membership inference attacksquantile regressionsingle-model quantile regression attackLiRA attackquantile consistencyfalse positive rateslikelihood ratioWide residual networks

Table of Contents

  • 1 Introduction
  • 1.1 Our Results
  • 1.2 Additional Related Work
  • 2 Preliminaries
  • 3 Our Attack
  • 4 Experiments
  • 4.1 Image Classification Experiments
  • 4.2 Tabular Classification Experiments
  • 5 Discussion
  • References
  • A Shadow Model Architecture Mismatch
  • B Target Model Accuracies
  • C Proofs
  • C.1 Proof of Theorem 1
  • C.2 Proof of Theorem 2
  • D Hyperparameters
  • E Additional Results

Knowls

  1. Knowl 1 — Quantile Regression Membership Inference Attack

    model/method

    The quantile regression membership inference attack formulates membership inference as a conditional quantile estimation problem using a single trained model. Let f:X→[0,1]Yf: \mathcal{X} \to [0, 1]^\mathcal{Y} be a black-box machine learning model trained on an unobserved private dataset Dprivate\mathcal{D}_{\text{private}} sampled from a data distribution D\mathcal{D}. An attacker with access to a public dataset Dpublic={(xi,yi)}i=1n∼DD_{\text{public}} = \{(x_i, y_i)\}_{i=1}^n \sim \mathcal{D} (disjoint from Dprivate\mathcal{D}_{\text{private}}) computes a confidence or logit-based scalar test statistic s(xi,yi)∈Rs(x_i, y_i) \in \mathbb{R} from ff.

    The attacker trains a single quantile regression model q:X→Rq: \mathcal{X} \to \mathbb{R} on the dataset of pairs {(xi,s(xi,yi))}i=1n\{(x_i, s(x_i, y_i))\}_{i=1}^n by minimizing the (1−α)(1-\alpha)-pinball loss to estimate the sample-dependent (1−α)(1-\alpha)-quantile of the confidence score distribution under the null hypothesis (that (x,y)∼D(x, y) \sim \mathcal{D} was not used in training).

    For any target example (x,y)(x, y), the attack evaluates s(x,y)s(x, y) using ff and outputs a membership decision Aq(x,y)A_q(x, y):

    Aq(x,y)={⊤ (declare non-member),if s(x,y)<q(x)⊥ (declare member),if s(x,y)≥q(x)A_q(x, y) = \begin{cases} \top \text{ (declare non-member)}, & \text{if } s(x, y) < q(x) \\ \bot \text{ (declare member)}, & \text{if } s(x, y) \ge q(x) \end{cases}

    Unlike shadow-model attacks, this approach requires training only one quantile model, does not require knowledge of ff's architecture or training algorithm, and directly produces sample-dependent classification thresholds q(x)q(x) corresponding to a target false positive rate α\alpha.

  2. Knowl 2 — Exact False Positive Rate Guarantee for Quantile Regression Membership Inference

    theoretical result

    Let D∈Δ(X×Y)\mathcal{D} \in \Delta(\mathcal{X} \times \mathcal{Y}) be a distribution over labeled instances, and let ff be a trained model with confidence score function s(x,y)∈Rs(x, y) \in \mathbb{R}. Assume that the marginal distribution of s(x,y)s(x, y) for (x,y)∼D(x, y) \sim \mathcal{D} is continuous. Let H\mathcal{H} be any hypothesis class of threshold models q:X→Rq: \mathcal{X} \to \mathbb{R} that is closed under additive constant shifts (i.e., for every q∈Hq \in \mathcal{H} and Δ∈R\Delta \in \mathbb{R}, the shifted function q′(x)=q(x)+Δq'(x) = q(x) + \Delta is also in H\mathcal{H}).

    If q∈Hq \in \mathcal{H} minimizes the expected (1−α)(1-\alpha)-pinball loss on non-member data:

    q∈arg⁡min⁡q′∈HE(x,y)∼D[PB1−α(q′(x),s(x,y))]q \in \arg\min_{q' \in \mathcal{H}} \mathbb{E}_{(x, y) \sim \mathcal{D}} \left[ \text{PB}_{1-\alpha}(q'(x), s(x, y)) \right]

    then the membership inference attack Aq(x,y)=1[s(x,y)≥q(x)]A_q(x, y) = \mathbf{1}[s(x, y) \ge q(x)] achieves an exact marginal false positive rate equal to α\alpha:

    FPR(Aq)=Pr⁡(x,y)∼D[Aq(x,y)=⊥]=α\text{FPR}(A_q) = \Pr_{(x, y) \sim \mathcal{D}} \left[ A_q(x, y) = \bot \right] = \alpha

  3. Knowl 3 — Group Conditional Quantile Consistency and Conditional False Positive Rate Guarantee

    theoretical result

    Let G\mathcal{G} be a collection of group indicator functions g:X→{0,1}g: \mathcal{X} \to \{0, 1\}, and let P∈Δ(X×R)\mathcal{P} \in \Delta(\mathcal{X} \times \mathbb{R}) be a joint distribution over inputs xx and scores ss. A model q:X→Rq: \mathcal{X} \to \mathbb{R} satisfies group conditional quantile consistency with respect to P\mathcal{P}, target quantile 1−α1-\alpha, and group collection G\mathcal{G} if for every g∈Gg \in \mathcal{G}:

    Pr⁡(x,s)∼P[s≤q(x)∣g(x)=1]=1−α\Pr_{(x, s) \sim \mathcal{P}} \left[ s \le q(x) \mid g(x) = 1 \right] = 1 - \alpha

    Let D∈Δ(X×Y)\mathcal{D} \in \Delta(\mathcal{X} \times \mathcal{Y}) be a data distribution, ff be a target model with score function s(x,y)s(x, y), and H\mathcal{H} be a model class such that:

    1. H\mathcal{H} is closed under shifts from G\mathcal{G}: for every h∈Hh \in \mathcal{H}, g∈Gg \in \mathcal{G}, and η∈R\eta \in \mathbb{R}, the function h′(x)=h(x)+ηg(x)h'(x) = h(x) + \eta g(x) is in H\mathcal{H}.
    2. The conditional distribution of s(x,y)s(x, y) given g(x)=1g(x) = 1 under (x,y)∼D(x, y) \sim \mathcal{D} is continuous for all g∈Gg \in \mathcal{G}.

    Then the membership inference attack AqA_q instantiated with the pinball loss minimizer q∈arg⁡min⁡q′∈HE(x,y)∼D[PB1−α(q′(x),s(x,y))]q \in \arg\min_{q' \in \mathcal{H}} \mathbb{E}_{(x, y) \sim \mathcal{D}} [\text{PB}_{1-\alpha}(q'(x), s(x, y))] achieves a conditional false positive rate of exactly α\alpha across every subpopulation g∈Gg \in \mathcal{G}:

    Pr⁡(x,y)∼D[Aq(x,y)=⊥∣g(x)=1]=α\Pr_{(x, y) \sim \mathcal{D}} \left[ A_q(x, y) = \bot \mid g(x) = 1 \right] = \alpha

  4. Knowl 4 — Pinball Loss Minimization for Quantile Threshold Estimation

    equation

    The (1−α)(1 - \alpha)-pinball loss function PB1−α:R×R→R≥0\text{PB}_{1-\alpha}: \mathbb{R} \times \mathbb{R} \to \mathbb{R}_{\ge 0} for a target quantile level 1−α∈(0,1)1 - \alpha \in (0, 1), predicted threshold y^∈R\hat{y} \in \mathbb{R}, and observed target score y∈Ry \in \mathbb{R} is defined as:

    PB1−α(y^,y)=max⁡{α(y^−y),(1−α)(y−y^)}\text{PB}_{1-\alpha}(\hat{y}, y) = \max \left\{ \alpha (\hat{y} - y), (1 - \alpha)(y - \hat{y}) \right\}

    Minimizing the expected pinball loss over a real-valued scalar elicits the (1−α)(1 - \alpha)-quantile of a continuous random variable Y∼PY \sim \mathcal{P}, satisfying arg⁡min⁡y^∈REy∼P[PB1−α(y^,y)]=FY−1(1−α)\arg\min_{\hat{y} \in \mathbb{R}} \mathbb{E}_{y \sim \mathcal{P}} [\text{PB}_{1-\alpha}(\hat{y}, y)] = F_Y^{-1}(1 - \alpha). In quantile regression membership inference, a parameterized function q:X→Rq: \mathcal{X} \to \mathbb{R} is trained by solving:

    min⁡q∈HE(x,y)∼D[PB1−α(q(x),s(x,y))]\min_{q \in \mathcal{H}} \mathbb{E}_{(x, y) \sim \mathcal{D}} \left[ \text{PB}_{1-\alpha}(q(x), s(x, y)) \right]

    where s(x,y)s(x, y) is the test statistic produced by the target model on non-member samples (x,y)∼D(x, y) \sim \mathcal{D}.

  5. Knowl 5 — Parametric Gaussian Quantile Estimation for Small Datasets

    model/method

    In low-data regimes (e.g., when the available public dataset contains around 25,000 samples), directly optimizing non-parametric quantile regressors across multiple fine-grained target false positive rates α\alpha can underfit or suffer from optimization instability. Instead, a parametric approach models the conditional confidence score distribution as a Gaussian.

    A single regression model is trained to output two predictions for each input xx: the conditional mean μ(x)∈R\mu(x) \in \mathbb{R} and the conditional log standard deviation log⁡σ(x)∈R\log \sigma(x) \in \mathbb{R}, assuming:

    s(x,y)∣x∼N(μ(x),(elog⁡σ(x))2)s(x, y) \mid x \sim \mathcal{N}\left(\mu(x), \left(e^{\log \sigma(x)}\right)^2\right)

    After optimizing the model parameters using Gaussian maximum likelihood (negative log-likelihood loss) on public non-training data, sample-dependent thresholds qα(x)q_\alpha(x) for any desired false positive rate α\alpha are computed analytically using the standard normal inverse cumulative distribution function Φ−1\Phi^{-1}:

    qα(x)=μ(x)+elog⁡σ(x)⋅Φ−1(1−α)q_\alpha(x) = \mu(x) + e^{\log \sigma(x)} \cdot \Phi^{-1}(1 - \alpha)

  6. Knowl 6 — Hinge Scoring Rule for Membership Inference

    definition

    Let f:X→[0,1]Yf: \mathcal{X} \to [0, 1]^\mathcal{Y} be a classification model, and let z(x)=(z1(x),…,z∣Y∣(x))∈R∣Y∣z(x) = (z_1(x), \dots, z_{|\mathcal{Y}|}(x)) \in \mathbb{R}^{|\mathcal{Y}|} denote the unnormalized logit vector output by the model prior to the softmax activation on input xx. The hinge scoring rule shinge(x,y)s_{\text{hinge}}(x, y) for a labeled instance (x,y)∈X×Y(x, y) \in \mathcal{X} \times \mathcal{Y} is defined as:

    shinge(x,y)=zy(x)−max⁡y′≠yzy′(x)s_{\text{hinge}}(x, y) = z_y(x) - \max_{y' \neq y} z_{y'}(x)

    This represents the logit margin between the true class yy and the most likely competing incorrect class y′≠yy' \neq y. For models with high prediction confidence, this difference closely approximates the log-odds (logit transformation) of the predicted probability:

    s(x,y)≈log⁡(fy(x))−log⁡(1−fy(x))s(x, y) \approx \log(f_y(x)) - \log(1 - f_y(x))

    and empirically follows an approximately normal distribution across non-training data points.

  7. Knowl 7 — Pinball Loss on Public Data as a Universal Predictor of MIA Success

    empirical result

    Across both computer vision and tabular classification benchmarks, the empirical pinball loss evaluated on held-out public non-member data (x,y)∼D(x, y) \sim \mathcal{D} serves as a reliable proxy and predictor for membership inference performance across different attack methodologies (including marginal quantile baselines, Likelihood Ratio Attacks (LiRA) with varying numbers of shadow models, and single-model quantile regression attacks).

    Specifically:

    1. The attack method that attains the lowest test pinball loss PB1−α\text{PB}_{1-\alpha} on public data consistently achieves the highest true positive rate (TPR) and precision at that false positive rate (FPR) α\alpha.
    2. On tasks where shadow-model attacks outperform directly trained quantile regression models (such as CIFAR-10), the shadow models produce decision thresholds that achieve lower pinball loss on public test data than the trained quantile regressor. This demonstrates that attack effectiveness is fundamentally driven by the quality of (1−α)(1-\alpha)-quantile estimation.

  8. Knowl 8 — Precision Comparison of Membership Inference Attacks on ResNet-50 Vision Benchmarks

    data/table

    Comparison of membership inference precision at 1% and 0.1% false positive rates (FPR) on ResNet-50 target models trained on 50% of the training split across four vision datasets: CIFAR-10 (C-10), CIFAR-100 (C-100), CINIC-10 (CIN10), and ImageNet-1k (IN-1k). Target models achieved test accuracies of 91.0% (C-10), 68.6% (C-100), and 67.5% (IN-1k). The quantile regression attack uses a single pretrained ConvNeXt-Tiny architecture (and ResNet-50 on CINIC-10) without knowledge of the target architecture.

    Precision @ 1% FPR Precision @ 0.1% FPR
    Method C-10 C-100 CIN10 IN-1k C-10 C-100 CIN10 IN-1k
    Marginal Baseline 48.56% 58.81% 49.02% 47.62% 60.94% 65.75% 45.76% 46.81%
    LiRA (n=2n=2) 78.55% 95.21% 55.43% 62.70% 83.18% 98.65% 54.07% 56.04%
    LiRA (n=4n=4) 80.52% 95.87% 78.71% 89.11% 91.48% 98.94% 86.99% 95.18%
    LiRA (n=6n=6) 83.19% 96.20% 88.75% 93.74% 93.17% 99.02% 96.40% 98.38%
    LiRA (n=8n=8) 83.00% 96.07% 91.86% 94.57% 93.70% 98.98% 97.94% 98.73%
    Quantile MIA (Ours) 62.95% 79.57% 76.67% 97.45% 64.48% 85.41% 85.46% 99.64%

    On the complex ImageNet-1k dataset, the single-model quantile regression attack outperforms LiRA with 8 shadow models at both 1% FPR (97.45% vs. 94.57%) and 0.1% FPR (99.64% vs. 98.73%). On CINIC-10, quantile MIA achieves performance comparable to 4 shadow models. On smaller CIFAR datasets, quantile regression improves substantially over the marginal baseline but achieves lower precision than multi-shadow-model LiRA.

  9. Knowl 9 — Precision of Quantile Regression Attacks on Tabular Benchmarks

    data/table

    Precision comparison between LiRA (evaluated with 16, 32, and 64 shadow models) and the single-model quantile regression attack on tabular datasets. Target classifiers and attack models are Gradient Boosted Decision Trees trained with CatBoost. Evaluations use 1% FPR on OpenML datasets (~5,000 samples) and 0.5% FPR on US Census American Community Survey (ACS) New York datasets (~20,000 samples).

    Precision @ 1% FPR (OpenML) Precision @ 0.5% FPR (ACS NY)
    Method Task 361057 Task 361064 Task 361067 Task 361070 Coverage Income Travel Mobility
    LiRA (n=16n=16) 70.33% 85.44% 85.52% 73.33% 66.07% 50.71% 66.07% 72.74%
    LiRA (n=32n=32) 76.22% 88.52% 89.62% 78.35% 66.28% 52.53% 67.52% 68.84%
    LiRA (n=64n=64) 82.73% 90.31% 89.46% 79.57% 69.23% 50.24% 65.64% 65.26%
    Quantile MIA (Ours) 83.35% 88.05% 87.35% 86.54% 67.31% 56.35% 63.98% 85.27%

    A single quantile regression tree model matches or exceeds the precision of LiRA utilizing 16 to 64 shadow models across tabular tasks (such as 85.27% vs 65.26% on ACS Mobility, and 86.54% vs 79.57% on OpenML Task 361070), while requiring only the computational budget of training a single shadow model.

  10. Knowl 10 — Degradation of Shadow Model Attacks Under Architecture Mismatch

    empirical result

    The effectiveness of shadow-model likelihood ratio attacks (LiRA) depends heavily on knowing the target model's architecture. When evaluated across 6 architecture configurations on CIFAR-10 and CIFAR-100 (comprising 3 CNN models with 32 filters and 1 to 3 pooling layers, and 4 Wide Residual Networks of depth 28 with widths 1, 2, 5, and 10):

    1. Small architecture mismatches within the same model family result in modest performance degradation.
    2. Substantial architecture mismatches—particularly deploying simpler shadow models against more complex target models or cross-family CNN vs. WRN mismatches—cause the attack precision at 0.1% FPR to decrease sharply from >90% down to 30%–45%.

    In contrast, the quantile regression membership inference attack is architecture-agnostic, maintaining high attack performance using a fixed independent architecture (such as ConvNeXt-Tiny) without requiring knowledge of the target model's internal structure or training configuration.

Coverage note — Extended per-architecture precision and pinball loss tables for ResNet-10, 18, and 34 on CIFAR-10 and CIFAR-100 from Appendix E were omitted in favor of the primary ResNet-50 and cross-dataset comparative tables.

References

  1. 1.T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
  2. 2.O. Bastani, V. Gupta, C. Jung, G. Noarov, R. Ramalingam, and A. Roth. Practical adversarial multivalid conformal prediction. In Neural Information Processing Systems (NeurIPS), 2022.
  3. 3.J. Bergstra, D. Yamins, and D. Cox. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In International conference on machine learning, pages 115–123. PMLR, 2013.
  4. 4.N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022.
  5. 5.L. N. Darlow, E. J. Crowley, A. Antoniou, and A. J. Storkey. Cinic-10 is not imagenet or cifar-10. arXiv preprint arXiv:1810.03505, 2018.
  6. 6.F. Ding, M. Hardt, J. Miller, and L. Schmidt. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021.
  7. 7.L. Grinsztajn, E. Oyallon, and G. Varoquaux. Why do tree-based models still outperform deep learning on tabular data? arXiv preprint arXiv:2207.08815, 2022.
  8. 8.V. Gupta, C. Jung, G. Noarov, M. M. Pai, and A. Roth. Online multivalid learning: Means, moments, and prediction intervals. In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022.
  9. 9.K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. arxiv 2015. arXiv preprint arXiv:1512.03385, 14, 2015.
  10. 10.U. Hébert-Johnson, M. Kim, O. Reingold, and G. Rothblum. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, pages 1939–1948. PMLR, 2018.
  11. 11.N. Homer, S. Szelinger, M. Redman, D. Duggan, W. Tembe, J. Muehling, J. V. Pearson, D. A. Stephan, S. F. Nelson, and D. W. Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLOS Genetics, 4(8): 1–9, 08 2008. doi: 10.1371/journal.pgen.1000167. URL https://doi.org/10.1371/journal.pgen.1000167.
  12. 12.B. Jayaraman, L. Wang, K. Knipmeyer, Q. Gu, and D. Evans. Revisiting membership inference under realistic assumptions. arXiv preprint arXiv:2005.10881, 2020.
  13. 13.B. Jayaraman, L. Wang, K. Knipmeyer, Q. Gu, and D. Evans. Revisiting membership inference under realistic assumptions. Proceedings on Privacy Enhancing Technologies, 2021:348–368, 04 2021.
  14. 14.C. Jung, G. Noarov, R. Ramalingam, and A. Roth. Batch multivalid conformal prediction. In International Conference on Learning Representations (ICLR), 2023.
  15. 15.A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
  16. 16.L. Li, K. Jamieson, A. Rostamizadeh, E. Gonina, J. Ben-Tzur, M. Hardt, B. Recht, and A. Talwalkar. A system for massively parallel hyperparameter tuning. Proceedings of Machine Learning and Systems, 2:230–246, 2020.
  17. 17.Z. Li and Y. Zhang. Membership leakage in label-only exposures. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2020.
  18. 18.R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, and I. Stoica. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118, 2018.
  19. 19.Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022.
  20. 20.Y. Long, V. Bindschaedler, L. Wang, D. Bu, X. Wang, H. Tang, C. A. Gunter, and K. Chen. Understanding membership inferences on well-generalized learning models. CoRR, abs/1802.04889, 2018. URL http://arxiv.org/abs/1802.04889.
  21. 21.Y. Long, L. Wang, D. Bu, V. Bindschaedler, X. Wang, H. Tang, C. A. Gunter, and K. Chen. A pragmatic approach to membership inferences on machine learning models. In 2020 IEEE European Symposium on Security and Privacy (EuroS&P), pages 521–534. IEEE, 2020.
  22. 22.J. Neyman and E. S. Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society A, 231:289–337, 1933.
  23. 23.G. Noarov and A. Roth. The scope of multicalibration: Characterizing multicalibration via property elicitation. International Conference on Machine Learning (ICML), 2023.
  24. 24.A. Roth. Uncertain: Modern topics in uncertainty estimation. https://www.cis.upenn.edu/ aaroth/uncertainty-notes.pdf, 2022.
  25. 25.O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015. doi: 10.1007/s11263-015-0816-y.
  26. 26.A. Sablayrolles, M. Douze, C. Schmid, Y. Ollivier, and H. Jégou. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pages 5558–5567. PMLR, 2019.
  27. 27.A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246, 2018.
  28. 28.R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017.
  29. 29.L. Song and P. Mittal. Systematic evaluation of privacy risks of machine learning models. In USENIX Security Symposium, volume 1, page 4, 2021.
  30. 30.L. Watson, C. Guo, G. Cormode, and A. Sablayrolles. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440, 2021.
  31. 31.Y. Wen, A. Bansal, H. Kazemi, E. Borgnia, M. Goldblum, J. Geiping, and T. Goldstein. Canary in a coalmine: Better membership inference with ensembled adversarial queries. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=b7SBTEBFnC.
  32. 32.J. Ye, A. Maddi, S. K. Murakonda, and R. Shokri. Enhanced membership inference attacks against machine learning models. CoRR, abs/2111.09679, 2021. URL https://arxiv.org/abs/2111.09679.
  33. 33.S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268–282. IEEE, 2018.
  34. 34.S. Zagoruyko and N. Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.

Citation

MLA
Bertran, M., et al. “Scalable Membership Inference Attacks via Quantile Regression”. Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 314–30, https://proceedings.neurips.cc/paper_files/paper/2023/file/01328d0767830e73a612f9073e9ff15f-Paper-Conference.pdf.
APA
Bertran, M., Tang, S., Roth, A., Kearns, M., Morgenstern, J. H., & Wu, S. (2023). Scalable Membership Inference Attacks via Quantile Regression. Advances in Neural Information Processing Systems, 36, 314–330. https://proceedings.neurips.cc/paper_files/paper/2023/file/01328d0767830e73a612f9073e9ff15f-Paper-Conference.pdf
Chicago
Bertran, M., S. Tang, A. Roth, M. Kearns, J. H. Morgenstern, and S. Wu. 2023. “Scalable Membership Inference Attacks via Quantile Regression”. Advances in Neural Information Processing Systems 36: 314–30. https://proceedings.neurips.cc/paper_files/paper/2023/file/01328d0767830e73a612f9073e9ff15f-Paper-Conference.pdf.
Harvard
Bertran, M. et al. (2023) “Scalable Membership Inference Attacks via Quantile Regression”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 314–330. Available at: https://proceedings.neurips.cc/paper_files/paper/2023/file/01328d0767830e73a612f9073e9ff15f-Paper-Conference.pdf.
Vancouver
1. Bertran M, Tang S, Roth A, Kearns M, Morgenstern JH, Wu S (2023) Scalable Membership Inference Attacks via Quantile Regression. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 314–330

BibTeX

@inproceedings{bertran2023scalable,
  title = {Scalable Membership Inference Attacks via Quantile Regression},
  author = {Bertran, Martin and Tang, Shuai and Roth, Aaron and Kearns, Michael and Morgenstern, Jamie H. and Wu, Steven},
  year = {2023},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {36},
  pages = {314-330},
  url = {https://proceedings.neurips.cc/paper_files/paper/2023/file/01328d0767830e73a612f9073e9ff15f-Paper-Conference.pdf}
}
Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: Authors