Federated Machine Learning

Qiang YangYang LiuTianjian ChenYongxin Tong

article2019ACM TIST3,335 citationsFrontiers of Science Award

Establishes a foundational categorization of horizontal, vertical, and transfer federated learning, providing the architectural blueprint for training machine learning models collaboratively across isolated organizational data silos without compromising user privacy.

Listen

Data fragmentation across organizations combined with stricter privacy regulations now blocks many AI projects that once relied on pooling raw records. Laws such as the EU’s GDPR and similar statutes in China and the United States prohibit the transfer of personal data to a central site, while competitive and administrative barriers keep useful data sets isolated even inside single firms. The paper therefore sets out to define and survey secure federated learning as a practical way for multiple parties to train a shared model without exposing their local data.

The authors review prior work on privacy-preserving machine learning, extend Google’s original on-device framework, and introduce a three-way classification—horizontal, vertical, and federated transfer learning—based on how data samples and features overlap among participants. They describe system architectures that rely on secure multi-party computation, differential privacy, or homomorphic encryption, and they outline the communication steps required for each category. The survey also covers incentive mechanisms that use blockchain-style ledgers to allocate benefits according to each party’s data contribution.

The central finding is that federated protocols can produce models whose accuracy is nearly identical to those trained on pooled data while satisfying formal privacy definitions under honest-but-curious or semi-honest assumptions. Horizontal learning works when organizations share the same features but different users; vertical learning aggregates complementary features for the same users; transfer learning extends coverage when both features and users differ. Concrete applications in retail recommendation, multi-bank fraud detection, and medical diagnosis illustrate that the approach can operate across industry and national boundaries.

These results imply that organizations can now form data alliances that comply with current law, reduce the legal and reputational risks of data sharing, and still capture most of the performance gains previously available only through centralized collection. The framework also shifts industry attention from incremental model tuning toward the design of privacy-compliant data networks.

The authors recommend that enterprises adopt federated mechanisms as the technical standard for cross-organization modeling and pair them with transparent contribution ledgers to encourage participation. They note that further work on malicious-party defenses, communication efficiency, and standardized evaluation benchmarks is still required before large-scale deployments can be considered routine.

Advanced Innovation Center for Big Data and Brain Computing Beihang University The Hong Kong University of Science and Technology WeBank federated learning network architecture privacy and security Federated Learning Secure Multi‑party Computation Differential Privacy

arXiv: 1902.04885 FederatedAI/FATE

Paper: Federated Learning: Strategies for Improving Communication Efficiency, Jakub Konečný et al. (2016). This foundational study on communication-efficient federated learning strategies provides the core gradient compression and subsampling techniques that the source paper categorizes and builds upon.
Paper: Federated Optimization in Heterogeneous Networks, Tian Li et al. (2018). Understanding this formulation of federated optimization over heterogeneous networks is essential for grasping the challenges of non-identical data distributions addressed in the source survey.

Paper: Federated Learning: Challenges, Methods, and Future Directions, Tian Li et al. (2019). This comprehensive follow-up survey expands directly upon the taxonomy and challenges established in the source framework, providing deeper methodological insights for federated learning.
Paper: SCAFFOLD: Stochastic Controlled Averaging for Federated Learning, Sai Praneeth Karimireddy et al. (2019). Building directly on the federated frameworks reviewed in the source, this work introduces stochastic controlled averaging to solve convergence issues caused by client data heterogeneity.

Abstract

Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.

2 AN OVERVIEW OF FEDERATED LEARNING
2.1 Definition of Federated Learning
2.2 Privacy of Federated Learning
2.3 A Categorization of Federated Learning
2.3.1 Horizontal Federated Learning
2.4 Architecture for a federated learning system
2.4.1 Horizontal Federated Learning.
3 RELATED WORKS
3.1 Privacy-preserving machine learning
3.2 Federated Learning vs Distributed Machine Learning
3.3 Federated Learning vs Edge Computing
3.4 Federated Learning vs Federated Database Systems
4 APPLICATIONS
5 FEDERATED LEARNING AND DATA ALLIANCE OF ENTERPRISES
6 CONCLUSIONS AND PROSPECTS

Knowls

Knowl 1 — Definition of Federated Learning and Delta-Accuracy Loss
definition
Federated learning is a distributed machine learning paradigm where $N$ data owners $\{F_1, \dots, F_N\}$ collaboratively train a joint model $M_{FED}$ using their respective private local datasets $\{D_1, \dots, D_N\}$ such that no data owner $F_i$ exposes its raw data $D_i$ to any other party.

Let $M_{SUM}$ denote the conventional centralized model trained by aggregating all data together, $D = D_1 \cup \dots \cup D_N$ , with performance metric (such as accuracy) denoted by $V_{SUM}$ , and let $V_{FED}$ denote the performance of the federated model $M_{FED}$ . For a non-negative real tolerance parameter $\delta \ge 0$ , a federated learning algorithm is defined to have $\delta$ -accuracy loss if:

$|V_{FED} - V_{SUM}| < \delta$

When $\delta = 0$ , the federated learning process is lossless, meaning the federated model achieves the exact performance of a model trained on fully centralized data.
Knowl 2 — Categorization of Federated Learning by Feature and Sample ID Partitioning
definition
Let $D_i$ denote the dataset held by data owner $i$ , represented by $(I_i, X_i, Y_i)$ , where $I_i$ is the sample ID space, $X_i$ is the feature space, and $Y_i$ is the label space. Federated learning is categorized into three distinct classes based on data partitioning across these spaces:
1. Horizontal Federated Learning (Sample-based FL): Applicable when participating datasets share identical feature spaces but contain different sample instances:
$X_i = X_j, \quad Y_i = Y_j, \quad I_i \neq I_j \quad (\forall D_i, D_j, i \neq j)$
1. Vertical Federated Learning (Feature-based FL): Applicable when participating datasets share the same or overlapping sample ID instances but differ in feature spaces:
$X_i \neq X_j, \quad Y_i \neq Y_j, \quad I_i = I_j \quad (\forall D_i, D_j, i \neq j)$
1. Federated Transfer Learning (FTL): Applicable when participating datasets differ in both feature spaces and sample ID instances, having only a small subset of overlapping features and samples:
$X_i \neq X_j, \quad Y_i \neq Y_j, \quad I_i \neq I_j \quad (\forall D_i, D_j, i \neq j)$
Knowl 3 — Loss and Gradient Formulation for Additively Encrypted Vertical Federated Linear Regression
model/method
In two-party vertical federated linear regression, Party A holds feature vectors $\{x_i^A\}_{i \in D_A}$ and Party B holds feature vectors and labels $\{x_i^B, y_i\}_{i \in D_B}$ for aligned common sample IDs $i \in D_A \cap D_B$ . The learning objective parameterized by weight vectors $\Theta_A$ and $\Theta_B$ with $L_2$ regularization parameter $\lambda$ is:

$\min_{\Theta_A, \Theta_B} \sum_i \|\Theta_A x_i^A + \Theta_B x_i^B - y_i\|^2 + \frac{\lambda}{2}\left(\|\Theta_A\|^2 + \|\Theta_B\|^2\right)$

Let $[[\cdot]]$ denote encryption under an additive homomorphic cryptosystem, and define $u_i^A = \Theta_A x_i^A$ and $u_i^B = \Theta_B x_i^B$ . The regularized loss decomposes into three additive components:

$[[L]] = [[L_A]] + [[L_B]] + [[L_{AB}]]$

where:

$[[L_A]] = \left[\left[\sum_i (u_i^A)^2 + \frac{\lambda}{2}\|\Theta_A\|^2\right]\right]$

$[[L_B]] = \left[\left[\sum_i (u_i^B - y_i)^2 + \frac{\lambda}{2}\|\Theta_B\|^2\right]\right]$

$[[L_{AB}]] = 2 \sum_i [[u_i^A]] (u_i^B - y_i)$

Defining intermediate error term $[[d_i]] = [[u_i^A]] + [[u_i^B - y_i]]$ , the gradients with respect to local parameters $\Theta_A$ and $\Theta_B$ are:

$\left[\left[\frac{\partial L}{\partial \Theta_A}\right]\right] = \sum_i [[d_i]] x_i^A + [[\lambda \Theta_A]]$

$\left[\left[\frac{\partial L}{\partial \Theta_B}\right]\right] = \sum_i [[d_i]] x_i^B + [[\lambda \Theta_B]]$

These formulations allow exact gradient computation without exposing raw features $x_i^A, x_i^B$ or label $y_i$ between participants.
Knowl 4 — Secure Training Protocol for Vertical Federated Linear Regression
algorithm
The training protocol for vertical federated linear regression coordinates two semi-honest data owners (Party A with features $x_i^A$ and Party B with features $x_i^B$ and labels $y_i$ ) and an honest/semi-honest third party (STP, Party C) who does not collude with either party. Additive homomorphic encryption is denoted by $[[\cdot]]$ .
Input: Aligned sample dataset $\{x_i^A\}_{i \in D_A}$ at Party A, $\{x_i^B, y_i\}_{i \in D_B}$ at Party B, regularization parameter $\lambda$ , learning rate $\eta$
Output: Learned model weights $\Theta_A$ at Party A and $\Theta_B$ at Party B
Party C creates an additively homomorphic encryption key pair and sends the public key to Party A and Party B
Party A initializes model weights $\Theta_A$ ; Party B initializes model weights $\Theta_B$
repeat
    Party A computes $[[u_i^A]] = [[\Theta_A x_i^A]]$ and $[[L_A]] = [[\sum_i (u_i^A)^2 + \frac{\lambda}{2}\|\Theta_A\|^2]]$ for all $i$
    Party A sends $\{[[u_i^A]]\}_i$ and $[[L_A]]$ to Party B
    Party B computes $[[u_i^B]] = [[\Theta_B x_i^B]]$ and $[[d_i^B]] = [[u_i^B - y_i]]$ for all $i$
    Party B computes $[[L_B]] = [[\sum_i (u_i^B - y_i)^2 + \frac{\lambda}{2}\|\Theta_B\|^2]]$ and $[[L_{AB}]] = 2 \sum_i [[u_i^A]](u_i^B - y_i)$
    Party B computes total loss $[[L]] = [[L_A]] + [[L_B]] + [[L_{AB}]]$ and sends $[[L]]$ to Party C
    Party B sends $\{[[d_i^B]]\}_i$ to Party A
    Party A computes $[[d_i]] = [[u_i^A]] + [[d_i^B]]$ and gradient $[[\frac{\partial L}{\partial \Theta_A}]] = \sum_i [[d_i]] x_i^A + [[\lambda \Theta_A]]$
    Party A samples random mask $R_A$ and sends $[[\frac{\partial L}{\partial \Theta_A}]] + [[R_A]]$ to Party C
    Party B computes gradient $[[\frac{\partial L}{\partial \Theta_B}]] = \sum_i [[d_i]] x_i^B + [[\lambda \Theta_B]]$
    Party B samples random mask $R_B$ and sends $[[\frac{\partial L}{\partial \Theta_B}]] + [[R_B]]$ to Party C
    Party C decrypts $[[L]]$ to obtain loss $L$
    Party C decrypts masked gradients to obtain $(\frac{\partial L}{\partial \Theta_A} + R_A)$ and $(\frac{\partial L}{\partial \Theta_B} + R_B)$
    Party C sends $(\frac{\partial L}{\partial \Theta_A} + R_A)$ to Party A, and $(\frac{\partial L}{\partial \Theta_B} + R_B)$ to Party B
    Party A updates $\Theta_A \leftarrow \Theta_A - \eta ((\frac{\partial L}{\partial \Theta_A} + R_A) - R_A)$
    Party B updates $\Theta_B \leftarrow \Theta_B - \eta ((\frac{\partial L}{\partial \Theta_B} + R_B) - R_B)$
until loss $L$ converges
Knowl 5 — Collaborative Inference Protocol for Vertical Federated Linear Regression
algorithm
During inference in vertical federated learning, each party holds only the sub-model parameter corresponding to its own feature space ( $\Theta_A$ at Party A and $\Theta_B$ at Party B). To predict the target output for a specific sample $i$ without revealing features or weights to each other, an inquisitor or coordinator C coordinates the joint prediction.
Input: Sample ID $i$ , Party A with feature $x_i^A$ and weight $\Theta_A$ , Party B with feature $x_i^B$ and weight $\Theta_B$
Output: Prediction result $\hat{y}_i$ for sample $i$ computed at Inquisitor C
Inquisitor C sends target user ID $i$ to Party A and Party B
Party A computes $u_i^A = \Theta_A x_i^A$ and sends $u_i^A$ to Inquisitor C
Party B computes $u_i^B = \Theta_B x_i^B$ and sends $u_i^B$ to Inquisitor C
Inquisitor C computes the final predicted value $\hat{y}_i = u_i^A + u_i^B$
This procedure allows prediction across vertically partitioned data while keeping local feature values and model parameters private to their respective owners.
Knowl 6 — Horizontal Federated Learning Architecture and Training Workflow
model/method
In horizontal federated learning (HFL), $K$ client participants holding datasets with identical feature spaces collaboratively train a shared global model with the coordination of a central parameter server. The standard system setting assumes honest participants and a semi-honest (honest-but-curious) server.

The iterative training workflow proceeds as follows:
1. Local Gradient Computation and Masking: Each participant $k$ computes local model training gradients on its private data. To prevent raw gradient leakage, the participant masks the gradient vector using additive homomorphic encryption, differential privacy noise injection, or secret sharing protocols before transmitting the masked results to the central server.
2. Secure Aggregation: The central server aggregates the masked gradients received from all $K$ clients without learning individual client gradients or raw data contributions.
3. Broadcast Updates: The server sends the securely aggregated global gradient update back to all participating clients.
4. Local Model Update: Each participant unmasks or decrypts the aggregated global update and applies it to update its local model parameters.
Upon training convergence, all participants obtain identical parameters for the universal federated model.
Knowl 7 — Federated Transfer Learning Framework
model/method
Federated Transfer Learning (FTL) enables multi-party collaborative learning when participating institutions have minimal overlap in both sample IDs ( $I_A \cap I_B$ is small) and feature spaces ( $X_A \cap X_B$ is small).

FTL operates through the following mechanisms:
- Common Representation Learning: Using the small subset of overlapping entities identified via encrypted entity alignment (private set intersection), FTL learns shared feature representations across disparate feature spaces $X_A$ and $X_B$ .
- Cross-Domain Knowledge Transfer: The model leverages supervisory label information available primarily in a source domain (e.g., Party B) to minimize prediction errors on the target domain (e.g., Party A), allowing Party A to obtain predictions for samples lacking target-side labels or overlapping features.
- Privacy-Preserving Gradient Exchange: Gradients and intermediate loss terms for the representation layers are exchanged across domains using homomorphic encryption or secret sharing with a semi-honest third party, ensuring that non-overlapping raw features and user records remain completely private.
- Collaborative Inference: At inference time, generating predictions for target instances requires both parties to collaboratively evaluate their respective parts of the learned shared representation.
Knowl 8 — Privacy and Security Guarantees in Horizontal and Vertical Federated Learning
theoretical result
The security guarantees and threat models of federated learning differ across architectures:
1. Horizontal Federated Learning Security:
  - Threat Model: Assumes honest participants and an honest-but-curious coordinator server.
  - Guarantees: Gradient encryption via additive homomorphic encryption or secure multi-party aggregation prevents the server from reconstructing local client data.
  - Vulnerabilities: Direct exposure of unmasked intermediate stochastic gradients can leak training data structure (e.g., image pixels). Collaborative deep learning under HFL is vulnerable to malicious participants executing Generative Adversarial Network (GAN) inference attacks or model-poisoning backdoor attacks.
2. Vertical Federated Learning Security:
  - Threat Model: Assumes two honest-but-curious, non-colluding participants (Party A and Party B) and an honest, non-colluding Semi-honest Third Party (STP, Party C).
  - Guarantees: Party C observes only masked gradients $[[\frac{\partial L}{\partial \Theta_A}]] + [[R_A]]$ and $[[\frac{\partial L}{\partial \Theta_B}]] + [[R_B]]$ , guaranteeing zero information disclosure about gradients or raw features to C. Neither Party A nor Party B can recover the other party's features $x_i$ or model parameters $\Theta$ from intermediate scalar products $u_i = \Theta x_i$ , because inferring $x_i$ from scalar products reduces to solving an underdetermined system of equations when the sample size $N_A$ is much larger than the feature dimension $n_A$ ( $N_A \gg n_A$ ).
  - Malicious Participant Detection: If a malicious participant injects forged sparse inputs (e.g., a single non-zero feature) to probe intermediate values, the resulting gradient distortions disrupt subsequent iterations, enabling divergence detection and protocol termination.
Knowl 9 — Incentive Mechanisms and Enterprise Data Alliances in Federated Learning
model/method
Federated learning incorporates an economic and business model for multi-enterprise data alliances, structuring collaborative AI as a closed-loop system:
- Contribution-Based Profit Allocation: Participating organizations contribute heterogeneous private datasets to train a collective model. The actual performance and commercial value generated by the joint model are recorded on an immutable ledger (e.g., via blockchain consensus mechanisms).
- Consensus and Incentive Alignment: Organizations contributing larger or higher-quality data partitions receive proportionally higher profit allocations and utility from the federated system.
- Closed-Loop Data Ecosystem: By decoupling model utility from centralized data pooling, the incentive mechanism motivates enterprises with isolated data silos to continually join the federation without risking data ownership, proprietary intellectual property, or regulatory non-compliance.

Coverage note — None omitted; all core conceptual contributions, formal definitions, categorical frameworks, loss/gradient equations, algorithms, and security analyses from the paper have been captured.

References

1.Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, New York, NY, USA, 308–318. https://doi.org/10.1145/2976749.2978318
2.Abbas Acar, Hidayet Aksu, A. Selcuk Uluagac, and Mauro Conti. 2018. A Survey on Homomorphic Encryption Schemes: Theory and Implementation. ACM Comput. Surv. 51, 4, Article 79 (July 2018), 35 pages. https://doi.org/10.1145/3214303
3.Rakesh Agrawal and Ramakrishnan Srikant. 2000. Privacy-preserving Data Mining. In Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data (SIGMOD ’00). ACM, New York, NY, USA, 439–450. https://doi.org/10.1145/342009.335438
4.Yoshinori Aono, Takuya Hayashi, Le Trieu Phong, and Lihua Wang. 2016. Scalable and Secure Logistic Regression via Homomorphic Encryption. In Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy (CODASPY ’16). ACM, New York, NY, USA, 142–144. https://doi.org/10.1145/2857705.2857731
5.Toshinori Araki, Jun Furukawa, Yehuda Lindell, Ariel Nof, and Kazuma Ohara. 2016. High-Throughput Semi-Honest Secure Three-Party Computation with an Honest Majority. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, New York, NY, USA, 805–817. https://doi.org/10.1145/2976749.2978331
6.Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. 2018. How To Backdoor Federated Learning. arXiv:cs.CR/1807.00459
7.Raad Bahmani, Manuel Barbosa, Ferdinand Brasser, Bernardo Portela, Ahmad-Reza Sadeghi, Guillaume Scerri, and Bogdan Warinschi. 2017. Secure Multiparty Computation from SGX. In Financial Cryptography and Data Security - 21st International Conference, FC 2017, Sliema, Malta, April 3-7, 2017, Revised Selected Papers. 477–497. https://doi.org/10.1007/978-3-319-70972-7_27
8.Dan Bogdanov, Sven Laur, and Jan Willemson. 2008. Sharemind: A Framework for Fast Privacy-Preserving Computations. In Proceedings of the 13th European Symposium on Research in Computer Security: Computer Security (ESORICS ’08). Springer-Verlag, Berlin, Heidelberg, 192–206. https://doi.org/10.1007/978-3-540-88313-5_13
9.Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17). ACM, New York, NY, USA, 1175–1191. https://doi.org/10.1145/3133956.3133982
10.Florian Bourse, Michele Minelli, Matthias Minihold, and Pascal Paillier. 2017. Fast Homomorphic Evaluation of Deep Discretized Neural Networks. IACR Cryptology ePrint Archive 2017 (2017), 1114.
11.Hervé Chabanne, Amaury de Wargny, Jonathan Milgram, Constance Morel, and Emmanuel Prouff. 2017. Privacy-Preserving Classification on Deep Neural Network. IACR Cryptology ePrint Archive 2017 (2017), 35.
12.Kamalika Chaudhuri and Claire Monteleoni. 2009. Privacy-preserving logistic regression. In Advances in Neural Information Processing Systems 21, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.). Curran Associates, Inc., 289–296. http://papers.nips.cc/paper/3486-privacy-preserving-logistic-regression.pdf
13.Fei Chen, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. 2018. Federated Meta-Learning for Recommendation. CoRR abs/1802.07876 (2018). arXiv:1802.07876 http://arxiv.org/abs/1802.07876
14.Nathan Dowlin, Ran Gilad-Bachrach, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. Technical Report. https://www.microsoft.com/en-us/research/publication/cryptonets-applying-neural-networks-to-encrypted-data-with-high-throughput-and-accuracy/
15.W. Du and M. Atallah. 2001. Privacy-Preserving Cooperative Statistical Analysis. In Proceedings of the 17th Annual Computer Security Applications Conference (ACSAC ’01). IEEE Computer Society, Washington, DC, USA, 102–. http://dl.acm.org/citation.cfm?id=872016.872181
16.Wenliang Du, Yunghsiang Sam Han, and Shigang Chen. 2004. Privacy-Preserving Multivariate Statistical Analysis: Linear Regression and Classification. In SDM.
17.Wenliang Du and Zhijun Zhan. 2002. Building Decision Tree Classifier on Private Data. In Proceedings of the IEEE International Conference on Privacy, Security and Data Mining - Volume 14 (CRPIT ’14). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 1–8. http://dl.acm.org/citation.cfm?id=850782.850784
18.Cynthia Dwork. 2008. Differential Privacy: A Survey of Results. In Proceedings of the 5th International Conference on Theory and Applications of Models of Computation (TAMC’08). Springer-Verlag, Berlin, Heidelberg, 1–19. http://dl.acm.org/citation.cfm?id=1791834.1791836
19.EU. 2016. REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Available at: https://eur-lex. europa. eu/legal-content/EN/TXT (2016).
20.Boi Faltings, Goran Radanovic, and Ronald Brachman. 2017. Game Theory for Data Science: Eliciting Truthful Information. Morgan & Claypool Publishers.
21.Jun Furukawa, Yehuda Lindell, Ariel Nof, and Or Weinstein. 2016. High-Throughput Secure Three-Party Computation for Malicious Adversaries and an Honest Majority. Cryptology ePrint Archive, Report 2016/944. https://eprint.iacr.org/2016/944.
22.Adrià Gascón, Phillipp Schoppmann, Borja Balle, Mariana Raykova, Jack Doerner, Samee Zahur, and David Evans. 2016. Secure Linear Regression on Vertically Partitioned Datasets. IACR Cryptology ePrint Archive 2016 (2016), 892.
23.Robin C. Geyer, Tassilo Klein, and Moin Nabi. 2017. Differentially Private Federated Learning: A Client Level Perspective. CoRR abs/1712.07557 (2017). arXiv:1712.07557 http://arxiv.org/abs/1712.07557
24.Irene Giacomelli, Somesh Jha, Marc Joye, C. David Page, and Kyonghwan Yoon. 2017. Privacy-Preserving Ridge Regression with only Linearly-Homomorphic Encryption. Cryptology ePrint Archive, Report 2017/979. https://eprint.iacr.org/2017/979.
25.O. Goldreich, S. Micali, and A. Wigderson. 1987. How to Play ANY Mental Game. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing (STOC ’87). ACM, New York, NY, USA, 218–229. https://doi.org/10.1145/28395.28420
26.Rob Hall, Stephen E. Fienberg, and Yuval Nardi. 2011. Secure multiple linear regression based on homomorphic encryption. Journal of Official Statistics 27, 4 (2011), 669–691.
27.Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne. 2017. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. CoRR abs/1711.10677 (2017).
28.Ehsan Hesamifard, Hassan Takabi, and Mehdi Ghasemi. 2017. CryptoDL: Deep Neural Networks over Encrypted Data. CoRR abs/1711.05189 (2017). arXiv:1711.05189 http://arxiv.org/abs/1711.05189
29.Briland Hitaj, Giuseppe Ateniese, and Fernando Pérez-Cruz. 2017. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. CoRR abs/1702.07464 (2017).
30.Qirong Ho, James Cipar, Henggang Cui, Jin Kyu Kim, Seunghak Lee, Phillip B. Gibbons, Garth A. Gibson, Gregory R. Ganger, and Eric P. Xing. 2013. More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 1 (NIPS’13). Curran Associates Inc., USA, 1223–1231. http://dl.acm.org/citation.cfm?id=2999611.2999748
31.Murat Kantarcioglu and Chris Clifton. 2004. Privacy-Preserving Distributed Mining of Association Rules on Horizontally Partitioned Data. IEEE Trans. on Knowl. and Data Eng. 16, 9 (Sept. 2004), 1026–1037. https://doi.org/10.1109/TKDE.2004.45
32.Alan F. Karr, X. Sheldon Lin, Ashish P. Sanil, and Jerome P. Reiter. 2004. Privacy-Preserving Analysis of Vertically Partitioned Data Using Secure Matrix Products.
33.Niki Kilbertus, Adria Gascon, Matt Kusner, Michael Veale, Krishna Gummadi, and Adrian Weller. 2018. Blind Justice: Fairness with Encrypted Sensitive Attributes. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol. 80. PMLR, Stockholmsmässan, Stockholm Sweden, 2630–2639. http://proceedings.mlr.press/v80/kilbertus18a.html
34.Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. 2018. On-Device Federated Learning via Blockchain and its Latency Analysis. arXiv:cs.IT/1808.03949
35.Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, and Xiaoqian Jiang. 2018. Secure Logistic Regression Based on Homomorphic Encryption: Design and Evaluation. JMIR Med Inform 6, 2 (17 Apr 2018), e19. https://doi.org/10.2196/medinform.8805
36.Jakub Konecný, H. Brendan McMahan, Daniel Ramage, and Peter Richtárik. 2016. Federated Optimization: Distributed Machine Learning for On-Device Intelligence. CoRR abs/1610.02527 (2016). arXiv:1610.02527 http://arxiv.org/abs/1610.02527
37.Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated Learning: Strategies for Improving Communication Efficiency. CoRR abs/1610.05492 (2016). arXiv:1610.05492 http://arxiv.org/abs/1610.05492
38.Gang Liang and Sudarshan S Chawathe. 2004. Privacy-preserving inter-database operations. In International Conference on Intelligence and Security Informatics. Springer, 66–82.
39.Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J. Dally. 2017. Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training. CoRR abs/1712.01887 (2017). arXiv:1712.01887 http://arxiv.org/abs/1712.01887
40.Jian Liu, Mika Juuti, Yao Lu, and N. Asokan. 2017. Oblivious Neural Network Predictions via MiniONN Transformations. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17). ACM, New York, NY, USA, 619–631. https://doi.org/10.1145/3133956.3134056
41.H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas. 2016. Federated Learning of Deep Networks using Model Averaging. CoRR abs/1602.05629 (2016). arXiv:1602.05629 http://arxiv.org/abs/1602.05629
42.H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. 2017. Learning Differentially Private Language Models Without Losing Accuracy. CoRR abs/1710.06963 (2017).
43.Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2018. Inference Attacks Against Collaborative Learning. CoRR abs/1805.04049 (2018). arXiv:1805.04049 http://arxiv.org/abs/1805.04049
44.Payman Mohassel and Peter Rindal. 2018. ABY3: A Mixed Protocol Framework for Machine Learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS ’18). ACM, New York, NY, USA, 35–52. https://doi.org/10.1145/3243734.3243760
45.Payman Mohassel, Mike Rosulek, and Ye Zhang. 2015. Fast and Secure Three-party Computation: The Garbled Circuit Approach. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS ’15). ACM, New York, NY, USA, 591–602. https://doi.org/10.1145/2810103.2813705
46.Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. In IEEE Symposium on Security and Privacy. IEEE Computer Society, 19–38.
47.Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. IACR Cryptology ePrint Archive 2017 (2017), 396.
48.Valeria Nikolaenko, Udi Weinsberg, Stratis Ioannidis, Marc Joye, Dan Boneh, and Nina Taft. 2013. Privacy-Preserving Ridge Regression on Hundreds of Millions of Records. In Proceedings of the 2013 IEEE Symposium on Security and Privacy (SP ’13). IEEE Computer Society, Washington, DC, USA, 334–348. https://doi.org/10.1109/SP.2013.30
49.Richard Nock, Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Giorgio Patrini, Guillaume Smith, and Brian Thorne. 2018. Entity Resolution and Federated Learning get a Federated Resolution. CoRR abs/1803.04035 (2018). arXiv:1803.04035 http://arxiv.org/abs/1803.04035
50.Sinno Jialin Pan and Qiang Yang. 2010. A Survey on Transfer Learning. IEEE Trans. on Knowl. and Data Eng. 22, 10 (Oct. 2010), 1345–1359. https://doi.org/10.1109/TKDE.2009.191
51.Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. 2018. Privacy-Preserving Deep Learning via Additively Homomorphic Encryption. IEEE Trans. Information Forensics and Security 13, 5 (2018), 1333–1345.
52.M. Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M. Songhori, Thomas Schneider, and Farinaz Koushanfar. 2018. Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications. CoRR abs/1801.03239 (2018).
53.R L Rivest, L Adleman, and M L Dertouzos. 1978. On Data Banks and Privacy Homomorphisms. Foundations of Secure Computation, Academia Press (1978), 169–179.
54.Bita Darvish Rouhani, M. Sadegh Riazi, and Farinaz Koushanfar. 2017. DeepSecure: Scalable Provably-Secure Deep Learning. CoRR abs/1705.08963 (2017). arXiv:1705.08963 http://arxiv.org/abs/1705.08963
55.Ashish P. Sanil, Alan F. Karr, Xiaodong Lin, and Jerome P. Reiter. 2004. Privacy Preserving Regression Modelling via Distributed Computation. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’04). ACM, New York, NY, USA, 677–682. https://doi.org/10.1145/1014052.1014139
56.Monica Scannapieco, Ilya Figotin, Elisa Bertino, and Ahmed K. Elmagarmid. 2007. Privacy Preserving Schema and Data Matching. In Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD ’07). ACM, New York, NY, USA, 653–664. https://doi.org/10.1145/1247480.1247553
57.Amit P. Sheth and James A. Larson. 1990. Federated Database Systems for Managing Distributed, Heterogeneous, and Autonomous Databases. ACM Comput. Surv. 22, 3 (Sept. 1990), 183–236. https://doi.org/10.1145/96602.96604
58.Reza Shokri and Vitaly Shmatikov. 2015. Privacy-Preserving Deep Learning. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS ’15). ACM, New York, NY, USA, 1310–1321. https://doi.org/10.1145/2810103.2813687
59.David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484–503. http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
60.Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. 2017. Federated Multi-Task Learning. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4424–4434. http://papers.nips.cc/paper/7029-federated-multi-task-learning.pdf
61.Shuang Song, Kamalika Chaudhuri, and Anand D. Sarwate. 2013. Stochastic gradient descent with differentially private updates. 2013 IEEE Global Conference on Signal and Information Processing (2013), 245–248.
62.Lili Su and Jiaming Xu. 2018. Securing Distributed Machine Learning in High Dimensions. CoRR abs/1804.10140 (2018). arXiv:1804.10140 http://arxiv.org/abs/1804.10140
63.Latanya Sweeney. 2002. K-anonymity: A Model for Protecting Privacy. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 10, 5 (Oct. 2002), 557–570. https://doi.org/10.1142/S0218488502001648
64.Jaideep Vaidya and Chris Clifton. [n. d.]. Privacy Preserving Naive Bayes Classifier for Vertically Partitioned Data. In in Proceedings of the fourth SIAM Conference on Data Mining, 2004. 330–334.
65.Jaideep Vaidya and Chris Clifton. 2002. Privacy Preserving Association Rule Mining in Vertically Partitioned Data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’02). ACM, New York, NY, USA, 639–644. https://doi.org/10.1145/775047.775142
66.Jaideep Vaidya and Chris Clifton. 2003. Privacy-preserving K-means Clustering over Vertically Partitioned Data. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’03). ACM, New York, NY, USA, 206–215. https://doi.org/10.1145/956750.956776
67.Jaideep Vaidya and Chris Clifton. 2005. Privacy-Preserving Decision Trees over Vertically Partitioned Data. In Data and Applications Security XIX, Sushil Jajodia and Duminda Wijesekera (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 139–152.
68.Li Wan, Wee Keong Ng, Shuguo Han, and Vincent C. S. Lee. 2007. Privacy-preservation for Gradient Descent Methods. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’07). ACM, New York, NY, USA, 775–783. https://doi.org/10.1145/1281192.1281275
69.Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan. 2018. When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning. CoRR abs/1804.05271 (2018). arXiv:1804.05271 http://arxiv.org/abs/1804.05271
70.Wikipedia. 2018. https://en.wikipedia.org/wiki/Facebook-Cambridge_Analytica_data_scandal.
71.Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2018. Federated Learning. Communications of The CCF 14, 11 (2018), 49–55.
72.Andrew C. Yao. 1982. Protocols for Secure Computations. In Proceedings of the 23rd Annual Symposium on Foundations of Computer Science (SFCS ’82). IEEE Computer Society, Washington, DC, USA, 160–164. http://dl.acm.org/citation.cfm?id=1382436.1382751
73.Hwanjo Yu, Xiaoqian Jiang, and Jaideep Vaidya. 2006. Privacy-preserving SVM Using Nonlinear Kernels on Horizontally Partitioned Data. In Proceedings of the 2006 ACM Symposium on Applied Computing (SAC ’06). ACM, New York, NY, USA, 603–610. https://doi.org/10.1145/1141277.1141415
74.Hwanjo Yu, Jaideep Vaidya, and Xiaoqian Jiang. 2006. Privacy-Preserving SVM Classification on Vertically Partitioned Data. In Proceedings of the 10th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining (PAKDD’06). Springer-Verlag, Berlin, Heidelberg, 647–656. https://doi.org/10.1007/11731139_74
75.Jiawei Yuan and Shucheng Yu. 2014. Privacy Preserving Back-Propagation Neural Network Learning Made Practical with Cloud Computing. IEEE Trans. Parallel Distrib. Syst. 25, 1 (Jan. 2014), 212–221. https://doi.org/10.1109/TPDS.2013.18
76.Qingchen Zhang, Laurence T. Yang, and Zhikui Chen. 2016. Privacy Preserving Deep Computation Model on Cloud for Big Data Feature Learning. IEEE Trans. Comput. 65, 5 (May 2016), 1351–1362. https://doi.org/10.1109/TC.2015.2470255
77.Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated Learning with Non-IID Data. arXiv:cs.LG/1806.00582

Citation

MLA

Yang, Q., et al. “Federated Machine Learning”. ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, 2019, pp. 1–9, https://doi.org/10.1145/3298981.

APA

Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019). Federated Machine Learning. ACM Transactions on Intelligent Systems and Technology, 10(2), 1–19. https://doi.org/10.1145/3298981

Chicago

Yang, Q., Y. Liu, T. Chen, and Y. Tong. 2019. “Federated Machine Learning”. ACM Transactions on Intelligent Systems and Technology 10 (2): 1–19. https://doi.org/10.1145/3298981.

Harvard

Yang, Q. et al. (2019) “Federated Machine Learning”, ACM Transactions on Intelligent Systems and Technology, 10(2), pp. 1–19. Available at: https://doi.org/10.1145/3298981.

Vancouver

1. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated Machine Learning. ACM Transactions on Intelligent Systems and Technology 10:1–19

BibTeX

@article{Yang_2019, title={Federated Machine Learning: Concept and Applications}, volume={10}, ISSN={2157-6912}, url={http://dx.doi.org/10.1145/3298981}, DOI={10.1145/3298981}, number={2}, journal={ACM Transactions on Intelligent Systems and Technology}, publisher={Association for Computing Machinery (ACM)}, author={Yang, Qiang and Liu, Yang and Chen, Tianjian and Tong, Yongxin}, year={2019}, month=Jan, pages={1–19} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Federated Machine Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Definition of Federated Learning and Delta-Accuracy Loss

Knowl 2 — Categorization of Federated Learning by Feature and Sample ID Partitioning

Knowl 3 — Loss and Gradient Formulation for Additively Encrypted Vertical Federated Linear Regression

Knowl 4 — Secure Training Protocol for Vertical Federated Linear Regression

Knowl 5 — Collaborative Inference Protocol for Vertical Federated Linear Regression

Knowl 6 — Horizontal Federated Learning Architecture and Training Workflow

Knowl 7 — Federated Transfer Learning Framework

Knowl 8 — Privacy and Security Guarantees in Horizontal and Vertical Federated Learning

Knowl 9 — Incentive Mechanisms and Enterprise Data Alliances in Federated Learning

References

Citation

BibTeX

Source Code

Access the Paper