Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Paul RollandVolkan CevherMatthäus KleindessnerChris RussellDominik JanzingBernhard SchölkopfFrancesco Locatello

article2022ICML140 citations

Demonstrates how to identify causal directed acyclic graphs in nonlinear additive noise models using data score functions, introducing an efficient Jacobian approximation method that achieves competitive discovery accuracy with linear complexity in the number of nodes.

Listen

Identifying cause-and-effect relationships from purely observational data is fundamental for guiding strategic decisions, evaluating potential interventions, and assessing operational risks. However, discovering these causal structures is computationally demanding because the number of possible graph configurations grows exponentially with each additional variable. Existing algorithms frequently depend on slow, greedy search heuristics or complex continuous optimization procedures that do not scale well to larger enterprise systems.

The article demonstrates that the score function of a data distribution—defined as the gradient of the log-probability density—contains direct structural information that enables exact recovery of causal graphs in nonlinear additive noise environments. To operationalize this insight, the authors introduce SCORE, a scalable algorithm that identifies the causal ordering of variables in linear time relative to the number of nodes before pruning unnecessary connections.

The authors develop a mathematical proof establishing that leaf variables (nodes without outgoing effects) can be uniquely detected because their corresponding diagonal entries in the score's derivative matrix (the Jacobian) remain constant across observations. To compute this from sample data without expensive neural network training, the authors design a closed-form estimator utilizing second-order Stein identities and kernel methods. Once an empirical leaf is identified, it is sequentially removed to establish a full causal sequence, which is subsequently pruned using standard regression techniques. The approach was evaluated on synthetic benchmarks of 10 to 50 variables across Gaussian, Laplace, and Gumbel noise distributions, as well as on biological and pseudo-real benchmark datasets.

The evaluation yields several key findings regarding speed, structural accuracy, and algorithmic robustness. First, SCORE delivers substantial computational speed improvements, running roughly 10 times faster than the leading CAM baseline on 20-variable networks (32.7 seconds versus 313 seconds) and more than 4 times faster than GraN-DAG on 50-variable systems (257 seconds versus 1,410 seconds). Within SCORE, ordering the variables takes only a small fraction of the overall runtime—such as 31 seconds out of 257 seconds for 50 nodes—shifting the primary computational cost almost entirely to the final pruning step. Second, the method achieves structural accuracy competitive with or superior to established baselines, showing notable improvements in denser network topologies where competing methods struggle to find the correct ordering. Third, SCORE proves highly robust against noise misspecification, maintaining strong accuracy even when noise deviates from normal distributions.

These results show that score matching can effectively bypass combinatorial bottlenecks in causal discovery, lowering the computational expense and timeline required to map intricate dependencies in data-rich domains. By avoiding non-convex optimization and combinatorial searches, organizations can analyze complex interconnected systems with significantly greater confidence and reduced infrastructure runtime. The ability to handle non-Gaussian distributions further indicates that the framework is practical for real-world scenarios where data rarely conform to standard Gaussian assumptions.

Organizations seeking to implement causal discovery pipelines should consider adopting score-based ordering methods as an efficient replacement for greedy topological searches, pairing the resulting ordering with domain-appropriate pruning modules. Before full deployment on massive datasets, practitioners should implement memory-efficient kernel approximations or explore amortized deep score estimators to manage large sample volumes. Future engineering should concentrate on extending direct score-based identification across wider non-additive generative settings and refining edge-pruning techniques, which currently represent the primary computational bottleneck.

Amazon École Polytechnique Fédérale de Lausanne additive noise gaussians jacobians Score matching Nonlinear additive noise models Causal discovery

arXiv: 2203.04413 paulrolland1307/SCORE

Paper: Estimation of Non-Normalized Statistical Models by Score Matching, Aapo Hyvärinen (2005). Hyvärinen establishes score matching as a tractable way to estimate log-density gradients, the central statistical tool SCORE adapts for recovering causal structure.
Paper: A Linear Non-Gaussian Acyclic Model for Causal Discovery, Shohei Shimizu et al. (2006). LiNGAM shows how noise assumptions make causal direction identifiable from observations, providing a useful contrast to SCORE’s nonlinear additive Gaussian-noise setting.

No sufficiently relevant recommendations were found.

Cover for Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Abstract

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal discovery methods. To showcase our approach, we also propose a new efficient method for approximating the score’s Jacobian, enabling to recover the causal graph. Empirically, we find that the new algorithm, called SCORE, is competitive with state-of-the-art causal discovery methods while being significantly faster.

Topics

additive noise gaussians jacobians matching algorithm matching methods bipartite graphs

Keywords

Score matching Nonlinear additive noise models Causal discovery Structural intervention distance

Abstract
1. Introduction
2. Related Work
3. Preliminaries
3.1. Causal discovery for non-linear additive Gaussian noise models
3.2. Score matching
4. Causal Discovery via Score Matching
4.1. Deduce the causal graph from the score of the data distribution
4.2. Approximation of the score's Jacobian
4.3. Extension to non-Gaussian additive noise models
5. Numerical Experiments
5.1. Synthetic data
5.2. Real data
6. Conclusion
7. Acknowledgements
References
A. Additional Experiments

Knowls

Knowl 1 — Nonlinear additive Gaussian causal model and its score
model/method
Let $X=(X_1,ldots,X_d)inmathbb R^d$ be generated by a directed acyclic graph (DAG) through

$X_i=f_i(X_{operatorname{pa}(i)})+varepsilon_i,$

where $X_{operatorname{pa}(i)}$ denotes the parent variables of node $i$ , the noises are mutually independent with $varepsilon_isimmathcal N(0,sigma_i^2)$ , and each $f_i$ is twice continuously differentiable and nonlinear in each parent argument. Writing $s(x)=nablalog p(x)$ for the score of the joint density, its $j$ th component is

$s_j(x)=-frac{x_j-f_j(x_{operatorname{pa}(j)})}{sigma_j^2}+sum_{iinoperatorname{children}(j)}frac{partial f_i}{partial x_j}(x_{operatorname{pa}(i)})frac{x_i-f_i(x_{operatorname{pa}(i)})}{sigma_i^2}.$

Here $operatorname{children}(j)$ is the set of nodes caused directly by $j$ . The first term depends on the structural equation for $j$ ; the remaining terms arise from its children. This decomposition exposes graph information in derivatives of the observational score.
Knowl 2 — Score-Jacobian characterization of leaves and their parents
theoretical result
For the nonlinear additive Gaussian model with independent noises and link functions nonlinear in every parent variable, node $j$ is a leaf (has no children) if and only if its diagonal score derivative is constant over all inputs:

$jtext{ is a leaf}quadLongleftrightarrowquadfrac{partial s_j(x)}{partial x_j}=cquadtext{for all }x,text{ for some constant }c.$

Equivalently, for a random vector $X$ from this model, $operatorname{Var}[partial s_j(X)/partial x_j]=0$ exactly for leaves. For a leaf $j$ , a node $i$ is a parent of $j$ exactly when $s_j$ depends on $x_i$ ; equivalently, $operatorname{Var}[partial s_j(X)/partial x_i]ne 0$ . Thus the diagonal score-Jacobian variation identifies leaves, while variation in the other entries of a leaf’s score component identifies its parents.
Knowl 3 — Sequential score-based causal ordering and graph pruning
algorithm
The ordering method repeatedly finds a leaf in the graph induced by the variables still present, removes that variable, and prepends it to the ordering. With exact score derivatives, every selected node is a leaf; with estimated derivatives, the method selects the coordinate whose diagonal score derivative has the smallest empirical variance. Prepending the successively removed leaves produces a source-to-sink topological order. Once the order is fixed, the possible edges are restricted to those consistent with it, and the resulting complete ordered DAG is pruned using CAM-style sparse regression and additive-model hypothesis tests (the experiments use cutoff $0.001$ ).
Input: Observations of $d$ variables and estimates of the diagonal score derivatives for each active-variable set
Initialize active variables $A = \{1, \ldots, d\}$ and ordering $\pi = []$
While $A$ is nonempty:
    Estimate the score of the distribution on the active variables
    For each $j$ in $A$ , compute $V_j$ , the empirical variance across observations of $\partial s_j(X)/\partial x_j$
    Set $\ell$ to the active variable with the smallest $V_j$
    Prepend $\ell$ to $\pi$
    Remove variable $\ell$ from $A$ and from the observations
    Re-estimate the score derivatives for the reduced variable set
Construct the DAG containing all edges consistent with $\pi$
Prune that DAG using sparse additive regression and additive-model hypothesis tests
Output: Topological ordering $\pi$ and the pruned DAG
Knowl 4 — Kernel Stein estimation of score values and diagonal score Hessians
model/method
For observations $x^1,\ldots,x^n\in\mathbb R^d$ , the paper estimates score values and the diagonal Hessian of the log density at every observation using kernel ridge regression based on first- and second-order Stein identities. Let $K\in\mathbb R^{n\times n}$ have entries $K_{ab}=\kappa(x^a,x^b)$ , and let $I_n$ be the $n\times n$ identity matrix. For a regularization parameter $\eta>0$ , define

$D_{aj}=\sum_{k=1}^n\frac{\partial\kappa(x^a,x^k)}{\partial x_j^k},\qquad B_{aj}=\sum_{k=1}^n\frac{\partial^2\kappa(x^a,x^k)}{\partial (x_j^k)^2},$

where $x_j^k$ is coordinate $j$ of observation $k$ . The estimated score matrix $\widehat G\in\mathbb R^{n\times d}$ and estimated diagonal score-Hessian matrix $\widehat J\in\mathbb R^{n\times d}$ are

$\widehat G=-(K+\eta I_n)^{-1}D,\qquad \widehat J=-\widehat G\odot\widehat G+(K+\eta I_n)^{-1}B.$

Row $k$ of $\widehat G$ estimates $\nabla\log p(x^k)$ ; entry $(k,j)$ of $\widehat J$ estimates $\partial^2\log p(x^k)/\partial x_j^2$ . The symbol $\odot$ denotes elementwise multiplication. The implementation uses the RBF kernel $\kappa(u,v)=\exp(-\|u-v\|_2^2/(2s^2))$ , with bandwidth $s$ set to the median pairwise Euclidean distance among the observations. In sequential ordering, the bandwidth is recomputed after each variable is removed, and the same $\eta$ is used for both score and Hessian estimation.
Knowl 5 — Score decomposition for non-Gaussian additive noise
theoretical result
For an additive structural model with mutually independent, identically distributed noises having a smooth density $p_{\varepsilon}$ , the score component for variable $j$ has the form

$s_j(x)=\frac{d\log p_{\varepsilon}}{du}\bigg|_{u=x_j-f_j(x_{\operatorname{pa}(j)})}-\sum_{i\in\operatorname{children}(j)}\frac{\partial f_i}{\partial x_j}(x_{\operatorname{pa}(i)})\frac{d\log p_{\varepsilon}}{du}\bigg|_{u=x_i-f_i(x_{\operatorname{pa}(i)})}.$

Thus the score still separates a term associated with node $j$ from terms contributed by its children. For Gaussian noise, the first term is linear in $x_j$ , enabling the exact leaf criterion based on diagonal score-Jacobian constancy. For general non-Gaussian noise, the paper does not establish an analogous formal leaf-identification theorem; it proposes that child terms may still make the ordering method useful and evaluates that possibility empirically.
Knowl 6 — Computational complexity of the ordering and pruning stages
theoretical result
For $n$ observations and $d$ variables, SCORE estimates the topological order by inverting an $n\times n$ kernel matrix once for each variable-removal step, giving ordering cost $O(dn^3)$ . If $r(n,d)$ is the cost of fitting a generalized additive model to $n$ observations in $d$ dimensions, the total cost including final pruning is $O(dn^3+d\,r(n,d))$ . The paper gives CAM’s corresponding complexity as $O(d^2r(n,d))$ . In measured settings, pruning was the larger component of SCORE’s runtime: the ordering stage accounted for 30% of total time at $(d,n)=(20,1000)$ and 5% at $(50,1000)$ .
Knowl 7 — Synthetic graph recovery accuracy and robustness to noise type
empirical result
Synthetic experiments averaged results over 10 independent runs. The link functions were sampled from Gaussian processes with a unit-bandwidth RBF kernel; Gaussian-noise variances were sampled independently from $[0.4,0.8]$ . Erdős–Rényi graphs had either $d$ edges on average (ER1) or $4d$ (ER4). Laplace-noise experiments tested noise-type misspecification. All order-based methods used the same CAM pruning procedure with cutoff $0.001$ . Structural Hamming distance (SHD) counts missing, false, and reversed edges; structural intervention distance (SID) counts incorrectly estimated interventional distributions; topological-order divergence $D_{\rm top}$ counts true edges incompatible with the estimated ordering and lower-bounds final SHD.

At $d=20$ with Gaussian noise, SCORE versus CAM obtained, respectively: ER1 SHD $2.6\pm1.9$ versus $3.5\pm1.6$ , SID $9.9\pm8.5$ versus $14.3\pm9.8$ , and $D_{\rm top}=1.2\pm1.7$ versus $0.8\pm1.0$ ; ER4 SHD $47.5\pm4.5$ versus $54.2\pm5.4$ , SID $177.5\pm11.6$ versus $201.9\pm29.0$ , and $D_{\rm top}=3.1\pm1.5$ versus $13.6\pm6.9$ . With Laplace noise at $d=20$ , SCORE versus CAM obtained ER1 SHD $1.6\pm1.2$ versus $2.3\pm1.4$ and ER4 SHD $48.0\pm4.0$ versus $52.4\pm3.9$ ; ER4 $D_{\rm top}$ was $4.9\pm1.8$ versus $11.6\pm7.9$ . These results show similar or better recovery than CAM in these settings, particularly for denser graphs, and comparable performance under Laplace noise. At $d=50$ , performance was mixed in ER1 (Gaussian SHD $10.4\pm3.9$ for SCORE and $8.3\pm2.9$ for CAM), while SCORE had lower ER4 SHD ( $131.5\pm7.5$ versus $140.8\pm5.5$ ). Additional experiments covered Gumbel noise and Gaussian noise on scale-free graphs and reported broadly competitive results.
Knowl 8 — Measured runtime advantage over CAM and GraN-DAG
empirical result
On ER1 synthetic graphs, reported mean runtime in seconds (with standard deviations) for SCORE’s ordering stage, complete SCORE, CAM, and GraN-DAG was as follows. At $d=10$ : $3.3\pm0.1$ , $6.3\pm0.2$ , $30.1\pm3.7$ , and $185\pm26$ . At $d=20$ : $8.5\pm0.8$ , $32.7\pm6.7$ , $313\pm80$ , and $357\pm47$ . At $d=50$ : $31\pm2.9$ , $257\pm17$ , $1143\pm79$ , and $1410\pm73$ . Thus complete SCORE was about ten times faster than CAM at 20 nodes and about five times faster than GraN-DAG at 50 nodes. The CAM run at 50 nodes used preliminary neighbor search and capped each node’s neighbors at 20 to keep runtime manageable.
Knowl 9 — Evaluation on Sachs and SynTReN datasets
empirical result
On the Sachs real-world dataset (11 nodes, 17 edges, 853 observations), SCORE obtained SHD 12 and SID 45; CAM obtained SHD 12 and SID 55; GraN-DAG obtained SHD 13 and SID 47. SCORE therefore matched CAM’s SHD and had the lowest SID among these three methods. On 10 datasets generated by SynTReN, SCORE’s mean SHD and SID were $36.2\pm4.7$ and $193.4\pm60.2$ ; CAM’s were $40.5\pm6.8$ and $152.3\pm48.0$ ; and GraN-DAG’s were $34.0\pm8.5$ and $161.7\pm53.4$ . GraN-DAG had the lowest mean scores on SynTReN, although the reported confidence intervals overlapped substantially.

Coverage note — The exhaustive cell-by-cell values in the supplementary Gumbel-noise and scale-free synthetic tables are not reproduced; their settings and broad comparative findings are included because the full grids reiterate the reported robustness and accuracy patterns.

References

1.Barabási, A.-L. and Albert, R. Emergence of scaling in random networks. science, 286(5439):509–512, 1999.
2.Barp, A., Briol, F.-X., Duncan, A. B., Girolami, M., and Mackey, L. Minimum stein discrepancy estimators. arXiv preprint arXiv:1906.08283, 2019.
3.Bouckaert, R. R. Optimizing causal orderings for generating dags from data. In Uncertainty in Artificial Intelligence, pp. 9–16. Elsevier, 1992.
4.Bühlmann, P., Peters, J., and Ernest, J. Cam: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics, 42(6):2526–2556, 2014.
5.Cai, R., Qiao, J., Zhang, Z., and Hao, Z. Self: structural equational likelihood framework for causal discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
6.Chickering, D. M. Learning bayesian networks is np-complete. In Learning from data, pp. 121–130. Springer, 1996.
7.Chickering, D. M. Optimal structure identification with greedy search. Journal of machine learning research, 3 (Nov):507–554, 2002.
8.Cooper, G. F. and Herskovits, E. A bayesian method for the induction of probabilistic networks from data. Machine learning, 9(4):309–347, 1992.
9.Diaconis, P., Stein, C., Holmes, S., and Reinert, G. Use of exchangeable pairs in the analysis of simulations. In Stein’s Method, pp. 1–25. Institute of Mathematical Statistics, 2004.
10.Erdős, P. and Rényi, A. On the evolution of random graphs, pp. 38–82. Princeton University Press, 2011. doi: doi:10.1515/9781400841356.38. URL https://doi.org/10.1515/9781400841356.38.
11.Garreau, D., Jitkrittum, W., and Kanagawa, M. Large sample analysis of the median heuristic. arXiv preprint arXiv:1707.07269, 2017.
12.Ghoshal, A. and Honorio, J. Learning linear structural equation models in polynomial time and sample complexity. In International Conference on Artificial Intelligence and Statistics, pp. 1466–1475. PMLR, 2018.
13.Hyvärinen, A. and Dayan, P. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
14.Lachapelle, S., Brouillard, P., Deleu, T., and Lacoste-Julien, S. Gradient-based neural dag learning. arXiv preprint arXiv:1906.02226, 2019.
15.Larranaga, P., Kuijpers, C. M., Murga, R. H., and Yurramendi, Y. Learning bayesian network structures by searching for the best ordering with genetic algorithms. IEEE transactions on systems, man, and cybernetics-part A: systems and humans, 26(4):487–493, 1996.
16.Li, Y. and Turner, R. E. Gradient estimators for implicit models. arXiv preprint arXiv:1705.07107, 2017.
17.Liu, Q., Lee, J., and Jordan, M. A kernelized stein discrepancy for goodness-of-fit tests. In International conference on machine learning, pp. 276–284. PMLR, 2016.
18.Loh, P.-L. and Bühlmann, P. High-dimensional learning of linear causal networks via inverse covariance estimation. The Journal of Machine Learning Research, 15(1):3065–3105, 2014.
19.Löwe, S., Madras, D., Zemel, R., and Welling, M. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006.10833, 2020.
20.Marra, G. and Wood, S. N. Practical variable selection for generalized additive models. Computational Statistics & Data Analysis, 55(7):2372–2387, 2011.
21.Peters, J. and Bühlmann, P. Structural intervention distance for evaluating causal graphs. Neural computation, 27(3):771–799, 2015.
22.Peters, J., Mooij, J. M., Janzing, D., and Schölkopf, B. Causal discovery with continuous additive noise models. 2014.
23.Peters, J., Janzing, D., and Schölkopf, B. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
24.Raskutti, G. and Uhler, C. Learning directed acyclic graph models based on sparsest permutations. Stat, 7(1):e183, 2018.
25.Reisach, A. G., Seiler, C., and Weichwald, S. Beware of the simulated dag! varsortability in additive noise models. NeurIPS, 2021.
26.Sachs, K., Perez, O., Pe’er, D., Lauffenburger, D. A., and Nolan, G. P. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308 (5721):523–529, 2005.
27.Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE, 109 (5):612–634, 2021.
28.Si, S., Hsieh, C.-J., and Dhillon, I. Memory efficient kernel approximation. In International Conference on Machine Learning, pp. 701–709. PMLR, 2014.
29.Singh, M. and Valtorta, M. An algorithm for the construction of bayesian network structures from data. In Uncertainty in Artificial Intelligence, pp. 259–265. Elsevier, 1993.
30.Solus, L., Wang, Y., and Uhler, C. Consistency guarantees for greedy permutation-based causal inference algorithms. Biometrika, 108(4):795–814, 2021.
31.Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pp. 11918–11930, 2019.
32.Song, Y. and Ermon, S. Improved techniques for training score-based generative models. arXiv preprint arXiv:2006.09011, 2020.
33.Song, Y., Garg, S., Shi, J., and Ermon, S. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 574–584. PMLR, 2020a.
34.Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
35.Spirtes, P., Glymour, C. N., Scheines, R., and Heckerman, D. Causation, prediction, and search. MIT press, 2000.
36.Stein, C. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the sixth Berkeley symposium on mathematical statistics and probability, volume 2: Probability theory, volume 6, pp. 583–603. University of California Press, 1972.
37.Strassen, V. Gaussian elimination is not optimal. Numerische mathematik, 13(4):354–356, 1969.
38.Teyssier, M. and Koller, D. Ordering-based search: A simple and effective algorithm for learning bayesian networks. arXiv preprint arXiv:1207.1429, 2012.
39.Van den Bulcke, T., Van Leemput, K., Naudts, B., van Remortel, P., Ma, H., Verschoren, A., De Moor, B., and Marchal, K. Syntren: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC bioinformatics, 7(1):1–12, 2006.
40.Wang, X., Du, Y., Zhu, S., Ke, L., Chen, Z., Hao, J., and Wang, J. Ordering-based causal discovery with reinforcement learning. arXiv preprint arXiv:2105.06631, 2021.
41.Wilks, S. S. Mathematical statistics, 1962.
42.Zhang, J. On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence, 172(16-17):1873–1896, 2008.
43.Zheng, X., Aragam, B., Ravikumar, P., and Xing, E. P. Dags with no tears: Continuous optimization for structure learning. arXiv preprint arXiv:1803.01422, 2018.
44.Zhu, J. Hessian estimation via stein’s identity in black-box problems. arXiv preprint arXiv:2104.01317, 2021.
45.Zhu, S., Ng, I., and Chen, Z. Causal discovery with reinforcement learning. arXiv preprint arXiv:1906.04477, 2019.
46.Zimmermann, R. S., Schott, L., Song, Y., Dunn, B. A., and Klindt, D. A. Score-based generative classifiers. arXiv preprint arXiv:2110.00473, 2021.

Citation

MLA

Rolland, P., et al. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models”. International Conference on Machine Learning, vol. 162, 2022, pp. 18741–53, https://proceedings.mlr.press/v162/rolland22a.html.

APA

Rolland, P., Cevher, V., Kleindessner, M., Russell, C., Janzing, D., Schölkopf, B., & Locatello, F. (2022). Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models. International Conference on Machine Learning, 162, 18741–18753. https://proceedings.mlr.press/v162/rolland22a.html

Chicago

Rolland, P., V. Cevher, M. Kleindessner, et al. 2022. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models”. International Conference on Machine Learning 162: 18741–53. https://proceedings.mlr.press/v162/rolland22a.html.

Harvard

Rolland, P. et al. (2022) “Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models”, International Conference on Machine Learning. PMLR, pp. 18741–18753. Available at: https://proceedings.mlr.press/v162/rolland22a.html.

Vancouver

1. Rolland P, Cevher V, Kleindessner M, Russell C, Janzing D, Schölkopf B, Locatello F (2022) Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models. In: International Conference on Machine Learning. PMLR, pp 18741–18753

BibTeX

@InProceedings{pmlr-v162-rolland22a,
  title = 	 {Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models},
  author =       {Rolland, Paul and Cevher, Volkan and Kleindessner, Matth{\"a}us and Russell, Chris and Janzing, Dominik and Sch{\"o}lkopf, Bernhard and Locatello, Francesco},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {18741--18753},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/rolland22a/rolland22a.pdf},
  url = 	 {https://proceedings.mlr.press/v162/rolland22a.html},
  abstract = 	 {This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a building block, we show how to design a new generation of scalable causal discovery methods. To showcase our approach, we also propose a new efficient method for approximating the score’s Jacobian, enabling to recover the causal graph. Empirically, we find that the new algorithm, called SCORE, is competitive with state-of-the-art causal discovery methods while being significantly faster.}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Nonlinear additive Gaussian causal model and its score

Knowl 2 — Score-Jacobian characterization of leaves and their parents

Knowl 3 — Sequential score-based causal ordering and graph pruning

Knowl 4 — Kernel Stein estimation of score values and diagonal score Hessians

Knowl 5 — Score decomposition for non-Gaussian additive noise

Knowl 6 — Computational complexity of the ordering and pruning stages

Knowl 7 — Synthetic graph recovery accuracy and robustness to noise type

Knowl 8 — Measured runtime advantage over CAM and GraN-DAG

Knowl 9 — Evaluation on Sachs and SynTReN datasets

References

Citation

BibTeX

Source Code

Access the Paper