Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure

Shaohua FanXiao WangYanhu MoChuan ShiJian Tang

article2022NeurIPS151 citations

Proposes a debiasing framework that separates causal from spurious graph substructures using an edge mask generator and counterfactual synthesis, significantly improving graph neural network generalization on biased datasets.

Listen

Graph Neural Networks (GNNs) are widely used for complex graph-classification tasks, such as predicting molecular properties and classifying social or image network data. However, real-world training datasets frequently contain severe biases where irrelevant, spurious background patterns strongly correlate with target labels. When exposed to such data, standard GNNs tend to rely on these superficial bias shortcuts rather than learning true causal substructures, leading to catastrophic performance failures when deployed in unbiased or shifting environments.

To address this vulnerability, the article investigates the impact of severe bias on GNNs and proposes DisC, a general debiasing framework designed to separate causal substructures from bias substructures and eliminate spurious correlations. The approach employs a shared edge mask generator to divide input graphs into causal and bias subgraphs from a global dataset perspective. It then trains two dedicated GNN modules using specialized loss functions—a generalized cross-entropy loss that forces the bias module to capture easy-to-learn spurious features, and a weighted loss that focuses the causal module on true predictive patterns. Finally, the framework generates counterfactual unbiased samples in the latent embedding space by randomly shuffling bias representations across graphs, breaking the dependency between causal and bias features.

The framework was evaluated across three newly constructed benchmark graph datasets (CMNIST-75sp, CFashion-75sp, and CKuzushiji-75sp) with controllable bias degrees ranging from 80% to 95%. The experimental results demonstrated that standard GNNs experienced severe performance drops as bias increased, whereas DisC significantly improved out-of-distribution generalization. Across the tested datasets, DisC achieved average performance improvements between 4.47% and 40.02% in moderate bias scenarios (80%) and dramatic average improvements between 14.67% and 169.17% in severe bias scenarios (90% to 95%) compared to base architectures. Additionally, DisC maintained stable predictive accuracy when exposed to entirely unseen biases in testing sets, while competing debiasing baselines struggled.

These findings indicate that relying on unmitigated GNNs in biased environments creates major operational and accuracy risks. By isolating causal substructures, DisC not only enhances reliability and robustness against distribution shifts but also improves model interpretability by generating identifiable causal subgraphs. Furthermore, the learned edge masks proved transferable, allowing organizations to purify biased datasets to boost the accuracy of other standard GNN models without retraining the full framework.

Organizations deploying GNNs in high-stakes domains should adopt causal debiasing frameworks to audit and mitigate shortcut learning before production release. While the framework demonstrates high empirical confidence and consistency across multiple architectures, evaluation was conducted on controlled visual graph benchmarks. Stakeholders should conduct pilot validation on complex real-world domain graphs, such as chemical molecules or financial networks, to confirm performance boundaries across varied graph topologies.

Beijing University of Posts and Telecommunications CIFAR HEC Montréal Mila – Québec Artificial Intelligence Institute bipartite graphs classification graph neural networks causal GNN causal substructure graph neural networks

arXiv: 2209.14107 googlebaba/DisC

Paper: GNNExplainer: Generating Explanations for Graph Neural Networks, Rex Ying et al. (2019). Introduces parameterized edge and feature masking optimization to isolate critical explanatory subgraphs in GNNs, providing the foundational substructure extraction technique adapted for causal and bias subgraph separation.
Paper: Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations, Francesco Locatello et al. (2018). Establishes essential theoretical foundations and necessary inductive biases for disentangling latent variables, underpinning the methodology for separating causal factors from spurious bias.
Paper: Isolating Sources of Disentanglement in Variational Autoencoders, Ricky T. Q. Chen et al. (2018). Provides key techniques for isolating sources of variation and penalizing total correlation across latent variables to achieve effective statistical decorrelation.
Paper: Equivalence and Synthesis of Causal Models, Tom S. Verma et al. (1990). Presents core causal modeling principles for distinguishing genuine causal mechanisms from spurious statistical associations in graphical structures.
Paper: How Powerful are Graph Neural Networks?, Keyulu Xu et al. (2019). Characterizes the expressive capacity of message-passing graph neural networks for whole-graph representation and classification tasks.
Paper: Semi-Supervised Classification with Graph Convolutional Networks, Thomas N. Kipf et al. (2017). Introduces the standard graph convolutional network architecture that serves as the baseline encoder module for graph representation learning.

No sufficiently relevant recommendations were found.

Cover for Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure

Abstract

Most Graph Neural Networks (GNNs) predict the labels of unseen graphs by learning the correlation between the input graphs and labels. However, by presenting a graph classification investigation on the training graphs with severe bias, surprisingly, we discover that GNNs always tend to explore the spurious correlations to make decision, even if the causal correlation always exists. This implies that existing GNNs trained on such biased datasets will suffer from poor generalization capability. By analyzing this problem in a causal view, we find that disentangling and decorrelating the causal and bias latent variables from the biased graphs are both crucial for debiasing. Inspired by this, we propose a general disentangled GNN framework to learn the causal substructure and bias substructure, respectively. Particularly, we design a parameterized edge mask generator to explicitly split the input graph into causal and bias subgraphs. Then two GNN modules supervised by causal/bias-aware loss functions respectively are trained to encode causal and bias subgraphs into their corresponding representations. With the disentangled representations, we synthesize the counterfactual unbiased training samples to further decorrelate causal and bias variables. Moreover, to better benchmark the severe bias problem, we construct three new graph datasets, which have controllable bias degrees and are easier to visualize and explain. Experimental results well demonstrate that our approach achieves superior generalization performance over existing baselines. Furthermore, owing to the learned edge mask, the proposed model has appealing interpretability and transferability.

Topics

bipartite graphs classification graph neural networks interpretability latent variable training sample

Keywords

causal GNN causal substructure graph neural networks bias subgraphs debiasing GNNs counterfactual unbiased samples disentangled causal substructure bias classifier

1 Introduction
2 Related Works
3 Preliminary Study and Analysis
3.1 Motivating Example
3.2 Problem Analysis
4 Methodology
4.1 Causal and Bias Substructure Generator
4.2 Learning Disentangled Graph Representations
4.3 Counterfactual Unbiased Sample Generation
5 Experiment
5.1 Quantitative Evaluation
5.2 Qualitative Evaluation
6 Conclusion
Acknowledgments and Disclosure of Funding
References
Checklist

Knowls

Knowl 1 — DisC: Debiasing Graph Neural Networks via Disentangled Causal Substructures
model/method
DisC is a general framework designed to eliminate spurious correlations in graph classification under severe bias distributions. Given an observed graph $G$ , DisC separates the causal substructure $G_c$ (which deterministically dictates the ground-truth label $Y$ ) from the bias substructure $G_b$ (which correlates with $Y$ during training but is non-causal).

DisC consists of three core components:
1. Parameterized Edge Mask Generator: A shared Multi-Layer Perceptron (MLP) operating on node features scores edge probabilities to decompose the input graph $G = \{A, X\}$ into a causal subgraph $G_c = \{M_c \odot A, X\}$ and a bias subgraph $G_b = \{M_b \odot A, X\}$ , where $A$ is the adjacency matrix, $X$ is the node feature matrix, $\odot$ is the Hadamard product, and $M_c, M_b$ are soft edge masks.
2. Disentangled Representation Learning: Two separate Graph Neural Network (GNN) modules—a causal GNN $g_c$ and a bias GNN $g_b$ —encode $G_c$ and $G_b$ into causal embeddings $z_c = g_c(G_c; \gamma_c)$ and bias embeddings $z_b = g_b(G_b; \gamma_b)$ . The bias model is supervised with Generalized Cross-Entropy (GCE) loss to capture dominant bias shortcuts, while the causal model is supervised with a sample-reweighted cross-entropy loss prioritizing samples on which the bias model fails.
3. Counterfactual Unbiased Sample Generation: In latent embedding space, bias vectors are randomly permuted across mini-batch instances to synthesize counterfactual composite embeddings $z_{\text{unbiased}} = [z_c; \hat{z}_b]$ paired with swapped bias labels $\hat{y}$ . Optimizing on these decorrelated representations removes the statistical dependence between causal and bias representations.
During inference, predictions are produced strictly using the causal branch classifier $C_c(z_c)$ , ensuring invariant and debiased graph classification.
Knowl 2 — Parameterized Edge Mask Generator for Causal Subgraph Extraction
model/method
In the DisC framework, an input graph $G = \{A, X\}$ with adjacency matrix $A \in \{0, 1\}^{N \times N}$ and node feature matrix $X = [x_1, x_2, \dots, x_N]^\top \in \mathbb{R}^{N \times d}$ is decomposed into a causal subgraph $G_c$ and a bias subgraph $G_b$ via a parameterized edge mask generator.

For each edge $(i, j)$ with $A_{ij} = 1$ , the generator uses a Multi-Layer Perceptron (MLP) operating on the concatenated node feature vectors $x_i, x_j \in \mathbb{R}^d$ : $\alpha_{ij} = \text{MLP}([x_i, x_j])$ $\begin{aligned} c_{ij} &= \sigma(\alpha_{ij}) \\ b_{ij} &= 1 - c_{ij} \end{aligned}$ where $\sigma(t) = \frac{1}{1 + e^{-t}}$ is the sigmoid activation function, $c_{ij} \in (0, 1)$ represents the probability that edge $(i, j)$ belongs to the causal subgraph, and $b_{ij}$ represents the probability that it belongs to the bias subgraph.

The soft causal and bias edge masks are defined as $M_c = [c_{ij}] \in (0, 1)^{N \times N}$ and $M_b = [b_{ij}] \in (0, 1)^{N \times N}$ , yielding the weighted subgraphs: $G_c = \{M_c \odot A, X\}, \quad G_b = \{M_b \odot A, X\}$ where $\odot$ denotes the element-wise Hadamard product. Sharing the MLP parameters across all edges and all graphs allows the generator to capture population-level statistical dependencies and generalize to prune unseen graph structures at inference time without retraining.
Knowl 3 — Disentangled Graph Representation Learning Objective
equation
Given causal subgraph $G_c$ and bias subgraph $G_b$ , a causal GNN $g_c(\cdot; \gamma_c)$ and a bias GNN $g_b(\cdot; \gamma_b)$ produce latent embeddings $z_c = g_c(G_c; \gamma_c)$ and $z_b = g_b(G_b; \gamma_b)$ . The concatenated embedding $z = [z_c; z_b]$ is passed to linear classifiers $C_c(z; \alpha_c)$ and $C_b(z; \alpha_b)$ .

To force $g_b$ and $C_b$ (parameterized by $\theta_b = [\gamma_b, \alpha_b]$ ) to focus on easily learnable shortcut bias patterns, they are trained using Generalized Cross-Entropy (GCE) loss with hyperparameter $q \in (0, 1]$ : $\text{GCE}(C_b(z; \alpha_b), y) = \frac{1 - \left(C_b^y(z; \alpha_b)\right)^q}{q}$ where $C_b^y(z; \alpha_b)$ is the softmax output assigned by $C_b$ to the ground-truth target class $y$ . The gradient with respect to $\theta_b$ scales with $(C_b^y)^q$ : $\frac{\partial \text{GCE}(C_b(z; \alpha_b), y)}{\partial \theta_b} = \left(C_b^y(z; \alpha_b)\right)^q \frac{\partial \text{CE}(C_b(z; \alpha_b), y)}{\partial \theta_b}$ which heavily up-weights gradients on easily predicted biased samples.

To train the causal branch $(g_c, C_c)$ , an unbias weight $W(z)$ is computed using the standard cross-entropy (CE) loss of the bias classifier: $W(z) = \frac{\text{CE}(C_b(z), y)}{\text{CE}(C_c(z), y) + \text{CE}(C_b(z), y)}$ where a higher $W(z)$ indicates that the sample violates the dominant bias pattern.

The overall disentanglement objective $L_D$ is: $L_D = W(z) \text{CE}(C_c(z), y) + \text{GCE}(C_b(z), y)$ Loss gradients from $C_b$ are not backpropagated into $g_c$ , and gradients from $C_c$ are not backpropagated into $g_b$ .
Knowl 4 — Counterfactual Latent Augmentation for Causal and Bias Decorrelation
model/method
Even after extracting causal embeddings $z_c$ and bias embeddings $z_b$ , they inherit statistical correlations from the biased training data. To decorrelate $z_c$ and $z_b$ , DisC generates counterfactual unbiased samples directly in the latent space.

Within each training mini-batch, the bias representations $\{z_b^{(k)}\}_{k=1}^B$ and their corresponding labels $\{y^{(k)}\}_{k=1}^B$ are randomly permuted into $\{\hat{z}_b^{(k)}\}_{k=1}^B$ and $\{\hat{y}^{(k)}\}_{k=1}^B$ . Unbiased counterfactual representations are formed by pairing the original causal embedding with the shuffled bias embedding: $z_{\text{unbiased}} = [z_c; \hat{z}_b]$

The generation objective $L_G$ trains the GNNs and classifiers on these counterfactual representations: $L_G = W(z) \text{CE}(C_c(z_{\text{unbiased}}), y) + \text{GCE}(C_b(z_{\text{unbiased}}), \hat{y})$ where $W(z) = \frac{\text{CE}(C_b(z), y)}{\text{CE}(C_c(z), y) + \text{CE}(C_b(z), y)}$ is the unbias sample weight, $\text{CE}$ is cross-entropy loss, and $\text{GCE}$ is generalized cross-entropy loss.

The overall training loss of DisC is: $L = L_D + \lambda_G L_G$ where $\lambda_G > 0$ is a weighting hyperparameter and $L_D$ is the disentanglement loss. During the initial training epochs, the model is trained exclusively with $L_D$ until representations are sufficiently disentangled, after which the model is trained on the full objective $L$ .
Knowl 5 — Structural Causal Model Formulation of Biased Graph Classification
theoretical result
The data-generating process and model prediction mechanism in biased graph classification are formalized via a Structural Causal Model (SCM) comprising five variables:
- $C$ : unobserved latent causal variable (e.g., ground-truth digit subgraph),
- $B$ : unobserved latent bias variable (e.g., background color subgraph),
- $G$ : observed graph,
- $E$ : learned graph embedding,
- $Y$ : ground-truth label / model prediction.
The structural dependencies in standard GNNs are:
1. $C \to G \leftarrow B$ : the observed graph $G$ is generated jointly by latent causal and bias variables.
2. $C \to Y$ : the causal variable $C$ is the sole endogenous parent determining the ground-truth label $Y$ .
3. $C \leftrightarrow B$ : spurious statistical correlation between $C$ and $B$ introduced during data collection.
4. $G \to E \to Y$ : standard GNNs construct an entangled embedding $E$ from $G$ to predict $Y$ .
By $d$ -connection theory, two unblocked paths induce spurious correlations between the bias variable $B$ and the prediction $Y$ :
- Path 1: $B \to G \to E \to Y$
- Path 2: $B \leftrightarrow C \to Y$
To make prediction $Y$ independent of bias $B$ , two interventions are required:
1. Block Path 1 by disentangling $C \leftarrow G \to B$ and predicting $Y$ strictly from $C$ via $C \to Y$ .
2. Block Path 2 by severing the correlation $C \leftrightarrow B$ via counterfactual latent sample generation.

Knowl 6 — Graph Classification Performance of DisC on Seen Biased Benchmarks

data/table

The classification accuracy (%) of DisC and baseline methods was evaluated on unbiased test sets for three biased graph datasets (CMNIST-75sp, CFashion-75sp, CKuzushiji-75sp) across training bias degrees $\rho \in \{0.8, 0.9, 0.95\}$ . The test sets share the same bias feature (color) domain as the training set, but with uniform, uncorrelated label-color distributions. Results are reported over 4 runs (mean $\pm$ standard error).

Dataset	CMNIST-75sp			CFashion-75sp			CKuzushiji-75sp
Bias Degree	0.8	0.9	0.95	0.8	0.9	0.95	0.8	0.9	0.95
FactorGCN	$72.30 \pm 1.18$	$62.35 \pm 5.07$	$42.50 \pm 4.91$	$61.23 \pm 1.11$	$53.50 \pm 1.29$	$45.78 \pm 2.40$	$42.87 \pm 1.19$	$32.35 \pm 2.79$	$23.87 \pm 0.12$
DiffPool	$73.79 \pm 0.02$	$66.45 \pm 0.78$	$47.12 \pm 1.04$	$62.82 \pm 0.53$	$57.50 \pm 0.39$	$50.86 \pm 0.20$	$45.46 \pm 0.65$	$36.18 \pm 0.19$	$27.45 \pm 0.26$
DIR	$9.98 \pm 0.33$	$9.96 \pm 0.23$	$10.03 \pm 0.27$	$13.02 \pm 1.92$	$12.80 \pm 1.67$	$11.98 \pm 1.41$	$10.35 \pm 0.32$	$10.72 \pm 0.27$	$10.59 \pm 0.46$
StableGNN	$77.65 \pm 1.64$	$68.87 \pm 1.74$	$51.33 \pm 0.87$	$64.03 \pm 0.29$	$58.26 \pm 0.09$	$51.46 \pm 0.39$	$49.41 \pm 0.09$	$39.30 \pm 0.12$	$28.26 \pm 0.14$
LDD $_{\text{GCN}}$	$64.95 \pm 1.22$	$56.65 \pm 2.18$	$46.83 \pm 2.88$	$63.85 \pm 1.17$	$64.30 \pm 0.89$	$62.28 \pm 0.48$	$42.38 \pm 0.33$	$38.75 \pm 0.49$	$33.08 \pm 0.59$
LDD $_{\text{GIN}}$	$64.88 \pm 1.45$	$50.59 \pm 1.07$	$31.23 \pm 2.48$	$64.65 \pm 0.63$	$57.10 \pm 0.43$	$53.38 \pm 0.47$	$37.83 \pm 0.54$	$28.97 \pm 0.18$	$22.13 \pm 0.34$
LDD $_{\text{GCNII}}$	$78.03 \pm 0.66$	$69.53 \pm 0.96$	$51.05 \pm 3.87$	$50.63 \pm 1.79$	$54.09 \pm 2.54$	$57.93 \pm 0.88$	$48.70 \pm 1.98$	$41.59 \pm 1.07$	$33.93 \pm 0.71$
GCN	$50.43 \pm 4.13$	$28.97 \pm 4.4$	$13.50 \pm 1.38$	$63.60 \pm 0.53$	$57.22 \pm 0.93$	$47.69 \pm 0.42$	$38.45 \pm 1.1$	$28.35 \pm 0.79$	$20.70 \pm 0.88$
DisC $_{\text{GCN}}$	$\mathbf{82.60 \pm 0.93}$	$\mathbf{78.14 \pm 2.14}$	$\mathbf{63.47 \pm 5.65}$	$\mathbf{66.85 \pm 1.11}$	$\mathbf{65.33 \pm 4.70}$	$\mathbf{63.93 \pm 1.50}$	$\mathbf{55.53 \pm 2.29}$	$\mathbf{48.13 \pm 2.59}$	$\mathbf{36.63 \pm 1.73}$
GIN	$57.75 \pm 0.78$	$36.78 \pm 5.55$	$16.04 \pm 1.14$	$64.25 \pm 0.46$	$58.03 \pm 0.40$	$49.74 \pm 0.60$	$41.83 \pm 0.78$	$30.09 \pm 0.87$	$21.18 \pm 1.63$
DisC $_{\text{GIN}}$	$\mathbf{82.10 \pm 1.50}$	$\mathbf{74.90 \pm 1.81}$	$\mathbf{58.58 \pm 4.24}$	$\mathbf{67.10 \pm 1.07}$	$\mathbf{59.90 \pm 1.31}$	$\mathbf{55.80 \pm 0.36}$	$\mathbf{55.18 \pm 1.00}$	$\mathbf{41.75 \pm 0.81}$	$\mathbf{30.25 \pm 1.63}$
GCNII	$69.70 \pm 1.73$	$57.68 \pm 1.68$	$41.00 \pm 3.75$	$66.68 \pm 0.59$	$60.58 \pm 0.28$	$53.18 \pm 0.08$	$48.53 \pm 0.25$	$36.23 \pm 0.20$	$25.60 \pm 0.76$
DisC $_{\text{GCNII}}$	$\mathbf{79.50 \pm 2.48}$	$\mathbf{76.00 \pm 1.90}$	$\mathbf{60.54 \pm 5.33}$	$\mathbf{66.47 \pm 1.77}$	$\mathbf{65.48 \pm 0.70}$	$\mathbf{61.75 \pm 0.27}$	$\mathbf{54.90 \pm 1.30}$	$\mathbf{44.73 \pm 1.55}$	$\mathbf{36.95 \pm 0.70}$

DisC consistently outperforms all base models (GCN, GIN, GCNII) and baseline debiasing techniques across all datasets, with performance improvements reaching up to 169.17% over base GNNs under high bias (bias degree 0.95).

Knowl 7 — Generalization Performance on Unseen Unbiased Graph Classification

data/table

The out-of-distribution robustness of DisC was evaluated on testing sets with unseen bias features, where the spurious background colors are entirely disjoint from those in the training set. Classification accuracy (%) is reported across training bias degrees $\rho \in \{0.8, 0.9, 0.95\}$ over 4 runs (mean $\pm$ standard error).

Dataset	CMNIST-75sp			CFashion-75sp			CKuzushiji-75sp
Bias Degree	0.8	0.9	0.95	0.8	0.9	0.95	0.8	0.9	0.95
DIR	$10.38 \pm 0.28$	$10.14 \pm 0.40$	$9.77 \pm 0.18$	$16.77 \pm 1.71$	$16.51 \pm 3.20$	$12.59 \pm 1.61$	$10.48 \pm 0.34$	$10.33 \pm 0.75$	$10.59 \pm 0.95$
GCN	$36.88 \pm 5.16$	$23.07 \pm 4.07$	$11.88 \pm 0.33$	$59.33 \pm 0.55$	$53.65 \pm 0.47$	$45.60 \pm 1.06$	$36.35 \pm 0.48$	$27.88 \pm 0.94$	$19.95 \pm 0.67$
DisC $_{\text{GCN}}$	$82.73 \pm 1.31$	$77.70 \pm 0.87$	$65.48 \pm 0.76$	$67.90 \pm 1.45$	$68.28 \pm 0.18$	$63.77 \pm 1.37$	$57.80 \pm 2.38$	$51.60 \pm 0.41$	$41.60 \pm 3.94$
GIN	$48.93 \pm 2.99$	$34.95 \pm 0.86$	$14.53 \pm 0.97$	$58.88 \pm 0.57$	$53.80 \pm 0.52$	$48.43 \pm 0.69$	$39.25 \pm 0.57$	$30.75 \pm 1.45$	$22.35 \pm 0.86$
DisC $_{\text{GIN}}$	$77.80 \pm 1.33$	$73.00 \pm 0.61$	$58.80 \pm 1.66$	$67.15 \pm 0.79$	$59.98 \pm 0.62$	$51.70 \pm 0.34$	$55.47 \pm 0.98$	$43.20 \pm 1.36$	$31.33 \pm 1.71$
GCNII	$53.50 \pm 6.23$	$45.52 \pm 2.26$	$32.60 \pm 5.66$	$58.85 \pm 1.89$	$53.98 \pm 0.85$	$46.97 \pm 1.38$	$39.93 \pm 0.88$	$30.33 \pm 1.17$	$23.09 \pm 1.83$
DisC $_{\text{GCNII}}$	$79.65 \pm 2.13$	$76.63 \pm 1.38$	$60.00 \pm 5.66$	$60.50 \pm 2.77$	$63.05 \pm 2.25$	$61.78 \pm 1.60$	$56.23 \pm 3.45$	$49.10 \pm 2.05$	$41.05 \pm 0.11$

While vanilla GNNs experience severe degradation under unseen bias shifts (e.g., GCN on CMNIST-75sp dropping to $11.88\%$ at $\rho = 0.95$ ), DisC achieves high, stable accuracy across all backbones and bias degrees, verifying that the learned causal edge masks identify true invariant substructures rather than memorizing bias features.

Knowl 8 — Colored Superpixel Graph Benchmarks for Biased Graph Classification
experimental setup
To benchmark GNN generalization under controlled spurious correlations, three graph classification datasets were established: CMNIST-75sp, CFashion-75sp, and CKuzushiji-75sp, derived from MNIST, Fashion-MNIST, and Kuzushiji-MNIST respectively.
1. Biased Image Generation: Each class category is assigned a pre-defined background color. For a chosen bias degree $\rho \in \{0.8, 0.85, 0.9, 0.95\}$ , a proportion $\rho$ of training images in each class receive their assigned color (biased samples), while the remaining $1 - \rho$ proportion receive random background colors (unbiased samples).
2. Superpixel Graph Construction: Images are converted into superpixel graphs with at most 75 nodes per graph using the SLIC algorithm. Node feature vectors $x_i \in \mathbb{R}^5$ are formed by concatenating the 2D spatial coordinates $(u_i, v_i)$ and mean RGB values $(r_i, g_i, b_i)$ of each superpixel. Edges are generated using $k$ -Nearest Neighbors ( $k$ -NN) over the 2D coordinates.
3. Evaluation Splits:
  - Biased Test Set: Matches the training bias degree $\rho$ and color palette to quantify model reliance on spurious shortcuts.
  - Seen Unbiased Test Set: Background colors from the training color palette are distributed uniformly and independently across classes.
  - Unseen Unbiased Test Set: Background colors are drawn from a disjoint color palette not seen during training, distributed uniformly across classes.
Knowl 9 — Transferability of DisC Learned Edge Masks for Graph Pruning
empirical result
The parameterized edge mask generator trained within $\text{DisC}_{\text{GCN}}$ extracts edge weights $c_{ij} \in (0, 1)$ that isolate causal graph substructures independently of the downstream GNN architecture.

To verify transferability, edges in biased training graphs were pruned by removing the edges with the lowest $p\%$ weights ( $p \in \{0\%, 20\%, 40\%, 60\%\}$ ) as predicted by the frozen $\text{DisC}_{\text{GCN}}$ mask generator, while preserving the remaining continuous edge weights. Training standard, vanilla GIN and GCNII architectures on these pruned graphs yielded substantial accuracy gains over the same models trained on the raw biased graphs across all pruning thresholds. This shows that the learned edge masker functions as an architecture-agnostic graph purifier that successfully removes spurious bias edges.
Knowl 10 — Ablation Analysis of DisC Subgraph Masking and Counterfactual Generation
empirical result
Ablation experiments comparing DisC with and without counterfactual sample generation (DisC vs. DisC w/o G) against feature-level debiasing LDD with and without generation (LDD vs. LDD w/o G) across CMNIST-75sp, CFashion-75sp, and CKuzushiji-75sp demonstrate two key properties:
1. Necessity of Edge Masking: DisC w/o G significantly outperforms LDD w/o G across all datasets and bias levels, showing that explicit topological subgraph decomposition via edge masking is essential for graph debiasing compared to feature-level representation debiasing alone.
2. Role of Well-Disentangled Embeddings in Sample Generation: Counterfactual sample generation boosts the performance of DisC across all datasets, whereas adding generation to LDD often degrades performance. Because counterfactual sample synthesis combines representations across graphs, it introduces disruptive noise unless the underlying causal and bias representations are cleanly disentangled by graph-level substructure extraction.

Coverage note — None was omitted; all core contributions including the theoretical SCM analysis, edge mask generator, disentanglement loss, counterfactual sample generation, synthetic benchmark construction, quantitative performance tables, transferability, and ablation studies are included.

References

1.Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In ICML, 2020.
2.Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. In NeurIPS, 2019.
3.Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In ICML, 2020.
4.Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01718, 2018.
5.Luke Darlow, Stanisław Jastrzębski, and Amos Storkey. Latent adversarial debiasing: Mitigating collider bias in deep neural networks. arXiv preprint arXiv:2011.11486, 2020.
6.Shaohua Fan, Xiao Wang, Chuan Shi, Peng Cui, and Bai Wang. Generalizing graph neural networks on out-of-distribution graphs. In arXiv preprint arXiv:2111.10657, 2021.
7.Shaohua Fan, Xiao Wang, Chuan Shi, Kun Kuang, Nian Liu, and Bai Wang. Debiased graph neural networks with agnostic label selection bias. TNNLS, 2022.
8.Shaohua Fan, Xiao Wang, Chuan Shi, Emiao Lu, Ken Lin, and Bai Wang. One2multi graph autoencoder for multi-view graph clustering. In WWW, pages 3070–3076, 2020.
9.Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. Metapath-guided heterogeneous graph neural network for intent recommendation. In SIGKDD, pages 2478–2486, 2019.
10.Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.
11.Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231, 2018.
12.Madelyn Glymour, Judea Pearl, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
13.Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, 2017.
14.Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.
15.Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
16.Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In CVPR, 2019.
17.Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2016.
18.Boris Knyazev, Graham W Taylor, and Mohamed Amer. Understanding attention and generalization in graph neural networks. In NeurIPS, 2019.
19.G.E. Kvan der Maaten, L.J.P.; Hinton. Understanding attention and generalization in graph neural networks. Journal of Machine Learning Research, 2008.
20.John Boaz Lee, Ryan Rossi, and Xiangnan Kong. Graph classification using structural attention. In SIGKDD, 2018.
21.Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debiased representation via disentangled feature augmentation. In NeurIPS, 2021.
22.Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Ood-gnn: Out-of-distribution generalized graph neural network. arXiv preprint arXiv:2112.03806, 2021.
23.Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In CVPR, 2019.
24.Renjie Liao, Raquel Urtasun, and Richard Zemel. A pac-bayesian approach to generalization bounds for graph neural networks. In ICLR, 2020.
25.Yanbei Liu, Xiao Wang, Shu Wu, and Zhitao Xiao. Independence promoted graph disentangled networks. In AAAI, 2020.
26.Ana Lucic, Maartje ter Hoeve, Gabriele Tolomei, Maarten de Rijke, and Fabrizio Silvestri. Cf-gnnexplainer: Counterfactual explanations for graph neural networks. In AISTATS, 2022.
27.Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurIPS, 2020.
28.Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In ICML, pages 4212–4221. PMLR, 2019.
29.Jiaqi Ma, Junwei Deng, and Qiaozhu Mei. Subgroup generalization and fairness of graph neural networks. In NeurIPS, 2021.
30.Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. In NeurIPS, 2020.
31.Judea Pearl. Causality. Cambridge university press, 2009.
32.Hans Reichenbach. The direction of time, volume 65. Univ of California Press, 1991.
33.Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR, 2019.
34.Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2008.
35.Petar Velićkovi ć, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2017.
36.Minh Vu and My T Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In NeurIPS, 2020.
37.Haohan Wang, Zexue He, Zachary C Lipton, and Eric P Xing. Learning robust representations by projecting superficial statistics out. arXiv preprint arXiv:1903.06256, 2019.
38.Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective. In ICLR, 2022.
39.Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. In ICLR, 2022.
40.Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
41.Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
42.Yiding Yang, Zunlei Feng, Mingli Song, and Xinchao Wang. Factorizable graph convolutional networks. In NeurIPS, volume 33, pages 20286–20296, 2020.
43.Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, 2019.
44.Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differentiable pooling. In NeurIPS, volume 31, 2018.
45.Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In ICML, 2021.
46.Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In AAAI, 2018.
47.Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. NeurIPS, 2018.

Citation

MLA

Fan, S., et al. “Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure”. Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 24934–46, https://proceedings.neurips.cc/paper_files/paper/2022/file/9e47a0bc530cc88b09b7670d2c130a29-Paper-Conference.pdf.

APA

Fan, S., Wang, X., Mo, Y., Shi, C., & Tang, J. (2022). Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure. Advances in Neural Information Processing Systems, 35, 24934–24946. https://proceedings.neurips.cc/paper_files/paper/2022/file/9e47a0bc530cc88b09b7670d2c130a29-Paper-Conference.pdf

Chicago

Fan, S., X. Wang, Y. Mo, C. Shi, and J. Tang. 2022. “Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure”. Advances in Neural Information Processing Systems 35: 24934–46. https://proceedings.neurips.cc/paper_files/paper/2022/file/9e47a0bc530cc88b09b7670d2c130a29-Paper-Conference.pdf.

Harvard

Fan, S. et al. (2022) “Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 24934–24946. Available at: https://proceedings.neurips.cc/paper_files/paper/2022/file/9e47a0bc530cc88b09b7670d2c130a29-Paper-Conference.pdf.

Vancouver

1. Fan S, Wang X, Mo Y, Shi C, Tang J (2022) Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 24934–24946

BibTeX

@inproceedings{fan2022debiasing,
  title = {Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure},
  author = {Fan, Shaohua and Wang, Xiao and Mo, Yanhu and Shi, Chuan and Tang, Jian},
  year = {2022},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {35},
  pages = {24934-24946},
  url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/9e47a0bc530cc88b09b7670d2c130a29-Paper-Conference.pdf}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — DisC: Debiasing Graph Neural Networks via Disentangled Causal Substructures

Knowl 2 — Parameterized Edge Mask Generator for Causal Subgraph Extraction

Knowl 3 — Disentangled Graph Representation Learning Objective

Knowl 4 — Counterfactual Latent Augmentation for Causal and Bias Decorrelation

Knowl 5 — Structural Causal Model Formulation of Biased Graph Classification