E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking

Yangtian ZhangHuiyu CaiChence ShiJian Tang

article2023ICLR55 citations

Proposes E3Bind, an end-to-end equivariant neural network that iteratively predicts flexible ligand docking poses directly from protein structures, outperforming traditional physics-based tools and existing multi-stage deep learning approaches on standard benchmarks.

Listen

Predicting how drug-like molecules bind to target proteins in the human body is a fundamental challenge in pharmaceutical research. While experimental structure determination is slow and covers only a tiny fraction of potential drug-protein pairs, computational docking methods are essential for screening vast chemical libraries. Traditional physics-based tools are slow and struggle with accuracy, whereas existing deep-learning approaches often fail because they either predict intermediate geometric distances that cannot be translated into valid 3D shapes or place molecules in a single step without allowing their shapes to adapt to the binding site. The article addresses these bottlenecks by developing and evaluating an end-to-end artificial intelligence model, named E3Bind, designed to directly and iteratively predict the precise 3D position, orientation, and flexible shape of small molecules within target proteins.

The approach introduces a specialized neural network architecture that combines a geometry-aware feature extractor with an iterative coordinate refinement module. Instead of predicting distances or placing the molecule in one shot, the model updates molecular coordinates across multiple steps, continually sensing the surrounding protein environment to resolve physical clashes and refine the fit. The model also incorporates a built-in confidence scoring mechanism to select optimal poses and flag uncertain predictions. To test the system, the authors trained and evaluated the framework on standard benchmark data comprising thousands of experimentally determined protein-ligand complexes from the PDBbind database, comparing its accuracy and speed against established physics-based software and state-of-the-art deep-learning models.

The evaluation revealed several key findings in order of importance. First, E3Bind demonstrated superior precision in flexible docking, achieving a 33% increase in high-accuracy predictions (poses within 2 angstroms of the true structure) compared to the previous leading deep-learning benchmark. Second, the system operated orders of magnitude faster than traditional physics-based tools, completing predictions in roughly two seconds on standard processors and under half a second on specialized graphics hardware, compared to minutes or hours for legacy tools. Third, iterative refinement significantly reduced physical errors; only 3.3% of test predictions produced atomic overlap clashes, compared to 21% for single-step deep-learning models. Fourth, the model showed robust binding site detection on large, challenging targets and maintained competitive performance even when tested on novel proteins that were never seen during training.

These findings suggest that end-to-end iterative modeling overcomes the major structural validity and speed trade-offs of previous docking methods. By eliminating time-consuming conformational sampling and multi-stage post-processing, the framework can substantially reduce computational costs and project timelines in virtual drug screening. Moreover, its self-confidence scoring provides drug discovery teams with a reliable filter to prioritize high-probability candidates and avoid pursuing flawed computational artifacts.

Organizations evaluating computational discovery pipelines should consider piloting iterative end-to-end models like E3Bind to accelerate early-stage screening workflows. However, decision-makers should note that the model treats the target protein as a rigid structure and exhibits lower accuracy on completely unseen proteins compared to familiar targets. Further research should focus on incorporating protein flexibility and testing the system across broader real-world biological targets before deploying it without experimental validation.

arXiv: 2210.06069
Cover for E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking

Abstract

In silico prediction of the ligand binding pose to a given protein target is a crucial but challenging task in drug discovery. This work focuses on blind flexible selfdocking, where we aim to predict the positions, orientations and conformations of docked molecules. Traditional physics-based methods usually suffer from inaccurate scoring functions and high inference cost. Recently, data-driven methods based on deep learning techniques are attracting growing interest thanks to their efficiency during inference and promising performance. These methods usually either adopt a two-stage approach by first predicting the distances between proteins and ligands and then generating the final coordinates based on the predicted distances, or directly predicting the global roto-translation of ligands. In this paper, we take a different route. Inspired by the resounding success of AlphaFold2 for protein structure prediction, we propose E3Bind, an end-to-end equivariant network that iteratively updates the ligand pose. E3Bind models the protein-ligand interaction through careful consideration of the geometric constraints in docking and the local context of the binding site. Experiments on standard benchmark datasets demonstrate the superior performance of our end-to-end trainable model compared to traditional and recently-proposed deep learning methods.

Table of Contents

  • 1 Introduction
  • 2 Related Works
  • 3 The E3Bind Model
  • 3.1 Preliminaries
  • 3.2 Extracting Geometry-Consistent Information with Trioformer
  • 3.3 Iterative Coordinate Update with Context-Aware E(3)-Equivariant Layer
  • 3.4 Training and Inference
  • 4 Experiments
  • 4.1 Flexible Self Docking
  • 4.2 Ablation Study
  • 4.3 Case Study
  • 5 Conclusion
  • References
  • A Experiment for blind re-docking
  • B Details of the Trioformer
  • C Details of self-confidence prediction
  • C.1 confidence loss
  • C.2 self-confidence prediction results
  • D Implementation Details
  • E Further Ablation Study Results
  • E.1 Ablation Study of E3Bind
  • E.2 Impact of Post Optimization
  • F Sensitivity to Initialization
  • G Inference Speed
  • H Additional trajectories and case studies
  • I Examining the Validity of Generated Structures
  • I.1 Bond Length Distribution
  • I.2 Steric Clash Problem

Knowls

  1. Knowl 1 — E3Bind Architecture for End-to-End Equivariant Protein-Ligand Docking

    model/method

    E3Bind is an end-to-end E(3)E(3)-equivariant neural network architecture for flexible blind protein-ligand docking. It directly predicts the 3D coordinates of a ligand bound to a target protein from the unbound molecular graph of the ligand and the fixed 3D structure of the protein, without relying on intermediate distance-to-coordinate optimization or post-hoc conformation sampling.

    Input Representations

    • Ligand Representation: The ligand is modeled as an atom-level graph Gl=(Vl,El)G^l = (V^l, E^l) with nln_l atoms. Initial node embeddings {hil}i=1nl∈Rnl×d\{h_i^l\}_{i=1}^{n_l} \in \mathbb{R}^{n_l \times d} are extracted using a Graph Isomorphism Network (GIN). Initial 3D Cartesian coordinates {xil}i=1nl∈Rnl×3\{x_i^l\}_{i=1}^{n_l} \in \mathbb{R}^{n_l \times 3} are obtained from an unbound conformation generated via Experimental-Torsion Knowledge Distance Geometry (ETKDG).
    • Protein Representation: The protein is modeled as a residue-level KK-nearest neighbor graph Gp=(Vp,Ep)G^p = (V^p, E^p) with npn_p residues. Node embeddings {hjp}j=1np∈Rnp×d\{h_j^p\}_{j=1}^{n_p} \in \mathbb{R}^{n_p \times d} are computed with a Geometric Vector Perceptron Graph Neural Network (GVP-GNN). Residue coordinates {xjp}j=1np∈Rnp×3\{x_j^p\}_{j=1}^{n_p} \in \mathbb{R}^{n_p \times 3} correspond to the CαC_\alpha atom positions and remain fixed.
    • Protein-Ligand Pair Embeddings: For every protein residue jj and ligand atom ii, an initial pair embedding zij∈Rdz_{ij} \in \mathbb{R}^{d} is constructed via an Outer Product Module (OPM): zij=Linear(vec(Linear(hil)⊗Linear(hjp)))z_{ij} = \text{Linear}\left( \text{vec}\left( \text{Linear}(h_i^l) \otimes \text{Linear}(h_j^p) \right) \right)

    Pipeline

    1. Trioformer Feature Extractor: An iterative geometric encoder that mixes residue, atom, and pair embeddings while injecting intra-protein and intra-ligand distance constraints into pair interactions.
    2. Context-Aware E(3)E(3)-Equivariant Decoder: An iterative refinement module that updates ligand coordinates and node features across TT iterations using Equivariant Graph Convolutional Layers (EGCL).
    3. Self-Confidence Predictor: A lightweight module predicting a scalar confidence score c^∈[0,1]\hat{c} \in [0, 1] from the pooled final ligand representations to assess pose quality and select the best candidate across multiple binding pockets.
  2. Knowl 2 — Trioformer Feature Extraction and Geometry-Aware Interaction Encoding

    algorithm

    The Trioformer is the feature extraction encoder of E3Bind. It iteratively refines protein residue embeddings {hjp}\{h_j^p\}, ligand atom embeddings {hil}\{h_i^l\}, and cross-entity pair embeddings {zij}\{z_{ij}\} across a stack of Trioformer blocks. In each block, node features are updated through multi-head cross-attention using pair representations as attention bias (MHAWithPairBias), followed by an Outer Product Module (OPM) update of pair features and geometry-aware pair attention constrained by intra-molecular distance matrices.

    Input: ligand node embeddings {hil}i=1nl\{h_i^l\}_{i=1}^{n_l}, protein residue embeddings {hjp}j=1np\{h_j^p\}_{j=1}^{n_p}, pair embeddings {zij}i=1,j=1nl,np\{z_{ij}\}_{i=1, j=1}^{n_l, n_p}, intra-ligand reference distances {dik∗}\{d_{ik}^*\}, intra-protein distances $\{d_{jk'}^*\}
    Output: updated embeddings $\{h_i^l\}, \{h_j^p\}, \{z_{ij}\}
    // 1. Cross-attention with pair bias on node representations
    {hil}←{hil}+MHAWithPairBias({hil},{hjp},{hjp},{zij})\{h_i^l\} \leftarrow \{h_i^l\} + \text{MHAWithPairBias}(\{h_i^l\}, \{h_j^p\}, \{h_j^p\}, \{z_{ij}\})
    {hjp}←{hjp}+MHAWithPairBias({hjp},{hil},{hil},{zij})\{h_j^p\} \leftarrow \{h_j^p\} + \text{MHAWithPairBias}(\{h_j^p\}, \{h_i^l\}, \{h_i^l\}, \{z_{ij}\})
    // 2. Node MLP transitions
    {hil}←{hil}+MLP({hil})\{h_i^l\} \leftarrow \{h_i^l\} + \text{MLP}(\{h_i^l\})
    {hjp}←{hjp}+MLP({hjp})\{h_j^p\} \leftarrow \{h_j^p\} + \text{MLP}(\{h_j^p\})
    // 3. Update pair embeddings via Outer Product Module (OPM)
    {zij}←{zij}+OPM({hil},{hjp})\{z_{ij}\} \leftarrow \{z_{ij}\} + \text{OPM}(\{h_i^l\}, \{h_j^p\})
    // 4. Geometry-aware attentive pair updates
    {zij}←{zij}+LigandConstrainedAttentivePairUpdate({zij},{dik∗})\{z_{ij}\} \leftarrow \{z_{ij}\} + \text{LigandConstrainedAttentivePairUpdate}(\{z_{ij}\}, \{d_{ik}^*\})
    {zij}←{zij}+ProteinConstrainedAttentivePairUpdate({zij},{djk′∗})\{z_{ij}\} \leftarrow \{z_{ij}\} + \text{ProteinConstrainedAttentivePairUpdate}(\{z_{ij}\}, \{d_{jk'}^*\})
    // 5. Pair MLP transition
    {zij}←{zij}+MLP({zij})\{z_{ij}\} \leftarrow \{z_{ij}\} + \text{MLP}(\{z_{ij}\})
    return {hil},{hjp},{zij}\{h_i^l\}, \{h_j^p\}, \{z_{ij}\}

    In MHAWithPairBias, the cross-attention weights between ligand atom ii and protein residue k′k' for attention head hh are computed as: aik′(h)=softmaxk′(1c(qi(h))⊤kk′(h)+bik′(h))a_{ik'}^{(h)} = \text{softmax}_{k'}\left( \frac{1}{\sqrt{c}} (q_i^{(h)})^\top k_{k'}^{(h)} + b_{ik'}^{(h)} \right) where qi(h)q_i^{(h)} is projected from hilh_i^l, kk′(h)k_{k'}^{(h)} and vk′(h)v_{k'}^{(h)} are projected from hk′ph_{k'}^p, and bik′(h)b_{ik'}^{(h)} is a linear projection of the pair embedding zik′z_{ik'}. Node features are then updated by projecting the aggregated values ∑k′=1npaik′(h)vk′(h)\sum_{k'=1}^{n_p} a_{ik'}^{(h)} v_{k'}^{(h)} across all HH attention heads.

  3. Knowl 3 — Geometry-Informed Attentive Pair Updates in Trioformer

    equation

    In E3Bind's Trioformer, protein-ligand pair representations zijz_{ij} are updated using triangular edge geometric constraints to ensure that cross-entity spatial relationships respect the triangle inequality with known intra-protein and intra-ligand distances.

    Protein-Constrained Attentive Pair Update

    For each ligand-protein pair (i,j)(i, j), attention is computed over all neighboring pairs (i,k′)(i, k') sharing the same ligand atom ii (1≤k′≤np1 \le k' \le n_p): aijk′(h)=softmaxk′(1c(qij(h))⊤kik′(h)+bij(h)+tjk′(h))a_{ijk'}^{(h)} = \text{softmax}_{k'}\left( \frac{1}{\sqrt{c}} (q_{ij}^{(h)})^\top k_{ik'}^{(h)} + b_{ij}^{(h)} + t_{jk'}^{(h)} \right) zij←zij+Linear(concat1≤h≤H(gij(h)⊙∑k′=1npaijk′(h)vik′(h)))z_{ij} \leftarrow z_{ij} + \text{Linear}\left( \text{concat}_{1 \le h \le H} \left( g_{ij}^{(h)} \odot \sum_{k'=1}^{n_p} a_{ijk'}^{(h)} v_{ik'}^{(h)} \right) \right) where:

    • qij(h),kik′(h),vik′(h),bij(h)q_{ij}^{(h)}, k_{ik'}^{(h)}, v_{ik'}^{(h)}, b_{ij}^{(h)} are learned linear projections of the pair embeddings zijz_{ij} and zik′z_{ik'} for attention head h∈{1,…,H}h \in \{1, \dots, H\}.
    • gij(h)=σ(Linear(h)(zij))g_{ij}^{(h)} = \sigma(\text{Linear}^{(h)}(z_{ij})) is a per-head sigmoid gating vector.
    • tjk′(h)=Linear(h)(djk′∗)t_{jk'}^{(h)} = \text{Linear}^{(h)}(d_{jk'}^*) is a head-specific distance bias computed from the embedded Euclidean distance djk′∗=∥xjp−xk′p∥d_{jk'}^* = \|x_j^p - x_{k'}^p\| between residues jj and k′k' in the fixed protein structure.
    • cc is the projection feature dimension.

    Ligand-Constrained Attentive Pair Update

    Analogously, for a fixed protein residue jj, pair (i,j)(i, j) attends to neighboring pairs (k,j)(k, j) sharing the same protein residue (1≤k≤nl1 \le k \le n_l) with an attention bias tik(h)=Linear(h)(dik∗)t_{ik}^{(h)} = \text{Linear}^{(h)}(d_{ik}^*) derived from the intra-ligand reference distance dik∗d_{ik}^* between atoms ii and kk (provided for atoms separated by ≤2\le 2 hops or within the same ring in the initial ETKDG unbound ligand structure).

  4. Knowl 4 — Context-Aware E(3)-Equivariant Coordinate Refinement Layer

    model/method

    E3Bind refines the ligand pose iteratively over TT steps using an E(3)E(3)-equivariant decoder. At step t∈{0,…,T−1}t \in \{0, \dots, T-1\}, the decoder operates on a heterogeneous context graph consisting of ligand-protein inter-edges and ligand-ligand intra-edges, updating both node embeddings and ligand coordinates:

    (h(t+1),{Δxi(t+1)}i=1nl)=DecoderLayer(t)(h(t),{xi(t)}i=1nl,{xjp}j=1np)\left( h^{(t+1)}, \{\Delta x_i^{(t+1)}\}_{i=1}^{n_l} \right) = \text{DecoderLayer}^{(t)}\left( h^{(t)}, \{x_i^{(t)}\}_{i=1}^{n_l}, \{x_j^p\}_{j=1}^{n_p} \right) xi(t+1)=xi(t)+Δxi(t+1),0≤t<Tx_i^{(t+1)} = x_i^{(t)} + \Delta x_i^{(t+1)}, \quad 0 \le t < T where xjpx_j^p are the fixed protein CαC_\alpha coordinates.

    Equivariant Graph Convolutional Message Passing

    Inter-edge messages between ligand atom ii and protein residue jj, and intra-edge messages between ligand atoms ii and kk, are computed via MLPs ϕm\phi^m and φm\varphi^m: (mij(t),mji(t))=ϕm(zij,hi(t),hj(t),∥xi(t)−xjp∥)(m_{ij}^{(t)}, m_{ji}^{(t)}) = \phi^m\left( z_{ij}, h_i^{(t)}, h_j^{(t)}, \|x_i^{(t)} - x_j^p\| \right) mik(t)=φm(hi(t),hk(t),∥xi(t)−xk(t)∥)m_{ik}^{(t)} = \varphi^m\left( h_i^{(t)}, h_k^{(t)}, \|x_i^{(t)} - x_k^{(t)}\| \right)

    Coordinate Update

    The E(3)E(3)-equivariant coordinate displacement Δxi(t)\Delta x_i^{(t)} is obtained by aggregating normalized directional vectors weighted by gated MLPs ϕx\phi^x and φx\varphi^x: Δxi(t)=∑j=1npxjp−xi(t)∥xjp−xi(t)∥ϕx(mij(t))+∑k=1nlxk(t)−xi(t)∥xk(t)−xi(t)∥φx(mik(t))\Delta x_i^{(t)} = \sum_{j=1}^{n_p} \frac{x_j^p - x_i^{(t)}}{\|x_j^p - x_i^{(t)}\|} \phi^x(m_{ij}^{(t)}) + \sum_{k=1}^{n_l} \frac{x_k^{(t)} - x_i^{(t)}}{\|x_k^{(t)} - x_i^{(t)}\|} \varphi^x(m_{ik}^{(t)})

    Node Embedding Update

    hi(t+1)=hi(t)+∑j=1npmij(t)+∑k=1nlmik(t)h_i^{(t+1)} = h_i^{(t)} + \sum_{j=1}^{n_p} m_{ij}^{(t)} + \sum_{k=1}^{n_l} m_{ik}^{(t)} hj(t+1)=hj(t)+∑i=1nlmji(t)h_j^{(t+1)} = h_j^{(t)} + \sum_{i=1}^{n_l} m_{ji}^{(t)}

    Recycling Mechanism

    The decoder holds parameters for 4 EGCL layers. For iterations beyond 4 (up to T=32T=32), parameter recycling is applied by feeding the previously predicted pose back into the decoder. During training, the number of cycles is sampled from Uniform(1,Ncycle)\text{Uniform}(1, N_{\text{cycle}}), and gradients are backpropagated solely through the final cycle by detaching earlier pose predictions.

  5. Knowl 5 — Self-Confidence Score Prediction and Training Loss

    equation

    E3Bind is trained end-to-end to directly optimize Cartesian coordinates and self-confidence estimation without distance matrix post-processing.

    Training Objective

    The overall training loss L\mathcal{L} balances coordinate deviation and self-confidence calibration: L=Lcoord+βLconfidence\mathcal{L} = \mathcal{L}_{\text{coord}} + \beta \mathcal{L}_{\text{confidence}} where β\beta is a weighting hyperparameter.

    Coordinate Loss

    Lcoord=∑i=1nl∥xi−xi∗∥2\mathcal{L}_{\text{coord}} = \sum_{i=1}^{n_l} \|x_i - x_i^*\|^2 where xi∈R3x_i \in \mathbb{R}^3 is the predicted coordinate of ligand atom ii and xi∗∈R3x_i^* \in \mathbb{R}^3 is the ground-truth coordinate in the bound complex.

    Self-Confidence Module and Loss

    The predicted confidence score c^∈(0,1)\hat{c} \in (0, 1) is computed from the pooled final decoder embeddings {hi(T)}i=1nl\{h_i^{(T)}\}_{i=1}^{n_l}: c^=σ(MLP(∑i=1nlhi(T)))\hat{c} = \sigma\left( \text{MLP}\left( \sum_{i=1}^{n_l} h_i^{(T)} \right) \right) where σ\sigma is the sigmoid function. The confidence loss is the mean squared error against target c∗c^*: Lconfidence=MSE(c^,c∗)\mathcal{L}_{\text{confidence}} = \text{MSE}(\hat{c}, c^*) where c∗c^* is defined by a piece-wise linear function of the detached root-mean-square deviation (RMSD) between predicted ligand coordinates {xi}\{x_i\} and ground truth {xi∗}\{x_i^*\}: c∗={1−12γ⋅RMSD({xi},{xi∗}),if RMSD({xi},{xi∗})≤γc0,otherwisec^* = \begin{cases} 1 - \frac{1}{2\gamma} \cdot \text{RMSD}(\{x_i\}, \{x_i^*\}), & \text{if } \text{RMSD}(\{x_i\}, \{x_i^*\}) \le \gamma \\ c_0, & \text{otherwise} \end{cases} where γ\gamma is an RMSD threshold cutoff and c0c_0 is a small constant floor.

  6. Knowl 6 — Multi-Pocket Inference and Self-Confidence-Guided Pose Selection

    algorithm

    To perform flexible blind self-docking on full target proteins that may have large surfaces or multiple candidate binding sites, E3Bind segments the receptor into functional blocks and uses its internal self-confidence module to select the best docked pose.

    Input: Protein structure GpG^p, unbound ligand molecular graph GlG^l, number of refinement iterations T=32T = 32
    Output: Predicted docked ligand coordinates {xi∗}i=1nl\{x_i^*\}_{i=1}^{n_l}
    // 1. Segment protein into candidate functional blocks using P2Rank
    B←SegmentProtein(Gp,max_blocks=10,radius=20 A˚)B \leftarrow \text{SegmentProtein}(G^p, \text{max\_blocks} = 10, \text{radius} = 20\text{ Å})
    candidates←[ ]\text{candidates} \leftarrow [\ ]
    for each functional block b∈Bb \in B do
        // 2. Initialize unbound ligand pose with random rotation and translation centered at block bb
        {xi(0)}i=1nl←RandomTransform(ETKDG(Gl),center=center(b))\{x_i^{(0)}\}_{i=1}^{n_l} \leftarrow \text{RandomTransform}(\text{ETKDG}(G^l), \text{center} = \text{center}(b))
        
        // 3. Extract protein-ligand features with Trioformer
        {hil},{hjp},{zij}←Trioformer(Gl,b,{xi(0)})\{h_i^l\}, \{h_j^p\}, \{z_{ij}\} \leftarrow \text{Trioformer}(G^l, b, \{x_i^{(0)}\})
        
        // 4. Iteratively refine coordinates with context-aware E(3)-equivariant decoder
        {xi(T)},{hi(T)}←DecoderRecycle({hil},{hjp},{zij},{xi(0)},T)\{x_i^{(T)}\}, \{h_i^{(T)}\} \leftarrow \text{DecoderRecycle}(\{h_i^l\}, \{h_j^p\}, \{z_{ij}\}, \{x_i^{(0)}\}, T)
        
        // 5. Predict self-confidence score
        c^b←σ(MLP(∑i=1nlhi(T)))\hat{c}_b \leftarrow \sigma\left(\text{MLP}\left(\sum_{i=1}^{n_l} h_i^{(T)}\right)\right)
        
        candidates.append(({xi(T)},c^b))\text{candidates.append}((\{x_i^{(T)}\}, \hat{c}_b))
    end for
    // 6. Select pose with the highest self-confidence score
    {xi∗}←arg⁡max⁡{xi(T)}c^b over ({xi(T)},c^b)∈candidates\{x_i^*\} \leftarrow \arg\max_{\{x_i^{(T)}\}} \hat{c}_b \text{ over } (\{x_i^{(T)}\}, \hat{c}_b) \in \text{candidates}
    return {xi∗}\{x_i^*\}

    Because pose selection is guided entirely by the self-confidence score c^\hat{c} predicted from structural embeddings, E3Bind operates without requiring an auxiliary binding affinity estimation model.

  7. Knowl 7 — Flexible Blind Self-Docking Benchmark on PDBbind v2020

    data/table

    Performance of E3Bind and baseline methods on the PDBbind v2020 time-split test set (363 complexes deposited after 2019, with training complexes sharing test ligands removed). Baselines include traditional scoring/sampling tools (QVina-W, GNINA, SMINA, GLIDE, Vina) and deep learning models (EquiBind, TankBind). Models suffixed with "-U" are uncorrected versions evaluated without post-optimization.

    LIGAND RMSD ()Ä CENTROID DISTANCE ()Ä
    Percentiles ↓\downarrow % Below ↑\uparrow Percentiles ↓\downarrow % Below ↑\uparrow
    Method 25% 50% 75% Mean 2Ä 5Ä 25% 50% 75% Mean 2Ä 5Ä
    QVina-W 2.5 7.7 23.7 13.6 20.9 40.2 0.9 3.7 22.9 11.9 41.0 54.6
    GNINA 2.8 8.7 22.1 13.3 21.2 37.1 1.0 4.5 21.2 11.5 36.0 52.0
    SMINA 3.8 8.1 17.9 12.1 13.5 33.9 1.3 3.7 16.2 9.8 38.0 55.9
    GLIDE 2.6 9.3 28.1 16.2 21.8 33.6 0.8 5.6 26.9 14.4 36.1 48.7
    Vina 5.7 10.7 21.4 14.7 5.5 21.2 1.9 6.2 20.1 12.1 26.5 47.1
    EquiBind 3.8 6.2 10.3 8.2 5.5 39.1 1.3 2.6 7.4 5.6 40.0 67.5
    TankBind 2.6 4.2 7.6 7.8 17.6 57.8 0.8 1.7 4.3 5.9 55.0 77.8
    E3Bind 2.1 3.8 7.8 7.2 23.4 60.0 0.8 1.5 4.0 5.1 60.0 78.8
    EquiBind-U 3.3 5.7 9.7 7.8 7.2 42.4 1.3 2.6 7.4 5.6 40.0 67.5
    TankBind-U 3.9 7.7 13.6 10.5 8.0 34.7 1.3 3.0 8.2 6.6 40.5 66.4
    E3Bind-U 2.0 3.8 7.7 7.2 25.6 60.6 0.8 1.5 4.0 5.1 59.0 78.8

    E3Bind achieves state-of-the-art results across most metrics. In particular, the proportion of high-resolution predictions with Ligand RMSD <2< 2 Å reaches 23.4% (and 25.6% for E3Bind-U), representing a 33% relative improvement over TankBind (17.6%). E3Bind-U achieves top performance without post-optimization, confirming that the end-to-end network directly outputs physically and geometrically valid poses.

  8. Knowl 8 — Flexible Blind Self-Docking on Unseen Target Proteins

    data/table

    Generalization benchmark on a test subset containing 144 protein-ligand complexes whose protein targets were entirely unseen during training, evaluating out-of-distribution transfer.

    LIGAND RMSD ()Ä CENTROID DISTANCE ()Ä
    Percentiles ↓\downarrow % Below ↑\uparrow Percentiles ↓\downarrow % Below ↑\uparrow
    Method 25% 50% 75% Mean 2Ä 5Ä 25% 50% 75% Mean 2Ä 5Ä
    QVina-W 3.4 10.3 28.1 16.9 15.3 31.9 1.3 6.5 26.8 15.2 35.4 47.9
    GNINA 4.5 13.4 27.8 16.7 13.9 27.8 2.0 10.1 27.0 15.1 25.7 39.5
    SMINA 4.8 10.9 26.0 15.7 9.0 25.7 1.6 6.5 25.7 13.6 29.9 41.7
    GLIDE 3.4 18.0 31.4 19.6 19.6 28.7 1.1 17.6 29.1 18.1 29.4 40.6
    Vina 7.9 16.6 27.1 18.7 1.4 12.0 2.4 15.7 26.2 16.1 20.4 37.3
    EquiBind 5.9 9.1 14.3 11.3 0.7 18.8 2.6 6.3 12.9 8.9 16.7 43.8
    TankBind 3.4 5.7 10.8 10.5 3.5 43.7 1.2 2.6 8.4 8.2 40.9 70.8
    E3Bind 3.0 6.1 10.2 10.1 6.3 38.9 1.2 2.3 7.0 7.6 43.8 66.0
    EquiBind-U 5.7 8.8 14.1 11.0 1.4 21.5 2.6 6.3 12.9 8.9 16.7 43.8
    TankBind-U 4.0 7.9 14.9 8.3 3.5 34.0 1.4 3.3 10.9 8.3 35.4 65.2
    E3Bind-U 3.1 6.0 10.6 10.1 5.6 41.0 1.2 2.3 7.8 7.7 42.4 65.3

    On unseen targets, E3Bind outperforms other deep learning baselines (achieving 6.3% RMSD <2< 2 Å vs. 3.5% for TankBind and 0.7% for EquiBind). While traditional physics-based scoring methods (e.g., GLIDE at 19.6% and QVina-W at 15.3%) maintain higher fractions of <2< 2 Å poses in this out-of-distribution regime, E3Bind substantially suppresses extreme docking failures (reducing mean RMSD to 10.1 Å compared to 16.9 Å for QVina-W and 19.6 Å for GLIDE).

  9. Knowl 9 — Ablation Study on E3Bind Architectural Components

    data/table

    Ablation analysis on the flexible blind self-docking task evaluating the contribution of geometry-aware constraints, Trioformer, iteration count, intra-ligand message passing, and P2Rank pocket segmentation.

    LIGAND RMSD ()Ä CENTROID DISTANCE ()Ä
    Percentiles ↓\downarrow % Below ↑\uparrow Percentiles ↓\downarrow % Below ↑\uparrow
    Method 25% 50% 75% Mean 2Ä 5Ä 25% 50% 75% Mean 2Ä 5Ä
    E3Bind 2.1 3.8 7.8 7.2 23.4 60.0 0.8 1.5 4.0 5.1 60.0 78.8
    w/o Geometric Constraint 2.3 4.0 7.8 7.6 20.1 57.6 0.9 1.7 4.0 5.3 58.2 76.4
    w/o Trioformer 2.8 4.5 8.1 7.9 11.7 55.2 1.2 2.0 4.5 5.6 54.2 77.4
    4 Iteration 2.6 4.0 7.8 7.5 14.9 58.9 1.1 1.8 4.5 5.5 52.6 77.9
    w/o Intra 2.4 4.0 7.8 7.6 17.6 56.4 0.9 1.7 4.5 5.6 55.6 78.3
    w/o P2Rank 2.0 4.2 7.8 7.6 24.2 56.4 0.8 1.7 4.1 5.6 54.8 79.1

    Key takeaways:

    1. Trioformer vs. Concatenation: Replacing Trioformer with simple concatenation of protein and ligand embeddings drops RMSD <2< 2 Å from 23.4% to 11.7%.
    2. Geometry Awareness: Removing intra-protein/ligand distance biases in pair updates reduces <2< 2 Å poses from 23.4% to 20.1%.
    3. Iterative Refinement: Reducing decoder iterations from 32 to 4 drops <2< 2 Å poses to 14.9%.
    4. Intra-Ligand Messages: Omitting ligand intra-edge message passing lowers <2< 2 Å poses to 17.6%.
    5. Pocket Segmentation Independence: Segmenting proteins into 30 random blocks without P2Rank preserves a 24.2% <2< 2 Å fraction, indicating low dependence on specialized pocket pre-selectors.
  10. Knowl 10 — Blind Rigid Re-Docking Benchmark

    data/table

    Blind rigid re-docking performance on the 363 PDBbind v2020 test complexes where the ground-truth bound ligand conformation is provided as prior knowledge and only global roto-translation into the pocket needs to be predicted. EquiBind-R predicts only global rigid rotation and translation without updating internal conformations.

    LIGAND RMSD ()Ä CENTROID DISTANCE ()Ä
    Percentiles ↓\downarrow % Below ↑\uparrow Percentiles ↓\downarrow % Below ↑\uparrow
    Method 25% 50% 75% Mean 2Ä 5Ä 25% 50% 75% Mean 2Ä 5Ä
    QVina-W 1.6 7.9 24.1 13.4 27.7 39.0 0.9 3.8 23.2 11.8 40.4 55.4
    GNINA 1.3 6.1 22.9 12.2 32.2 46.8 0.7 2.8 22.1 10.9 43.8 58.4
    SMINA 1.4 6.2 15.2 10.3 30.1 46.7 0.8 2.6 12.7 8.5 45.3 63.5
    GLIDE 0.5 8.3 29.5 15.7 43.4 45.7 0.3 4.9 28.5 14.8 45.4 50.4
    Vina 4.5 9.7 19.9 13.4 13.2 26.7 1.7 5.5 18.7 11.2 29.8 47.9
    EquiBind-R 2.0 5.1 9.8 7.4 25.1 49.0 1.4 2.6 7.3 5.8 40.8 66.9
    TankBind 1.4 3.4 7.0 7.0 37.2 63.9 0.8 1.7 4.1 5.6 55.1 78.2
    E3Bind 1.2 3.3 7.2 6.7 38.3 63.9 0.6 1.3 3.8 5.1 61.4 79.0
    TankBind-U 4.5 8.0 15.5 10.9 8.0 27.5 1.5 3.3 8.3 6.7 33.3 60.3
    E3Bind-U 1.4 3.2 7.1 6.8 34.7 65.6 0.6 1.4 3.7 5.1 61.4 79.3

    Even though E3Bind is designed for fully flexible modeling, enforcing a rigid structure through post-optimization configuration loss produces competitive RMSD (38.3% <2< 2 Å) and superior centroid localization (61.4% <2< 2 Å Centroid Distance) compared to both deep learning and classical sampling baselines.

  11. Knowl 11 — Inference Efficiency and Structural Validity via Steric Clash Mitigation

    empirical result

    E3Bind provides substantial inference speedup over traditional physics-based docking software and resolves steric overlap issues common in one-shot geometric deep learning.

    Inference Runtime

    Average execution time per protein-ligand complex on the test set:

    • Classical score-based methods (16-CPU): QVina-W: 49 s; GNINA: 247 s (146 s on GPU); SMINA: 146 s; Vina: 205 s; GLIDE (single-threaded): 1405 s.
    • Deep learning methods: EquiBind-U: 0.14 s (CPU) / 0.03 s (GPU); TankBind-U: 1.2 s (CPU) / 0.87 s (GPU); E3Bind-U: 2.2 s (CPU) / 0.44 s (GPU). E3Bind-U runs 100x to 3000x faster than traditional docking baselines while including P2Rank functional block segmentation time (~0.15 s in parallel).

    Steric Clash Mitigation

    A steric clash is defined as any heavy-atom pair distance <0.4< 0.4 Å between protein and ligand.

    • Occurrence Rates: EquiBind yields severe clashes on 21.0% of test complexes due to its unrefined one-shot keypoint alignment. In contrast, E3Bind exhibits clashes on only 3.3% of test complexes.
    • Mechanism: The iterative refinement process dynamically senses local steric conflicts and corrects them across update cycles. Without self-confidence filtering and with only 4 iterations, 13.8% of poses contain clashes; adding confidence filtering reduces this to 6.6%, and extending refinement to 32 iterations further drops clash occurrence to 3.3%.

Coverage note — None was omitted; all core model components (Trioformer encoder, E(3)-equivariant iterative decoder, self-confidence module), training objectives, multi-pocket inference algorithms, and primary benchmark tables (flexible self-docking, unseen receptors, ablations, re-docking, and speed/clash analysis) were converted into standalone knowls.

References

  1. 1.Jingxiao Bao, Xiao He, and John ZH Zhang. Deepbsp—a machine learning method for accurate prediction of protein–ligand docking structures. Journal of Chemical Information and Modeling, 61(5):2231–2240, 2021.
  2. 2.Helen M Berman, John Westbrook, Zukang Feng, Gary Gilliland, Talapady N Bhat, Helge Weissig, Ilya N Shindyalov, and Philip E Bourne. The protein data bank. Nucleic acids research, 28(1): 235–242, 2000.
  3. 3.Ryan G Coleman, Michael Carchia, Teague Sterling, John J Irwin, and Brian K Shoichet. Ligand pose and orientational sampling in molecular docking. PloS one, 8(10):e75992, 2013.
  4. 4.UniProt Consortium. Uniprot: a worldwide hub of protein knowledge. Nucleic acids research, 47 (D1):D506–D515, 2019.
  5. 5.Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-equivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12200–12209, 2021.
  6. 6.Jacob D Durrant and J Andrew McCammon. Nnscore 2.0: a neural-network receptor–ligand scoring function. Journal of chemical information and modeling, 51(11):2897–2903, 2011.
  7. 7.Leonardo G Ferreira, Ricardo N Dos Santos, Glaucius Oliva, and Adriano D Andricopulo. Molecular docking and structure-based drug design strategies. Molecules, 20(7):13384–13421, 2015.
  8. 8.Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B Iovanisci, Ian Snyder, and David R Koes. Three-dimensional convolutional neural networks and a cross-docked data set for structure-based drug design. Journal of chemical information and modeling, 60(9):4200– 4215, 2020.
  9. 9.Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems, 34:13757–13769, 2021a.
  10. 10.Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi Jaakkola, and Andreas Krause. Independent se (3)-equivariant models for end-to-end rigid protein docking. arXiv preprint arXiv:2111.07786, 2021b.
  11. 11.Pascale Gaudet, Pierre-Andre Michel, Monique Zahn-Zabal, Aurore Britan, Isabelle Cusin, Marcin Domagalski, Paula D Duek, Alain Gateau, Anne Gleizes, Valerie Hinard, et al. The nextprot knowledgebase on human proteins: 2017 update. Nucleic acids research, 45(D1):D177–D182, 2017.
  12. 12.Thomas A Halgren, Robert B Murphy, Richard A Friesner, Hege S Beard, Leah L Frye, W Thomas Pollard, and Jay L Banks. Glide: a new approach for rapid, accurate docking and scoring. 2. enrichment factors in database screening. Journal of medicinal chemistry, 47(7):1750–1759, 2004.
  13. 13.Nafisa M Hassan, Amr A Alhossary, Yuguang Mu, and Chee-Keong Kwoh. Protein-ligand blind docking using quickvina-w with inter-process spatio-temporal integration. Scientific reports, 7 (1):1–13, 2017.
  14. 14.Jer´ ome Hert, John J Irwin, Christian Laggner, Michael J Keiser, and Brian K Shoichet. Quantifying biogenic bias in screening libraries. Nature chemical biology, 5(7):479–483, 2009.
  15. 15.Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  16. 16.Emiel Hoogeboom, Vıctor Garcia Satorras, Cl ´ ement Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pp. 8867–8887. PMLR, 2022.
  17. 17.Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. Equivariant graph mechanics networks with constraints. arXiv preprint arXiv:2203.06442, 2022.
  18. 18.Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. arXiv preprint arXiv:2009.01411, 2020.
  19. 19.Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. arXiv preprint arXiv:2206.01729, 2022.
  20. 20.John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zˇ´ıdek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021.
  21. 21.David Ryan Koes, Matthew P Baumgartner, and Carlos J Camacho. Lessons learned in empirical scoring with smina from the csar 2011 benchmarking exercise. Journal of chemical information and modeling, 53(8):1893–1904, 2013.
  22. 22.Radoslav Krivak and David Hoksza. P2rank: machine learning based tool for rapid and accurate prediction of ligand binding sites from protein structure. Journal of cheminformatics, 10(1):1–12, 2018.
  23. 23.Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 2013.
  24. 24.Leo Liberti, Carlile Lavor, Nelson Maculan, and Antonio Mucherino. Euclidean distance geometry and applications. SIAM review, 56(1):3–69, 2014.
  25. 25.Yu Liu, Lei Zhao, Wentao Li, Dongyu Zhao, Miao Song, and Yongliang Yang. Fipsdock: a new molecular docking technique driven by fully informed swarm optimization algorithm. Journal of computational chemistry, 34(1):67–75, 2013.
  26. 26.Zhihai Liu, Minyi Su, Li Han, Jie Liu, Qifan Yang, Yan Li, and Renxiao Wang. Forging the basis for developing protein–ligand interaction scoring functions. Accounts of chemical research, 50 (2):302–309, 2017.
  27. 27.Wei Lu, Qifeng Wu, Jixian Zhang, Jiahua Rao, Chengtao Li, and Shuangjia Zheng. Tankbind: Trigonometry-aware neural networks for drug-protein binding structure prediction. bioRxiv, 2022.
  28. 28.Jiankun Lyu, Sheng Wang, Trent E Balius, Isha Singh, Anat Levit, Yurii S Moroz, Matthew J O’Meara, Tao Che, Enkhjargal Algaa, Kateryna Tolmachova, et al. Ultra-large library docking for discovering new chemotypes. Nature, 566(7743):224–229, 2019.
  29. 29.Matthew Masters, Amr H Mahmoud, Yao Wei, and Markus Alexander Lill. Deep learning model for flexible and efficient protein-ligand docking. In ICLR2022 Machine Learning for Drug Discovery, 2022.
  30. 30.Andrew T McNutt, Paul Francoeur, Rishal Aggarwal, Tomohide Masuda, Rocco Meli, Matthew Ragoza, Jocelyn Sunseri, and David Ryan Koes. Gnina 1.0: molecular docking with deep learning. Journal of cheminformatics, 13(1):1–20, 2021.
  31. 31.Oscar Mendez-Lucio, Mazen Ahmad, Ehecatl Antonio del Rio-Chanona, and J ´ org Kurt Wegner. A geometric deep learning approach to predict binding conformations of bioactive molecules. Nature Machine Intelligence, 3(12):1033–1039, 2021.
  32. 32.Garrett M Morris, David S Goodsell, Ruth Huey, and Arthur J Olson. Distributed automated docking of flexible ligands to proteins: parallel applications of autodock 2.4. Journal of computer-aided molecular design, 10(4):293–304, 1996.
  33. 33.Matthew Ragoza, Joshua Hochuli, Elisa Idrobo, Jocelyn Sunseri, and David Ryan Koes. Protein– ligand scoring with convolutional neural networks. Journal of chemical information and modeling, 57(4):942–957, 2017.
  34. 34.Srinivas Ramachandran, Pradeep Kota, Feng Ding, and Nikolay V Dokholyan. Automated minimization of steric clashes in protein structures. Proteins: Structure, Function, and Bioinformatics, 79(1):261–270, 2011.
  35. 35.Jean-Louis Reymond and Mahendra Awale. Exploring chemical space for drug discovery using the chemical universe database. ACS chemical neuroscience, 3(9):649–657, 2012.
  36. 36.Sereina Riniker and Gregory A Landrum. Better informed distance geometry: using what we know to improve conformation generation. Journal of chemical information and modeling, 55(12): 2562–2574, 2015.
  37. 37.Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323–9332. PMLR, 2021.
  38. 38.Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zˇ´ıdek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706–710, 2020a.
  39. 39.Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zˇ´ıdek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706–710, 2020b.
  40. 40.Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International Conference on Machine Learning, pp. 9558–9568. PMLR, 2021.
  41. 41.Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
  42. 42.Hannes Stark, Octavian Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International Conference on Machine Learning, pp. 20503–20521. PMLR, 2022.
  43. 43.Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):455–461, 2010.
  44. 44.Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674, 2011.
  45. 45.Sheng Wang, Siqi Sun, Zhen Li, Renyu Zhang, and Jinbo Xu. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLoS computational biology, 13(1):e1005324, 2017.
  46. 46.Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  47. 47.Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923, 2022.
  48. 48.Wenyi Zhang, Eric W Bell, Minghao Yin, and Yang Zhang. Edock: blind protein–ligand docking by replica-exchange monte carlo simulation. Journal of cheminformatics, 12(1):1–17, 2020.
  49. 49.Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. ChemRxiv, 2022. doi: 10.26434/chemrxiv-2022-jjm0j-v3.
  50. 50.Zhaocheng Zhu, Chence Shi, Zuobai Zhang, Shengchao Liu, Minghao Xu, Xinyu Yuan, Yangtian Zhang, Junkun Chen, Huiyu Cai, Jiarui Lu, et al. Torchdrug: A powerful and flexible machine learning platform for drug discovery. arXiv preprint arXiv:2202.08320, 2022.

Citation

MLA
Zhang, Y., et al. “E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking”. arXiv, 2022, http://arxiv.org/abs/2210.06069v2.
APA
Zhang, Y., Cai, H., Shi, C., Zhong, B., & Tang, J. (2022). E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking. arXiv. http://arxiv.org/abs/2210.06069v2
Chicago
Zhang, Y., H. Cai, C. Shi, B. Zhong, and J. Tang. 2022. “E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking”. arXiv. http://arxiv.org/abs/2210.06069v2.
Harvard
Zhang, Y. et al. (2022) “E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2210.06069v2.
Vancouver
1. Zhang Y, Cai H, Shi C, Zhong B, Tang J (2022) E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking. arXiv

BibTeX

@article{zhang2022e3bind,
  title = {E3Bind: An End-to-End Equivariant Network for Protein-Ligand Docking},
  author = {Zhang, Yangtian and Cai, Huiyu and Shi, Chence and Zhong, Bozitao and Tang, Jian},
  year = {2022},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2210.06069v2},
  eprint = {2210.06069}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: Authors