Back

Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition

Lei ShiYifan ZhangJian ChengHanqing Lu

article2018CVPR1,898 citations

Proposes a two-stream adaptive graph convolutional network that dynamically learns graph structures end-to-end and combines joint coordinates with bone vectors to achieve state-of-the-art accuracy in skeleton-based action recognition.

Listen

Skeleton-based action recognition is an important computer vision capability for applications such as surveillance, human-computer interaction, and robotics because skeletal representations remain robust against complex visual backgrounds and varying lighting. Previous graph convolutional network approaches successfully treated human joints as nodes on a graph. However, those older methods relied on static, manually designed graph topologies limited strictly to natural physical bone connections. This rigid structure prevented models from discovering relationships between physically separated body parts (such as two hands clapping) and failed to adapt across different network layers or varied action categories. Additionally, earlier models focused only on raw joint coordinates, leaving the lengths and orientations of bones largely unexploited.

To address these limitations, the article presents a two-stream adaptive graph convolutional network. The objective of this research is to demonstrate that dynamically learning graph structures and jointly processing both joint positions and bone vectors substantially improves action recognition accuracy across benchmark evaluations.

The authors evaluated their method using extensive experimental benchmarking on two large-scale public datasets: NTU-RGBD, containing 56,000 indoor action video clips across 60 categories, and Kinetics-Skeleton, containing over 260,000 video clips across 400 action classes derived from real-world online videos. The proposed architecture introduces an adaptive layer combining three distinct graph representations: a fixed physical human body graph, a learnable global graph shared across samples, and a data-dependent graph that computes unique connection strengths for each specific action sample. Furthermore, the framework integrates a dual-stream design that processes both first-order joint coordinates and second-order bone vectors, combining their final prediction scores.

The experimental findings show that the proposed two-stream adaptive architecture outperforms prior state-of-the-art methods by a significant margin. On the NTU-RGBD cross-subject benchmark, the model achieved an 88.5% accuracy rate, and it reached 95.1% on the cross-view benchmark, surpassing standard spatial-temporal graph networks by several percentage points. On the challenging Kinetics-Skeleton dataset, the network achieved 36.1% top-1 accuracy and 58.7% top-5 accuracy, delivering improvements of more than 5 percentage points over prior baselines. Ablation analyses confirmed that both the adaptive graph components and the combination of joint and bone data streams were essential to achieving peak accuracy.

These results indicate that artificial intelligence systems for motion and action recognition can achieve higher operational reliability by letting neural networks learn relational physical structures directly from data rather than relying strictly on handcrafted anatomical models. In practice, adopting this data-driven graph formulation can improve classification accuracy in automated monitoring and human-machine collaboration systems without requiring completely new sensor hardware.

Organizations implementing skeleton-based recognition pipelines should consider adopting adaptive graph formulations and dual-stream inputs that leverage both joint coordinates and bone vectors. Technical teams should evaluate the computational trade-offs of running parallel network streams for real-time edge deployments and run targeted pilot studies on their domain-specific video streams to confirm real-world performance gains.

Confidence in these findings is supported by thorough ablation studies and validation on two established, large-scale benchmarks. However, decision-makers should note that the system depends on the quality of upstream 2D or 3D pose estimation tools, and severe occlusions or tracking failures in noisy environments could affect real-world recognition reliability.

CAS Center for Excellence in Brain Science and Intelligence TechnologyInstitute of Automation, Chinese Academy of SciencesUniversity of Chinese Academy of Sciencesaction recognitionbp algorithmdata streamGraphRecognitionAction
arXiv: 1805.07694lshiwjx/2s-AGCN
Cover for Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition

Abstract

In skeleton-based action recognition, graph convolutional networks (GCNs), which model the human body skeletons as spatiotemporal graphs, have achieved remarkable performance. However, in existing GCN-based methods, the topology of the graph is set manually, and it is fixed over all layers and input samples. This may not be optimal for the hierarchical GCN and diverse samples in action recognition tasks. In addition, the second-order information (the lengths and directions of bones) of the skeleton data, which is naturally more informative and discriminative for action recognition, is rarely investigated in existing methods. In this work, we propose a novel two-stream adaptive graph convolutional network (2s-AGCN) for skeleton-based action recognition. The topology of the graph in our model can be either uniformly or individually learned by the BP algorithm in an end-to-end manner. This data-driven method increases the flexibility of the model for graph construction and brings more generality to adapt to various data samples. Moreover, a two-stream framework is proposed to model both the first-order and the second-order information simultaneously, which shows notable improvement for the recognition accuracy. Extensive experiments on the two large-scale datasets, NTU-RGBD and Kinetics-Skeleton, demonstrate that the performance of our model exceeds the state-of-the-art with a significant margin.

Topics

action recognitionbp algorithmdata streamgraph constructiontopologybipartite graphsconvolutional neural networksattention mechanism

Keywords

GraphRecognitionActionAction RecognitionGraph ConvolutionalAdaptive GraphAdaptive Graph ConvolutionalNetworksSkeleton-Based ActionSkeleton

Table of Contents

  • 1 Introduction
  • 2 Related work
  • 2.1 Skeleton-based action recognition
  • 2.2 Graph convolutional neural networks
  • 3 Graph Convolutional Networks
  • 3.1 Graph construction
  • 3.2 Graph convolution
  • 3.3 Implementation
  • 4 Two-stream adaptive graph convolutional network
  • 4.1 Adaptive graph convolutional layer
  • 4.2 Adaptive graph convolutional block
  • 4.3 Adaptive graph convolutional network
  • 4.4 Two-stream networks
  • 5 Experiments
  • 5.1 Datasets
  • 5.2 Training details
  • 5.3 Ablation Study
  • 5.3.1 Adaptive graph convolutional block.
  • 5.3.2 Visualization of the learned graphs
  • 5.3.3 Two-stream framework
  • 5.4 Comparison with the state-of-the-art
  • 5.5 Conclusion
  • References

Knowls

  1. Knowl 1 — Adaptive Graph Convolutional Layer

    model/method

    In the adaptive graph convolutional network (AGCN), graph convolution learns graph topologies adaptively for different network layers and different input samples rather than relying solely on a fixed physical skeleton graph. For an input feature map fin∈RCin×T×N\mathbf{f}_{in} \in \mathbb{R}^{C_{in} \times T \times N} (where CinC_{in} is input channel count, TT is the number of temporal frames, and NN is the number of joints/vertices), the output feature map fout∈RCout×T×N\mathbf{f}_{out} \in \mathbb{R}^{C_{out} \times T \times N} across KvK_v spatial partition subsets is computed as:

    fout=∑k=1KvWkfin(Ak+Bk+Ck)\mathbf{f}_{out} = \sum_{k=1}^{K_v} \mathbf{W}_k \mathbf{f}_{in} (\mathbf{A}_k + \mathbf{B}_k + \mathbf{C}_k)

    where:

    • KvK_v is the spatial kernel size (empirically set to 33, corresponding to root, centripetal, and centrifugal partition subsets).
    • Wk∈RCout×Cin×1×1\mathbf{W}_k \in \mathbb{R}^{C_{out} \times C_{in} \times 1 \times 1} is a learnable weight tensor implementing a 1×11 \times 1 convolution across channel dimensions for subset kk.
    • Ak∈RN×N\mathbf{A}_k \in \mathbb{R}^{N \times N} is the normalized adjacency matrix representing the physical human body connectivity for subset kk.
    • Bk∈RN×N\mathbf{B}_k \in \mathbb{R}^{N \times N} is an unconstrained, learnable parameter matrix optimized end-to-end via backpropagation to capture global inter-joint relations across all samples for that layer (initialized to 00).
    • Ck∈RN×N\mathbf{C}_k \in \mathbb{R}^{N \times N} is a sample-dependent adaptive adjacency matrix dynamically computed for each individual input sample using an embedded Gaussian similarity function.

    A residual connection directly links the input feature map fin\mathbf{f}_{in} to the output fout\mathbf{f}_{out}, transformed by a 1×11 \times 1 convolution when Cin≠CoutC_{in} \neq C_{out}.

  2. Knowl 2 — Sample-Dependent Graph Matrix Computation via Embedded Gaussian Similarity

    equation

    In the adaptive graph convolutional layer, the sample-specific adjacency matrix Ck∈RN×N\mathbf{C}_k \in \mathbb{R}^{N \times N} dynamically captures inter-joint dependencies conditioned on the input feature representation fin∈RCin×T×N\mathbf{f}_{in} \in \mathbb{R}^{C_{in} \times T \times N}. It is computed using a normalized embedded Gaussian similarity:

    Ck=softmax(finTWθkTWϕkfin)\mathbf{C}_k = \text{softmax}\left(\mathbf{f}_{in}^T \mathbf{W}_{\theta k}^T \mathbf{W}_{\phi k} \mathbf{f}_{in}\right)

    Elementwise, the pairwise similarity f(vi,vj)f(v_i, v_j) between joint vertex viv_i and joint vertex vjv_j is given by:

    f(vi,vj)=exp⁡(θk(vi)Tϕk(vj))∑j=1Nexp⁡(θk(vi)Tϕk(vj))f(v_i, v_j) = \frac{\exp\left(\theta_k(v_i)^T \phi_k(v_j)\right)}{\sum_{j=1}^N \exp\left(\theta_k(v_i)^T \phi_k(v_j)\right)}

    where θk\theta_k and ϕk\phi_k are embedding functions parameterized by 1×11 \times 1 convolutions with learnable weights Wθk∈RCe×Cin×1×1\mathbf{W}_{\theta k} \in \mathbb{R}^{C_e \times C_{in} \times 1 \times 1} and Wϕk∈RCe×Cin×1×1\mathbf{W}_{\phi k} \in \mathbb{R}^{C_e \times C_{in} \times 1 \times 1}, projecting input features from CinC_{in} channels to an embedding channel dimension CeC_e. The input fin\mathbf{f}_{in} is rearranged into an N×CeTN \times C_e T matrix and multiplied with a CeT×NC_e T \times N matrix to obtain the N×NN \times N similarity matrix, normalized across rows using softmax into values in [0,1][0, 1]. The embedding weights Wθk\mathbf{W}_{\theta k} and Wϕk\mathbf{W}_{\phi k} are initialized to 00.

  3. Knowl 3 — Skeleton Bone Representation and Root Joint Padding

    model/method

    To exploit second-order geometric information (lengths and orientations of body parts) alongside first-order 3D joint coordinates, human skeleton data is explicitly represented as directional bone vectors.

    Each bone is bounded by two adjacent joints: a source joint v1=(x1,y1,z1)v_1 = (x_1, y_1, z_1) located closer to the skeleton center of gravity, and a target joint v2=(x2,y2,z2)v_2 = (x_2, y_2, z_2) located farther from the center of gravity. The bone vector ev1,v2\mathbf{e}_{v_1, v_2} pointing from the source to the target joint is defined as:

    ev1,v2=(x2−x1, y2−y1, z2−z1)\mathbf{e}_{v_1, v_2} = (x_2 - x_1,\, y_2 - y_1,\, z_2 - z_1)

    Because the human skeleton graph contains no cycles, every bone is uniquely associated with its target joint. In a skeleton of NN joints, there are N−1N - 1 physical bones because the central root joint (at the center of gravity) is not a target of any bone. An empty bone with coordinate vector (0,0,0)(0, 0, 0) is assigned to the central root joint, making the total number of bone nodes equal to NN and allowing the bone network to share the exact graph topology and vertex indexing of the joint network.

  4. Knowl 4 — Two-Stream Adaptive Graph Convolutional Network Architecture

    model/method

    The Two-Stream Adaptive Graph Convolutional Network (2s-AGCN) processes first-order coordinate data and second-order bone vector data through two parallel streams:

    1. Joint Stream (J-Stream): Receives raw 2D or 3D coordinate sequences of human skeleton joints as input and processes them through an adaptive graph convolutional network to produce class prediction scores.
    2. Bone Stream (B-Stream): Receives directional bone vectors (with a zero vector padding the root joint) as input and processes them through an identical adaptive graph convolutional network architecture to produce class prediction scores.

    For any given skeleton sequence, bone vectors are computed directly from joint coordinates. The joint sequence and bone sequence are fed in parallel into the J-Stream and B-Stream, respectively. The resulting softmax score vectors from both streams are summed to generate the final prediction:

    sfinal=sJ+sB\mathbf{s}_{final} = \mathbf{s}_{J} + \mathbf{s}_{B}

    where sJ\mathbf{s}_{J} is the softmax probability vector of the J-Stream and sB\mathbf{s}_{B} is the softmax probability vector of the B-Stream.

  5. Knowl 5 — Adaptive Graph Convolutional Block and 9-Stage Backbone Network

    model/method

    The Adaptive Graph Convolutional Network (AGCN) is constructed by stacking 9 spatio-temporal adaptive graph convolutional blocks:

    • Basic Block Structure: Each basic block comprises a spatial adaptive graph convolutional layer (Convs), followed by Batch Normalization (BN) and ReLU activation, followed by a temporal convolutional layer (Convt) applying a Kt×1K_t \times 1 1D convolution across the temporal dimension, followed by BN, ReLU, and a Dropout layer with drop rate 0.50.5. A residual shortcut wraps around the entire block.
    • Backbone Stacking:
      • A data Batch Normalization layer normalizes raw input data at the entrance of the network.
      • Nine consecutive basic blocks (B1 to B9) are stacked with channel configurations:
        • Block 1 (B1): 64 output channels, temporal stride 1 (input channels 3 for coordinates/bones)
        • Blocks 2–3 (B2, B3): 64 output channels, temporal stride 1
        • Block 4 (B4): 128 output channels, temporal stride 2
        • Blocks 5–6 (B5, B6): 128 output channels, temporal stride 1
        • Block 7 (B7): 256 output channels, temporal stride 2
        • Blocks 8–9 (B8, B9): 256 output channels, temporal stride 1
      • A Global Average Pooling (GAP) layer pools spatial and temporal dimensions into a fixed-size vector, followed by a fully connected layer with a softmax classifier.

  6. Knowl 6 — Distance-Based Spatial Graph Partitioning and Adjacency Normalization

    model/method

    In spatial skeleton graph convolution, the 1-distance neighborhood BiB_i of each vertex viv_i is partitioned into Kv=3K_v = 3 subsets relative to the skeleton's center of gravity:

    1. Si1S_{i1} (Root subset): The vertex viv_i itself.
    2. Si2S_{i2} (Centripetal subset): Neighboring vertices in BiB_i that are closer to the skeleton center of gravity than viv_i.
    3. Si3S_{i3} (Centrifugal subset): Neighboring vertices in BiB_i that are farther from the skeleton center of gravity than viv_i.

    For each subset k∈{1,2,3}k \in \{1, 2, 3\}, an N×NN \times N adjacency matrix Aˉk\bar{\mathbf{A}}_k is defined where element Aˉkij=1\bar{A}_k^{ij} = 1 if vertex vj∈Sikv_j \in S_{ik} and 00 otherwise. To normalize the matrix, a diagonal matrix Λk\mathbf{\Lambda}_k is formed with Λkii=∑jAˉkij+α\Lambda_k^{ii} = \sum_j \bar{A}_k^{ij} + \alpha, where α=0.001\alpha = 0.001 prevents empty rows. The normalized adjacency matrix for subset kk is:

    Ak=Λk−12AˉkΛk−12\mathbf{A}_k = \mathbf{\Lambda}_k^{-\frac{1}{2}} \bar{\mathbf{A}}_k \mathbf{\Lambda}_k^{-\frac{1}{2}}

  7. Knowl 7 — Ablation Study on Adaptive Graph Components

    data/table

    An ablation study on the Cross-View (X-View) benchmark of the NTU-RGBD dataset compares the contribution of the physical adjacency matrix A\mathbf{A}, the globally learned parameter matrix B\mathbf{B}, the sample-dependent matrix C\mathbf{C}, and the attention mask M\mathbf{M} used in ST-GCN.

    Methods Accuracy (%)
    ST-GCN 92.7
    ST-GCN wo/M 91.1
    AGCN wo/A 93.4
    AGCN wo/B 93.3
    AGCN wo/C 93.4
    AGCN 93.7

    The results demonstrate:

    1. Removing the attention mask M\mathbf{M} from the baseline ST-GCN decreases top-1 accuracy by 1.6%1.6\% (92.7%→91.1%92.7\% \to 91.1\%), indicating the necessity of learnable connection strengths.
    2. Removing any individual component among A\mathbf{A}, B\mathbf{B}, or C\mathbf{C} from AGCN harms accuracy, dropping performance to 93.3%–93.4%93.3\%\text{--}93.4\%.
    3. Using all three components jointly (A+B+C)(\mathbf{A} + \mathbf{B} + \mathbf{C}) achieves the highest accuracy of 93.7%93.7\%.

  8. Knowl 8 — Ablation Study on Input Modalities in Two-Stream Framework

    data/table

    An ablation study on the NTU-RGBD Cross-View (X-View) benchmark compares single-stream AGCN models using only joint coordinates (Js-AGCN) or only bone vectors (Bs-AGCN) against the fused two-stream architecture (2s-AGCN).

    Methods Accuracy (%)
    Js-AGCN 93.7
    Bs-AGCN 93.2
    2s-AGCN 95.1

    The joint-only network achieves 93.7%93.7\% and the bone-only network achieves 93.2%93.2\%. Fusing both streams via score summation yields 95.1%95.1\%, an absolute improvement of 1.4%1.4\% over the joint stream alone and 1.9%1.9\% over the bone stream alone, showing that first-order position and second-order bone vectors provide complementary discriminative cues.

  9. Knowl 9 — Action Recognition Performance Comparison on NTU-RGBD Benchmark

    data/table

    Evaluation of 2s-AGCN on the 60-class NTU-RGBD benchmark under Cross-Subject (X-Sub) and Cross-View (X-View) validation protocols compared against handcrafted, RNN-based, CNN-based, and GCN-based models.

    Methods X-Sub (%) X-View (%)
    Lie Group 50.1 82.8
    HBRNN 59.1 64.0
    Deep LSTM 60.7 67.3
    ST-LSTM 69.2 77.7
    STA-LSTM 73.4 81.2
    VA-LSTM 79.2 87.7
    ARRN-LSTM 80.7 88.8
    Ind-RNN 81.8 88.0
    Two-Stream 3DCNN 66.8 72.6
    TCN 74.3 83.1
    Clips+CNN+MTLN 79.6 84.8
    Synthesized CNN 80.0 87.2
    CNN+Motion+Trans 83.2 89.3
    3scale ResNet152 85.0 92.3
    ST-GCN 81.5 88.3
    DPRL+GCNN 83.5 89.8
    2s-AGCN (ours) 88.5 95.1

    2s-AGCN achieves 88.5%88.5\% on X-Sub and 95.1%95.1\% on X-View, outperforming the previous top-performing approach (3scale ResNet152 at 85.0%85.0\% / 92.3%92.3\%) by 3.5%3.5\% and 2.8%2.8\%, and exceeding the original ST-GCN baseline (81.5%81.5\% / 88.3%88.3\%) by 7.0%7.0\% and 6.8%6.8\%, respectively.

  10. Knowl 10 — Action Recognition Performance Comparison on Kinetics-Skeleton Dataset

    data/table

    Evaluation of single-stream (Js-AGCN, Bs-AGCN) and two-stream (2s-AGCN) models on the 400-class Kinetics-Skeleton dataset compared against existing skeleton-based action recognition methods.

    Methods Top-1 (%) Top-5 (%)
    Feature Enc. 14.9 25.8
    Deep LSTM 16.4 35.3
    TCN 20.3 40.0
    ST-GCN 30.7 52.8
    Js-AGCN (ours) 35.1 57.1
    Bs-AGCN (ours) 33.3 55.7
    2s-AGCN (ours) 36.1 58.7

    On the Kinetics-Skeleton validation set, 2s-AGCN achieves 36.1%36.1\% Top-1 and 58.7%58.7\% Top-5 accuracy. This outperforms the baseline ST-GCN (30.7%30.7\% Top-1 / 52.8%52.8\% Top-5) by 5.4%5.4\% on Top-1 accuracy and 5.9%5.9\% on Top-5 accuracy.

Coverage note — No substantial contributed material was omitted; all key architectural components, mathematical formulations, and primary empirical evaluations are covered.

References

  1. 1.James Atwood and Don Towsley. Diffusion-convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1993–2001, 2016. 1, 2
  2. 2.Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral Networks and Locally Connected Networks on Graphs. In ICLR, 2014. 2
  3. 3.C. Cao, C. Lan, Y. Zhang, W. Zeng, H. Lu, and Y. Zhang. Skeleton-Based Action Recognition with Gated Convolutional Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology, pages 1–1, 2018. 2
  4. 4.Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017. 6
  5. 5.Michal Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3844–3852. Curran Associates, Inc., 2016. 2
  6. 6.Yong Du, Wei Wang, and Liang Wang. Hierarchical recurrent neural network for skeleton based action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1110–1118, 2015. 1, 2, 8
  7. 7.David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzik, and Ryan P Adams. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2224–2232. Curran Associates, Inc., 2015. 1, 2
  8. 8.Basura Fernando, Efstratios Gavves, Jose M. Oramas, Amir Ghodrati, and Tinne Tuytelaars. Modeling video evolution for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5378–5387, 2015. 1, 2, 8
  9. 9.Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1025–1035, 2017. 1, 2
  10. 10.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 5
  11. 11.Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015. 2
  12. 12.Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, and others. The Kinetics Human Action Video Dataset. arXiv preprint arXiv:1705.06950, 2017. 2, 5, 6
  13. 13.Qiuhong Ke, Mohammed Bennamoun, Senjian An, Ferdous Ahmed Sohel, and Farid Boussad. A New Representation of Skeleton Sequences for 3d Action Recognition. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4570–4579, 2017. 1, 2, 8
  14. 14.Tae Soo Kim and Austin Reiter. Interpretable 3d human action analysis with temporal convolutional networks. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on, pages 1623–1631, 2017. 1, 2, 8
  15. 15.Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In International Conference on Machine Learning (ICML), 2018. 1, 2
  16. 16.Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. arXiv preprint arXiv:1609.02907, Sept. 2016. 1, 2
  17. 17.Bo Li, Yuchao Dai, Xuelian Cheng, Huahui Chen, Yi Lin, and Mingyi He. Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep CNN. In Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on, pages 601–604. IEEE, 2017. 1, 2, 8
  18. 18.Chao Li, Qiaoyong Zhong, Di Xie, and Shiliang Pu. Skeleton-based action recognition with convolutional neural networks. In Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on, pages 597–600. IEEE, 2017. 1, 2, 8
  19. 19.Lin Li, Wu Zheng, Zhaoxiang Zhang, Yan Huang, and Liang Wang. Skeleton-Based Relational Modeling for Action Recognition. arXiv:1805.02556 [cs], 2018. 1, 2, 8
  20. 20.Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. Independently recurrent neural network (indrnn): Building A longer and deeper RNN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5457–5466, 2018. 1, 2, 8
  21. 21.Hong Liu, Juanhui Tu, and Mengyuan Liu. Two-Stream 3d Convolutional Neural Network for Skeleton-Based Action Recognition. arXiv:1705.08106 [cs], May 2017. 1, 2, 8
  22. 22.Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. Spatio-Temporal LSTM with Trust Gates for 3d Human Action Recognition. In Computer Vision ECCV 2016, volume 9907, pages 816–833. Springer International Publishing, Cham, 2016. 1, 2, 8
  23. 23.Mengyuan Liu, Hong Liu, and Chen Chen. Enhanced skeleton visualization for view invariant human action recognition. Pattern Recognition, 68:346–362, 2017. 1, 2, 8
  24. 24.Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proc. CVPR, volume 1, page 3, 2017. 1, 2
  25. 25.Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In International conference on machine learning, pages 2014–2023, 2016. 1, 2
  26. 26.Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS-W, 2017. 6
  27. 27.Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+D: A Large Scale Dataset for 3d Human Activity Analysis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 1, 2, 5, 6, 8
  28. 28.David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3):83–98, 2013. 2
  29. 29.Sijie Song, Cuiling Lan, Junliang Xing, Wenjun Zeng, and Jiaying Liu. An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data. In AAAI, volume 1, pages 4263–4270, 2017. 1, 2, 8
  30. 30.Yansong Tang, Yi Tian, Jiwen Lu, Peiyang Li, and Jie Zhou. Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 1, 8
  31. 31.Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3d skeletons as points in a lie group. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 588–595, 2014. 1, 2, 8
  32. 32.Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. In AAAI, 2018. 1, 2, 3, 5, 6, 7, 8
  33. 33.Pengfei Zhang, Cuiling Lan, Junliang Xing, Wenjun Zeng, Jianru Xue, and Nanning Zheng. View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition From Skeleton Data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117–2126, 2017. 1, 2, 8
  34. 34.Yifan Zhang, Congqi Cao, Jian Cheng, and Hanqing Lu. EgoGesture: A New Dataset and Benchmark for Egocentric Hand Gesture Recognition. IEEE Transactions on Multimedia, pages 1–1, 2018. 1

Citation

MLA
Shi, L., et al. “Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition”. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Pp. 12026-12035, 2018, http://arxiv.org/abs/1805.07694v3.
APA
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2018). Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Pp. 12026-12035. http://arxiv.org/abs/1805.07694v3
Chicago
Shi, L., Y. Zhang, J. Cheng, and H. Lu. 2018. “Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition”. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Pp. 12026-12035. http://arxiv.org/abs/1805.07694v3.
Harvard
Shi, L. et al. (2018) “Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition”, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 12026-12035 [Preprint]. Available at: http://arxiv.org/abs/1805.07694v3.
Vancouver
1. Shi L, Zhang Y, Cheng J, Lu H (2018) Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 12026-12035

BibTeX

@article{shi2018two,
  title = {Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition},
  author = {Shi, Lei and Zhang, Yifan and Cheng, Jian and Lu, Hanqing},
  year = {2018},
  journal = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 12026-12035},
  url = {http://arxiv.org/abs/1805.07694v3},
  eprint = {1805.07694}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE