Back

Deep Learning for Sensor-based Activity Recognition: A Survey

Jindong WangYiqiang ChenShuji HaoXiaohui PengLisha Hu

article2017Pattern Recognition Letters1,890 citations

Systematizes deep learning techniques for sensor-based human activity recognition across sensor modalities, neural network architectures, and application domains while outlining key challenges in unsupervised and incremental learning.

Listen

Sensor-based human activity recognition plays an increasingly vital role in modern healthcare, smart environments, and assisted living by converting continuous readings from devices like accelerometers into high-level behavioral insights. While traditional pattern recognition methods provided initial progress, they rely heavily on manual, hand-crafted feature extraction. This dependency limits their ability to generalize across diverse conditions, detect complex human behaviors, or learn from real-world data streams that lack detailed annotations.

This article comprehensively evaluates how deep learning architectures overcome the bottlenecks of traditional sensor-based recognition. The analysis synthesizes literature across sensor modalities, deep model designs, and emerging applications while reviewing standard benchmark datasets.

To conduct this evaluation, the article reviews several dozen published studies spanning multiple application areas, including daily activities, sports, and healthcare conditions such as Parkinson's disease. The authors categorize hardware setups into body-worn, object-attached, and ambient sensors, and systematically examine leading deep learning architectures, notably convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks, autoencoders, and hybrid structures.

Key findings show that deep learning substantially outperforms traditional shallow models by automating feature extraction and simultaneously learning high-level representations directly from raw multi-channel sensor signals. Hybrid models that combine spatial and temporal processing—specifically CNNs paired with recurrent layers—consistently achieve superior performance across diverse benchmarks, reaching accuracy levels between 91% and 96% on standard datasets. Model efficacy depends heavily on the activity type: recurrent networks excel at short activities with defined sequences, whereas convolutional models perform best on repetitive, long-duration tasks. In addition, generative architectures like autoencoders and deep belief networks offer distinct advantages by effectively utilizing unlabeled data during initial training phases.

These results demonstrate that moving toward end-to-end deep learning lowers the engineering cost of designing bespoke statistical features and reduces the operational risks of deploying rigid, poorly generalizing models. However, realizing these performance gains in commercial products requires balancing higher computational and power demands against battery constraints on smart wearables and edge devices.

Organizations planning sensor-based systems should implement hybrid architectures for complex multi-modal tracking, while selecting dedicated convolutional networks for repetitive fitness metrics or lightweight recurrent models for sequential workflows. To enable on-device execution, teams should explore model compression, combined shallow-deep pipelines, and transfer learning to minimize the need for expensive manually labeled data.

The findings are bounded by the current literature's reliance on offline remote server training and clean public benchmarks. Readers should remain cautious when translating these results to real-time mobile environments, as streaming data, context-aware tracking, and on-chip learning continue to pose practical engineering constraints.

Agency for Science, Technology and ResearchInstitute of Computing Technology, Chinese Academy of SciencesUniversity of Chinese Academy of Sciencesactivity recognitionconvolutional neural networksdeep learningDeepActivityActivity Recognition
arXiv: 1707.03502
Cover for Deep Learning for Sensor-based Activity Recognition: A Survey

Abstract

Sensor-based activity recognition seeks the profound high-level knowledge about human activities from multitudes of low-level sensor readings. Conventional pattern recognition approaches have made tremendous progress in the past years. However, those methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. Additionally, existing methods are undermined for unsupervised and incremental learning tasks. Recently, the recent advancement of deep learning makes it possible to perform automatic high-level feature extraction thus achieves promising performance in many areas. Since then, deep learning based methods have been widely adopted for the sensor-based activity recognition tasks. This paper surveys the recent advance of deep learning based sensor-based activity recognition. We summarize existing literature from three aspects: sensor modality, deep model, and application. We also present detailed insights on existing work and propose grand challenges for future research.

Topics

activity recognitionconvolutional neural networksdeep learningfeature extractionsensor data

Keywords

DeepActivityActivity RecognitionLearningDeep LearningCNNSensorsHARADLBody-worn

Table of Contents

  • 1 Introduction
  • 2 Background
  • 2.1 Sensor-based Activity Recognition
  • 2.2 Why Deep Learning?
  • 3 Sensor Modality
  • 3.1 Body-worn Sensor
  • 3.2 Object Sensor
  • 3.3 Ambient Sensor
  • 3.4 Hybrid Sensor
  • 4 Deep Model
  • 4.1 Deep Neural Network
  • 4.2 Convolutional Neural Network
  • 4.3 Autoencoder
  • 4.4 Restricted Boltzmann Machine
  • 4.5 Recurrent Neural Network
  • 4.6 Hybrid Model
  • 5 Applications
  • 5.1 Featured Applications
  • 5.2 Benchmark Datasets
  • 6 Summary and Discussion
  • 7 Grand Challenges
  • 8 Conclusion
  • References

Knowls

  1. Knowl 1 — Mathematical Formulation of Sensor-Based Human Activity Recognition

    definition

    Sensor-based Human Activity Recognition (HAR) aims to infer user behavior from sequential multi-dimensional sensor data.

    Given a predefined discrete activity label set with mm distinct activity classes: A={Ai}i=1mA = \{A_i\}_{i=1}^m and a time-series sequence of multi-dimensional sensor readings spanning nn time steps: s={d1,d2,…,dt,…,dn}\mathbf{s} = \{\mathbf{d}_1, \mathbf{d}_2, \dots, \mathbf{d}_t, \dots, \mathbf{d}_n\} where dt\mathbf{d}_t denotes the sensor measurement vector captured at discrete time index tt, the objective is to learn a predictive mapping function F\mathcal{F} such that the predicted activity sequence: A^={A^j}j=1n=F(s),A^j∈A\hat{A} = \{\hat{A}_j\}_{j=1}^n = \mathcal{F}(\mathbf{s}), \quad \hat{A}_j \in A accurately matches the ground truth activity sequence: A∗={Aj∗}j=1n,Aj∗∈AA^* = \{A_j^*\}_{j=1}^n, \quad A_j^* \in A where n≥mn \ge m.

    In standard formulation, the model does not operate directly on raw reading sequences s\mathbf{s}. Instead, a feature projection function Φ\Phi maps each reading di∈s\mathbf{d}_i \in \mathbf{s} to a dd-dimensional feature vector Φ(di)∈Rd\Phi(\mathbf{d}_i) \in \mathbb{R}^d. The learning objective corresponds to minimizing a discrepancy loss function L(F(Φ(di)),A∗)\mathcal{L}(\mathcal{F}(\Phi(\mathbf{d}_i)), A^*) over the dataset.

  2. Knowl 2 — Sensor Modality Taxonomy for Activity Recognition

    definition

    Sensor modalities employed in sensor-based Human Activity Recognition are grouped into four primary categories based on physical deployment:

    • Body-worn sensors: Attached directly to the user's body or integrated into personal wearable devices (such as accelerometers, gyroscopes, and magnetometers within smartphones, smartwatches, fitness bands, or smart glasses). They capture kinematic body motion and are predominantly utilized for recognizing activities of daily living (ADL) and sports.
    • Object sensors: Attached to physical objects to track object displacement and infer human interaction (e.g., accelerometers placed on drinking cups, or passive/active RFID tags on household items). These capture fine-grained manipulations to infer specific user tasks.
    • Ambient sensors: Embedded within the surrounding environment to capture human-environment interactions without direct physical contact (e.g., acoustic sensors, radar, pressure sensors, temperature sensors, and Wi-Fi signal alterations).
    • Hybrid sensing systems: Integrate multiple sensor categories simultaneously (combining body-worn, object, and ambient sensors) to capture comprehensive behavioral and environmental context required for complex multi-occupant or smart home activity recognition.

  3. Knowl 3 — Input Adaptation Strategies for 1D Sensor Data in Convolutional Neural Networks

    model/method

    Because raw sensor-based HAR inputs consist of multi-dimensional 1D temporal series rather than native 2D image matrices, applying Convolutional Neural Networks (CNNs) requires input adaptation into virtual image representations via two primary strategies:

    • Data-driven approach: Treats each individual sensor measurement axis or modality as an independent 1D channel (analogous to color channels in images) and executes 1D convolutions along the temporal sliding window. Feature maps from distinct channels are merged and flattened before feedforward dense layers. This method preserves raw 1D temporal dynamics directly but ignores explicit spatial correlations across different sensor dimensions.
    • Model-driven approach: Reshapes multi-dimensional time series into a virtual 2D image matrix to enable standard 2D convolutions. Approaches include arranging multiple sensor axes as rows/columns of a 2D matrix, generating time-frequency spectrogram representations, or transforming pressure distribution arrays into 2D spatial maps. This allows 2D convolution kernels to exploit both inter-sensor and temporal correlations, though it requires domain-specific mapping schemes.

  4. Knowl 4 — Deep Learning Architectural Paradigms for Sensor-Based HAR

    model/method

    Deep learning architectures applied to sensor-based activity recognition fall into six major paradigms:

    • Deep Neural Networks (DNN): Multilayer fully-connected architectures that perform feature learning and classification directly from raw inputs or extracted features, with additional hidden layers improving representation capacity for multi-dimensional signals.
    • Convolutional Neural Networks (CNN): Exploit local dependency across adjacent sensor readings and scale invariance across varying activity paces. They use 1D/2D convolution kernels, weight sharing (such as relaxed partial weight sharing across sensor units), and pooling layers.
    • Stacked Autoencoders (SAE): Unsupervised representation learners trained via greedy layer-wise pre-training and encoding-decoding reconstruction, often incorporating sparsity constraints (e.g., Kullback-Leibler divergence penalties) to extract discriminative features prior to supervised fine-tuning.
    • Restricted Boltzmann Machines (RBM) and Deep Belief Networks (DBN): Generative bipartite graphical models using continuous Gaussian visible units in the input layer and binary units in subsequent layers, optionally structured as multi-modal RBMs with separate branches per sensor modality.
    • Recurrent Neural Networks (RNN) and LSTM: Sequential neural networks that leverage recurrent memory cells to track temporal dependencies across streaming sensor data, including lightweight binarized-BLSTM architectures for resource-constrained embedded platforms.
    • Hybrid Architectures (e.g., DeepConvLSTM): Cascaded networks where early convolutional layers extract local spatial/spectral feature hierarchies, followed by recurrent LSTM layers and dense output units to model temporal transitions across variable time spans.

  5. Knowl 5 — Empirical Performance Comparison of Deep Learning Models across HAR Benchmark Protocols

    data/table

    Comparative performance of deep learning architectures across standardized benchmark evaluation protocols on OPPORTUNITY, Skoda, and UCI Smartphone datasets:

    Protocol Model Result Metric
    OPPORTUNITY 1 b-LSTM-S 92.70% Weighted F1F_1-score
    OPPORTUNITY 1 CNN 85.10% Weighted F1F_1-score
    OPPORTUNITY 1 CNN 88.30% Weighted F1F_1-score
    OPPORTUNITY 1 DeepConvLSTM 91.70% Weighted F1F_1-score
    OPPORTUNITY 2 DBN 73.20% Accuracy
    OPPORTUNITY 2 CNN 76.80% Accuracy
    OPPORTUNITY 2 DBN 83.30% Accuracy
    Skoda CNN 86.10% Accuracy
    Skoda CNN 89.30% Accuracy
    Skoda DeepConvLSTM 95.80% Accuracy
    UCI smartphone CNN 94.61% Accuracy
    UCI smartphone CNN 95.18% Accuracy
    UCI smartphone CNN 94.79% Accuracy
    UCI smartphone CNN 90.00% Accuracy

    The comparisons indicate that no single architecture dominates across all scenarios. Hybrid models combining convolutional and recurrent units (such as DeepConvLSTM) attain superior performance on complex multimodal activity benchmarks (achieving 91.70% weighted F1F_1-score on OPPORTUNITY 1 and 95.80% accuracy on Skoda). For simple repetitive daily activities with smartphone inertial sensors, pure CNN models consistently deliver high accuracy (exceeding 95.18%). Generative networks (DBNs) and recurrent architectures (b-LSTM-S) achieve high efficacy on sequential and composite activities.

  6. Knowl 6 — Model Selection and Preprocessing Guidelines for Sensor-Based HAR

    theoretical result

    Synthesizing extensive empirical evaluations yields systematic guidelines for deep model selection and sensor data preprocessing in HAR:

    • Temporal Characteristics vs. Model Choice: Recurrent neural networks (RNN, LSTM) are optimal for short-duration activities that follow natural, sequential temporal orders. Convolutional neural networks (CNN) are optimal for long-term repetitive activities (e.g., walking, running, cycling) because their kernels extract deep features from periodic and recursive signal patterns.
    • Multimodal Sensor Fusion: For inputs comprising heterogeneous and multi-axis sensor streams, CNN architectures are preferred due to their ability to integrate signals across multi-channel convolutions.
    • Input Adaptation Choice: Data-driven 1D channel representations generally surpass model-driven 2D image transformations by preserving the native temporal correlation of time-series data without imposing artificial 2D spatial adjacency.
    • Generative Pre-training: Multi-layer generative models (RBMs and Stacked Autoencoders) require greedy layer-wise unsupervised pre-training prior to supervised fine-tuning to achieve stable convergence on sparse labeled datasets.
    • Windowing and Channelization: Preprocessing must incorporate temporal sliding windows proportional to sensor sampling rates and structure multi-axis readings into discrete channels to maximize neural representation capacity.

  7. Knowl 7 — Public Benchmark Datasets for Sensor-Based Activity Recognition

    data/table

    Characteristics of prominent public benchmark datasets utilized in deep learning-based sensor activity recognition research:

    Dataset Activity Domain # Subjects Sampling Rate Sensors
    OPPORTUNITY ADL 4 32 Hz Accelerometer, Gyroscope, Magnetometer, Object, Ambient
    Skoda Checkpoint Factory 1 96 Hz Accelerometer
    UCI Smartphone ADL 30 50 Hz Accelerometer, Gyroscope
    PAMAP2 ADL 9 100 Hz Accelerometer, Gyroscope, Magnetometer
    USC-HAD ADL 14 100 Hz Accelerometer, Gyroscope
    WISDM ADL 29 20 Hz Accelerometer
    DSADS ADL 8 25 Hz Accelerometer, Gyroscope, Magnetometer
    Ambient Kitchen Food preparation 20 40 Hz Object sensors
    Darmstadt Daily ADL 1 100 Hz Accelerometer
    Actitracker ADL 36 20 Hz Accelerometer
    SHO ADL 10 50 Hz Accelerometer, Gyroscope, Magnetometer
    BIDMC Heart failure 15 125 Hz ECG
    MHEALTH ADL 10 50 Hz Accelerometer, ECG, Gyroscope
    Daphnet Gait Gait (Parkinson's) 10 64 Hz Accelerometer
    ActiveMiles ADL 10 50–200 Hz Accelerometer
    HASC ADL 1 200 Hz Accelerometer
    PAF PAF disease 48 128 Hz EEG
    ActRecTut Gesture 2 32 Hz Accelerometer, Gyroscope
    Heterogeneous ADL / Cartrack 9 100–200 Hz Accelerometer, Gyroscope

    These standardized datasets provide public benchmarks spanning basic Activities of Daily Living (ADL), industrial gestures, gait disorders, and clinical physiological diagnostics across sampling rates ranging from 20 Hz to 200 Hz.

  8. Knowl 8 — Open Challenges and Future Directions in Deep Sensor-Based HAR

    limitation

    Six fundamental open challenges define future research in sensor-based deep activity recognition:

    • Online and Real-Time Mobile Deployment: Existing approaches predominantly train deep models offline on central servers, executing only forward inference on mobile hardware. Enabling on-device training and real-time incremental adaptation requires reducing server-device communication overhead and optimizing edge execution.
    • Label Scarcity and Unsupervised Representation Learning: Fully supervised training is bottlenecked by the difficulty of continuous labeling. Overcoming this necessitates crowdsourcing annotation schemes and deep transfer learning algorithms to transfer invariant representations across users, sensor placements, and disparate domains.
    • High-Level and Context-Aware Activity Recognition: Inferring semantic, complex daily actions (e.g., "having coffee" or social interactions) exceeds the capabilities of pure body-worn kinematics and requires integrating hybrid sensor configurations with contextual signals (Wi-Fi, Bluetooth, GPS).
    • Light-Weight Deep Architectures: Deep networks require memory and compute beyond the capacity of low-power wearables. Solutions include combining handcrafted statistical domain features with deep representations and designing collaborative deep-shallow architectures.
    • Non-Invasive and Opportunistic Sensing: Preserving natural user behavior requires opportunistic computing models capable of handling non-continuous, intermittent sensor streams with robust backpropagation strategies.
    • Expansion to Assessment and Assistance: Moving beyond basic discrete activity classification toward skill assessment (e.g., sports proficiency, surgical evaluation) and proactive clinical assistance systems.

Coverage note — All primary survey contributions—including mathematical formulations, sensor modality and architectural taxonomies, input adaptation strategies, model selection guidelines, benchmark dataset characteristics, empirical comparative evaluation tables, and key open challenges—are included. Specific citations and individual design variations from the extensive survey table (Table 4) were summarized into the respective architectural and modality taxonomies.

References

  1. 1.Abowd, G.D., Dey, A.K., Brown, P.J., Davies, N., Smith, M., Steggles, P., 1999. Towards a better understanding of context and context-awareness, in: International Symposium on Handheld and Ubiquitous Computing, Springer. pp. 304–307.
  2. 2.Almaslukh, B., AlMuhtadi, J., Artoli, A., 2017. An effective deep autoencoder approach for online smartphone-based human activity recognition. International Journal of Computer Science and Network Security 17, 160.
  3. 3.Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., Tan, H.P., 2016. Deep activity recognition models with triaxial accelerometers. AAAI workshop .
  4. 4.Bao, L., Intille, S.S., 2004. Activity recognition from user-annotated acceleration data, in: International Conference on Pervasive Computing, Springer. pp. 1–17.
  5. 5.Bengio, Y., 2013. Deep learning of representations: Looking forward, in: International Conference on Statistical Language and Speech Processing, Springer. pp. 1–37.
  6. 6.Bhattacharya, S., Lane, N.D., 2016. From smart to deep: Robust activity recognition on smartwatches using deep learning, in: 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), IEEE. pp. 1–6.
  7. 7.Bulling, A., Blanke, U., Schiele, B., 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 33.
  8. 8.Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S.T., Tröster, G., Millán, J.d.R., Roggen, D., 2013. The opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 2033–2042.
  9. 9.Chen, Y., Gu, Y., Jiang, X., Wang, J., 2016a. Ocean: a new opportunistic computing model for wearable activity recognition, in: UbiComp, ACM. pp. 33–36.
  10. 10.Chen, Y., Xue, Y., 2015. A deep learning approach to human activity recognition based on single accelerometer, in: Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on, IEEE. pp. 1488–1492.
  11. 11.Chen, Y., Zhong, K., Zhang, J., Sun, Q., Zhao, X., 2016b. Lstm networks for mobile human activity recognition .
  12. 12.Cheng, W.Y., Scotland, A.e.a., 2017. Human activity recognition from sensor-based large-scale continuous monitoring of parkinsons disease patients, in: Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2017 IEEE/ACM International Conference on, pp. 249–250.
  13. 13.Cook, D., Feuz, K.D., Krishnan, N.C., 2013. Transfer learning for activity recognition: A survey. Knowledge and information systems 36, 537–556.
  14. 14.Edel, M., Köppe, E., 2016. Binarized-blstm-rnn based human activity recognition, in: Indoor Positioning and Indoor Navigation (IPIN), 2016 International Conference on, IEEE. pp. 1–7.
  15. 15.Fang, H., Hu, C., 2014. Recognizing human activity in smart home using deep learning algorithm, in: Chinese Control Conference (CCC), pp. 4716–4720.
  16. 16.Gjoreski, H., Bizjak, J., Gjoreski, M., Gams, M., 2016. Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer, in: IJCAI-16 workshop on Deep Learning for Artificial Intelligence (DLAI).
  17. 17.Guan, Y., Ploetz, T., 2017. Ensembles of deep lstm learners for activity recognition using wearables. arXiv preprint arXiv:1703.09370 .
  18. 18.Ha, S., Choi, S., 2016. Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors, in: Neural Networks (IJCNN), 2016 International Joint Conference on, IEEE. pp. 381–388.
  19. 19.Ha, S., Yun, J.M., Choi, S., 2015. Multi-modal convolutional neural networks for activity recognition, in: Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on, IEEE. pp. 3017–3022.
  20. 20.Hammerla, N.Y., Fisher, J., Andras, P., Rochester, L., Walker, R., Plötz, T., 2015. Pd disease state assessment in naturalistic environments using deep learning, in: Twenty-Ninth AAAI Conference on Artificial Intelligence.
  21. 21.Hammerla, N.Y., Halloran, S., Ploetz, T., 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables, in: IJCAI.
  22. 22.Hannink, J., Kautz, T., Pasluosta, C.F., Gaßmann, K.G., et al., 2017. Sensor-based gait parameter extraction with deep convolutional neural networks. IEEE journal of biomedical and health informatics 21, 85–93.
  23. 23.Hayashi, T., Nishida, M., Kitaoka, N., Takeda, K., 2015. Daily activity recognition based on dnn using environmental sound and acceleration signals, in: Signal Processing Conference (EUSIPCO), 23rd European, pp. 2306–2310.
  24. 24.Hinton, G.E., Osindero, S., Teh, Y.W., 2006. A fast learning algorithm for deep belief nets. Neural computation 18, 1527–1554.
  25. 25.Hu, L., Chen, Y., Wang, S., Wang, J., Shen, J., Jiang, X., Shen, Z., 2016. Less annotation on personalized activity recognition using context data, in: UIC, pp. 327–332.
  26. 26.Inoue, M., Inoue, S., Nishida, T., 2016. Deep recurrent neural network for mobile human activity recognition with high throughput. arXiv preprint arXiv:1611.03607 .
  27. 27.Jiang, W., Yin, Z., 2015. Human activity recognition using wearable sensors by deep convolutional neural networks, in: MM, ACM. pp. 1307–1310.
  28. 28.Khan, A., Mellor, S., Berlin, E., Thompson, R., McNaney, R., Olivier, P., Plötz, T., 2015. Beyond activity recognition: skill assessment from accelerometer data, in: UbiComp, ACM. pp. 1155–1166.
  29. 29.Khan, U.M., Kabir, Z., Hassan, S.A., Ahmed, S.H., 2017. A deep learning framework using passive wifi sensing for respiration monitoring. arXiv preprint arXiv:1704.05708 .
  30. 30.Kim, Y., Li, Y., 2017. Human activity classification with transmission and reflection coefficients of on-body antennas through deep convolutional neural networks. IEEE Transactions on Antennas and Propagation 65, 2764–2768.
  31. 31.Kim, Y., Toomajian, B., 2016. Hand gesture recognition using micro-doppler signatures with convolutional neural network. IEEE Access .
  32. 32.Lane, N.D., Georgiev, P., 2015. Can deep learning revolutionize mobile sensing?, in: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, ACM. pp. 117–122.
  33. 33.Lane, N.D., Georgiev, P., Qendro, L., 2015. Deepear: robust smartphone audio sensing in unconstrained acoustic environments using deep learning, in: UbiComp, ACM. pp. 283–294.
  34. 34.Lara, O.D., Labrador, M.A., 2013. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys & Tutorials 15, 1192–1209.
  35. 35.LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521, 436–444.
  36. 36.Lee, S.M., Yoon, S.M., Cho, H., 2017. Human activity recognition from accelerometer data using convolutional neural network, in: Big Data and Smart Computing (BigComp), IEEE International Conference on, pp. 131–134.
  37. 37.Li, X., Zhang, Y., Li, M., Marsic, I., Yang, J., Burd, R.S., 2016a. Deep neural network for rfid based activity recognition, in: Wireless of the Students, by the Students, and for the Students (S3) Workshop with MobiCom.
  38. 38.Li, X., Zhang, Y., Marsic, I., Sarcevic, A., Burd, R.S., 2016b. Deep learning for rfid-based activity recognition, in: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, ACM. pp. 164–175.
  39. 39.Li, Y., Shi, D., Ding, B., Liu, D., 2014. Unsupervised feature learning for human activity recognition using smartphone sensors, in: Mining Intelligence and Knowledge Exploration. Springer, pp. 99–107.
  40. 40.Liu, C., Zhang, L., Liu, Z., Liu, K., Li, X., Liu, Y., 2016. Lasagna: towards deep hierarchical understanding and searching over mobile sensing data, in: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, ACM. pp. 334–347.
  41. 41.Mohammed, S., Tashev, I., 2017. Unsupervised deep representation learning to remove motion artifacts in free-mode body sensor networks, in: Wearable and Implantable Body Sensor Networks (BSN), 2017 IEEE 14th International Conference on, IEEE. pp. 183–188.
  42. 42.Morales, F.J.O., Roggen, D., 2016. Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations, in: Proceedings of the 2016 ACM International Symposium on Wearable Computers, ACM. pp. 92–99.
  43. 43.Murad, A., Pyun, J.Y., 2017. Deep recurrent neural networks for human activity recognition. Sensors 17, 2556.
  44. 44.Ordóñez, F.J., Roggen, D., 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 115.
  45. 45.Pan, S.J., Yang, Q., 2010. A survey on transfer learning. Knowledge and Data Engineering, IEEE Transactions on 22, 1345–1359.
  46. 46.Panwar, M., Dyuthi, S.R., Prakash, K.C., Biswas, D., Acharyya, A., Maharatna, K., Gautam, A., Naik, G.R., 2017. Cnn based approach for activity recognition using a wrist-worn accelerometer, in: Engineering in Medicine and Biology Society (EMBC), 2017 39th Annual International Conference of the IEEE, IEEE. pp. 2438–2441.
  47. 47.Plötz, T., Hammerla, N.Y., Olivier, P., 2011. Feature learning for activity recognition in ubiquitous computing, in: IJCAI, p. 1729.
  48. 48.Pourbabaee, B., Roshtkhari, M.J., Khorasani, K., 2017. Deep convolution neural networks and learning ecg features for screening paroxysmal atrial fibrillatio patients. IEEE Trans. on Systems, Man, and Cybernetics .
  49. 49.Prelec, D., Seung, H.S., McCoy, J., 2017. A solution to the single-question crowd wisdom problem. Nature 541, 532–535.
  50. 50.Qin, J., Liu, L., Zhang, Z., Wang, Y., Shao, L., 2016. Compressive sequential learning for action similarity labeling. IEEE Transactions on Image Processing 25, 756–769.
  51. 51.Radu, V., Lane, N.D., Bhattacharya, S., Mascolo, C., Marina, M.K., Kawsar, F., 2016. Towards multimodal deep learning for activity recognition on mobile devices, in: UbiComp, ACM. pp. 185–188.
  52. 52.Ravi, D., Wong, C., Lo, B., Yang, G.Z., 2016. Deep learning for human activity recognition: A resource efficient implementation on low-power devices, in: Wearable and Implantable Body Sensor Networks (BSN), 2016 IEEE 13th International Conference on, IEEE. pp. 71–76.
  53. 53.Ravì, D., Wong, C., Lo, B., Yang, G.Z., 2017. A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE journal of biomedical and health informatics 21, 56–64.
  54. 54.Ronao, C.A., Cho, S.B., 2015a. Deep convolutional neural networks for human activity recognition with smartphone sensors, in: International Conference on Neural Information Processing, Springer. pp. 46–53.
  55. 55.Ronao, C.A., Cho, S.B., 2015b. Evaluation of deep convolutional neural network architectures for human activity recognition with smartphone sensors, in: Proc. of the KIISE Korea Computer Congress, pp. 858–860.
  56. 56.Ronao, C.A., Cho, S.B., 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications 59, 235–244.
  57. 57.Sathyanarayana, A., Joty, S., Fernandez-Luque, L., Ofli, F., Srivastava, J., Elmagarmid, A., Taheri, S., Arora, T., 2016. Impact of physical activity on sleep: A deep learning based exploration. arXiv preprint:1607.07034 .
  58. 58.Schmidhuber, J., 2015. Deep learning in neural networks: An overview. Neural Networks 61, 85–117.
  59. 59.Singh, M.S., Pondenkandath, V., Zhou, B., Lukowicz, P., Liwicki, M., 2017. Transforming sensor data to the image domain for deep learning-an application to footstep detection. arXiv preprint arXiv:1701.01077 .
  60. 60.Stewart, R., Ermon, S., 2017. Label-free supervision of neural networks with physics and domain knowledge., in: AAAI, pp. 2576–2582.
  61. 61.Vepakomma, P., De, D., Das, S.K., Bhansali, S., 2015. A-wristocracy: Deep learning on wrist-worn sensing for recognition of user complex activities, in: 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), IEEE. pp. 1–6.
  62. 62.Walse, K.H., Dharaskar, R.V., Thakare, V.M., 2016. Pca based optimal ann classifiers for human activity recognition using mobile sensors data, in: Proceedings of First International Conference on Information and Communication Technology for Intelligent Systems: Volume 1, Springer. pp. 429–436.
  63. 63.Wang, A., Chen, G., Shang, C., Zhang, M., Liu, L., 2016a. Human activity recognition in a smart home environment with stacked denoising autoencoders, in: International Conference on Web-Age Information Management, Springer. pp. 29–40.
  64. 64.Wang, J., Chen, Y., Hao, S., Feng, W., Shen, Z., 2017. Balanced distribution adaptation for transfer learning, in: The IEEE International conference on data mining (ICDM), pp. 1129–1134.
  65. 65.Wang, J., Zhang, X., Gao, Q., Yue, H., Wang, H., 2016b. Device-free wireless localization and activity recognition: A deep learning approach. IEEE Transactions on Vehicular Technology .
  66. 66.Yang, J.B., Nguyen, M.N., San, P.P., Li, X.L., Krishnaswamy, S., 2015. Deep convolutional neural networks on multichannel time series for human activity recognition, in: IJCAI, Buenos Aires, Argentina, pp. 25–31.
  67. 67.Yang, Q., 2009. Activity recognition: Linking low-level sensors to high-level intelligence., in: IJCAI, pp. 20–25.
  68. 68.Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T., 2017. Deepsense: A unified deep learning framework for time-series mobile sensing data processing, in: WWW, pp. 351–360.
  69. 69.Zebin, T., Scully, P.J., Ozanyan, K.B., 2016. Human activity recognition with inertial sensors using a deep learning approach, in: SENSORS, pp. 1–3.
  70. 70.Zeng, M., Nguyen, L.T., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P., Zhang, J., 2014. Convolutional neural networks for human activity recognition using mobile sensors, in: Mobile Computing, Applications and Services (MobiCASE), 2014 6th International Conference on, IEEE. pp. 197–205.
  71. 71.Zhang, L., Wu, X., Luo, D., 2015a. Human activity recognition with hmm-dnn model, in: Cognitive Informatics & Cognitive Computing (ICCI* CC), 2015 IEEE 14th International Conference on, IEEE. pp. 192–197.
  72. 72.Zhang, L., Wu, X., Luo, D., 2015b. Real-time activity recognition on smartphones using deep neural networks, in: UIC, IEEE. pp. 1236–1242.
  73. 73.Zhang, L., Wu, X., Luo, D., 2015c. Recognizing human activities from raw accelerometer data using deep neural networks, in: IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 865–870.
  74. 74.Zhang, S., Ng, W.W., Zhang, J., Nugent, C.D., 2017a. Human activity recognition using radial basis function neural network trained via a minimization of localized generalization error, in: International Conference on Ubiquitous Computing and Ambient Intelligence, Springer. pp. 498–507.
  75. 75.Zhang, Y., Li, X., Zhang, J., et al., 2017b. Car-a deep learning structure for concurrent activity recognition, in: IPSN, pp. 299–300.
  76. 76.Zheng, Y., Liu, Q., Chen, E., Ge, Y., Zhao, J.L., 2014. Time series classification using multi-channels deep convolutional neural networks, in: International Conference on Web-Age Information Management, Springer. pp. 298–310.
  77. 77.Zheng, Y., Liu, Q., Chen, E., Ge, Y., Zhao, J.L., 2016. Exploiting multi-channels deep convolutional neural networks for multivariate time series classification. Frontiers of Computer Science 10, 96–112.

Citation

MLA
Wang, J., et al. “Deep Learning for Sensor-based Activity Recognition: A Survey”. Pattern Recognition Letters, vol. 119, 2019, pp. 3–1, https://doi.org/10.1016/j.patrec.2018.02.010.
APA
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019). Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters, 119, 3–11. https://doi.org/10.1016/j.patrec.2018.02.010
Chicago
Wang, J., Y. Chen, S. Hao, X. Peng, and L. Hu. 2019. “Deep Learning for Sensor-based Activity Recognition: A Survey”. Pattern Recognition Letters 119: 3–11. https://doi.org/10.1016/j.patrec.2018.02.010.
Harvard
Wang, J. et al. (2019) “Deep learning for sensor-based activity recognition: A survey”, Pattern Recognition Letters, 119, pp. 3–11. Available at: https://doi.org/10.1016/j.patrec.2018.02.010.
Vancouver
1. Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters 119:3–11

BibTeX

@article{Wang_2019, title={Deep learning for sensor-based activity recognition: A survey}, volume={119}, ISSN={0167-8655}, url={http://dx.doi.org/10.1016/j.patrec.2018.02.010}, DOI={10.1016/j.patrec.2018.02.010}, journal={Pattern Recognition Letters}, publisher={Elsevier BV}, author={Wang, Jindong and Chen, Yiqiang and Hao, Shuji and Peng, Xiaohui and Hu, Lisha}, year={2019}, month=Mar, pages={3–11} }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF