research paper on mobile devices

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 18 October 2021

A systematic review of smartphone-based human activity recognition methods for health research

• Marcin Straczkiewicz   ORCID: orcid.org/0000-0002-8703-4451 1 ,
• Peter James 2 , 3 &
• Jukka-Pekka Onnela 1  

npj Digital Medicine volume  4 , Article number:  148 ( 2021 ) Cite this article

18k Accesses

75 Citations

23 Altmetric

Metrics details

• Predictive markers
• Public health
• Quality of life

Smartphones are now nearly ubiquitous; their numerous built-in sensors enable continuous measurement of activities of daily living, making them especially well-suited for health research. Researchers have proposed various human activity recognition (HAR) systems aimed at translating measurements from smartphones into various types of physical activity. In this review, we summarized the existing approaches to smartphone-based HAR. For this purpose, we systematically searched Scopus, PubMed, and Web of Science for peer-reviewed articles published up to December 2020 on the use of smartphones for HAR. We extracted information on smartphone body location, sensors, and physical activity types studied and the data transformation techniques and classification schemes used for activity recognition. Consequently, we identified 108 articles and described the various approaches used for data acquisition, data preprocessing, feature extraction, and activity classification, identifying the most common practices, and their alternatives. We conclude that smartphones are well-suited for HAR research in the health sciences. For population-level impact, future studies should focus on improving the quality of collected data, address missing data, incorporate more diverse participants and activities, relax requirements about phone placement, provide more complete documentation on study participants, and share the source code of the implemented methods and algorithms.

Similar content being viewed by others

Best practices for analyzing large-scale health data from wearables and smartphone apps

Jennifer L. Hicks, Tim Althoff, … Scott L. Delp

A “one-size-fits-most” walking recognition method for smartphones, smartwatches, and wearable accelerometers

Marcin Straczkiewicz, Emily J. Huang & Jukka-Pekka Onnela

Quantification of acceleration as activity counts in ActiGraph wearable

Ali Neishabouri, Joe Nguyen, … Christine C. Guo

Introduction

Progress in science has always been driven by data. More than 5 billion mobile devices were in use in 2020 1 , with multiple sensors (e.g., accelerometer and GPS) that can capture detailed, continuous, and objective measurements on various aspects of our lives, including physical activity. Such proliferation in worldwide smartphone adoption presents unprecedented opportunities for the collection of data to study human behavior and health. Along with sufficient storage, powerful processors, and wireless transmission, smartphones can collect a tremendous amount of data on large cohorts of individuals over extended time periods without additional hardware or instrumentation.

Smartphones are promising data collection instruments for objective and reproducible quantification of traditional and emerging risk factors for human populations. Behavioral risk factors, including but not limited to sedentary behavior, sleep, and physical activity, can all be monitored by smartphones in free-living environments, leveraging the personal or lived experiences of individuals. Importantly, unlike some wearable activity trackers 2 , smartphones are not a niche product but instead have become globally available, increasingly adopted by users of all ages both in advanced and emerging economies 3 , 4 . Their adoption in health research is further supported by encouraging findings made with other portable devices, primarily wearable accelerometers, which have demonstrated robust associations between physical activity and health outcomes, including obesity, diabetes, various cardiovascular diseases, mental health, and mortality 5 , 6 , 7 , 8 , 9 . However, there are some important limitations to using wearables for studying population health: (1) their ownership is much lower than that of smartphones 10 ; (2) most people stop using their wearables after 6 months of use 11 ; and (3) raw data are usually not available from wearable devices. The last point often forces investigators to rely on proprietary device metrics, which lowers the already low rate of reproducibility of biomedical research in general 12 and makes uncertainty quantification in the measurements nearly impossible.

Human activity recognition (HAR) is a process aimed at the classification of human actions in a given period of time based on discrete measurements (acceleration, rotation speed, geographical coordinates, etc.) made by personal digital devices. In recent years, this topic has been proliferating within the machine learning research community; at the time of writing, over 400 articles had been published on HAR methods using smartphones. This is a substantial increase from just a handful of articles published a few years earlier (Fig. 1 ). As data collection using smartphones becomes easier, analysis of the collected data is increasingly identified as the main bottleneck in health research 13 , 14 , 15 . To tackle the analytical challenges of HAR, researchers have proposed various algorithms that differ substantially in terms of the type of data they use, how they manipulate the collected data, and the statistical approaches used for inference and/or classification. Published studies use existing methods and propose new methods for the collection, processing, and classification of activities of daily living. Authors commonly discuss data filtering and feature selection techniques and compare the accuracy of various machine learning classifiers either on previously existing datasets or on datasets they have collected de novo for the purposes of the specific study. The results are typically summarized using classification accuracy within different groups of activities, such as ambulation, locomotion, and exercise.

Articles were published between January 2008 and December 2020, based on a search of PubMed, Scopus, and Web of Science databases (for details, see “Methods”).

To successfully incorporate developments in HAR into research in public health and medicine, there is a need to understand the approaches that have been developed and identify their potential limitations. Methods need to accommodate physiological (e.g., weight, height, age) and habitual (e.g., posture, gait, walking speed) differences of smartphone users, as well as differences in the built environment (e.g., buildings and green spaces) that provide the physical and social setting for human activities. Moreover, the data collection and statistical approaches typically used in HAR may be affected by location (where the user wears the phone on their body) and orientation of the device 16 , which complicates the transformation of collected data into meaningful and interpretable outputs.

In this paper, we systematically review the emerging literature on the use of smartphones for HAR for health research in free-living settings. Given that the main challenge in this field is shifting from data collection to data analysis, we focus our analysis on the approaches used for data acquisition, data preprocessing, feature extraction, and activity classification. We provide insight into the complexity and multidimensionality of HAR utilizing smartphones, the types of data collected, and the methods used to translate digital measurements into human activities. We discuss the generalizability and reproducibility of approaches, i.e., the features that are essential and applicable to large and diverse cohorts of study participants. Lastly, we identify challenges that need to be tackled to accelerate the wider utilization of smartphone-based HAR in public health studies.

Our systematic review was conducted by searching for articles published up to December 31, 2020, on PubMed, Scopus, and Web of Science databases. The databases were screened for titles, abstracts, and keywords containing phrases “activity” AND (“recognition” OR “estimation” OR “classification”) AND (“smartphone” OR “cell phone” OR “mobile phone”). The search was limited to full-length journal articles written in English. After removing duplicates, we read the titles and abstracts of the remaining publications. Studies that did not investigate HAR approaches were excluded from further screening. We then filtered out studies that employed auxiliary equipment, like wearable or ambient devices, and studies that required carrying multiple smartphones. Only studies that made use of commercially available consumer-grade smartphones (either personal or loaner) were read in full. We excluded studies that used the smartphone microphone or video camera for activity classification as they might record information about an individual’s surroundings, including information about unconsented individuals, and thus hinder the large-scale application of the approach due to privacy concerns. To focus on studies that mimicked free-living settings, we excluded studies that utilized devices strapped or glued to the body in a fixed position.

Our search resulted in 1901 hits for the specified search criteria (Fig. 2 ). After removal of articles that did not discuss HAR algorithms ( n  = 793), employed additional hardware ( n  = 150), or utilized microphones, cameras, or body-affixed smartphones ( n  = 149), there were 108 references included in this review.

The search was conducted in PubMed, Scopus, and Web of Science databases and included full-length peer-reviewed articles written in English. The search was carried out on January 2, 2021.

Most HAR approaches consist of four stages: data acquisition, data preprocessing, feature extraction, and activity classification (Fig. 3 ). Here, we provide an overview of these steps and briefly point to significant methodological differences among the reviewed studies for each step. Figure 4 summarizes specific aspects of each study. Of note, we decomposed data acquisition processes into sensor type, experimental environment, investigated activities, and smartphone location; we indicated which studies preprocessed collected measurements using signal correction methods, noise filtering techniques, and sensor orientation-invariant transformations; we marked investigations based on the types of signal features they extracted, as well as the feature selection approaches used; we indicated the adopted activity classification principles, utilized classifiers, and practices for accuracy reporting; and finally, we highlighted efforts supporting reproducibility and generalizability of the research. Before diving into these technical considerations, we first provide a brief description of study populations.

The map displays common aspects of HAR systems together with their operational definitions. The methodological differences between the reviewed studies are highlighted in Figure 4 .

The columns correspond to the 108 reviewed studies and the rows correspond to different technical aspects of each study. Cells marked with a cross (x) indicate that the given study used the given method, algorithm, or approach. Rows have been grouped to correspond to different stages of HAR, such as data processing, and color shading of rows indicates how frequently a particular aspect is present among the studies (darker shade corresponds to higher frequency).

Study populations

We use the term study population to refer to the group of individuals investigated in any given study. In the reviewed studies, data were usually collected from fewer than 30 individuals, although one larger study analyzed data from 440 healthy individuals 17 . Studies often included healthy adults in their 20s and 30s, with only a handful of studies involving older individuals. Most studies did not report the full distribution of ages, only the mean age or the age range of participants (Fig. 5 ). To get a sense of the distribution of participant ages, we attempted to reconstruct an overall approximate age distribution by assuming that the participants in each study are evenly distributed in age between the minimum and maximum ages, which may not be the case. A comparison of the reconstructed age distribution of study participants with nationwide age distributions clearly demonstrates that future HAR research in health settings needs to broaden the age spectrum of the participants. Less effort was devoted in the studies to investigating populations with different demographic and disease characteristics, such as elders 18 , 19 , 20 and individuals with Parkinson’s disease 21 .

Panel a displays age of the population corresponding to individual studies, typically described by its range (lines) or mean (dots). Panel b displays the reconstructed age distribution in the reviewed studies (see the text). Nationwide age distributions displayed in panel c of three countries offer a stark contrast with the reconstructed distribution of study participant ages.

Data acquisition

We use the term data acquisition to refer to a process of collecting and storing raw sub-second-level smartphone measurements for the purpose of HAR. The data are typically collected from individuals by an application that runs on the device and samples data from built-in smartphone sensors according to a predefined schedule. We carefully examined the selected literature for details on the investigated population, measurement environment, performed activities, and smartphone settings.

In the reviewed studies, data acquisition typically took place in a research facility and/or nearby outdoor surroundings. In such environments, study participants were asked to perform a series of activities along predefined routes and to interact with predefined objects. The duration and order of performed activities were usually determined by the study protocol and the participant was supervised by a research team member. A less common approach involved observation conducted in free-living environments, where individuals performed activities without specific instructions. Such studies were likely to provide more insight into diverse activity patterns due to individual habits and unpredictable real-life conditions. Compared to a single laboratory visit, studies conducted in free-living environments also allowed investigators to monitor behavioral patterns over many weeks 22 or months 23 .

Activity selection is one of the key aspects of HAR. The studies in our review tended to focus on a small set of activities, including sitting, standing, walking, running, and stair climbing. Less common activities involved various types of mobility, locomotion, fitness, and household routines, e.g., slow, normal, and brisk walking 24 , multiple transportation modes, such as by car, bus, tram, train, metro, and ferry 25 , sharp body-turns 26 , and household activities, like sweeping a floor or walking with a shopping bag 27 . More recent studies concentrated solely on walking recognition 28 , 29 . As shown in Fig. 4 , the various measured activities in the reviewed studies can be grouped into classes: “posture” refers to lying, sitting, standing, or any pair of these activities; “mobility” refers to walking, stair climbing, body-turns, riding an elevator or escalator, running, cycling, or any pair of these activities; “locomotion” refers to motorized activities; and “other” refers to various household and fitness activities or singular actions beyond the described groups.

The spectrum of investigated activities determines the choice of sensors used for data acquisition. At the time of writing, a standard smartphone is equipped with a number of built-in hardware sensors and protocols that can be used for activity monitoring, including an accelerometer, gyroscope, magnetometer, GPS, proximity sensor, and light sensor, as well as to collect information on ambient pressure, humidity, and temperature (Fig. 6 ). Accurate estimation of commonly available sensors over time is challenging given a large number of smartphone manufacturers and models, as well as the variation in their adoption in different countries. Based on global statistics on smartphone market shares 30 and specifications of flagship models 31 , it appears that accelerometer, gyroscope, magnetometer, GPS, and proximity and light sensors were fairly commonly available by 2010. Other smartphone sensors were introduced a couple of years later; for example, the barometer was included in Samsung Galaxy S III released in 2012, and thermometer and hygrometer were included in Samsung Galaxy S4 released in 2013.

Inertial sensors (accelerometer, gyroscope, and magnetometer) provide measurements with respect to the three orthogonal axes ( x , y , z ) of the body of the phone; the remaining sensors are orientation-invariant.

Our literature review revealed that the most commonly used sensors for HAR are the accelerometer, gyroscope, and magnetometer, which capture data about acceleration, angular velocity, and phone orientation, respectively, and provide temporally dense, high-resolution measurements for distinguishing among activity classes (Fig. 7 ). Inertial sensors were often used synchronously to provide more insight into the dynamic state of the device. Some studies showed that the use of a single sensor can yield similar accuracy of activity recognition as using multiple sensors in combination 32 . To alleviate the impact of sensor position, some researchers collected data using the built-in barometer and GPS sensors to monitor changes in altitude and geographic location 33 , 34 , 35 . Certain studies benefited from using the broader set of capabilities of smartphones; for example, some researchers additionally exploited the proximity sensor and light sensor to allow recognition of a measurement’s context, e.g., the distance between a smartphone and the individual’s body, and changes between in-pocket and out-of-pocket locations based on changes in illumination 36 , 37 . The selection of sensors was also affected by secondary research goals, such as simplicity of classification and minimization of battery drain. In these studies, data acquisition was carried out using a single sensor (e.g., accelerometer 22 ), a small group of sensors (e.g., accelerometer and GPS 38 ), or a purposely modified sampling frequency or sampling scheme (e.g., alternating between data collection and non-collection cycles) to reduce the volume of data collected and processed 39 . Supplementing GPS data with other sensor data was motivated by the limited indoor reception of GPS; satellite signals may be absorbed or attenuated by walls and ceilings 17 up to 60% of the time inside buildings and up to 70% of the time in underground trains 23 .

a A person is sitting by the desk with the smartphone placed in the front pants pocket; b a person is walking normally (~1.9 steps per second) with the smartphone placed in a jacket pocket; c a person is ascending stairs with the smartphone placed in the backpack; d a person is walking slowly (~1.4 steps per second) holding the smartphone in hand; e a person is jogging (~2.8 steps per second) with the smartphone placed in back short’s pocket.

Sampling frequency specifies how many observations are collected by a sensor within a 1-s time interval. The selection of sampling frequency is usually performed as a trade-off between measurement accuracy and battery drain. Sampling frequency in the reviewed studies typically ranged between 20 and 30 Hz for inertial sensors and 1 and 10 Hz for the barometer and GPS. The most significant variations were seen in studies where limited energy consumption was a priority (e.g., accelerometer sampled at 1 Hz 40 ) or if investigators used advanced signal processing methods, such as time-frequency decomposition methods, or activity templates that required higher sampling frequency (e.g., accelerometer sampled at 100 Hz 41 ). Some studies stated that inertial sensors sampled at 20 Hz provided enough information to distinguish between various types of transportation 42 , while 10 Hz sampling rate was sufficient to distinguish between various types of mobility 43 . One study demonstrated that reducing the sampling rate from 100 Hz to 12.5 Hz increased the duration of data collection by a factor of three on a single battery charge 44 .

A crucial parameter in the data acquisition process is the smartphone’s location on the body. This is important mainly because of the nonstationary nature of real-life conditions and the strong effect it has on the smartphone’s inertial sensors. The main challenge in HAR in free-living conditions is that data recorded by the accelerometer, gyroscope, and magnetometer sensors differ between the upper and lower body as the device is not affixed to any specific location or orientation 45 . Therefore, it is essential that studies collect data from as many body locations as possible to ensure the generalizability of results. In the reviewed literature, study participants were often instructed to carry the device in a pants pocket (either front or back), although a number of studies also considered other placements, such as jacket pocket 46 , bag or backpack 47 , 48 , and holding the smartphone in the hand 49 or in a cupholder 50 .

To establish the ground truth for physical activity in HAR studies, data were usually annotated manually by trained research personnel or by the study participants themselves 51 , 52 . However, we also noted several approaches that automated this process both in controlled and free-living conditions, e.g., through a designated smartphone application 22 or built-in step counter combined paired with GPS data 53 ., used a built-in step counter and GPS data to produce “weak” labels. The annotation was also done using the built-in microphone 54 , video camera 18 , 20 , or an additional body-worn sensor 29 .

Finally, the data acquisition process in the reviewed studies was carried out on purposely designed applications that captured data. In studies with online activity classification, the collected data did not leave the device, but instead, the entire HAR pipeline was implemented on the smartphone; in contrast, studies using offline classification transmitted data to an external (remote) server for processing using a cellular, Wi-Fi, Bluetooth, or wired connection.

Data preprocessing

We use the term data preprocessing to refer to a collection of procedures aimed at repairing, cleaning, and transforming measurements recorded for HAR. The need for such step is threefold: (1) measurement systems embedded in smartphones are often less stable than research-grade data acquisition units, and the data might therefore be sampled unevenly or there might be missingness or sudden spikes that are unrelated to an individual’s actual behavior; (2) the spatial orientation (how the phone is situated in a person’s pocket, say) of the device influences tri-axial measurements of inertial sensors, thus potentially degrading the performance of the HAR system; and (3) despite careful planning and execution of the data acquisition stage, data quality may be compromised due to other unpredictable factors, e.g., lack of compliance by the study participants, unequal duration of activities in the measurement (i.e., dataset imbalance), or technological issues.

In our literature review, the first group of obstacles was typically addressed using signal processing techniques (in Fig. 4 , see “standardization”). For instance, to alleviate the mismatch between requested and effective sampling frequency, researchers proposed the use of linear interpolation 55 or spline interpolation 56 (Fig. 8 ). Such procedures were imposed on a range of affected sensors, typically the accelerometer, gyroscope, magnetometer, and barometer. Further time-domain preprocessing considered data trimming, carried out to remove unwanted data components. For this purpose, the beginning and end of each activity bout, a short period of activity of a specified kind, were clipped as nonrepresentative for the given activity 46 . During this stage, the researchers also dealt with dataset imbalance, which occurs when there are different numbers of observations for different activity classes in the training dataset. Such a situation makes the classifier susceptible to overfitting in favor of the larger class; in the reviewed studies, this issue was resolved using up-sampling or down-sampling of data 17 , 57 , 58 , 59 . In addition, the measurements were processed for high-frequency noise cancellation (i.e., “denoising”). The literature review identified several methods suitable for this task, including the use of low-pass finite impulse response filters (with a cutoff frequency typically equal to 10 Hz for inertial sensors and 0.1 Hz for barometers) 60 , 61 , which remove the portion of the signal that is unlikely to result from the activities of interest; weighted moving average 55 ; moving median 45 , 62 ; and singular-value decomposition 63 . GPS data were sometimes de-noised based on the maximum allowed positional accuracy 64 .

Standardization includes relabeling ( a ), when labels are reassigned to better match transitions between activities; trimming ( b ), when part of the signal is removed to balance the dataset for system training; interpolation ( c ), when missing data are filled in based on adjacent observations; and denoising ( d ), when the signal is filtered from redundant components. The transformation includes normalization ( e ), when the signal is normalized to unidimensional vector magnitude; rotation ( f ), when the signal is rotated to a different coordinate system; and separation ( g ), when the signal is separated into linear and gravitational components. Raw accelerometer data are shown in gray, and preprocessed data are shown using different colors.

Another element of data preprocessing considers device orientation (in Fig. 4 , see “transformation”). Smartphone measurements are sensitive to device orientation, which may be due to clothing, body shape, and movement during dynamic activities 57 . One of the popular solutions reported in the literature was to transform the three-dimensional signal into a univariate vector magnitude that is invariant to rotations and more robust to translations. This procedure was often applied to accelerometer, gyroscope, and magnetometer data. Accelerometer data were also subjected to digital filtering by separating the signal into linear (related to body motions) and gravitational (related to device spatial orientation) acceleration 65 . This separation was typically performed using a high-pass Butterworth filter of low order (e.g., order 3) with a cutoff frequency below 1 Hz. Other approaches transformed tri-axial into bi-axial measurement with horizontal and vertical axes 49 , or projected the data from the device coordinate system into a fixed coordinate system (e.g., the coordinate system of a smartphone that lies flat on the ground) using a rotation matrix (Euler angle-based 66 or quaternion 47 , 67 ).

Feature extraction

We use the term feature extraction to refer to a process of selecting and computing meaningful summaries of smartphone data for the goal of activity classification. A typical extraction scheme includes data visualization, data segmentation, feature selection, and feature calculation. A careful feature extraction step allows investigators not only to understand the physical nature of activities and their manifestation in digital measurements, but also, and more importantly, to help uncover hidden structures and patterns in the data. The identified differences are later quantified through various statistical measures to distinguish between activities. In an alternative approach, the process of feature extraction is automated using deep learning, which handles feature selection using simple signal processing units, called neurons, that have been arranged in a network structure that is multiple layers deep 59 , 68 , 69 , 70 . As with many applications of deep learning, the results may not be easily interpretable.

The conventional approach to feature extraction begins with data exploration. For this purpose, researchers in our reviewed studies employed various graphical data exploration techniques like scatter plots, lag plots, autocorrelation plots, histograms, and power spectra 71 . The choice of tools was often dictated by the study objectives and methods. For example, research on inertial sensors typically presented raw three-dimensional data from accelerometers, gyroscopes, and magnetometers plotted for the corresponding activities of standing, walking, and stair climbing 50 , 72 , 73 . Acceleration data were often inspected in the frequency domain, particularly to observe periodic motions of walking, running, and cycling 45 , and the impact of the external environment, like natural vibration frequencies of a bus or a subway 74 . Locomotion and mobility were investigated using estimates of speed derived from GPS. In such settings, investigators calculated the average speed of the device and associated it with either the group of motorized (car, bus, train, etc.) or non-motorized (walking, cycling, etc.) modes of transportation.

In the next step, measurements are divided into smaller fragments (also, segments or epochs) and signal features are calculated for each fragment (Fig. 9 ). In the reviewed studies, this segmentation was typically conducted using a windowing technique that allows consecutive windows to overlap. The window size usually had a fixed length that varied from 1 to 5 s, while the overlap of consecutive windows was often set to 50%. Several studies that investigated the optimal window size supported this common finding: short windows (1–2 s) were sufficient for recognizing posture and mobility, whereas somewhat longer windows (4–5 s) had better classification performance 75 , 76 , 77 . Even longer windows (10 s or more) were recommended for recognizing locomotion modes or for HAR systems employing frequency-domain features calculated with the Fourier transform (resolution of the resulting frequency spectrum is inversely proportional to window length) 42 . In principle, this calibration aims to closely match the window size with the duration of a single instance of the activity (e.g., one step). Similar motivation led researchers to seek more adaptive segmentation methods. One idea was to segment data based on specific time-domain events, like zero-cross points (when the signal changes value from positive to negative or vice versa), peak points (local maxima), or valley points (local minima), which represent the start and endpoints of a particular activity bout 55 , 57 . This allowed for segments to have different lengths corresponding to a single fundamental period of the activity in question. Such an approach was typically used to recognize quasiperiodic activities like walking, running, and stair climbing 63 .

An analyzed measurement ( a ) is segmented into smaller fragments using a sliding window ( b ). Depending on the approach, each segment may then be used to compute time-domain ( c ) or frequency-domain features ( d ), but also it may serve as the activity template ( e ), or as input for deep learning networks that compute hidden (“deep”) features ( f ). The selected feature extraction approach determines the activity classifier: time- and frequency-domain features are paired with machine learning classifiers ( g ) and activity templates are investigated using distance metrics ( h ), while deep features are computed within embedded layers of convolutional neural networks ( i ).

The literature described a large variety of signal features used for HAR, which can be divided into several categories based on the initial signal processing procedure. This enables one to distinguish between activity templates (i.e., raw signal), deep features (i.e., hidden features calculated within layers of deep neural networks), time-domain features (i.e., statistical measures of time-series data), and frequency-domain features (i.e., statistical measures of frequency representation of time-series data). The most popular features in the reviewed papers were calculated from time-domain signals as descriptive statistics, such as local mean, variance, minimum and maximum, interquartile range, signal energy (defined as the area under the squared magnitude of the considered continuous signal), and higher-order statistics. Other time-domain features included mean absolute deviation, mean (or zero) crossing rate, regression coefficients, and autocorrelation. Some studies described novel and customized time-domain features, like histograms of gradients 78 , and the number of local maxima and minima, their amplitude, and the temporal distance between them 39 . Time-domain features were typically calculated over each axis of the three-dimensional measurement or orientation-invariant vector magnitude. Studies that used GPS also calculated average speed 64 , 79 , 80 , while studies that used the barometer analyzed the pressure derivative 81 .

Signals transformed to the frequency domain were less exploited in the literature. A commonly performed signal decomposition used the fast Fourier transform (FFT) 82 , 83 , an algorithm that converts a temporal sequence of samples to a sequence of frequencies present in that sample. The essential advantage of frequency-domain features over time-domain features is their ability to identify and isolate certain periodic components of performed activities. This enabled researchers to estimate (kinetic) energy within particular frequency bands associated with human activities, like gait and running 51 , as well as with different modes of locomotion 74 . Other frequency-domain features included spectral entropy and parameters of the dominant peak, e.g., its frequency and amplitude.

Activity templates function essentially as blueprints for different types of physical activity. In the HAR systems, we reviewed, these templates were compared to patterns of observed raw measurements using various distance metrics 38 , 84 , such as the Euclidean or Manhattan distance. Given the heterogeneous nature of human activities, activity templates were often enhanced using techniques similar to dynamic time warping 29 , 57 , which measures the similarity of two temporal sequences that may vary in speed. As an alternative to raw measurements, some studies used signal symbolic approximation, which translates a segmented time-series signal into sequences of symbols based on a predefined mapping rule (e.g., amplitude between −1 and −0.5 g represents symbol “a”, amplitude between −0.5 and 0 g represents symbol “b”, and so on) 85 , 86 , 87 .

More recent studies utilized deep features. In these approaches, smartphone data were either fed to deep neural networks as raw univariate or multivariate time series 35 , 48 , 60 or preprocessed into handcrafted time- and frequency-domain feature vectors 82 , 83 . Within the network layers, the input data were then transformed (e.g., using convolution) to produce two-dimensional activation maps that revealed hidden spatial relations between axes and sensors specific to a given activity. To improve the resolution of input data, one study proposed to split the integer and decimal values of accelerometer measurements 41 .

In the reviewed articles, the number of extracted features typically varied from a few to a dozen. However, some studies purposely calculated too many features (sometimes hundreds) and let the analytical method perform variable selection, i.e., identify those features that were most informative for HAR 88 . Support vector machines 81 , 89 , gain ratio 43 , recursive feature elimination 38 , correlation-based feature selection 51 , and principal component analysis 90 were among the popular feature selection/dimension reduction methods used.

Activity classification

We use the term activity classification to refer to a process of associating extracted features with particular activity classes based on the adopted classification principle. The classification is typically performed by a supervised learning algorithm that has been trained to recognize patterns between features and labeled physical activities in the training dataset. The fitted model is then validated on separate observations, using a validation dataset, usually data obtained from the same group of study participants. The comparison between predictions made by the model and the known true labels allows one to assess the accuracy of the approach. This section summarizes the methods used in classification and validation, and also provides some insights into reporting on HAR performance.

The choice of classifier aims to identify a method that has the highest classification accuracy for the collected datasets and for the given data processing environment (e.g., online vs. offline). The reviewed literature included a broad range of classifiers, from simple decision trees 18 , k-nearest neighbors 65 , support vector machines 91 , 92 , 93 , logistic regression 21 , naïve Bayes 94 , and fuzzy logic 64 to ensemble classifiers such as random forest 76 , XGBoost 95 , AdaBoost 45 , 96 , bagging 24 , and deep neural networks 48 , 60 , 82 , 97 , 98 , 99 . Simple classifiers were frequently compared to find the best solution in the given measurement scenario 43 , 53 , 100 , 101 , 102 . A similar type of analysis was implemented for ensemble classifiers 79 . Incremental learning techniques were proposed to adapt the classification model to new data streams and unseen activities 103 , 104 , 105 . Other semi-supervised approaches were proposed to utilize unlabeled data to improve the personalization of HAR systems 106 and data annotation 53 , 70 . To increase the effectiveness of HAR, some studies used a hierarchical approach, where the classification was performed in separate stages and each stage could use a different classifier. The multi-stage technique was used for gradual decomposition of activities (coarse-grained first, then fine-grained) 22 , 37 , 52 , 60 and to handle the predicament of changing sensor location (body location first, then activity) 91 . Multi-instance multi-label approaches were adapted for the classification of complex activities (i.e., activities that consist of several basic activities) 62 , 107 as well as for recognition of basic activities paired with different sensor locations 108 .

Classification accuracy could also be improved by using post-processing, which relies on modifying the initially assigned label using the rules of logic and probability. The correction was typically performed based on activity duration 74 , activity sequence 25 , and activity transition probability and classification confidence 80 , 109 .

The selected method is typically cross-validated, which splits the collected dataset into two or more parts—training and testing—and only uses the part of the data for testing that was not used for training. The literature mentions a few cross-validation procedures, with k -fold and leave-one-out cross-validation being the most common 110 . Popular train-test proportions were 90–10, 70–30, and 60–40. A validation is especially valuable if it is performed using studies with different demographics and smartphone use habits. Such an approach allows one to understand the generalizability of the HAR system to real-life conditions and populations. We found a few studies that followed this validation approach 18 , 21 , 71 .

Activity classification is the last stage of HAR. In our review, we found that analysis results were typically reported in terms of classification accuracy using various standard metrics like precision, recall, and F-score. Overall, the investigated studies reported very high classification accuracies, typically above 95%. Several comparisons revealed that ensemble classifiers tended to outperform individual or single classifiers 27 , 77 , and deep-learning classifiers tended to outperform both individual and ensemble classifiers 48 . More nuanced summaries used the confusion matrix, which allows one to examine which activities are more likely to be classified incorrectly. This approach was particularly useful for visualizing classification differences between similar activities, such as normal and fast walking or bus and train riding. Additional statistics were usually provided in the context of HAR systems designed to operate on the device. In this case, activity classification needed to be balanced among acceptable classifier performance, processing time, and battery drain 44 . The desired performance optimum was obtained by making use of dataset remodeling (e.g., by replacing the oldest observations with the newest ones), low-cost classification algorithms, limited preprocessing, and conscientious feature selection 45 , 86 . Computation time was sometimes reported for complex methods, such as deep neural networks 20 , 82 , 111 and extreme learning machine 112 , as well as for symbolic representation 85 , 86 and in comparative analyses 46 . A comprehensive comparison of results was difficult or impossible, as discussed below.

Over the past decade, many studies have investigated HAR using smartphones. The reviewed literature provides detailed descriptions of essential aspects of data acquisition, data preprocessing, feature extraction, and activity classification. Studies were conducted with one or more objectives, e.g., to limit technological imperfections (e.g., no GPS signal reception indoors), to minimize computational requirements (e.g., for online processing of data directly on the device), and to maximize classification accuracy (all studies). Our review summarizes the most frequently used methods and offers available alternatives.

As expected, no single activity recognition procedure was found to work in all settings, which underlines the importance of designing methods and algorithms that address specific research questions in health while keeping the specifics of the study cohort in mind (e.g., age distribution, the extent of device use, and nature of disability). While datasets were usually collected in laboratory settings, there was little evidence that algorithms trained using data collected in these controlled settings could be generalized to free-living conditions 113 , 114 . In free-living settings, duration, frequency, and specific ways of performing any activity are subject to context and individual ability, and these degrees of freedom need to be considered in the development of HAR systems. Validation of these data in free-living settings is essential, as the true value of HAR systems for public health will come through transportable and scalable applications in large, long-term observational studies or real-world interventions.

Some studies were conducted with a small number of able-bodied volunteers. This makes the process of data handling and classification easier but also limits the generalizability of the approach to more diverse populations. The latter point was well demonstrated in two of the investigated studies. In the first study, the authors observed that the performance of a classifier trained on a young cohort significantly decreases if validated on an older cohort 18 . Similar conclusions can be drawn from the second study, where the observations on healthy individuals did not replicate in individuals with Parkinson’s disease 21 . These facts highlight the role of algorithmic fairness (or fairness of machine learning), the notion that the performance of an algorithm should not depend on variables considered sensitive, such as race, ethnicity, sexual orientation, age, and disability. A highly visible example of this was the decision of some large companies, including IBM, to stop providing facial recognition technology to police departments for mass surveillance 115 , and the European Commission has considered a ban on the use of facial recognition in public spaces 116 . These decisions followed findings demonstrating the poor performance of facial recognition algorithms when applied to individuals with dark-skin tones.

The majority of the studies we reviewed utilized stationary smartphones at a single-body position (i.e., a specific pants pocket), sometimes even with a fixed orientation. However, such scenarios are rarely observed in real-life settings, and these types of studies should be considered more as proofs of concept. Indeed, as demonstrated in several studies, inertial sensor data might not share similar features across body locations 49 , 117 , and smartphone orientation introduces additional artifacts to each axis of measurement which make any distribution-based features (e.g., mean, range, skewness) difficult to use without appropriate data preprocessing. Many studies provided only incomplete descriptions of the experimental setup and study protocol and provided few details on demographics, environmental context, and the details of the performed activities. Such information should be reported as fully and accurately as possible.

Only a few studies considered classification in a context that involves activities outside the set of activities the system was trained on; for example, if the system was trained to recognize walking and running, these were the only two activities that the system was later tested on. However, real-life activities are not limited to a prescribed set of behaviors, i.e., we do not just sit still, stand still, walk, and climb stairs. These classifiers, when applied to free-living conditions, will naturally miss the activities they were not trained on but will also likely overestimate those activities they were trained on. An improved scheme could assume that the observed activities are a sample from a broader spectrum of possible behaviors, including periods when the smartphone is not on a person, or assess the uncertainty associated with the classification of each type of activity 84 . This could also provide for an adaptive approach that would enable observation/interventions suited to a broad range of activities relevant for health, including decreasing sedentary behavior, increasing active transport (i.e., walking, bicycling, or public transit), and improving circadian patterns/sleep.

The use of personal digital devices, in particular smartphones, makes it possible to follow large numbers of individuals over long periods of time, but invariably investigators need to consider approaches to missing sensor data, which is a common problem. The importance of this problem is illustrated in a recent paper that introduced a resampling approach to imputing missing smartphone GPS data; the authors found that relative to linear interpolation—the naïve approach to missing spatial data—imputation resulted in a tenfold reduction in the error averaged across all daily mobility features 118 . On the flip side of missing data is the need to propagate uncertainty, in a statistically principled way, from the gaps in the raw data to the inferences that investigators wish to draw from the data. It is a common observation that different people use their phones differently, and some may barely use their phones at all; the net result is not that the data collected from these individuals are not useful, but instead the data are less informative about the behavior of this individual than they ideally might be. Dealing with missing data and accounting for the resulting uncertainty is important because it means that one does not have to exclude participants from a study because their data fail meet some arbitrary threshold of completeness; instead, everyone counts, and every bit of data from each individual counts.

The collection of behavioral data using smartphones understandably raises concerns about privacy; however, investigators in health research are well-positioned to understand and address these concerns given that health data are generally considered personal and private in nature. Consequently, there are established practices and common regulations on human subjects’ research, where informed consent of the individual to participate is one of the key foundations of any ethically conducted study. Federated learning is a machine learning technique that can be used to train an algorithm across decentralized devices, here smartphones, using only local data (data from the individual) and without the need to exchange data with other devices. This approach appears at first to provide a powerful solution to the privacy problem: the personal data never leave the person’s phone and only the outputs of the learning process, generally parameter estimates, are shared with others. This is where the tension between privacy and the need for reproducible research arises, however. The reason for data collection is to produce generalizable knowledge, but according to an often-cited study, 65% of medical studies were inconsistent when retested and only 6% were completely reproducible 12 . In the studies reviewed here, only 4 out of 108 made the source code or the methods used in the study publicly available. For a given scientific question, studies that are not replicable require the collection of more private and personal data; this highlights the importance of reproducibility of studies, especially in health, where there are both financial and ethical considerations when conducting research. If federated learning provides no possibility to confirm data analyses, to re-analyze data using different methods, or to pool data across studies, it by itself cannot be the solution to the privacy problem. Nevertheless, the technique may act as inspiration for developing privacy-preserving methods that also enable future replication of studies. One possibility is to use publicly available datasets (Table 1 ). If sharing of source code were more common, HAR methods could be tested on these publicly available datasets, perhaps in a similar way as datasets of handwritten digits are used to test classification methods in machine learning research. Although some efforts have been made in this area 42 , 119 , 120 , 121 , the recommended course of action assumes collecting and analyzing data from a large spectrum of sensors on diverse and understudied populations and validating classifiers against widely accepted gold standards.

When accurate, reproducible, and transportable methods coalesce to recognize a range of relevant activity patterns, smartphone-based HAR approaches will provide a fundamental tool for public health researchers and practitioners alike. We hope that this paper has provided to the reader some insights into how smartphones may be used to quantify human behavior in health research and the complexities that are involved in the collection and analysis of such data in this challenging but important field.

Data availability

Aggregated data analyzed in this study are available from the corresponding author upon request.

Code availability

Scripts used to process the aggregated data are available from the corresponding author upon request.

Association, G. The mobile economy 2020. https://www.gsma.com/mobileeconomy/wp-content/uploads/2020/03/GSMA_MobileEconomy2020_Global.pdf (2020).

Mercer, K. et al. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: a mixed-methods evaluation. JMIR mHealth uHealth 4 , e7 (2016).

Article   PubMed   PubMed Central   Google Scholar  

Anderson, M. & Perrin, A. Tech adoption climbs among older adults. http://www.pewinternet.org/wp-content/uploads/sites/9/2017/05/PI_2017.05.17_Older-Americans-Tech_FINAL.pdf (2017).

Taylor, K. & Silver, L. Smartphone ownership is growing rapidly around the world, but not always equally. http://www.pewresearch.org/global/wp-content/uploads/sites/2/2019/02/Pew-Research-Center_Global-Technology-Use-2018_2019-02-05.pdf (2019).

Cooper, A. R., Page, A., Fox, K. R. & Misson, J. Physical activity patterns in normal, overweight and obese individuals using minute-by-minute accelerometry. Eur. J. Clin. Nutr. 54 , 887–894 (2000).

Article   CAS   PubMed   Google Scholar  

Ekelund, U., Brage, S., Griffin, S. J. & Wareham, N. J. Objectively measured moderate- and vigorous-intensity physical activity but not sedentary time predicts insulin resistance in high-risk individuals. Diabetes Care 32 , 1081–1086 (2009).

Legge, A., Blanchard, C. & Hanly, J. G. Physical activity, sedentary behaviour and their associations with cardiovascular risk in systemic lupus erythematosus. Rheumatology https://doi.org/10.1093/rheumatology/kez429 (2019).

Loprinzi, P. D., Franz, C. & Hager, K. K. Accelerometer-assessed physical activity and depression among U.S. adults with diabetes. Ment. Health Phys. Act. 6 , 79–82 (2013).

Article   Google Scholar  

Smirnova, E. et al. The predictive performance of objective measures of physical activity derived from accelerometry data for 5-year all-cause mortality in older adults: National Health and Nutritional Examination Survey 2003–2006. J. Gerontol. Ser. A https://doi.org/10.1093/gerona/glz193 (2019).

Wigginton, C. Global Mobile Consumer Trends, 2nd edition. Deloitte, https://www2.deloitte.com/content/dam/Deloitte/us/Documents/technology-media-telecommunications/us-global-mobile-consumer-survey-second-edition.pdf (2017).

Coorevits, L. & Coenen, T. The rise and fall of wearable fitness trackers. Acad. Manag . 2016 , https://doi.org/10.5465/ambpp.2016.17305abstract (2016).

Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10 , 712 (2011).

Kubota, K. J., Chen, J. A. & Little, M. A. Machine learning for large-scale wearable sensor data in Parkinson’s disease: concepts, promises, pitfalls, and futures. Mov. Disord. 31 , 1314–1326 (2016).

Article   PubMed   Google Scholar  

Iniesta, R., Stahl, D. & McGuffin, P. Machine learning, statistical learning and the future of biological research in psychiatry. Psychol. Med. 46 , 2455–2465 (2016).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kuehn, B. M. FDA’s foray into big data still maturing. J. Am. Med. Assoc. 315 , 1934–1936 (2016).

Straczkiewicz, M., Glynn, N. W. & Harezlak, J. On placement, location and orientation of wrist-worn tri-axial accelerometers during free-living measurements. Sensors 19 , 2095 (2019).

Esmaeili Kelishomi, A., Garmabaki, A. H. S., Bahaghighat, M. & Dong, J. Mobile user indoor-outdoor detection through physical daily activities. Sensors 19 , 511 (2019).

Article   PubMed Central   Google Scholar  

Del Rosario, M. B. et al. A comparison of activity classification in younger and older cohorts using a smartphone. Physiol. Meas. 35 , 2269–2286 (2014).

Del Rosario, M. B., Lovell, N. H. & Redmond, S. J. Learning the orientation of a loosely-fixed wearable IMU relative to the body improves the recognition rate of human postures and activities. Sensors 19 , 2845 (2019).

Nan, Y. et al. Deep learning for activity recognition in older people using a pocket-worn smartphone. Sensors 20 , 7195 (2020).

Albert, M. V., Toledo, S., Shapiro, M. & Kording, K. Using mobile phones for activity recognition in Parkinson’s patients. Front. Neurol. 3 , 158 (2012).

Liang, Y., Zhou, X., Yu, Z. & Guo, B. Energy-efficient motion related activity recognition on mobile devices for pervasive healthcare. Mob. Netw. Appl. 19 , 303–317 (2014).

Gjoreski, H. et al. The university of Sussex-Huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access 6 , 42592–42604 (2018).

Wu, W., Dasgupta, S., Ramirez, E. E., Peterson, C. & Norman, G. J. Classification accuracies of physical activities using smartphone motion sensors. J. Med. Internet Res. 14 , e130 (2012).

Guvensan, M. A., Dusun, B., Can, B. & Turkmen, H. I. A novel segment-based approach for improving classification performance of transport mode detection. Sensors 18 , 87 (2018).

Pei, L. et al. Human behavior cognition using smartphone sensors. Sensors 13 , 1402–1424 (2013).

Della Mea, V., Quattrin, O. & Parpinel, M. A feasibility study on smartphone accelerometer-based recognition of household activities and influence of smartphone position. Inform. Heal. Soc. Care 42 , 321–334 (2017).

Klein, I. Smartphone location recognition: a deep learning-based approach. Sensors 20 , 214 (2020).

Casado, F. E. et al. Walking recognition in mobile devices. Sensors 20 , 1189 (2020).

O’Dea, S. Global smartphone market share worldwide by vendor 2009–2020. https://www.statista.com/statistics/271496/global-market-share-held-by-smartphone-vendors-since-4th-quarter-2009/ (2021).

GSMArena. https://www.gsmarena.com/ (2021). Accessed 24 March 2021.

Shoaib, M., Bosch, S., Durmaz Incel, O., Scholten, H. & Havinga, P. J. M. Fusion of smartphone motion sensors for physical activity recognition. Sensors 14 , 10146–10176 (2014).

Vanini, S., Faraci, F., Ferrari, A. & Giordano, S. Using barometric pressure data to recognize vertical displacement activities on smartphones. Comput. Commun. 87 , 37–48 (2016).

Wan, N. & Lin, G. Classifying human activity patterns from smartphone collected GPS data: a fuzzy classification and aggregation approach. Trans. GIS 20 , 869–886 (2016).

Gu, Y., Li, D., Kamiya, Y. & Kamijo, S. Integration of positioning and activity context information for lifelog in urban city area. Navigation 67 , 163–179 (2020).

Miao, F., He, Y., Liu, J., Li, Y. & Ayoola, I. Identifying typical physical activity on smartphone with varying positions and orientations. Biomed. Eng. Online 14 , 32 (2015).

Lee, Y.-S. & Cho, S.-B. Layered hidden Markov models to recognize activity with built-in sensors on Android smartphone. Pattern Anal. Appl. 19 , 1181–1193 (2016).

Martin, B. D., Addona, V., Wolfson, J., Adomavicius, G. & Fan, Y. Methods for real-time prediction of the mode of travel using smartphone-based GPS and accelerometer data. Sensors 17 , 2058 (2017).

Oshin, T. O., Poslad, S. & Zhang, Z. Energy-efficient real-time human mobility state classification using smartphones. IEEE Trans. Comput. 64 , 1680–1693 (2015).

Google Scholar  

Shin, D. et al. Urban sensing: Using smartphones for transportation mode classification. Comput. Environ. Urban Syst. 53 , 76–86 (2015).

Hur, T. et al. Iss2Image: a novel signal-encoding technique for CNN-based human activity recognition. Sensors 18 , 3910 (2018).

Gjoreski, M. et al. Classical and deep learning methods for recognizing human activities and modes of transportation with smartphone sensors. Inf. Fusion 62 , 47–62 (2020).

Wannenburg, J. & Malekian, R. Physical activity recognition from smartphone accelerometer data for user context awareness sensing. IEEE Trans. Syst. Man, Cybern. Syst. 47 , 3143–3149 (2017).

Yurur, O., Labrador, M. & Moreno, W. Adaptive and energy efficient context representation framework in mobile sensing. IEEE Trans. Mob. Comput. 13 , 1681–1693 (2014).

Li, P., Wang, Y., Tian, Y., Zhou, T.-S. & Li, J.-S. An automatic user-adapted physical activity classification method using smartphones. IEEE Trans. Biomed. Eng. 64 , 706–714 (2017).

PubMed   Google Scholar  

Awan, M. A., Guangbin, Z., Kim, C.-G. & Kim, S.-D. Human activity recognition in WSN: a comparative study. Int. J. Networked Distrib. Comput. 2 , 221–230 (2014).

Chen, Z., Zhu, Q., Soh, Y. C. & Zhang, L. Robust human activity recognition using smartphone sensors via CT-PCA and online SVM. IEEE Trans. Ind. Inform. 13 , 3070–3080 (2017).

Zhu, R. et al. Efficient human activity recognition solving the confusing activities via deep ensemble learning. IEEE Access 7 , 75490–75499 (2019).

Yang, R. & Wang, B. PACP: a position-independent activity recognition method using smartphone sensors. Inf 7 , 72 (2016).

Gani, M. O. et al. A light weight smartphone based human activity recognition system with high accuracy. J. Netw. Comput. Appl. 141 , 59–72 (2019).

Reddy, S. et al. Using mobile phones to determine transportation modes. ACM Trans. Sens. Networks 6 , 1–27 (2010).

Guidoux, R. et al. A smartphone-driven methodology for estimating physical activities and energy expenditure in free living conditions. J. Biomed. Inform. 52 , 271–278 (2014).

Cruciani, F. et al. Automatic annotation for human activity recognition in free living using a smartphone. Sensors 18 , 2203 (2018).

Micucci, D., Mobilio, M. & Napoletano, P. UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7 , 1101 (2017).

Derawi, M. & Bours, P. Gait and activity recognition using commercial phones. Comput. Secur. 39 , 137–144 (2013).

Gu, F., Khoshelham, K., Valaee, S., Shang, J. & Zhang, R. Locomotion activity recognition using stacked denoising autoencoders. IEEE Internet Things J. 5 , 2085–2093 (2018).

Chen, Y. & Shen, C. Performance analysis of smartphone-sensor behavior for human activity recognition. IEEE Access 5 , 3095–3110 (2017).

Javed, A. R. et al. Analyzing the effectiveness and contribution of each axis of tri-axial accelerometer sensor for accurate activity recognition. Sensors 20 , 2216 (2020).

Mukherjee, D., Mondal, R., Singh, P. K., Sarkar, R. & Bhattacharjee, D. EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications. Multimed. Tools Appl. 79 , 31663–31690 (2020).

Avilés-Cruz, C., Ferreyra-Ramírez, A., Zúñiga-López, A. & Villegas-Cortéz, J. Coarse-fine convolutional deep-learning strategy for human activity recognition. Sensors 19 , 1556 (2019).

Guiry, J. J., van de Ven, P. & Nelson, J. Multi-sensor fusion for enhanced contextual awareness of everyday activities with ubiquitous devices. Sensors 14 , 5687–5701 (2014).

Saha, J., Chowdhury, C., Ghosh, D. & Bandyopadhyay, S. A detailed human activity transition recognition framework for grossly labeled data from smartphone accelerometer. Multimed. Tools Appl . https://doi.org/10.1007/s11042-020-10046-w (2020).

Ignatov, A. D. & Strijov, V. V. Human activity recognition using quasiperiodic time series collected from a single tri-axial accelerometer. Multimed. Tools Appl. 75 , 7257–7270 (2016).

Das, R. D. & Winter, S. Detecting urban transport modes using a hybrid knowledge driven framework from GPS trajectory. ISPRS Int. J. Geo-Information 5 , 207 (2016).

Arif, M., Bilal, M., Kattan, A. & Ahamed, S. I. Better physical activity classification using smartphone acceleration sensor. J. Med. Syst. 38 , 95 (2014).

Heng, X., Wang, Z. & Wang, J. Human activity recognition based on transformed accelerometer data from a mobile phone. Int. J. Commun. Syst. 29 , 1981–1991 (2016).

Gao, Z., Liu, D., Huang, K. & Huang, Y. Context-aware human activity and smartphone position-mining with motion sensors. Remote Sensing 11 , 2531 (2019).

Kang, J., Kim, J., Lee, S. & Sohn, M. Transition activity recognition using fuzzy logic and overlapped sliding window-based convolutional neural networks. J. Supercomput. 76 , 8003–8020 (2020).

Shojaedini, S. V. & Beirami, M. J. Mobile sensor based human activity recognition: distinguishing of challenging activities by applying long short-term memory deep learning modified by residual network concept. Biomed. Eng. Lett. 10 , 419–430 (2020).

Mairittha, N., Mairittha, T. & Inoue, S. On-device deep personalization for robust activity data collection. Sensors 21 , 41 (2021).

Khan, A. M., Siddiqi, M. H. & Lee, S.-W. Exploratory data analysis of acceleration signals to select light-weight and accurate features for real-time activity recognition on smartphones. Sensors 13 , 13099–13122 (2013).

Ebner, M., Fetzer, T., Bullmann, M., Deinzer, F. & Grzegorzek, M. Recognition of typical locomotion activities based on the sensor data of a smartphone in pocket or hand. Sensors 20 , 6559 (2020).

Voicu, R.-A., Dobre, C., Bajenaru, L. & Ciobanu, R.-I. Human physical activity recognition using smartphone sensors. Sensors 19 , 458 (2019).

Hur, T., Bang, J., Kim, D., Banos, O. & Lee, S. Smartphone location-independent physical activity recognition based on transportation natural vibration analysis. Sensors 17 , 931 (2017).

Bashir, S. A., Doolan, D. C. & Petrovski, A. The effect of window length on accuracy of smartphone-based activity recognition. IAENG Int. J. Comput. Sci. 43 , 126–136 (2016).

Lu, D.-N., Nguyen, D.-N., Nguyen, T.-H. & Nguyen, H.-N. Vehicle mode and driving activity detection based on analyzing sensor data of smartphones. Sensors 18 , 1036 (2018).

Wang, G. et al. Impact of sliding window length in indoor human motion modes and pose pattern recognition based on smartphone sensors. Sensors 18 , 1965 (2018).

Jain, A. & Kanhangad, V. Human activity classification in smartphones using accelerometer and gyroscope sensors. IEEE Sens. J. 18 , 1169–1177 (2018).

Bedogni, L., Di Felice, M. & Bononi, L. Context-aware Android applications through transportation mode detection techniques. Wirel. Commun. Mob. Comput. 16 , 2523–2541 (2016).

Ferreira, P., Zavgorodnii, C. & Veiga, L. edgeTrans—edge transport mode detection. Pervasive Mob. Comput. 69 , 101268 (2020).

Gu, F., Kealy, A., Khoshelham, K. & Shang, J. User-independent motion state recognition using smartphone sensors. Sensors 15 , 30636–30652 (2015).

Li, X., Wang, Y., Zhang, B. & Ma, J. PSDRNN: an efficient and effective HAR scheme based on feature extraction and deep learning. IEEE Trans. Ind. Inform. 16 , 6703–6713 (2020).

Zhao, B., Li, S., Gao, Y., Li, C. & Li, W. A framework of combining short-term spatial/frequency feature extraction and long-term IndRNN for activity recognition. Sensors 20 , 6984 (2020).

Huang, E. J. & Onnela, J.-P. Augmented movelet method for activity classification using smartphone gyroscope and accelerometer data. Sensors 20 , 3706 (2020).

Montero Quispe, K. G., Sousa Lima, W., Macêdo Batista, D. & Souto, E. MBOSS: a symbolic representation of human activity recognition using mobile sensors. Sensors 18 , 4354 (2018).

Sousa Lima, W., de Souza Bragança, H. L., Montero Quispe, K. G. & Pereira Souto, E. J. Human activity recognition based on symbolic representation algorithms for inertial sensors. Sensors 18 , 4045 (2018).

Bragança, H., Colonna, J. G., Lima, W. S. & Souto, E. A smartphone lightweight method for human activity recognition based on information theory. Sensors 20 , 1856 (2020).

Saeedi, S. & El-Sheimy, N. Activity recognition using fusion of low-cost sensors on a smartphone for mobile navigation application. Micromachines 6 , 1100–1134 (2015).

Bilal, M., Shaikh, F. K., Arif, M. & Wyne, M. F. A revised framework of machine learning application for optimal activity recognition. Clust. Comput. 22 , 7257–7273 (2019).

Shi, D., Wang, R., Wu, Y., Mo, X. & Wei, J. A novel orientation- and location-independent activity recognition method. Pers. Ubiquitous Comput. 21 , 427–441 (2017).

Antos, S. A., Albert, M. V. & Kording, K. P. Hand, belt, pocket or bag: practical activity tracking with mobile phones. J. Neurosci. Methods 231 , 22–30 (2014).

Shi, J., Zuo, D. & Zhang, Z. Transition activity recognition system based on standard deviation trend analysis. Sensors 20 , 3117 (2020).

Garcia-Gonzalez, D., Rivero, D., Fernandez-Blanco, E. & Luaces, M. R. A public domain dataset for real-life human activity recognition using smartphone sensors. Sensors 20 , 2200 (2020).

Saeedi, S., Moussa, A. & El-Sheimy, N. Context-aware personal navigation using embedded sensor fusion in smartphones. Sensors 14 , 5742–5767 (2014).

Zhang, W., Zhao, X. & Li, Z. A comprehensive study of smartphone-based indoor activity recognition via Xgboost. IEEE Access 7 , 80027–80042 (2019).

Ferrari, A., Micucci, D., Mobilio, M. & Napoletano, P. On the personalization of classification models for human activity recognition. IEEE Access 8 , 32066–32079 (2020).

Zhou, B., Yang, J. & Li, Q. Smartphone-based activity recognition for indoor localization using a convolutional neural network. Sensors 19 , 621 (2019).

Pires, I. M. et al. Pattern recognition techniques for the identification of activities of daily living using a mobile device accelerometer. Electronics 9 , 509 https://www.mdpi.com/2079-9292/9/3/509#cite (2020).

Alo, U. R., Nweke, H. F., Teh, Y. W. & Murtaza, G. Smartphone motion sensor-based complex human activity identification using deep stacked autoencoder algorithm for enhanced smart healthcare system. Sensors 20 , 6300 https://www.mdpi.com/2079-9292/9/3/509#cite (2020).

Otebolaku, A. M. & Andrade, M. T. User context recognition using smartphone sensors and classification models. J. Netw. Comput. Appl. 66 , 33–51 (2016).

Zhuo, S. et al. Real-time smartphone activity classification using inertial sensors—recognition of scrolling, typing, and watching videos while sitting or walking. Sensors 20 , 655 (2020).

Asim, Y., Azam, M. A., Ehatisham-ul-Haq, M., Naeem, U. & Khalid, A. Context-aware human activity recognition (CAHAR) in-the-wild using smartphone accelerometer. IEEE Sens. J. 20 , 4361–4371 (2020).

Zhao, Z., Chen, Z., Chen, Y., Wang, S. & Wang, H. A class incremental extreme learning machine for activity recognition. Cogn. Comput. 6 , 423–431 (2014).

Abdallah, Z. S., Gaber, M. M., Srinivasan, B. & Krishnaswamy, S. Adaptive mobile activity recognition system with evolving data streams. Neurocomputing 150 , 304–317 (2015).

Guo, H., Chen, L., Chen, G. & Lv, M. Smartphone-based activity recognition independent of device orientation and placement. Int. J. Commun. Syst. 29 , 2403–2415 (2016).

Cruciani, F. et al. Personalizing activity recognition with a clustering based semi-population approach. IEEE ACCESS 8 , 207794–207804 (2020).

Saha, J., Ghosh, D., Chowdhury, C. & Bandyopadhyay, S. Smart handheld based human activity recognition using multiple instance multiple label learning. Wirel. Pers. Commun . https://doi.org/10.1007/s11277-020-07903-0 (2020).

Mohamed, R., Zainudin, M. N. S., Sulaiman, M. N., Perumal, T. & Mustapha, N. Multi-label classification for physical activity recognition from various accelerometer sensor positions. J. Inf. Commun. Technol. 17 , 209–231 (2018).

Wang, C., Xu, Y., Liang, H., Huang, W. & Zhang, L. WOODY: a post-process method for smartphone-based activity recognition. IEEE Access 6 , 49611–49625 (2018).

Garcia-Ceja, E. & Brena, R. F. An improved three-stage classifier for activity recognition. Int. J. Pattern Recognit. Artif. Intell . 32 , 1860003 (2018).

Ravi, D., Wong, C., Lo, B. & Yang, G.-Z. A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE J. Biomed. Heal. Inform. 21 , 56–64 (2017).

Chen, Z., Jiang, C. & Xie, L. A novel ensemble ELM for human activity recognition using smartphone sensors. IEEE Trans. Ind. Inform. 15 , 2691–2699 (2019).

van Hees, V. T., Golubic, R., Ekelund, U. & Brage, S. Impact of study design on development and evaluation of an activity-type classifier. J. Appl. Physiol. 114 , 1042–1051 (2013).

Sasaki, J. et al. Performance of activity classification algorithms in free-living older adults. Med. Sci. Sports Exerc. 48 , 941–950 (2016).

Allyn, B. IBM abandons facial recognition products, condemns racially biased surveillance. https://www.npr.org/2020/06/09/873298837/ibm-abandons-facial-recognition-products-condemns-racially-biased-surveillance (2020).

Chee, F. Y. EU mulls five-year ban on facial recognition tech in public areas. https://www.reuters.com/article/uk-eu-ai/eu-mulls-five-year-ban-on-facial-recognition-tech-in-public-areas-idINKBN1ZF2QN (2020).

Saha, J., Chowdhury, C., Chowdhury, I. R., Biswas, S. & Aslam, N. An ensemble of condition based classifiers for device independent detailed human activity recognition using smartphones. Information 9 , 94 (2018).

Barnett, I. & Onnela, J.-P. Inferring mobility measures from GPS traces with missing data. Biostatistics 21 , e98–e112 (2020).

Wang, L. et al. Enabling reproducible research in sensor-based transportation mode recognition with the Sussex-Huawei dataset. IEEE ACCESS 7 , 10870–10891 (2019).

Lee, M. H., Kim, J., Jee, S. H. & Yoo, S. K. Integrated solution for physical activity monitoring based on mobile phone and PC. Healthc. Inform. Res. 17 , 76–86 (2011).

Fahim, M., Fatima, I., Lee, S. & Park, Y.-T. EFM: evolutionary fuzzy model for dynamic activities recognition using a smartphone accelerometer. Appl. Intell. 39 , 475–488 (2013).

Yurur, O., Liu, C. H. & Moreno, W. Light-weight online unsupervised posture detection by smartphone accelerometer. IEEE Internet Things J. 2 , 329–339 (2015).

Awan, M. A., Guangbin, Z., Kim, H.-C. & Kim, S.-D. Subject-independent human activity recognition using Smartphone accelerometer with cloud support. Int. J. Ad Hoc Ubiquitous Comput. 20 , 172–185 (2015).

Chen, Z., Wu, J., Castiglione, A. & Wu, W. Human continuous activity recognition based on energy-efficient schemes considering cloud security technology. Secur. Commun. Netw. 9 , 3585–3601 (2016).

Guo, J. et al. Smartphone-based patients’ activity recognition by using a self-learning scheme for medical monitoring. J. Med. Syst. 40 , 140 (2016).

Walse, K. H., Dharaskar, R. V. & Thakare, V. M. A study of human activity recognition using AdaBoost classifiers on WISDM dataset. IIOAB J. 7 , 68–76 (2016).

Lee, K. & Kwan, M.-P. Physical activity classification in free-living conditions using smartphone accelerometer data and exploration of predicted results. Comput. Environ. Urban Syst. 67 , 124–131 (2018).

Ahmad, N. et al. SARM: salah activities recognition model based on smartphone. Electronics 8 , 881 (2019).

Usman Sarwar, M. et al. Recognizing physical activities having complex interclass variations using semantic data of smartphone. Softw. Pract. Exp . 51 , 532–549 (2020).

Kwapisz, J. R., Weiss, G. M. & Moore, S. A. Activity recognition using cell phone accelerometers. in Proceedings of the Fourth International Workshop on Knowledge Discovery from Sensor Data 10–18 https://doi.org/10.1145/1964897.1964918 (Association for Computing Machinery, 2010).

Sharma, A., Singh, S. K., Udmale, S. S., Singh, A. K. & Singh, R. Early transportation mode detection using smartphone sensing data. IEEE Sens. J . 1 , https://doi.org/10.1109/JSEN.2020.3009312 (2020).

Chen, Z. et al. Smartphone sensor-based human activity recognition using feature fusion and maximum full a posteriori. IEEE Trans. Instrum. Meas. 69 , 3992–4001 (2020).

Vavoulas, G. Chatzaki, C. Malliotakis, T. Pediaditis, M. & Tsiknakis, M. The MobiAct Dataset: recognition of activities of daily living using smartphones. In Proceedings of the International Conference on Information and Communication Technologies for Ageing Well and e-Health (eds. Röcker, C., Ziefle, M., O’Donoghue, J, Maciaszek, L. & Molloy W.) Vol. 1: ICT4AWE, (ICT4AGEINGWELL 2016) 143–151, https://www.scitepress.org/ProceedingsDetails.aspx?ID=VhZYzluZTNE=&t=1 (SciTePress, 2016).

Shoaib, M., Bosch, S., Incel, O. D., Scholten, H. & Havinga, P. J. M. Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors 16 , 426 (2016).

Lockhart, J. W. et al. Design considerations for the WISDM smart phone-based sensor mining architecture. in Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data. 25–33 https://doi.org/10.1145/2003653.2003656 (Association for Computing Machinery, 2011).

Vaizman, Y., Ellis, K. & Lanckriet, G. Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Comput. 16 , 62–74 (2017).

Sztyler, T. & Stuckenschmidt, H. On-body localization of wearable devices: an investigation of position-aware activity recognition. in 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) 1–9 https://ieeexplore.ieee.org/document/7456521 (IEEE, 2016).

Malekzadeh, M., Clegg, R. G., Cavallaro, A. & Haddadi, H. Mobile sensor data anonymization. in Proceedings of the International Conference on Internet of Things Design and Implementation. 49–58 https://doi.org/10.1145/3302505.3310068 (ACM, 2019).

Carpineti, C., Lomonaco, V., Bedogni, L., Felice, M. D. & Bononi, L. Custom dual transportation mode detection by smartphone devices exploiting sensor diversity. in 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) 367–372 https://ieeexplore.ieee.org/document/8480119 (IEEE, 2018).

Ichino, H., Kaji, K., Sakurada, K., Hiroi, K. & Kawaguchi, N. HASC-PAC2016: large scale human pedestrian activity corpus and its baseline recognition. in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 705–714 https://doi.org/10.1145/2968219.2968277 (Association for Computing Machinery, 2016).

Download references

Acknowledgements

Drs. Straczkiewicz and Onnela are supported by NHLBI award U01HL145386 and NIMH award R37MH119194. Dr. Onnela is also supported by the NIMH award U01MH116928. Dr. James is supported by NCI award R00CA201542 and NHLBI award R01HL150119.

Author information

Authors and affiliations.

Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, 02115, USA

Marcin Straczkiewicz & Jukka-Pekka Onnela

Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, 02215, USA

Peter James

Department of Environmental Health, Harvard T.H. Chan School of Public Health, Boston, MA, 02115, USA

You can also search for this author in PubMed   Google Scholar

Contributions

M.S. conducted the review, prepared figures, and wrote the initial draft. P.J. and J.P.O. revised the manuscript. J.P.O. supervised the project. All authors reviewed the manuscript.

Corresponding author

Correspondence to Marcin Straczkiewicz .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Straczkiewicz, M., James, P. & Onnela, JP. A systematic review of smartphone-based human activity recognition methods for health research. npj Digit. Med. 4 , 148 (2021). https://doi.org/10.1038/s41746-021-00514-4

Download citation

Received : 24 March 2021

Accepted : 13 September 2021

Published : 18 October 2021

DOI : https://doi.org/10.1038/s41746-021-00514-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Unlocking the potential of smartphone and ambient sensors for adl detection.

• Marija Stojchevska
• Mathias De Brouwer
• Femke Ongenae

Scientific Reports (2024)

Activity recognition based on smartphone sensor data using shallow and deep learning techniques: A Comparative Study

• Asif Iqbal Middya
• Sarvajit Kumar
• Sarbani Roy

Multimedia Tools and Applications (2024)

• Marcin Straczkiewicz
• Emily J. Huang
• Jukka-Pekka Onnela

npj Digital Medicine (2023)

Data filtering and deep learning for enhanced human activity recognition from UWB radars

• Julien Maitre
• Kévin Bouchard
• Sébastien Gaboury

Journal of Ambient Intelligence and Humanized Computing (2023)

New incremental SVM algorithms for human activity recognition in smart homes

• Mourad Oussalah
• Anthony Fleury

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Disclaimer » Advertising

• HealthyChildren.org

• Previous Article
• Next Article

Overall Study Design

Participants, survey measures, mobile device sampling methods, data analysis, conclusions, young children’s use of smartphones and tablets.

POTENTIAL CONFLICTS OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose

FINANCIAL DISCLOSURE: Dr Radesky is a consultant for and is on the Board of Directors of Melissa & Doug Toys and receives research support from Common Sense Media; the other authors have indicated they have no financial relationships relevant to this article to disclose.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• CME Quiz Close Quiz
• Open the PDF for in another window
• Get Permissions
• Cite Icon Cite
• Search Site

Jenny S. Radesky , Heidi M. Weeks , Rosa Ball , Alexandria Schaller , Samantha Yeo , Joke Durnez , Matthew Tamayo-Rios , Mollie Epstein , Heather Kirkorian , Sarah Coyne , Rachel Barr; Young Children’s Use of Smartphones and Tablets. Pediatrics July 2020; 146 (1): e20193518. 10.1542/peds.2019-3518

Download citation file:

• Ris (Zotero)
• Reference Manager

Video Abstract

Child mobile device use is increasingly prevalent, but research is limited by parent-report survey methods that may not capture the complex ways devices are used. We aimed to implement mobile device sampling, a set of novel methods for objectively measuring child mobile device use.

We recruited 346 English-speaking parents and guardians of children aged 3 to 5 years to take part in a prospective cohort study of child media use. All interactions with participants were through e-mail, online surveys, and mobile device sampling; we used a passive-sensing application (Chronicle) in Android devices and screenshots of the battery feature in iOS devices. Baseline data were analyzed to describe usage behaviors and compare sampling output with parent-reported duration of use.

The sample comprised 126 Android users (35 tablets, 91 smartphones) and 220 iOS users (143 tablets, 77 smartphones); 35.0% of children had their own device. The most commonly used applications were YouTube, YouTube Kids, Internet browser, quick search or Siri, and streaming video services. Average daily usage among the 121 children with their own device was 115.3 minutes/day (SD 115.1; range 0.20–632.5) and was similar between Android and iOS devices. Compared with mobile device sampling output, most parents underestimated (35.7%) or overestimated (34.8%) their child’s use.

Mobile device sampling is an unobtrusive and accurate method for assessing mobile device use. Parent-reported duration of mobile device use in young children has low accuracy, and use of objective measures is needed in future research.

Previous studies of young children’s mobile device use rely on parent recall or time-use diaries, which may be inaccurate or carry high participant burden. No previous studies in children have harnessed application usage data already collected by mobile devices.

Mobile device sampling (passive sensing for Android and screenshots from iOS devices) is an acceptable and feasible objective method for assessing mobile device use. Parent-reported duration of their child’s mobile device use had low accuracy compared with objective output.

Children’s use of mobile and interactive media has increased rapidly over the past decade. 1   Recent estimates reveal that the majority of parents own smartphones, 2   on which they allow their children to play games or watch videos. Up to 75% of young children have their own tablets, 3   and infants are estimated to start handling mobile devices during the first year of life, 1   but research on modern media has been limited by a lack of precise measurement tools.

Research on traditional screen media, such as television, historically used parent recall of child media use duration to test associations with outcomes such as sleep problems, obesity, and externalizing behavior. 4   Similarly, studies of the benefits of educational television programming relied on parent recall and content analysis of linear, noninteractive programs. 5 , 6   As the proportion of time that children spend on mobile platforms increases, 1   media researchers are posed with a challenge of measuring on-demand, portable, and intermittent mobile device usage. 7 , 8   Participant recall accuracy of mobile device use may be low because exposure occurs in small bursts 8   (less likely to be remembered than longer interactions 9   ), and parents may find it difficult to monitor content when children use handheld devices individually. 10  

Mobile devices collect usage data that could feasibly be harnessed for the purposes of research studies. Analysis of various data streams (eg, accelerometer, Bluetooth, location) has been used in public health research to predict patterns of human behavior 11   but collects more data than is necessary for the purposes of media use measurement. In a few studies, researchers have used commercially available or prototype applications (apps) (ie, created by researchers) to test hypotheses in adults regarding mental health and smartphone use 12   or motivations for using different apps, 13   but no previous research has been conducted by using similar measures on the devices of children. Harnessing mobile data from children’s devices may provide more accurate data collection with lower participant and researcher burden.

Our objective for the current study was to implement novel mobile device sampling methods in a community-based sample of preschool-aged children to describe their mobile device usage and compare parent report of child use with mobile device sampling output. We describe the development of this method, important considerations during implementation, and types of variables that can be generated for research. On the basis of pilot research revealing that most of parents’ recall of their own mobile device use is inaccurate, 14   we hypothesized that most parents would be inaccurate in reporting their child’s mobile device use.

The Preschooler Tablet Study is a longitudinal cohort study ( Eunice Kennedy Shriver National Institute of Child Health and Human Development grant R21HD094051) in which associations between early childhood digital media use, emotion regulation, and executive functioning are examined. Data were collected through online surveys and e-mail communication with participants, mobile device sampling, and an online time-use diary completed by parents at baseline and at the 3- and 6-month follow-up. Data from the baseline data collection wave (August 2018 to May 2019) are included in the present article. The study was approved by the University of Michigan Institutional Review Board.

Parents of young children were recruited via flyers posted in community centers, preschools, child care centers, and pediatric clinics in southeast Michigan as well as our university’s online participant registry and social media advertisements. Interested parents who contacted the study team were e-mailed a link to an eligibility questionnaire. Eligibility criteria included the following: (1) the parent was the legal guardian of a 3- to 4.99-year-old child, (2) the parent lived with the child at least 5 days/week, (3) the parent understood English sufficiently to complete questionnaires and provide consent, and (4) the family owned at least one Android or iOS tablet or smartphone. Children did not need to regularly use mobile devices to be included in the study. Exclusion criteria included presence of child developmental delays, use of psychotropic medication, and the child’s mobile device being a Kindle or Amazon Fire ( n = 43 interested but excluded), which do not use the standard Android operating system.

Because all interactions with the research team were electronic, we anticipated a high rate of attrition. Of 487 parents who consented to take part in the study, 64 (13%) submitted no study data after providing informed consent and receiving electronic reminders.

After providing online consent for themselves and their child, parents were e-mailed study instructions and a link to online Research Electronic Data Capture 15 , 16   surveys, in which parents reported their child’s age, sex, race and/or ethnicity, preschool or child care enrollment, and prematurity; their own age, sex, educational attainment, marital status, and employment status; and household income and size (from which we calculated the income-to-needs ratio).

Parents then completed an abbreviated version (36 items) of the Media Assessment Qualtrics Survey, which is used to assess child, parent, and household media use practices. In this survey, parents were asked, “Thinking about a typical [weekday or weekend], how much time does your child spend using 1) an iPad, tablet, LeapPad, iTouch, or similar mobile device (not including a smartphone) and 2) a smartphone for things like texting, playing games, watching videos, or surfing the Internet (don’t count time spent talking on the phone)?” Responses were never, <30 minutes, 30 minutes to 1 hour, 1 to 2 hours, 2 to 3 hours, 3 to 4 hours, 4 to 5 hours, and >5 hours. Because mobile sampling included both weekdays and weekends, we created a weighted categorical variable that reflected parent estimates of their child’s usual smartphone or tablet use throughout the week.

During eligibility screening, parents indicated what type(s) of mobile device(s), if any, the child regularly had access to or used. If the child used >1 device, we sampled the device used more frequently and asked the parent to avoid letting the child play on other devices that week. We provided video and visual instructions specific for tracking the device on a study Web site (see Mobile Device Sampling Methods: Installation and Data Collection in the Supplemental Information ).

Android Devices

Android users were instructed to download a study app, Chronicle, from the Google Play store ( Supplemental Figs 1 and 2 ). The Chronicle app was developed by OpenLattice, Inc, in collaboration with the Comprehensive Assessment of Family Media Exposure Consortium. It queries the Google UsageStatsManager application programming interface (API), which provides data about app usage on all Android devices running version 5.0 or later and transmits data automatically to the OpenLattice platform. Chronicle was pilot tested on a range of Android devices in June 2018 to July 2018, which allowed app debugging to ensure accuracy by comparing handwritten usage logs with raw output.

In the informed consent document, parents were informed that Chronicle only collects app name, timestamp, and a masked device identification but does not collect personal information (eg, contacts, content of messages, Web sites viewed) and that data are stored in a secure server and not shared with third-party companies. After installing Chronicle, parents were e-mailed a unique link routing their app data to the research team on the OpenLattice platform. The app user interface is simple, only providing a timestamp of the last data upload (see Android Mobile Devices in the Supplemental Information ), and runs in the background with no need for user interaction. Data are continually collected locally on the device and uploaded every 15 minutes when connected to WiFi. After 9 days, participants were instructed to uninstall Chronicle after confirming that data had been uploaded that day (ie, in case the devices had been recently disconnected from WiFi). The study team then exported the Chronicle data file through the Chronicle Web application in comma-separated values (CSV) format and conducted data cleaning and processing steps as described in the Chronicle Data Cleaning Methods section of the Supplemental Information .

iOS Devices

For children who used an iPhone or iPad, we asked parents to take a screenshot of the device’s battery page (under “Settings”) 7 days after completing the surveys. Instructions for taking screenshots, including the specific buttons that need to be tapped to visualize app usage over the past 7 to 10 days, were provided via the study Web site (see Apple Mobile Devices in the Supplemental Information ; see also Supplemental Fig 3 ).

When parents sent screenshots that did not follow study instructions, the study team responded by e-mail the same day, offering clarification on screenshot methods and requesting that new ones be sent. However, if screenshots were still incorrectly taken at this point, they were flagged for potential errors and manually inspected before inclusion in final data sets. Research assistants manually entered all screenshot data (app name, number of minutes) into spreadsheets.

Shared Devices

At the end of the sampling period, parents were asked whether the device had been shared with any other family members that week. If the parents responded yes (70.6% of Android users; 61.8% of iOS users), they completed a data form listing the names of the apps their child used that week. We created a subset of data files to include only the apps that children used during the sampling period.

App Category Coding

We developed a coding scheme to categorize apps on the basis of app store labels (eg, educational, age category), video chat, YouTube, streaming video, and other common categories such as eBooks or music (see Supplemental Table 4 for coding scheme; interrater reliability = 0.72–0.94).

First, for all children with complete mobile device data ( n = 346), we analyzed differences in sociodemographic characteristics by operating system and shared or unshared status. We calculated frequencies of the most commonly played apps and the number of different apps played by each child during the sampling week.

For children with their own, unshared mobile Android or iOS device ( n = 121), we created summary variables representing each child’s average daily duration of device use, average daily duration of app categories, and average daily duration of specific apps played during the sampling period. We chose not to calculate daily duration from shared mobile devices because of the risk of overestimating duration of apps such as YouTube, Safari, or Netflix, which are commonly used by both children and parents.

For children with unshared Android devices ( n = 37), whose output provides date and timestamps, we additionally calculated average usage by day of the week, proportion of days the child used the device, and average number of daily pickups. For illustrative purposes, we plotted the average hourly app category usage of 6 child participants (4 with heavy use, 2 with lighter use) to demonstrate diurnal visualizations of mobile device usage.

Finally, we calculated accuracy of parent-reported mobile device use by determining if each child’s average daily usage (based on mobile sampling output) fell within the weighted parent-reported time category. If parent report was inaccurate, we calculated the difference between actual daily usage and the upper or lower bounds of the parent-reported category.

All processing of raw timestamped data into user logs was performed in Python, 17   all mobile device sampling analyses were conducted by using data.table in R 3.5.2, 18 , 19   and analyses of demographics and comparison of parent report with sampling output were conducted by using SAS version 9.4 (SAS Institute, Inc, Cary, NC). 20  

Of the 423 parents who provided any data, 58 (13%) were excluded because of incomplete mobile device data. Reasons for missing mobile device data included the following: could not ( n = 7) or decided not to ( n = 2) install Chronicle, <2 days of data appeared on server (usually because of server maintenance; n = 13), failed to send iOS screenshots ( n = 20), screenshots were incorrect ( n = 4) or blank ( n = 4), and the app list for shared devices was not submitted ( n = 8). Participants with missing mobile device data had no significant sociodemographic differences compared with included participants. In addition, 19 children were reported to have never used mobile devices at baseline, so mobile device sampling was not performed; these children were more likely to have parents with higher educational attainment (χ 2 test; P = .02).

Characteristics of the full sample ( N = 346) and the unshared device subsample ( n = 121) are shown in Table 1 . Participants comprised 126 Android users (35 tablets, 91 smartphones) and 220 iOS users (143 tablets, 77 smartphones). Children with iOS devices were more likely to have higher-income families (2-sample Wilcoxon rank test; P < .0001), married parents (χ 2 test; P = .03), and parents with higher educational attainment (χ 2 test; P < .0001).

Participant Demographic Characteristics and Mobile Device Usage

GED, general equivalency diploma; ITN, income-to-needs ratio; —, not applicable.

ITN of 1 = 100% of the federal poverty level for the family’s size; ITN of 2 = 200% of the federal poverty level, etc.

In the full sample, children used between 1 and 85 different apps over the course of the sampling week; the 20 most commonly played apps are listed in Table 2 .

Most Commonly Played Apps Among 346 Preschool-Aged Children Who Underwent Mobile Device Sampling for 1 Week

PBS, Public Broadcasting Service.

Average daily usage among the 121 children with their own tablet ( n = 100) or smartphone ( n = 21) was 115.3 minutes (SD 115.1; range 0.20–632.5) and was similar between Android (117.7; SD 143.2) and iOS (114.2; SD 101.3) users. More than half (59.5%) of children used their device for an average of ≥1 hour/day, including 18 (14.9%) who averaged ≥4 hours/day ( Table 1 ).

Average daily use of the most commonly played apps by children with unshared devices is shown in Table 3 ; YouTube, YouTube Kids, and streaming video services revealed the highest daily duration, whereas the browser and Quick Search Box or Siri were accessed by a large number of children but used for briefer periods of time.

Average Daily Duration of Most Commonly Played Apps Among 121 Preschool-Aged Children With Their Own Mobile Devices

n/a, not applicable.

Includes the Samsung video app and iOS video app.

Among Android users, average pickup frequency was 3.82 per day (SD 5.48), children used devices on most (69.0%) days of sampling (SD 27.1%; range 25%–100%), and duration was longest on Fridays and Saturdays ( Supplemental Fig 4 ). Example data visualizations of average usage of different app categories (eg, educational apps, streaming video) and diurnal patterns for specific participants are available in Supplemental Figs 5 and 6 , respectively.

Of 115 participants with unshared devices and complete parent-report data, 41 (35.7%) parents underestimated, 34 (29.6%) were accurate, and 40 (34.8%) overestimated their child’s device use. Accuracy did not vary by operating system (Android 25.7% versus iOS 31.3%; P = .49). For inaccurate reporters, actual usage was on average 69.7 minutes (SD 67.5) above or below the parent-reported category bounds (median 50.7; range 0.86–332.5 minutes). Parents were more likely to overreport when their child’s average usage was <1 hour/day and underreport if their child’s average usage was ≥1 hour/day (χ 2 test; P = .001).

This is the first study to use an objective form of mobile device–based data collection (a method we term “mobile device sampling”) to examine young children’s tablet and smartphone usage. We found high variability in daily mobile device usage in children with their own smartphones or tablets, with ∼15% of children averaging ≥4 hours per day. The most commonly used apps were YouTube and YouTube Kids, followed by browsers, the camera and photograph gallery, and video streaming services such as Netflix.

Compared with our previous pilot research in which we used passive sensing in parents, 14   we had significantly lower rates of missing data when using the Chronicle app for Android and screenshot-based data collection for iOS. However, we had an ∼10% missing data rate for Chronicle, which we are addressing by (1) screening participants to ensure Chronicle compatibility before enrollment, (2) developing new features on the OpenLattice platform to increase stability and reliability of data uploads, and (3) providing in-person installation or phone troubleshooting.

Strengths of this approach include highly reliable data because the Google usage statistics API is maintained by Google and used by thousands of vendors. Participating parents found the mobile sampling methods highly acceptable and were informed of how their child’s data would be collected, handled, and destroyed.

A main limitation of our current app is that it cannot identify the user of shared devices, which is important in early childhood when many children do not have their own devices. However, our subset approach allowed us to generate a list of apps used by children who share mobile devices with family members that can be coded for educational value, 21   presence of advertising, 22   or age-appropriate content. For example, we documented that preschool-aged children use YouTube (36.7% of our sample), general audience apps such as Cookie Jam and Candy Crush (30.6% of our sample), gambling apps such as Cashman, and violent apps such as Terrorist Shooter, Flip the Gun, and Granny, which are intended for use by teenagers and adults. These findings also have implications for child privacy because general audience apps and platforms may not place restrictions on the data they collect or distribute to third-party advertising companies. 23  

We found low accuracy of parent-reported mobile device duration compared with mobile sampling output, which is consistent with our previous research in parents. 24   Inaccurate parents showed an average error of >60 minutes compared with their child’s actual daily device use. We therefore suggest that mobile device sampling may be an important future data collection tool for pediatric, adolescent, or adult research. For example, by using Chronicle, it is possible to define variables such as the number of checks of specific apps (eg, social media) per hour, usage during time periods when family meals or routines might occur, or overnight usage. At present, timestamped data are not available for iOS, and data transfer from screenshots is labor intensive; development of similar iOS tracking tools will therefore be necessary to fully assess children’s media landscapes. Mobile sampling will need to be used in combination with methods that capture media use on other platforms (eg, television, video game consoles) and other sensors that detect whether the user is awake (eg, Fitbit) or interacting with others (eg, LENA).

Limitations of our overall study design are worthy of mention. Use of online recruitment allowed for rapid enrollment of multiple families simultaneously because we did not have to schedule study visits, but it also led to higher rates of attrition immediately after enrollment. Our sample was more highly educated and had lower racial and/or ethnic diversity than the general population; future research in non–English-speaking populations is needed once our app interface is updated for other languages. Parents were aware of their child’s mobile device usage being tracked, which may have changed their usage behavior. Children may have used other mobile devices during the sampling period, so our results represent a minimum estimate of their true usage. Our app categorization approach was also limited by the fact that apps commonly disappear from app stores and may no longer appear when searched for several months later.

We describe development of a novel mobile device sampling method in which implementation allowed for description of the smartphone and tablet use behaviors of preschool-aged children. Given the limitations of parent report, such objective measurement tools must be developed and refined so that health research (and evidence-based guidelines) can reflect the complex ways modern media are used.

Dr Radesky conceptualized the data collection methods, designed the cohort study, supervised data collection, drafted the initial manuscript, and reviewed and revised the manuscript; Dr Weeks performed the data analysis and critically reviewed the manuscript for important intellectual content; Ms Ball, Ms Schaller, and Ms Yeo coordinated and conducted data collection, performed application coding, and reviewed and revised the manuscript; Dr Durnez, Mr Tomayo-Rios, and Ms Epstein developed and piloted the passive-sensing data collection methods, contributed to the data analysis, and reviewed and revised the manuscript; Drs Kirkorian, Coyne, and Barr helped develop the passive-sensing data collection methods and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Funded by Children and Screens: Institute of Digital Media and Child Development Inc for development of the passive-sensing technology and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grant 1R21HD094051) for the Preschooler Tablet Study. Research Electronic Data Capture and recruitment support was provided through the Michigan Institute for Clinical and Health Research (Clinical and Translational Science Award UL1TR002240). Funded by the National Institutes of Health (NIH).

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2020-1242 .

application programming interface

comma-separated values

Competing Interests

Supplementary data.

Advertising Disclaimer »

Citing articles via

Email alerts.

Affiliations

• Editorial Board
• Editorial Policies
• Journal Blogs
• Pediatrics On Call
• Online ISSN 1098-4275
• Print ISSN 0031-4005
• Pediatrics Open Science
• Hospital Pediatrics
• Pediatrics in Review
• AAP Grand Rounds
• Latest News
• Pediatric Care Online
• Red Book Online
• Pediatric Patient Education
• AAP Toolkits
• AAP Pediatric Coding Newsletter

First 1,000 Days Knowledge Center

Institutions/librarians, group practices, licensing/permissions, integrations, advertising.

• Privacy Statement | Accessibility Statement | Terms of Use | Support Center | Contact Us
• © Copyright American Academy of Pediatrics

This Feature Is Available To Subscribers Only

Sign In or Create an Account

The effects of using mobile devices on language learning: a meta-analysis

• Published: 27 June 2020
• Volume 68 , pages 1769–1789, ( 2020 )

Cite this article

• Zhenzhen Chen   ORCID: orcid.org/0000-0002-6773-1620 1 , 2 ,
• Weichao Chen   ORCID: orcid.org/0000-0001-8964-2568 3 ,
• Jiyou Jia 2 &
• Huili An 1  

4074 Accesses

45 Citations

3 Altmetric

Explore all metrics

Despite the rapid development of the field of Mobile-assisted Language Learning (MALL), research synthesis and systematic meta-analyses on MALL are still lacking. It remains unclear how effective mobile devices are for language learning under different conditions. Review studies on the overall effectiveness of the latest smart mobile devices are still scant. In order to evaluate the learning outcomes of MALL and the impact of moderator variables, we systematically searched journal articles, conference proceedings, and doctoral dissertations published during 2008–2018 and performed a meta-analysis based on a synthesis of 84 effect sizes from 80 experimental and quasi-experimental studies. A medium-to-high effect size of 0.722 was found for the overall effectiveness of using mobile devices for language learning. The findings indicate that the use of mobile devices for language learning is more effective than conventional methods. The effects of nine moderator variables were analyzed. The target language skill, target language and first/second language were found to be significant moderators. Implications for language teaching and research are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

A Meta-Analysis of the Effects of Learning Languages with Mobile Devices

Mobile-Assisted Language Assessment for Adult EFL Learners: Recommendations from a Systematic Review

Mobile Assisted Chinese Learning as a Foreign Language: An Overview of Publications between 2007 and 2019

Wenying Zhou

Agbatogun, A. (2014). Developing learners’ second language communicative competence through active learning: Clickers or communicative approach? Educational Technology & Society, 17 (2), 257–269.

Google Scholar  

Alemi, M., Sarab, M. R. A., & Lari, Z. (2012). Successful learning of academic word list via MALL: Mobile assisted language learning. International Education Studies, 5 (6), 99–109.

Andujar, A. (2016). Benefits of mobile instant messaging to develop ESL writing. System, 62 , 63–76.

Andújar-Vaca, A., & Cruz-Martínez, M. S. (2017). Mobile instant messaging: Whatsapp and its potential to develop oral skills. Comunicar, 25 (50), 43–52.

Asmali, M. (2018). Integrating technology into ESP classes: Use of student response system in English for specific purposes instruction. Teaching English with Technology, 18 (3), 86–104.

Binks, S. (2018). Testing enhances learning: A review of the literature. Journal of Professional Nursing, 34 (3), 205–210.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis . Chichester: Wiley.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18 (1), 32–42.

Burston, J. (2013). Language learning technology MALL: Future directions for BYOD applications. The IALLT Journal for Language Learning Technologies, 43 (2), 89–96.

Burston, J. (2014a). The reality of MALL: Still on the fringes. CALICO Journal, 31 (1), 103–125.

Burston, J. (2014b). MALL: The pedagogical challenges. Computer Assisted Language Learning, 27 (4), 344–357.

Burston, J. (2015). Twenty years of MALL project implementation: A meta-analysis of learning outcomes. ReCALL, 27 (1), 4–20.

Chang, C. C. (2018). Outdoor ubiquitous learning or indoor CAL? Achievement and different cognitive loads of college students. Behaviour and Information Technology, 37 (1), 38–49.

Chen, C. M., & Chung, C. J. (2008). Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle. Computers & Education, 51 (2), 624–645.

Chen Hsieh, J. S., Wu, W. C. V., & Marek, M. W. (2017). Using the flipped classroom to enhance EFL learning. Computer Assisted Language Learning, 30 (1–2), 1–21.

Cho, K., Lee, S., Joo, M.-H., & Becker, B. J. (2018). The effects of using mobile devices on student achievement in language learning: A meta-analysis. Education Sciences, 8 (3), 105.

Chou, P. N., Chang, C. C., & Lin, C. H. (2017). BYOD or not: A comparison of two assessment strategies for student learning. Computers in Human Behavior, 74 , 63–71.

Coe, R. (2002). It’s the effect size, stupid: What effect size is and why it is important. Paper presented at the British Educational Research Association Annual Conference, England 12–14 September 2002. Retrieved January 25, 2020, from https://www.leeds.ac.uk/educol/documents/00002182.htm .

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Cook, V. (1973). The comparison of language development in native children and foreign adults. IRAL, 15 , 1–20.

Cook, V. (2010). The relationship between first and second language learning revisited. In E. Macaro (Ed.), The Continuum companion to second language acquisition (pp. 137–157). London: Continuum.

de la Fuente, M. J. (2014). Learners’ attention to input during focus on form listening tasks: The role of mobile technology in the second language classroom. Computer Assisted Language Learning, 27 (3), 261–276.

DeCoster, J. (2004). Meta-analysis notes . Retrieved October 15, 2019, from  http://www.stat-help.com .

Dickersin, K., Chan, S., Chalmers, T. C., Sacks, H. S., & Smith, H. (1987). Publication bias in clinical trials. Controlled Clinical Trials, 8 , 348–353.

Ducate, L., & Lomicka, L. (2013). Going mobile: Language learning with an ipod touch in intermediate French and German classes. Foreign Language Annals, 46 (3), 445–468.

Duman, G., Orhon, G., & Gedik, N. (2015). Research trends in mobile assisted language learning from 2000 to 2012. ReCALL, 27 (2), 197–216.

Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337 (8746), 867–872.

Ellis, R. (1994). The study of second language acquisition . Oxford: Oxford University Press.

Godwin-Jones, R. (2016). Augmented reality and language learning: From annotated vocabulary to place-based mobile games. Language Learning & Technology, 20 (3), 9–19.

Godwin-Jones, R. (2017). Smartphones and language learning. Language Learning & Technology, 21 (2), 3–17.

Grgurovic, M., Chapelle, C., & Shelley, M. (2013). A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL, 25 (2), 165–198.

Grimshaw, J., & Cardoso, W. (2018). Activate space rats! Fluency development in a mobile game-assisted environment. Language Learning & Technology, 22 (3), 159–175.

Herrington, A., Herrington, J., & Mantei, J. (2009). Design principles for mobile learning. In Herrington, J., Herrington, A., Mantei, J., Olney, I. & Ferry, B. (Eds.), New technologies, new pedagogies: Mobile learning in higher education (p. 138). Wollongong: Faculty of Education, University of Wollongong.

Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327 , 557–560.

Hung, H. T. (2017). Clickers in the flipped classroom: Bring your own device (BYOD) to promote student learning. Interactive Learning Environments, 25 (8), 983–995.

Hwang, G.-J., & Fu, Q.-K. (2019). Trends in the research design and application of mobile language learning: A review of 2007–2016 publications in selected SSCI journals. Interactive Learning Environments, 27 (4), 567–581.

Hwang, G. J., & Tsai, C. C. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42 (4), 65–70.

Hwang, W. Y., & Chen, H. S. L. (2013). Users’ familiar situational contexts facilitate the practice of EFL in elementary schools with mobile devices. Computer Assisted Language Learning, 26 (2), 101–125.

Hwang, W. Y., Chen, H. S. L., Shadiev, R., Huang, R. Y. M., & Chen, C. Y. (2014). Improving English as a foreign language writing in elementary schools using mobile devices in familiar situational contexts. Computer Assisted Language Learning, 27 (5), 359–378.

Kukulska-Hulme, A. (2009). Will mobile learning change language learning? ReCall, 21 (2), 157–165.

Kukulska-hulme, A., & Shield, L. (2008). An overview of mobile assisted language learning: From content delivery to supported collaboration and interaction. ReCALL, 20 (3), 271–289.

Lai, A. (2016). Mobile immersion: An experiment using mobile instant messenger to support second-language learning. Interactive Learning Environments, 24 (2), 277–290.

Lan, Y. J., & Lin, Y. T. (2016). Mobile seamless technology enhanced CSL oral communication. Educational Technology & Society, 19 (3), 335–350.

Levy, M., & Kennedy, C. (2005). Learning Italian via mobile SMS. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 76–83). London: Taylor & Francis.

Li, L., Chen, G., & Yang, S. (2013). Construction of cognitive maps to improve e-book reading and navigation. Computers & Education, 60 (1), 32–39.

Liakin, D., Cardoso, W., & Liakina, N. (2017). The pedagogical use of mobile speech synthesis (TTS): Focus on French liaison. Computer Assisted Language Learning, 30 (3–4), 348–365.

Lin, C. C. (2017). Learning English with electronic textbooks on tablet PCs. Interactive Learning Environments, 25 (8), 1035–1047.

Lin, C. J., & Hwang, G. J. (2018). A learning analytics approach to investigating factors affecting EFL students’ oral performance in a flipped classroom. Educational Technology & Society, 21 (2), 205–219.

Lin, J., & Lin, H. (2019). Mobile-assisted ESL/EFL vocabulary learning: A systematic review and meta-analysis. Computer Assisted Language Learning, 32 (8), 878–919.

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis . Thousand Oaks, CA: Sage.

Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation. Computers & Education, 55 (2), 630–643.

Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of Computer Assisted Learning, 24 (6), 515–525.

Mahdi, H. S. (2018). Effectiveness of mobile devices on vocabulary learning. Journal of Educational Computing Research, 56 (1), 134–154.

Naismith, L., Sharples, M., Vavoula, G. & Lonsdale, P. (2004). Literature review in mobile technologies and learning . Bristol: University of Birmingham. (FutureLab Series, Report, 11). Retrieved October 20, 2019, from www.futurelab.org.uk/research/lit_reviews.htm .

Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8 (2), 157–159.

Quinn, C. (2000). mLearning: mobile, wireless, in your pocket learning. LineZine. Retrieved October 20, 2019, from https://www.linezine.com/2.1/features/cqmmwiyp.htm .

Reinders, H., & Pegrum, M. (2017). Supporting language learning on the move: An evaluative framework for mobile language learning resources. In B. Tomlinson (Ed.),  Second language acquisition research and materials development for language learning  (pp. 116–141). (Second Language Acquisition Research Series). London: Routledge.

Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86 (3), 638–641.

Rubin, J. (1975). What the “good language learner” can teach us? TESOL Quarterly, 9 (1), 41–51.

Sandberg, J., Maris, M., & De Geus, K. (2011). Mobile English learning: An evidence-based study with fifth graders. Computers and Education, 57 (1), 1334–1347.

Saran, M., Seferoǧlu, G., & Çaǧiltay, K. (2012). Mobile language learning: Contribution of multimedia messages via mobile phones in consolidating vocabulary. The Asia-Pacific Education Researcher, 21 (1), 181–190.

Shadiev, R., Hwang, W. Y., & Liu, T. Y. (2018). A study of the use of wearable devices for healthy and enjoyable English as a foreign language learning in authentic contexts. Educational Technology & Society, 21 , 217–231.

Sharples, M., Milrad, M., Arnedillo-Sánchez, I., Milrad, M., & Vavoula, G. (2009). Mobile learning: Small devices, big issues. In N. Balacheff, S. Ludvigsen, T. Jong, A. Lazonder, & S. Barnes (Eds.), Technology enhanced learning: Principles and products (pp. 233–249). Heidelberg: Springer.

Sharples, M., Taylor, J., & Vavoula, G. (2005). Towards a theory of mobile learning. Proceedings of mLearn, 1 , 1–9.

Silverio-Fernández, M., Renukappa, S., & Suresh, S. (2018). What is a smart device?—A conceptualisation within the paradigm of the internet of things. Visualization in Engineering, 6 (1), 3.

Song, Y., & Fox, R. (2008). Using PDA for undergraduate student incidental vocabulary testing. ReCALL, 20 (3), 290–314.

Stockwell, G., & Hubbard, P. (2013). Some emerging principles for mobile-assisted language learning. Monterey, CA: The International Research Foundation for English Language Education. Retrieved November 1, 2019, from https://www.tirfonline.org/english-in-the-workforce/mobile-assisted-language-learning .

Sung, Y., Chang, K., & Liu, T. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94 , 252–275.

Sung, Y., Chang, K., & Yang, J. (2015). How effective are mobile devices for language learning ? A meta-analysis. Educational Research Review, 16 , 68–84. https://doi.org/10.1016/j.edurev.2015.09.001 .

Article   Google Scholar  

Thornton, P., & Houser, C. (2005). Using mobile phones in English education in Japan. Journal of Computer Assisted Learning, 21 , 217–228.

Traxler, J. (2005). Defining mobile learning . Paper presented at IADIS International Conference Mobile Learning 2005 (pp. 261–266).

Van Lier, L. (2004). The ecology and semiotics of language learning: A sociocultural perspective . Dordrecht: Springer.

Viberg, O., & Grönlund, A. (2012). Mobile assisted language learning: A literature review. Paper presented at the 11th World Conference on Mobile and Contextual Learning, Helsinki, Finland. mLearn 2012. 1–8.

Wang, Y. H. (2016). Could a mobile-assisted learning system support flipped classrooms for classical Chinese learning? Journal of Computer Assisted Learning, 32 (5), 391–415.

Winters, N. (2006). What is mobile learning? In M. Sharples (Ed.), Big issues in mobile learning: Report of a workshop by the kaleidoscope network of excellence mobile learning initiative (pp. 5–9). Nottingham: University of Nottingham.

Wu, Q. (2015). Designing a smartphone app to teach English (L2) vocabulary. Computers & Education, 85 , 170–179.

Wu, T. T., Sung, T. W., Huang, Y. M., Yang, C. S., & Yang, J. T. (2011). Ubiquitous English learning system with dynamic personalized guidance of learning portfolio. Educational Technology & Society, 14 (4), 164–180.

Wu, W. C., Chen Hsieh, J., & Yang, J. C. (2017). Creating an online learning community in a flipped classroom to enhance EFL learners’ oral proficiency. Educational Technology & Society, 20 (2), 142–157.

Zheng, D., Liu, Y., Lambert, A., Lu, A., Tomei, J., & Holden, D. (2018). An ecological community becoming: Language learning as first-order experiencing with place and mobile technologies. Linguistics and Education, 44 , 45–57.

Zheng, D., & Newgarden, K. (2017). Dialogicality, ecology, and learning in online game worlds. In S. L. Thorne & S. May (Eds.), Language, education and technology (pp. 345–359). Cham: Springer International Publishing.

Zheng, L., Li, X., & Chen, F. (2016). Effects of a mobile self-regulated learning approach on students’ learning achievements and self-regulated learning skills. Innovations in Education and Teaching International, 55 (6), 1–9.

Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computer & Education, 42 , 289–314.

Download references

Acknowledgements

This research is part of the project entitled Classroom Interaction in Smartphone-supported College English Classes supported by the Key Projects of Ministry of Education, National Education Sciences Planning Projects of 2017, China (Project Number: DCA170308). We would also like to acknowledge the constructive suggestions provided by the reviewers and the valuable editing suggestions given by Mr. Mitchell Wagner from Indiana University.

Author information

Authors and affiliations.

School of Humanities, Beijing University of Posts and Telecommunications, No. 10 Xitucheng Road, Beijing, 100876, China

Zhenzhen Chen & Huili An

Graduate School of Education, Peking University, No. 5 Summer Palace Road, Beijing, 100871, China

Zhenzhen Chen & Jiyou Jia

Office of Medical Education, School of Medicine, University of Virginia, P.O. Box 800382, Charlottesville, MA, 22908, USA

Weichao Chen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Zhenzhen Chen .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Chen, Z., Chen, W., Jia, J. et al. The effects of using mobile devices on language learning: a meta-analysis. Education Tech Research Dev 68 , 1769–1789 (2020). https://doi.org/10.1007/s11423-020-09801-5

Download citation

Published : 27 June 2020

Issue Date : August 2020

DOI : https://doi.org/10.1007/s11423-020-09801-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile learning
• Language learning
• Meta-analysis
• Literature review
• Find a journal
• Publish with us
• Track your research

Mobile Forensics: A Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Pharm Investig
• v.7(3); Jul-Sep 2017

Smartphone usage and increased risk of mobile phone addiction: A concurrent study

Subramani parasuraman.

Unit of Pharmacology, AIMST University, Kedah, Malaysia

Aaseer Thamby Sam

1 Unit of Pharmacy Practice, Faculty of Pharmacy, AIMST University, Kedah, Malaysia

Stephanie Wong Kah Yee

Bobby lau chik chuon.

This study aimed to study the mobile phone addiction behavior and awareness on electromagnetic radiation (EMR) among a sample of Malaysian population.

This online study was conducted between December 2015 and 2016. The study instrument comprised eight segments, namely, informed consent form, demographic details, habituation, mobile phone fact and EMR details, mobile phone awareness education, psychomotor (anxious behavior) analysis, and health issues. Frequency of the data was calculated and summarized in the results.

Totally, 409 respondents participated in the study. The mean age of the study participants was 22.88 (standard error = 0.24) years. Most of the study participants developed dependency with smartphone usage and had awareness (level 6) on EMR. No significant changes were found on mobile phone addiction behavior between the participants having accommodation on home and hostel.

Conclusion:

The study participants were aware about mobile phone/radiation hazards and many of them were extremely dependent on smartphones. One-fourth of the study population were found having feeling of wrist and hand pain because of smartphone use which may lead to further physiological and physiological complication.

INTRODUCTION

Mobile/hand phones are powerful communication devices, first demonstrated by Motorola in 1973, and made commercially available from 1984.[ 1 ] In the last few years, hand phones have become an integral part of our lives. The number of mobile cellular subscriptions is constantly increasing every year. In 2016, there were more than seven billion users worldwide. The percentage of internet usage also increased globally 7-fold from 6.5% to 43% between 2000 and 2015. The percentage of households with internet access also increased from 18% in 2005 to 46% in 2015.[ 2 ] Parlay, the addiction behavior to mobile phone is also increasing. In 2012, new Time Mobility Poll reported that 84% people “couldn't go a single day without their mobile devices.”[ 3 ] Around 206 published survey reports suggest that 50% of teens and 27% of parents feel that they are addicted to mobiles.[ 4 ] The recent studies also reported the increase of mobile phone dependence, and this could increase internet addiction.[ 5 ] Overusage of mobile phones may cause psychological illness such as dry eyes, computer vision syndrome, weakness of thumb and wrist, neck pain and rigidity, increased frequency of De Quervain's tenosynovitis, tactile hallucinations, nomophobia, insecurity, delusions, auditory sleep disturbances, insomnia, hallucinations, lower self-confidence, and mobile phone addiction disorders.[ 6 ] In animals, chronic exposure to Wi-Fi radiation caused behavioral alterations, liver enzyme impairment, pyknotic nucleus, and apoptosis in brain cortex.[ 7 ] Kesari et al . concluded that the mobile phone radiation may increase the reactive oxygen species, which plays an important role in the development of metabolic and neurodegenerative diseases.[ 8 ]

In recent years, most of the global populations (especially college and university students), use smartphones, due to its wide range of applications. While beneficial in numerous ways, smartphones have disadvantages such as reduction in work efficacy, personal attention social nuisance, and psychological addiction. Currently, the addiction to smartphones among students is 24.8%–27.8%, and it is progressively increasing every year.[ 9 ] Mobile phone is becoming an integral part to students with regard to managing critical situations and maintaining social relationships.[ 10 ] This behavior may reduce thinking capabilities, affect cognitive functions, and induce dependency. The signs of smartphone addiction are constantly checking the phone for no reason, feeling anxious or restless without the phone, waking up in the middle of night to check the mobile and communication updates, delay in professional performance as a result of prolonged phone activities, and distracted with smartphone applications.[ 11 ]

Mobile phone is the most dominant portal of information and communication technology. A mental impairment resulting from modern technology has come to the attention of sociologists, psychologists, and scholars of education on mobile addiction.[ 12 ] Mobile phone addiction and withdrawal from mobile network may increase anger, tension, depression, irritability, and restlessness which may alter the physiological behavior and reduce work efficacy. Hence, the present study was planned to study the addiction behavior of mobile phone usage using an online survey.

This study was approved by Human and Animal Ethics Committee of AIMST University (AUHAEC/FOP/2016/05) and conducted according to the Declaration of Helsinki. The study was conducted among a sample of Malaysian adults. The study participants were invited through personal communications to fill the online survey form. The study was conducted between December 2015 and 2016. The study instrument comprised eight segments, namely, informed consent information, consent acceptance page, demographic details, habituation, mobile phone fact and electromagnetic radiation (EMR) details, mobile phone awareness education, psychomotor (anxious behavior) analysis, and health issues. If any of the participants were not willing to continue in the study, they could decline as per their discretion.

Totally, 450 participants were informed about the study and 409 participated in the study. The demographic details of the study participants are summarized in Table 1 . The incomplete forms were excluded from the study. The participants' details were maintained confidentially.

Demographic details of the study participants

Statistical analysis

Frequency of the data was calculated and the data were analyzed using two-sided Chi-square test with Yate's continuity correction.

Totally, 409 individuals participated in the study, of which 42.3% were males and 57.7% were females, between the age group of 18 and 55 years. Nearly 75.6% of the respondents were between the age group of 21 and 25 years. The mean age of the study participants was 22.88 (standard error = 0.24) years. The study participants' demographic details are summarized in Table 1 .

About 95% of the study participants were using smart phones, with 81.7% of them having at least one mobile phone. Most of the study participants used mobile phone for more than 5 years. Around 64.3% of the study participants use mobile phone for an hour (approximately) and remaining use it for more than an hour. Nearly 36.7% of the study participants have the habit of checking mobile phones in between sleep, while 27.1% felt inconvenience with mobile phone use. Majority of the respondents were using mobile phone for communication purposes (87.8%), photo shooting (59.7%), entertainment (58.2%), and educational/academic purposes (43.8%). Habits of mobile phone usage among the study participants are summarized in Table 2 .

Habituation analysis of mobile phone usage

The study results indicate that 86.8% of the participants are aware about EMR and 82.6% of the study participants are aware about the dangers of EMR. The prolonged use/exposure to EMR may cause De Quervain's syndrome, pain on wrist and hand, and ear discomfort. Among the study participants, 46.2% were having awareness on De Quervain's syndrome, 53.8% were feeling ear discomfort, and 25.9% were having mild-to-moderate wrist/hand pain. Almost 34.5% of the study participants felt pain in the wrist or at the back of the neck while utilizing smartphones [ Table 3a ]. Many of the study participants also agreed that mobile phone usage causes fatigue (12% agreed; 67.5% strongly agreed), sleep disturbance (16.9% agreed; 57.7% strongly agreed), and psychological disturbance (10.8% agreed; 54.8% strongly agreed) [ Table 3b ]. The study participants were having level 6 of awareness on mobile phone usage and EMR.

Analysis of awareness of mobile phone hazards

The behavioral analysis of the smartphone usage revealed that 70.4% of the study participants use smartphone longer than intended and 66.5% of the study participants are engaged for longer duration with smartphone. Nearly 57.7% of the study participants exercise control using their phones only for specific important functions. More number of study participants (58.2%) felt uncomfortable without mobile and were not able to withstand not having a smartphone, feeling discomfort with running out of battery (73.8%), felt anxious if not browsing through their favorite smartphone application (41.1%), and 50.4% of the study participants declared that they would never quit using smartphones even though their daily lifestyles were being affected by it. The study also revealed another important finding that 74.3% of smartphone users are feeling dependency on the use of smartphone. The addiction behavior analysis data of mobile phone are summarized in Table 4 .

Addiction behavior analysis of mobile phone

The study results also suggest that female participants were having more awareness than male participants ( P < 0.001) [ Table 5a ] and were more dependent on smartphones than male participants ( P < 0.05) [ Table 5b ]. Female participants were ready to quit using smartphones, if it affected daily lifestyle compared with male participants ( P < 0.05) [ Table 5b ]. Habituation of mobile phone use and addiction behavior were compared between both genders of the study participants and are summarized in Table 5a and ​ andb, b , respectively.

Comparison of habituation of mobile phone usage between genders

Comparison of addiction behavior between genders

A total of 297 participants were having accommodation in hostel, among them 39.6% of the study participants checked their mobile phone on an average of 21–30 times, a day, and 11.7% of the study participants checked their mobile phone more than 30 times a day. A total of 112 participants have accommodation in home, among them 28.6% of the study participants checked their mobile phone 21–30 times a day, and 13.4% of the study participants checked their mobile phone more than 30 times a day.

A total of 66.1% of participants having accommodation in home use their phones longer than intended, whereas 71.8% of participants having accommodation in hostel are using phone longer than intended. Forty-one (36.6%) and 109 (36.6%) participants from home and hotel checked mobile phone in-between sleep, respectively. About 67.9% of participants having accommodation in home felt dependent on mobile and it was the same for participants having accommodation in hostel (76.5%).

The study results suggest that a significant number of the participants had addiction to mobile phone usage, but were not aware on it, as mobile phones have become an integral part of life. No significant differences were found on addiction behavior between the participants residing in hostel and homes. Mobile phone abuse is rising as an important issue among the world population including physical problems such as eye problems, muscular pain, and psychological problem such as tactile and auditory delusions.[ 13 ] Along with mobile phone, availability of Wi-Fi facility in residence place and work premises also increases mobile phone dependence. The continuous and constant usage of mobile phone reduces intellectual capabilities and work efficacy. A study conducted in Chinese population (160 million out of the total 1.3 billion people) showed that people affected by mobile phone dependence have difficulty in focusing on work and are unsociable, eccentric, and use phones in spite of facing hazards or having knowledge of harmful effects of this form of electromagnetic pollution.[ 14 ]

The statement “I will never quit using my smartphone even though my daily lifestyles are affected by it” was statistically significant ( P = 0.0229). This points to a trend of mobile phone addiction among the respondents. This finding was discussed by Salehan and Negahban. They stated that this trend is due to the fast growth in the use of online social networking services (SNS). Extensive use of technology can lead to addiction. The use of SNS mobile applications is a significant predictor of mobile addiction. Their result showed that the use of SNS mobile applications is affected by both SNS network size and SNS intensity of the user. It has implications for academia as well as governmental and non-for-profit organizations regarding the effect of mobile phones on individual's and public health.[ 15 ] The health risks associated with mobile phones include increased chances of low self-esteem, anxiety or depression, bullying, eye strain and “digital or mobile phone thumb,” motor vehicle accidents, nosocomial infections, lack of sleep, brain tumors and low sperm counts, headache, hearing loss, expense, and dishonesty. The prevalence of cell phone dependence is unknown, but it is prevalent in all cultures and societies and is rapidly rising.[ 16 ] Relapse rate with mobile phone addiction is also high, which may also increase the health risk and affect cognitive function. Sahin et al . studied mobile phone addiction level and sleep quality in 576 university students and found that sleep quality worsens with increasing addiction level.[ 17 ]

The statement “Feeling dependent on the use of smartphone” was also statistically significant ( P = 0.0373). This was also explored by Richard et al . among 404 university students regarding their addiction to smartphones. Half of the respondents were overtly addicted to their phones, while one in five rated themselves totally dependent on their smartphones. Interestingly, higher number of participants felt more secure with their phones than without. Using their phones as an escapism was reported by more than half of the respondents. This study revealed an important fact that people are not actually addicted to their smartphones per se ; however, it is to the entertainment, information, and personal connections that majority of the respondents were addicted to.[ 18 ]

The 2015 statistical report from the British Chiropractic Association concluded that 45% of young people aged 16–24 years suffered with back pain. Long-term usage of smart phone may also cause incurable occipital neuralgia, anxiety and depression, nomophobia, stress, eyesight problem, hearing problems, and many other health issues.[ 19 ]

A study conducted among university students of Shahrekord, Iran, revealed that 21.49% of the participants were addicted to mobile phones, 17.30% participants had depressive disorder, 14.20% participants had obsessive-compulsive disorder, and 13.80% had interpersonal sensitivity.[ 12 ] Nearly 72% of South Korean children aged 11–12 years spend 5.4 h a day on mobile phones, 25% of those children were considered addicts to smartphones.[ 20 ] Thomée et al . collected data from 4156 adults aged between 20 and 24 years and observed no clear association between availability demands or being awakened at night and the mental health outcomes.[ 21 ] Overuse of mobile phone can lead to reduced quality of interpersonal relationships and lack of productivity in daily life. The study outcome from different studies showed variable results on addictive behavior on mobile phone usage. The fact is over-/long-time usage of mobile phone may cause behavioral alteration and induce addictive behavior.

This study suggests that most of the study participants are aware about mobile phone/radiation hazards and many of them developed dependent behavior with smartphone. No significant changes were found on mobile phone dependency behavior between participants having accommodation in house and hostel. One-fourth of the study population is having a feeling of wrist and hand because of smartphone usage which may lead to further physiological and physiological complications.

Limitations

• Cluster sampling from a wider population base could have provided a more clear idea regarding the topic of interest
• Increasing the time frame and number of study phases was not possible due to logistical issues
• Impact of smartphone addiction on sleep pattern could have been studied in-depth.

Financial support and sponsorship

Conflicts of interest.

There are no conflflicts of interest.