systematic literature review on mobile applications

Usability of mobile learning applications: a systematic literature review

• Published: 12 October 2017
• Volume 5 , pages 1–17, ( 2018 )

Cite this article

• Bimal Aklesh Kumar 1 &
• Priya Mohite 1  

3352 Accesses

67 Citations

3 Altmetric

Explore all metrics

Usability evaluation of mobile learning applications is an active area of research. This paper reports on the systematic literature review (SLR) carried out to investigate the state of art in conducting usability evaluation of mobile learning applications. SLR was performed on papers obtained from commonly used databases. In total, 42 publications were retrieved; out of which 23 were relevant to our research questions. The results of SLR provided an insight into main contributions of the field, gaps, and opportunities, which motivated the discussion for important research directions in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Mobile Applications Usability Evaluation: Systematic Review and Reappraisal

Usability Challenges in Smartphone Web Access: A Systematic Literature Review

Mobile Application Software Usability Evaluation: Issues, Methods and Future Research Directions

Explore related subjects.

• Artificial Intelligence
• Digital Education and Educational Technology

Ali, A. A. (2013). A framework for measuring the usability issues and criteria of mobile learning applications.

Al-Wabil, N. (2015). Usability of mobile applications in saudi higher education: An exploratory study. International Conference on Human-Computer Interaction, 529, 201–205.

Google Scholar  

Arain, A. A., Hussain, Z., Rizvi, W. H., & Vighio, M. S. (2016). Evaluating usability of M-learning application in the context of higher education institute. In International conference on learning and collaboration technologies (pp. 259–268). Springer.

Armstrong, P., & Wilkinson, B. (2016). Preliminary usability testing of ClaMApp: A classroom management app for tablets. Proceedings of the 28th Australian Conference on Computer - Human Interaction (pp. 654–656).

Asabere, N. Y. (2013). Benefits and challenges of mobile learning implementation: Story of developing nations. International Journal of Computer Applications, 73 (1), 23.

Article   Google Scholar  

Carroll, J. M. (1997). Human-computer interaction: Psychology as a science of design. Annual Review of Psychology, 48 (1), 61–83.

Carroll, J., Howard, S., Peck, J., & Murphy, J. (2002). A field study of perceptions and use of mobile telephones by 16–22 year olds. Journal of Information Technology Theory and Application, 4 (2), 49.

Chandhok, S., & Babbar, P. (2011). M-learning in distance education libraries: A case scenario of Indira Gandhi National Open University. The Electronic Library, 29 (5), 637–650.

Cole, J., Bergin, D., & Whittaker, T. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33 (4), 609–624.

Corlett, D., Sharples, M., Bull, S., & Chan, T. (2005). Evaluation of a mobile learning organiser for university students. Journal of Computer Assisted Learning, 21 (3), 162–170.

Danesh, A., Inkpen, K., Lau, F., Shu, K., & Booth, K. (2001). GeneyTM: Designing a collaborative activity for the palmTM handheld computer. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 388–395).

Fetaji, M., & Fetaji, B. (2011). Comparing developed MLUAT (mobile learning usability attribute testing) methodology with qualitative user testing method and heuristics evaluation. Proceedings of the 12th International Conference on Computer Systems and Technologies (pp. 516–523).

Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science, 1 (1), 1.

Hashim, A. S., & Ahmad, W. F. W. (2012). A comparison of architectures for a usability-aware customized mobile learning management system (CMLMS). 2012 Sixth UKSim/AMSS European Symposium on Computer Modeling and Simulation (EMS) (pp. 511–516).

Hashim, A. S., Ahmad, W. F. W., & Ahmad, R. (2011a). Mobile learning course content application as a revision tool: The effectiveness and usability. 2011 International Conference on Pattern Analysis and Intelligent Robotics (ICPAIR) (Vol. 2, pp. 184–187).

Heng, L. E., Sangodiah, A., & Ahmad, Wan Fatimah Bt Wan. (2012). End user’s perspective of usability in mobile learning system. 2012 International Conference on Computer & Information Science (ICCIS) (Vol. 2, pp. 1095–1098).

Hwang, G., Chu, H., & Lai, C. (2017). Prepare your own device and determination (PYOD): A successfully promoted mobile learning mode in Taiwan. International Journal of Mobile Learning and Organisation, 11 (2), 87–107.

Hwang, G., & Tsai, C. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42 (4), E65–E70.

Hwang, G., & Wu, P. (2014). Applications, impacts and trends of mobile technology-enhanced learning: A review of 2008–2012 publications in selected SSCI journals. International Journal of Mobile Learning and Organisation, 8 (2), 83–95.

Ivanc, D., Vasiu, R., & Onita, M. (2012). Usability evaluation of a LMS mobile web interface. International Conference on Information and Software Technologies (pp. 348–361).

Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33 (2004), 1–26.

Kondratova, I., & Goldfarb, I. (2006). M-learning: Overcoming the usability challenges of mobile devices. International Conference on Networking, Systems and International Conference on Mobile Communications and Learning Technologies, 2006. ICN/ICONS/MCL 2006 (Vol. 223).

Kukulska-Hulme, A., & Traxler, J. (2005). Mobile learning: A handbook for educators and trainers . London: Psychology Press.

Kumar, B. A., & Hussein, S. (2014). Heuristic based user interface evaluation of mobile money application: A case study. International Journal of Handheld Computing Research, 5 (2), 75–86.

Kumar, B. A., & Mohite, P. (2016). Usability guideline for mobile learning apps: An empirical study. International Journal of Mobile Learning and Organisation, 10 (4), 223–237.

Lai, C., & Hwang, G. (2015). High school teachers’ perspectives on applying different mobile learning strategies to science courses: The national mobile learning program in Taiwan. International Journal of Mobile Learning and Organisation, 9 (2), 124–145.

Lee, J. H. (2014). Development and usability assessment of tablet-based synchronous mobile learning system. Ubiquitous information technologies and applications (pp. 301–306). Springer.

Lee, M. J., & Chan, A. (2007). Pervasive, lifestyle-integrated mobile learning for distance learners: An analysis and unexpected results from a podcasting study. Open Learning, 22 (3), 201–218.

Li, Q., Wang, T., Wang, J., & Li, Y. (2011). Case study of usability testing methodology on mobile learning course.

Masood, M., & Thigambaram, M. (2015). The usability of mobile applications for pre-schoolers. Procedia-Social and Behavioral Sciences, 197, 1818–1826.

Nedungadi, P., & Raman, R. (2012). A new approach to personalization: Integrating e-learning and M-learning. Educational Technology Research and Development, 60 (4), 659–678.

Nielsen, J. (1994). Usability engineering . Elsevier.

Penzenstadler, B., Bauer, V., Calero, C., & Franch, X. (2012). Sustainability in software engineering: A systematic literature review.

Preece, J., & Rombach, H. D. (1994). A taxonomy for combining software engineering and human-computer interaction measurement approaches: Towards a common framework. International Journal of Human-Computer Studies, 41 (4), 553–583.

Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P. (2007). Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Medical Informatics and Decision Making, 7 (1), 16.

Sharples, M., Arnedillo-Sánchez, I., Milrad, M., & Vavoula, G. (2009). Mobile learning (pp. 233–249). Springer.

Snchez, J., & Espinoza, M. (2011). mGuides, design and usability of a mobile system to assist learning in critical situations. Universal Access in Human - Computer Interaction. Context Diversity (pp. 415–424).

Snchez, J., Flores, H., & Senz, M. (2008). Mobile science learning for the blind. CHI’08 Extended Abstracts on Human Factors in Computing Systems (pp. 3201–3206).

Traxler, J. (2009). Learning in a mobile age. International Journal of Mobile and Blended Learning, 1 (1), 1–12.

Tsuei, M., Chou, H., & Chen, B. (2013). Measuring usability of the mobile mathematics curriculum-based measurement application with children. International Conference of Design, User Experience, and Usability (pp. 304–310).

Wagner, E. D. (2005). Enabling mobile learning. EDUCAUSE Review , 40 (3), 41.

Wei, J., Zhang, H., & Zhuo, J. (2013). Mobile learning usability comparison between the US and chinese online education. International Journal of Innovation and Learning, 13 (1), 96–120.

Wei, J., Zhuo, J., & Zhang, H. (2008). Development of a mobile learning model with usability features for online education. International Journal of Mobile Learning and Organisation, 2 (1), 18–35.

Wyeth, P., McEwan, M., Roe, P., & MacColl, I. (2011). Expressive interactions: Tablet usability for young mobile learners. Proceedings of the 23rd Australian Computer - Human Interaction Conference (pp. 311–314).

Zaza, S., Wright-De Agero, L. K., Briss, P. A., Truman, B. I., Hopkins, D. P., Hennessy, M. H., et al. (2000). Data collection instrument and procedure for systematic reviews in the guide to community preventive services. American Journal of Preventive Medicine, 18 (1), 44–74.

Zbick, J., Nake, I., Milrad, M., & Jansen, M. (2015). A web-based framework to design and deploy mobile learning activities: Evaluating its usability, learnability and acceptance. 2015 IEEE 15th International Conference on Advanced Learning Technologies (ICALT) (pp. 88–92).

Zhang, D., & Adipat, B. (2005). Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction, 18 (3), 293–308.

Download references

Author information

Authors and affiliations.

Department of Computer Science and Information Systems, Fiji National University, Suva, Fiji

Bimal Aklesh Kumar & Priya Mohite

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Bimal Aklesh Kumar .

Rights and permissions

Reprints and permissions

About this article

Kumar, B.A., Mohite, P. Usability of mobile learning applications: a systematic literature review. J. Comput. Educ. 5 , 1–17 (2018). https://doi.org/10.1007/s40692-017-0093-6

Download citation

Received : 12 June 2017

Revised : 26 September 2017

Accepted : 28 September 2017

Published : 12 October 2017

Issue Date : March 2018

DOI : https://doi.org/10.1007/s40692-017-0093-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile learning
• Human computer interaction
• Mobile usability
• Mobile user experience
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PeerJ Comput Sci

A systematic literature review on the usability of mobile applications for visually impaired users

Muna al-razgan.

1 King Saud University, Riyadh, Saudi Arabia

Sarah Almoaiqel

Nuha alrajhi.

2 Imam Muhammad Ibn Saud University, Riyadh, Saudi Arabia

Alyah Alhumegani

Abeer alshehri, bashayr alnefaie, raghad alkhamiss, shahad rushdi, associated data.

The following information was supplied regarding data availability:

This is a systematic literature review; there is no raw data.

Interacting with mobile applications can often be challenging for people with visual impairments due to the poor usability of some mobile applications. The goal of this paper is to provide an overview of the developments on usability of mobile applications for people with visual impairments based on recent advances in research and application development. This overview is important to guide decision-making for researchers and provide a synthesis of available evidence and indicate in which direction it is worthwhile to prompt further research. We performed a systematic literature review on the usability of mobile applications for people with visual impairments. A deep analysis following the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) guidelines was performed to produce a set of relevant papers in the field. We first identified 932 papers published within the last six years. After screening the papers and employing a snowballing technique, we identified 60 studies that were then classified into seven themes: accessibility, daily activities, assistive devices, navigation, screen division layout, and audio guidance. The studies were then analyzed to answer the proposed research questions in order to illustrate the different trends, themes, and evaluation results of various mobile applications developed in the last six years. Using this overview as a foundation, future directions for research in the field of usability for the visually impaired (UVI) are highlighted.

Introduction

The era of mobile devices and applications has begun. With the widespread use of mobile applications, designers and developers need to consider all types of users and develop applications for their different needs. One notable group of users is people with visual impairments. According to the World Health Organization, there are approximately 285 million people with visual impairments worldwide ( World Health Organization, 2020 ). This is a huge number to keep in mind while developing new mobile applications.

People with visual impairments have urged more attention from the tech community to provide them with the assistive technologies they need ( Khan & Khusro, 2021 ). Small tasks that we do daily, such as picking out outfits or even moving from one room to another, could be challenging for such individuals. Thus, leveraging technology to assist with such tasks can be life changing. Besides, increasing the usability of applications and developing dedicated ones tailored to their needs is essential. The usability of an application refers to its efficiency in terms of the time and effort required to perform a task, its effectiveness in performing said tasks, and its users’ satisfaction ( Ferreira et al., 2020 ). Researchers have been studying this field intensively and proposing different solutions to improve the usability of applications for people with visual impairments.

This paper provides a systematic literature review (SLR) on the usability of mobile applications for people with visual impairments. The study aims to find discussions of usability issues related to people with visual impairments in recent studies and how they were solved using mobile applications. By reviewing published works from the last six years, this SLR aims to update readers on the newest trends, limitations of current research, and future directions in the research field of usability for the visually impaired (UVI).

This SLR can be of great benefit to researchers aiming to become involved in UVI research and could provide the basis for new work to be developed, consequently improving the quality of life for the visually impaired. This review differs from previous review studies ( i.e.,   Khan & Khusro, 2021 ) because we classified the studies into themes in order to better evaluate and synthesize the studies and provide clear directions for future work. The following themes were chosen based on the issues addressed in the reviewed papers: “Assistive Devices,” “Navigation,” “Accessibility,” “Daily Activities,” “Audio Guidance,” and “Gestures.” Figure 1 illustrates the percentage of papers classified in each theme.

The remainder of this paper is organized as follows: the next section specifies the methodology, following this, the results section illustrates the results of the data collection, the discussion section consists of the research questions with their answers and the limitations and potential directions for future work, and the final section summarizes this paper’s main findings and contribution.

Survey Methodology

This systematic literature review used the Meta-Analyses (PRISMA, 2009) guidelines to produce a set of relevant papers in the field. This SLR was undertaken to address the research questions described below. A deep analysis was performed based on a group of studies; the most relevant studies were documented, and the research questions were addressed.

A. Research questions

The research questions addressed by this study are presented in Table 1 with descriptions and the motivations behind them.

B. Search strategy

This review analysed and synthesised studies on usability for the visually impaired from a user perspective following a systematic approach. As proposed by Tanfield, Denyer & Smart (2003) , the study followed a three-stage approach to ensure that the findings were both reliable and valid. These stages were planning the review, conducting the review by analysing papers, and reporting emerging themes and recommendations. These stages will be discussed further in the following section.

1. Planning stage

The planning stage of this review included defining data sources and the search string protocol as well as inclusion and exclusion criteria.

Data sources.

We aimed to use two types of data sources: digital libraries and search engines. The search process was manually conducted by searching through databases. The selected databases and digital libraries are as follows:

• • ACM Library
• • IEEE Xplore
• • ScienceDirect
• • SpringerLink
• • ISI Web of Knowledge
• • Scopus.

The selected search engines were as follows:

• • DBLP (Computer Science Bibliography Website)
• • Google Scholar
• • Microsoft Academic

Search string.

The above databases were initially searched using the following keyword protocol: (“Usability” AND (”visual impaired” OR ”visually impaired” OR “blind” OR “impairment”) AND “mobile”). However, in order to generate a more powerful search string, the Network Analysis Interface for Literature Studies (NAILS) project was used. NAILS is an automated tool for literature analysis. Its main function is to perform statistical and social network analysis (SNA) on citation data ( Knutas et al., 2015 ). In this study, it was used to check the most important work in the relevant fields as shown in Fig. 2 .

NAILS produced a report displaying the most important authors, publications, and keywords and listed the references cited most often in the analysed papers ( Knutas et al., 2015 ) . The new search string was generated after using the NAILS project as follows: (“Usability” OR “usability model” OR “usability dimension” OR “Usability evaluation model” OR “Usability evaluation dimension”) AND (“mobile” OR “Smartphone”) AND (“Visually impaired” OR “Visual impairment” OR “Blind” OR “Low vision” OR “Blindness”).

Inclusion and exclusion criteria.

To be included in this systematic review, each study had to meet the following screening criteria:

• • The study must have been published between 2015 and 2020.
• • The study must be relevant to the main topic (Usability of Mobile Applications for Visually Impaired Users).
• • The study must be a full-length paper.
• • The study must be written in English because any to consider any other languages, the research team will need to use the keywords of this language in this topic and deal with search engines using that language to extract all studies related to our topic to form an SLR with a comprehensive view of the selected languages. Therefore, the research team preferred to focus on studies in English to narrow the scope of this SLR.

A research study was excluded if it did not meet one or more items of the criteria.

2. Conducting stage

The conducting stage of the review involved a systematic search based on relevant search terms. This consisted of three substages: exporting citations, importing citations into Mendeley, and importing citations into Rayyan.

Exporting citations.

First, in exporting the citations and conducting the search through the mentioned databases, a total of 932 studies were found. The numbers are illustrated in Fig. 3 below. The highest number of papers was found in Google Scholar, followed by Scopus, ISI Web of Knowledge, ScienceDirect, IEEE Xplore, Microsoft Academic, and DBLP and ACM Library with two studies each. Finally, SpringerLink did not have any studies that met the inclusion criteria.

The chance of encountering duplicate studies was determined to be high. Therefore, importing citations into Mendeley was necessary in order to eliminate the duplicates.

Importing citations into mendeley.

Mendeley is an open-source reference and citation manager. It can highlight paragraphs and sentences, and it can also list automatic references on the end page. Introducing the use of Mendeley is also expected to avoid duplicates in academic writing, especially for systematic literature reviews ( Basri & Patak, 2015 ). Hence, in the next step, the 932 studies were imported into Mendeley, and each study’s title and abstract were screened independently for eligibility. A total of 187 duplicate studies were excluded. 745 total studies remained after the first elimination process. The search stages are shown in Fig. 4 below.

Importing citations into rayyan.

Rayyan QCRI is a free web and mobile application that helps expedite the initial screening of both abstracts and titles through a semi-automated process while incorporating a high level of usability. Its main benefit is to speed up the most tedious part of the systematic literature review process: selecting studies for inclusion in the review ( Ouzzani et al., 2016 ). Therefore, for the last step, another import was done using Rayyan to check for duplications a final time. Using Rayyan, a total of 124 duplicate studies were found, resulting in a total of 621 studies. Using Rayyan, a two-step filtration was conducted to guarantee that the papers have met the inclusion criteria of this SLR. After filtering based on the abstracts, 564 papers did not meet the inclusion criteria. At this stage, 57 studies remained. The second step of filtration eliminated 11 more studies by reading the full papers; two studies were not written in the English language, and nine were inaccessible.

Snowballing.

Snowballing is an emerging technique used to conduct systematic literature reviews that are considered both efficient and reliable using simple procedures. The procedure for snowballing consisted of three phases in each cycle. The first phase is refining the start set, the second phase is backward snowballing, and the third is forward snowballing. The first step, forming the start set, is basically identifying relevant papers that can have a high potential of satisfying the criteria and research question. Backward snowballing was conducted using the reference list to identify new papers to include. It shall start by going through the reference list and excluding papers that do not fulfill the basic criteria; the rest that fulfil criteria shall be added to the SLR. Forward snowballing refers to identifying new papers based on those papers that cited the paper being examined ( Juneja & Kaur, 2019 ). Hence, in order to be sure that we concluded all related studies after we got the 46 papers, a snowballing step was essential. Forward and backward snowballing were conducted. Each of the 46 studies was examined by checking their references to take a look at any possible addition of sources and examining all papers that cited this study. The snowballing activity added some 38 studies, but after full reading, it became 33 that matched the inclusion criteria. A total of 79 studies were identified through this process.

Quality assessment.

A systematic literature review’s quality is determined by the content of the papers included in the review. As a result, it is important to evaluate the papers carefully ( Zhou et al., 2015 ). Many influential scales exist in the software engineering field for evaluating the validity of individual primary studies and grading the overall intensity of the body of proof. Hence, we adapted the comprehensive guidelines specified by Kitchenhand and Charters ( Keele, 2007 ), and the quasi-gold standard (QGS) ( Keele, 2007 ) was used to establish the quest technique, where a robust search strategy for enhancing the validity and reliability of a SLR’s search process is devised using the QGS. By applying this technique, our quality assessment questions were focused and aligned with the research questions mentioned earlier.

In our last step, we had to verify the papers’ eligibility; we conducted a quality check for each of the 79 studies. For quality assessment, we considered whether the paper answered the following questions:

QA1: Is the research aim clearly stated in the research?

QA2: Does the research contain a usability dimension or techniques for mobile applications for people with visual impairments?

QA3: Is there an existing issue with mobile applications for people with visual impairments that the author is trying to solve?

QA4: Is the research focused on mobile application solutions?

After discussing the quality assessment questions and attempting to find an answer in each paper, we agreed to score each study per question. If the study answers a question, it will be given 2 points; if it only partially answers a question, it will be given 1 point; and if there is no answer for a given question in the study, it will have 0 points.

The next step was to calculate the weight of each study. If the total weight was higher or equal to four points, the paper was accepted in the SLR; if not, the paper was discarded since it did not reach the desired quality level. Figure 5 below illustrates the quality assessment process. After applying the quality assessment, 39 papers were rejected since they received less than four points, which resulted in a final tally of 60 papers.

To summarize, this review was conducted according to the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) ( Liberati et al., 2009 ). The PRISMA diagram shown in Fig. 6 illustrates all systematic literature processes used in this study.

3. Analysing stage

All researchers involved in this SLR collected the data. The papers were distributed equally between them, and each researcher read each paper completely to determine its topic, extract the paper’s limitations and future work, write a quick summary about it, and record this information in an Excel spreadsheet.

All researchers worked intensively on this systematic literature review. After completing the previously mentioned steps, the papers were divided among all the researchers. Then, each researcher read their assigned papers completely and then classified them into themes according to the topic they covered. The researchers held several meetings to discuss and specify those themes. The themes were identified by the researchers based on the issues addressed in the reviewed papers. In the end, the researchers resulted in seven themes, as shown in Fig. 7 below. The references selected for each theme can be found in the Table A1 . Afterwards, each researcher was assigned one theme to summarize its studies and report the results. In this section, we review the results.

A. Accessibility

Of a total of 60 studies, 10 focused on issues of accessibility. Accessibility is concerned with whether all users are able to have equivalent user experiences, regardless of abilities. Six studies, Darvishy, Hutter & Frei (2019) , Morris et al. (2016) , Qureshi & Hooi-Ten Wong (2020) , Khan, Khusro & Alam (2018) , Paiva et al. (2020) , and Pereda, Murillo & Paz (2020) , gave suggestions for increasing accessibility, ( Darvishy, Hutter & Frei, 2019 ; Morris et al., 2016 ), gave some suggestions for making mobile map applications and Twitter accessible to visually impaired users, and ( Qureshi & Hooi-Ten Wong, 2020 ; Khan, Khusro & Alam, 2018 ) focused on user interfaces and provided accessibility suggestions suitable for blind people. Paiva et al. (2020) and Pereda, Murillo & Paz (2020) proposed a set of heuristics to evaluate the accessibility of mobile applications. Two studies, Khowaja et al. (2019) and Carvalho et al. (2018) , focused on evaluating usability and accessibility issues on some mobile applications, comparing them, and identifying the number and types of problems that visually impaired users faced. Aqle, Khowaja & Al-Thani (2020) proposed a new web search interface designed for visually impaired users. One study, McKay (2017) , focused on accessibility challenges by applying usability tests on a hybrid mobile app with some visually impaired university students.

B. Assistive devices

People with visual impairments have an essential need for assistive technology since they face many challenges when performing activities in daily life. Out of the 60 studies reviewed, 13 were related to assistive technology. The studies Smaradottir, Martinez & Håland (2017) , Skulimowski et al. (2019) , Barbosa, Hayes & Wang, (2016) , Rosner & Perlman (2018) , Csapó et al. (2015) , Khan & Khusro (2020) , Sonth & Kallimani (2017) , Kim et al. (2016) , Vashistha et al. (2015) ; Kameswaran et al. (2020) , Griffin-Shirley et al. (2017) , and Rahman, Anam & Yeasin (2017) were related to screen readers (voiceovers). On the other hand, Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) were related to proposing an assistant device for the visually impaired. Of the studies related to screening readers, Sonth & Kallimani, (2017) , Vashistha et al. (2015) , Khan & Khusro (2020) Lewis et al. (2016) cited challenges faced by visually impaired users. Barbosa, Hayes & Wang (2016) , Kim et al. (2016) , Rahman, Anam & Yeasin (2017) suggested new applications, while Smaradottir, Martinez & Håland (2017) , Rosner & Perlman (2018) , Csapó et al. (2015) and Griffin-Shirley et al. (2017) evaluated current existing work. The studies Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) proposed using wearable devices to improve the quality of life for people with visual impairments.

C. Daily activities

In recent years, people with visual impairments have used mobile applications to increase their independence in their daily activities and learning, especially those based on the braille method. We divide the daily activity section into braille-based applications and applications designed to enhance the independence of the visually impaired. Four studies, Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , Araújo et al. (2016) and Gokhale et al. (2017) , implemented and evaluated the usability of mobile phone applications that use braille to help visually impaired people in their daily lives. Seven studies, Vitiello et al. (2018) , Kunaratana-Angkul, Wu & Shin-Renn (2020) , Ghidini et al. (2016) , Madrigal-Cadavid et al. (2019) , Marques, Carriço & Guerreiro (2015) , Oliveira et al. (2018) and Rodrigues et al. (2015) , focused on building applications that enhance the independence and autonomy of people with visual impairments in their daily life activities.

D. Screen division layout

People with visual impairments encounter various challenges in identifying and locating non-visual items on touch screen interfaces like phones and tablets. Incidents of accidentally touching a screen element and frequently following an incorrect pattern in attempting to access objects and screen artifacts hinder blind people from performing typical activities on smartphones ( Khusro et al., 2019 ). In this review, 9 out of 60 studies discuss screen division layout: ( Khusro et al., 2019 ; Khan & Khusro, 2019 ; Grussenmeyer & Folmer, 2017 ; Palani et al., 2018 ; Leporini & Palmucci, 2018 ) discuss touch screen (smartwatch tablets, mobile phones, and tablet) usability among people with visual impairments, while ( Cho & Kim, 2017 ; Alnfiai & Sampalli, 2016 ; Niazi et al., 2016 ; Alnfiai & Sampalli, 2019 ) concern text entry methods that increase the usability of apps among visually impaired people. Khusro et al. (2019) provides a novel contribution to the literature regarding considerations that can be used as guidelines for designing a user-friendly and semantically enriched user interface for blind people. An experiment in Cho & Kim (2017) was conducted comparing the two-button mobile interface usability with the one-finger method and voiceover. Leporini & Palmucci (2018) gathered information on the interaction challenges faced by visually impaired people when answering questions on a mobile touch-screen device, investigated possible solutions to overcome the accessibility and usability challenges.

E. Gestures

In total, 3 of 60 studies discuss gestures in usability. Alnfiai & Sampalli (2017) compared the performance of BrailleEnter, a gesture based input method to the Swift Braille keyboard, a method that requires finding the location of six buttons representing braille dot, while Buzzi et al. (2017) and Smaradottir, Martinez & Haland (2017) provide an analysis of gesture performance on touch screens among visually impaired people.

F. Audio guidance

People with visual impairment primarily depend on audio guidance forms in their daily lives; accordingly, audio feedback helps guide them in their interaction with mobile applications.

Four studies discussed the use of audio guidance in different contexts: one in navigation ( Gintner et al., 2017 ), one in games ( Ara’ujo et al., 2017 ), one in reading ( Sabab & Ashmafee, 2016 ), and one in videos ( Façanha et al., 2016 ). These studies were developed and evaluated based on usability and accessibility of the audio guidance for people with visual impairments and aimed to utilize mobile applications to increase the enjoyment and independence of such individuals.

G. Navigation

Navigation is a common issue that visually impaired people face. Indoor navigation is widely discussed in the literature. Nair et al. (2020) , Al-Khalifa & Al-Razgan (2016) and De Borba Campos et al. (2015) discuss how we can develop indoor navigation applications for visually impaired people. Outdoor navigation is also common in the literature, as seen in Darvishy et al. (2020) , Hossain, Qaiduzzaman & Rahman (2020) , Long et al. (2016) , Prerana et al. (2019) and Bandukda et al. (2020) . For example, in Darvishy et al. (2020) , Touch Explorer, an accessible digital map application, was presented to alleviate many of the problems faced by people with visual impairments while using highly visually oriented digital maps. Primarily, it focused on using non-visual output modalities like voice output, everyday sound, and vibration feedback. Issues with navigation applications were also presented in Maly et al. (2015) . Kameswaran et al. (2020) discussed commonly used technologies in navigation applications for blind people and highlighted the importance of using complementary technologies to convey information through different modalities to enhance the navigation experience. Interactive sonification of images for navigation has also been shown in Skulimowski et al. (2019) .

In this section, the research questions are addressed in detail to clearly achieve the research objective. Also, a detailed overview of each theme will be mentioned below.

Answers to the research questions

This section will answer the research question proposed:

RQ1: What existing UVI issues did authors try to solve with mobile devices?

Mobile applications can help people with visual impairments in their daily activities, such as navigation and writing. Additionally, mobile devices may be used for entertainment purposes. However, people with visual impairments face various difficulties while performing text entry operations, text selection, and text manipulation on mobile applications ( Niazi et al., 2016 ). Thus, the authors of the studies tried to increase touch screens’ usability by producing prototypes or simple systems and doing usability testing to understand the UX of people with visual impairments.

RQ2: What is the role of mobile devices in solving those issues?

Mobile phones are widely used in modern society, especially among users with visual impairments; they are considered the most helpful tool for blind users to communicate with people worldwide ( Smaradottir, Martinez & Håland, 2017 ). In addition, the technology of touch screen assistive technology enables speech interaction between blind people and mobile devices and permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities perform actions or interact with systems ( Niazi et al., 2016 ).

RQ3: What are the publication trends on the usability of mobile applications among the visually impaired?

As shown in Fig. 8 below, research into mobile applications’ usability for the visually impaired has increased in the last five years, with a slight dip in 2018. Looking at the most frequent themes, we find that “Assistive Devices” peaked in 2017, while “Navigation” and “Accessibility” increased significantly in 2020. On the other hand, we see that the prevalence of “Daily Activities” stayed stable throughout the research years. The term “Audio Guidance” appeared in 2016 and 2017 and has not appeared in the last three years. “Gestures” also appeared only in 2017. “Screen Layout Division” was present in the literature in the last five years and increased in 2019 but did not appear in 2020.

RQ4: What are the current research limitations and future research directions regarding usability among the visually impaired?

We divide the answer to this question into two sections: first, we will discuss limitations; then, we will discuss future work for each proposed theme.

A. Limitations

Studies on the usability of mobile applications for visually impaired users in the literature have various limitations, and most of them were common among the studies. These limitations were divided into two groups. The first group concerns proposed applications; for example, Rahman, Anam & Yeasin (2017) , Oliveira et al. (2018) and Madrigal-Cadavid et al. (2019) faced issues regarding camera applications in mobile devices due to the considerable effort needed for its usage and being heavily dependent on the availability of the internet. The other group of studies, Rodrigues et al. (2015) , Leporini & Palmucci (2018) , Alnfiai & Sampalli (2016) , and Ara’ujo et al. (2017) , have shown limitations in visually impaired users’ inability to comprehend a graphical user interface. Alnfiai & Sampalli (2017) and Alnfiai & Sampalli (2019) evaluated new braille input methods and found that the traditional braille keyboard, where knowing the exact position of letters QWERTY is required, is limited in terms of usability compared to the new input methods. Most studies faced difficulties regarding the sample size and the fact that many of the participants were not actually blind or visually impaired but only blindfolded. This likely led to less accurate results, as blind or visually impaired people can provide more useful feedback as they experience different issues on a daily basis and are more ideal for this type of study. So, the need for a good sample of participants who actually have this disability is clear to allow for better evaluation results and more feedback and recommendations for future research.

B. Future work

A commonly discussed future work in the chosen literature is to increase the sample sizes of people with visual impairment and focus on various ages and geographical areas to generalize the studies. Table 2 summarizes suggestions for future work according to each theme. Those future directions could inspire new research in the field.

RQ5: What is the focus of research on usability for visually impaired people, and what are the research outcomes in the studies reviewed?

There are a total of 60 outcomes in this research. Of these, 40 involve suggestions to improve usability of mobile applications; four of them address problems that are faced by visually impaired people that reduce usability. Additionally, 16 of the outcomes are assessments of the usability of the prototype or model. Two of the results are recommendations to improve usability. Finally, the last two outcomes are hardware solutions that may help the visually impaired perform their daily activities. Figure 9 illustrates these numbers.

Overview of the reviewed studies

In the following subsections, we summarize all the selected studies based on the classified theme: accessibility, assistive devices, daily activities, screen division layout, gestures, audio guidance, and navigation. The essence of the studies will be determined, and their significance in the field will be explored.

For designers dealing with mobile applications, it is critical to determine and fix accessibility issues in the application before it is delivered to the users ( Khowaja et al., 2019 ). Accessibility refers to giving the users the same user experience regardless of ability. In Khowaja et al. (2019) and Carvalho et al. (2018) , the researchers focused on comparing the levels of accessibility and usability in different applications. They had a group of visually impaired users and a group of sighted users test out the applications to compare the number and type of problems they faced and determine which applications contained the most violations. Because people with visual impairments cannot be ignored in the development of mobile applications, many researchers have sought solutions for guaranteeing accessibility. For example, in Qureshi & Hooi-Ten Wong (2020) , the study contributed to producing a new, effective design for mobile applications based on the suggestions of people with visual impairments and with the help of two expert mobile application developers. In Khan, Khusro & Alam (2018) , an adaptive user interface model for visually impaired people was proposed and evaluated in an empirical study with 63 visually impaired people. In Aqle, Khowaja & Al-Thani (2020) , the researchers proposed a new web search interface for users with visual impairments that is based on discovering concepts through formal concept analysis (FCA). Users interact with the interface to collect concepts, which are then used as keywords to narrow the search results and target the web pages containing the desired information with minimal effort and time. The usability of the proposed search interface (InteractSE) was evaluated by experts in the field of HCI and accessibility, with a set of heuristics by Nielsen and a set of WCAG 2.0 guidelines.

In Darvishy, Hutter & Frei (2019) , the researchers proposed a solution for making mobile map applications accessible for people with blindness or visual impairment. They suggested replacing forests in the map with green color and birds’ sound, replacing water with blue color and water sounds, replacing streets with grey color and vibration, and replacing buildings with yellow color and pronouncing the name of the building. The prototype showed that it was possible to explore a simple map through vibrations, sounds, and speech.

In Morris et al. (2016) the researchers utilized a multi-faceted technique to investigate how and why visually impaired individuals use Twitter and the difficulties they face in doing so. They noted that Twitter had become more image-heavy over time and that picture-based tweets are largely inaccessible to people with visual impairments. The researchers then made several suggestions for how Twitter could be amended to continue to be usable for people with visual impairments.

The researchers in Paiva et al. (2020) focused on how to evaluate proposed methods for ensuring the accessibility and usability of mobile applications. Their checklist, Acc-MobileCheck, contains 47 items that correspond to issues related to comprehension (C), operation (O), perception (P), and adaptation (A) in mobile interface interaction. To validate Acc-MobileCheck, it was reviewed by five experts and three developers and determined to be effective. In Pereda, Murillo & Paz (2020) , the authors also suggest a set of heuristics to evaluate the accessibility of mobile e-commerce applications for visually impaired people. Finally, McKay (2017) conducted an accessibility test for hybrid mobile apps and found that students with blindness faced many barriers to access based on how they used hybrid mobile applications. While hybrid apps can allow for increased time for marketing, this comes at the cost of app accessibility for people with disabilities.

A significant number of people with visual impairments use state-of-the-art software to perform tasks in their daily lives. These technologies are made up of electronic devices equipped with sensors and processors that can make intelligent decisions.

One of the most important and challenging tasks in developing such technologies is to create a user interface that is appropriate for the sensorimotor capabilities of users with blindness ( Csapó et al., 2015 ). Several new hardware tools have proposed to improve the quality of life for people with visual impairments. Three tools were presented in this SLR: a smart stick that can notify the user of any obstacle, helping them to perform tasks easily and efficiently ( Bharatia, Ambawane & Rane, 2019 ), and an eye that can allow users to detect colors (medical evaluation is still required) ( Lewis et al., 2016 ).

The purpose of the study in Griffin-Shirley et al. (2017) was to understand how people with blindness use smartphone applications as assistive technology and how they perceive them in terms of accessibility and usability. An online survey with 259 participants was conducted, and most of the participants rated the applications as useful and accessible and were satisfied with them.

The researchers in Rahman, Anam & Yeasin (2017) designed and implemented EmoAssist, which is a smartphone application that assists with natural dyadic conversations and aims to promote user satisfaction by providing options for accessing non-verbal communication that predicts behavioural expressions and contains interactive dimensions to provide valid feedback. The usability of this application was evaluated in a study with ten people with blindness where several tools were applied in the application. The study participants found that the usability of EmoAssist was good, and it was an effective assistive solution.

This theme contains two main categories: braille-based application studies and applications to enhance the independence of VIU. Both are summarized below.

1- Braille-based applications

Braille is still the most popular method for assisting people with visual impairments in reading and studying, and most educational mobile phone applications are limited to sighted people. Recently, however, some researchers have developed assistive education applications for students with visual impairments, especially those in developing countries. For example, in India, the number of children with visual impairments is around 15 million, and only 5% receive an education ( Gokhale et al., 2017 ). Three of the braille studies focused on education: ( Nahar, Sulaiman & Jaafar, 2020 ; Nahar, Jaafar & Sulaiman, 2019 , and Araújo et al., 2016 ). These studies all used smartphone touchscreens and action gestures to gain input from the student, and then output was provided in the form of audio feedback. In Nahar, Sulaiman & Jaafar (2020) , vibrational feedback was added to guide the users. The participants in Nahar, Sulaiman & Jaafar (2020) ; Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) included students with blindness of visual impairment and their teachers. The authors in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) evaluated the usability of their applications following the same criteria (efficiency, learnability, memorability, errors, and satisfaction). The results showed that in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) , the applications met the required usability criteria. The authors in Gokhale et al. (2017) presented a braille-based solution to help people with visual impairments call and save contacts. A braille keypad on the smartphone touchscreen was used to gain input from the user, which was then converted into haptic and auditory feedback to let the user know what action was taken. The usability of this application was considered before it was designed. The participants’ responses were positive because this kind of user-centric design simplifies navigation and learning processes.

2- Applications to enhance the independence of people with visual impairments

The authors in the studies explored in this section focused on building applications that enhance independence and autonomy in daily life activities for users with visual impairments.

In Vitiello et al. (2018) , the authors presented their mobile application, an assistive solution for visually impaired users called “Crania”, which uses machine learning techniques to help users with visual impairments get dressed by recognizing the colour and texture of their clothing and suggesting suitable combinations. The system provides feedback through voice synthesis. The participants in the study were adults and elderly people, some of whom were completely blind and the rest of whom had partial sight. After testing for usability, all the participants with blindness agreed that using the application was better than their original method, and half of the participants with partial sight said the same thing. At the end of the study, the application was determined to be accessible and easy to use.

In Kunaratana-Angkul, Wu & Shin-Renn (2020) , an application which allows elderly people to measure low vision status at home through their smartphones instead of visiting hospitals was tested, and most of the participants considered it to be untrustworthy because the medical information was insufficient. Even when participants were able to learn how to use the application, most of them were still confused while using it and needed further instruction.

In Ghidini et al. (2016) , the authors studied the habits of people with visual impairments when using their smartphones in order to develop an electronic calendar with different interaction formats, such as voice commands, touch, and vibration interaction. The authors presented the lessons learned and categorized them based on usability heuristics such as feedback, design, user freedom and control, and recognition instead of remembering.

In Madrigal-Cadavid et al. (2019) , the authors developed a drug information application for people with visual impairments to help them access the labels of medications. The application was developed based on a user-centered design process. By conducting a usability test, the authors recognized some usability issues for people with visual impairments, such as difficulty in locating the bar code. Given this, a new version will include a search function that is based on pictures. The application is searched by capturing the bar code or text or giving voice commands that allow the user to access medication information. The participants were people with visual impairments, and most of them required assistance using medications before using the application. This application will enhance independence for people with visual impairments in terms of using medications.

In Marques, Carriço & Guerreiro (2015) , an authentication method is proposed for users with visual impairments that allows them to protect their passwords. It is not secure when blind or visually impaired users spell out their passwords or enter the numbers in front of others, and the proposed solution allows the users to enter their password with one hand by tapping the screen. The blind participants in this study demonstrated that this authentication method is usable and supports their security needs.

In Oliveira et al. (2018) , the author noted that people with visual impairments face challenges in reading, thus he proposed an application called LeR otulos. This application was developed and evaluated for the Android operating system and recognizes text from photos taken by the mobile camera and converts them into an audio description. The prototype was designed to follow the guidelines and recommendations of usability and accessibility. The requirements of the application are defined based on the following usability goals: the steps are easy for the user to remember; the application is efficient, safe, useful, and accessible; and user satisfaction is achieved.

Interacting with talkback audio devices is still difficult for people with blindness, and it is unclear how much benefit they provide to people with visual impairments in their daily activities. The author in Rodrigues et al. (2015) investigates the smartphone adoption process of blind users by conducting experiments, observations, and weekly interviews. An eight-week study was conducted with five visually impaired participants using Samsung and an enabled talkback 2 screen reader. Focusing on understanding the experiences of people with visual impairments when using touchscreen smartphones revealed accessibility and usability issues. The results showed that the participants have difficulties using smartphones because they fear that they cannot use them properly, and that impacts their ability to communicate with family. However, they appreciate the benefits of using smartphones in their daily activities, and they have the ability to use them.

People with visual impairments encounter various challenges identifying and locating non-visual items on touch screen interfaces, such as phones and tablets. Various specifications for developing a user interface for people with visual impairments must be met, such as having touch screen division to enable people with blindness to easily and comfortably locate objects and items that are non-visual on the screen ( Khusro et al., 2019 ). Article ( Khusro et al., 2019 ) highlighted the importance of aspects of the usability analysis, such as screen partitioning, to meet specific usability requirements, including orientation, consistency, operation, time consumption, and navigation complexity when users want to locate objects on their touchscreen. The authors of Khan & Khusro (2019) describe the improvements that people with blindness have experienced in using the smartphone while performing their daily tasks. This information was determined through an empirical study with 41 people with blindness who explained their user and interaction experiences operating a smartphone.

The authors in Palani et al. (2018) provide design guidelines governing the accurate display of haptically perceived graphical materials. Determining the usability parameters and the various cognitive abilities required for optimum and accurate use of device interfaces is crucial. Also the authors of Grussenmeyer & Folmer (2017) highlight the importance of usability and accessibility of smartphones and touch screens for people with visual impairments. The primary focus in Leporini & Palmucci (2018) is on interactive tasks used to finish exercises and to answer questionnaires or quizzes. These tools are used for evaluation tests or in games. When using gestures and screen readers to interact on a mobile device, difficulties may arise ( Leporini & Palmucci, 2018 ), The study has various objectives, including gathering information on the difficulties encountered by people with blindness during interactions with mobile touch screen devices to answer questions and investigating practicable solutions to solve the detected accessibility and usability issues. A mobile app with an educational game was used to apply the proposed approach. Moreover, in Alnfiai & Sampalli (2016) and Niazi et al. (2016) , an analysis of the single-tap braille keyboard created to help people with no or low vision while using touch screen smartphones was conducted. The technology used in Alnfiai & Sampalli (2016) was the talkback service, which provides the user with verbal feedback from the application, allowing users with blindness to key in characters according to braille patterns. To evaluate single tap braille, it was compared to the commonly used QWERTY keyboard. In Niazi et al. (2016) , it was found that participants adapted quickly to single-tap Braille and were able to type on the touch screen within 15 to 20 min of being introduced to this system. The main advantage of single tap braille is that it allows users with blindness to enter letters based on braille coding, which they are already familiar with. The average error rate is lower using single-tap Braille than it is on the QWERTY keyboard. The authors of Niazi et al. (2016) found that minimal typing errors were made using the proposed keypad, which made it an easier option for people with blindness ( Niazi et al., 2016 ). In Cho & Kim (2017) , the authors describe new text entry methods for the braille system including a left touch and a double touch scheme that form a two-button interface for braille input so that people with visual impairments are able to type textual characters without having to move their fingers to locate the target buttons.

One of the main problems affecting the visually impaired is limited mobility for some gestures. We need to know what gestures are usable by people with visual impairments. Moreover, the technology of assistive touchscreen-enabled speech interaction between blind people and mobile devices permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities to perform actions or interact with systems. Smaradottir, Martinez & Haland (2017) analyses a voiceover screen reader used in Apple Inc.’s products. An assessment of this assistive technology was conducted with six visually impaired test participants. The main objectives were to pinpoint the difficulties related to the performance of gestures applicable in screen interactions and to analyze the system’s response to the gestures. In this study, a user evaluation was completed in three phases. The first phase entailed training users regarding different hand gestures, the second phase was carried out in a usability laboratory where participants were familiarized with technological devices, and the third phase required participants to solve different tasks. In Knutas et al. (2015) , the vital feature of the system is that it enables the user to interactively select a 3D scene region for sonification by merely touching the phone screen. It uses three different modes to increase usability. Alnfiai & Sampalli (2017) explained a study done to compare the use of two data input methods to evaluate their efficiency with completely blind participants who had prior knowledge of braille. The comparison was made between the braille enter input method that uses gestures and the swift braille keyboard, which necessitates finding six buttons representing braille dots. Blind people typically prefer rounded shapes to angular ones when performing complex gestures, as they experience difficulties performing straight gestures with right angles. Participants highlighted that they experienced difficulties particularly with gestures that have steep or right angles. In Buzzi et al. (2017) , 36 visually impaired participants were selected and split into two groups of low-vision and blind people. They examined their touch-based gesture preferences in terms of the number of strokes, multitouch, and shape angles. For this reason, a wireless system was created to record sample gestures from various participants simultaneously while monitoring the capture process.

People with visual impairment typically cannot travel without guidance due to the inaccuracy of current navigation systems in describing roads and especially sidewalks. Thus, the author of Gintner et al. (2017) aims to design a system to guide people with visual impairments based on geographical features and addresses them through a user interface that converts text to audio using a built-in voiceover engine (Apple iOS). The system was evaluated positively in terms of accessibility and usability as tested in a qualitative study involving six participants with visual impairment.

Based on challenges faced by visually impaired game developers, Ara’ujo et al. (2017) provides guidance for developers to provide accessibility in digital games by using audio guidance for players with visual impairments. The interactions of the player can be conveyed through audio and other basic mobile device components with criteria focused on the game level and speed adjustments, high contrast interfaces, accessible menus, and friendly design. Without braille, people with visual impairments cannot read, but braille is expensive and takes effort, and so it is important to propose technology to facilitate reading for them. In Sabab & Ashmafee (2016) , the author proposed developing a mobile application called “Blind Reader” that reads an audio document and allows the user to interact with the application to gain knowledge. This application was evaluated with 11 participants, and the participants were satisfied with the application. Videos are an important form of digital media, and unfortunately people with visual impairment cannot access these videos. Therefore, Façanha et al. (2016) aims to discover sound synthesis techniques to maximize and accelerate the production of audio descriptions with low-cost phonetic description tools. This tool has been evaluated based on usability with eight people and resulted in a high acceptance rate among users.

1- Indoor navigation

Visually impaired people face critical problems when navigating from one place to another. Whether indoors or outdoors, they tend to stay in one place to avoid the risk of injury or seek the help of a sighted person before moving ( Al-Khalifa & Al-Razgan, 2016 ). Thus, aid in navigation is essential for those individuals. In Nair et al. (2020) , Nair developed an application called ASSIST, which leverages Bluetooth low energy (BLE) beacons and augmented reality (AR) to help visually impaired people move around cluttered indoor places ( e.g. , subways) and provide the needed safe guidance, just like having a sighted person lead the way. In the subway example, the beacons will be distributed across the halls of the subway and the application will detect them. Sensors and cameras attached to the individual will detect their exact location and send the data to the application. The application will then give a sequence of audio feedback explaining how to move around the place to reach a specific point ( e.g. , “in 50 ft turn right”, “now turn left”, “you will reach the destination in 20 steps”). The application also has an interface for sighted and low-vision users that shows the next steps and instructions. A usability study was conducted to test different aspects of the proposed solution. The majority of the participants agreed that they could easily reach a specified location using the application without the help of a sighted person. A survey conducted to give suggestions from the participants for future improvements showed that most participants wanted to attach their phones to their bodies and for the application to consider the different walking speeds of users. They were happy with the audio and vibration feedback that was given before each step or turn they had to take.

In Al-Khalifa & Al-Razgan (2016) , the main purpose of the study was to provide an Arabic-language application for guidance inside buildings using Google Glass and an associated mobile application. First, the building plan must be set by a sighted person who configures the different locations needed. Ebsar will ask the map builder to mark each interesting location with a QR code and generate a room number, and the required steps and turns are tracked using the mobile device’s built-in compass and accelerometer features. All of these are recorded in the application for the use of a visually impaired individual, and at the end, a full map is generated for the building. After setting the building map, a user can navigate inside the building with the help of Ebsar, paired with Google Glass, for input and output purposes. The efficiency, effectiveness, and levels of user satisfaction with this solution were evaluated. The results showed that the errors made were few, indicating that Ebsar is highly effective. The time consumed in performing tasks ranged from medium to low depending on the task; this can be improved later. Interviews with participants indicated the application’s ease of use. De Borba Campos et al. (2015) shows an application simulating a museum map for people with visual impairments. It discusses whether mental maps and interactive games can be used by people with visual impairments to recognize the space around them. After multiple usability evaluation sessions, the mobile application showed high efficiency among participants in understanding the museum’s map without repeating the visitation. The authors make a few suggestions based on feedback from the participants regarding enhancing usability, including using audio cues, adding contextual help to realise the activities carried around in a space, and focusing on audio feedback instead of graphics.

2- Outdoor navigation

Outdoor navigation is also commonly discussed in the literature. In Darvishy et al. (2020) , Touch Explorer was presented to alleviate many of the problems faced by visually impaired people in navigation by developing a non-visual mobile digital map. The application relies on three major methods of communication with the user: voice output, vibration feedback, and everyday sounds. The prototype was developed using simple abstract visuals and mostly relies on voice for explanation of the content. Usability tests show the great impact the prototype had on the understanding of the elements of the map. Few suggestions were given by the participants to increase usability, including GPS localization to locate the user on the map, a scale element for measuring the distance between two map elements, and an address search function.

In Hossain, Qaiduzzaman & Rahman (2020) , a navigation application called Sightless Helper was developed to provide a safe navigation method for people with visual impairments. It relies on footstep counting and GPS location to provide the needed guidance. It can also ensure safe navigation by detect objects and unsafe areas and can detect unusual shaking of the user and alert an emergency contact about the problem. The user interaction categories are voice recognition, touchpad, buttons, and shaking sensors. After multiple evaluations, the application was found to be useful in different scenarios and was considered usable by people with visual impairments. The authors in Long et al. (2016) propose an application that uses both updates from users and information about the real world to help visually impaired people navigate outdoor settings. After interviews with participants, some design goals were set, including the ability to tag an obstacle on the map, check the weather, and provide an emergency service. The application was evaluated and was found to be of great benefit; users made few errors and found it easy to use. In Prerana et al. (2019) , a mobile application called STAVI was presented to help visually impaired people navigate from a source to a destination safely and avoid issues of re-routing. The application depends on voice commands and voice output. The application also has additional features, such as calling, messages, and emergency help. The authors in Bandukda et al. (2020) helped people with visual impairments explore parks and natural spaces using a framework called PLACES. Different interviews and surveys were conducted to identify the issues visually impaired people face when they want to do any leisure activity. These were considered in the development of the framework, and some design directions were presented, such as the use of audio to share an experience.

3- General issues

The authors in Maly et al. (2015) discuss implementing an evaluation model to assess the usability of a navigation application and to understand the issues of communication with mobile applications that people with visual impairments face. The evaluation tool was designed using a client–server architecture and was applied to test the usability of an existing navigation application. The tool was successful in capturing many issues related to navigation and user behavior, especially the issue of different timing between the actual voice instruction and the position of the user. The authors in Kameswaran et al. (2020) conducted a study to find out which navigation technologies blind people can use and to understand the complementarity between navigation technologies and their impact on navigation for visually impaired users. The results of the study show that visually impaired people use both assistive technologies and those designed for non-visually impaired users. Improving voice agents in navigation applications was discussed as a design implication for the visually impaired. In Skulimowski et al. (2019) , the authors show how interactive sonification can be used in simple travel aids for the blind. It uses depth images and a histogram called U-depth, which is simple auditory representations for blind users. The vital feature of this system is that it enables the user to interactively select a 3D scene region for sonification by touching the phone screen. This sonic representation of 3D scenes allows users to identify the environment’s general appearance and determine objects’ distance. The prototype structure was tested by three blind individuals who successfully performed the indoor task. Among the test scenes used included walking along an empty corridor, walking along a corridor with obstacles, and locating an opening between obstacles. However, the results showed that it took a long time for the testers to locate narrow spaces between obstacles.

RQ6: What evaluation methods were used in the studies on usability for visually impaired people that were reviewed?

The most prevalent methods to evaluate the usability of applications were surveys and interviews. These were used to determine the usability of the proposed solutions and obtain feedback and suggestions regarding additional features needed to enhance the usability from the participants’ points of view. Focus groups were also used extensively in the literature. Many of the participants selected were blindfolded and were not actually blind or visually impaired. Moreover, the samples selected for the evaluation methods mentioned above considered the age factor depending on the study’s needs.

Limitation and future work

The limitations of this paper are mainly related to the methodology followed. Focusing on just eight online databases and restricting the search with the previously specified keywords and string may have limited the number of search results. Additionally, a large number of papers were excluded because they were written in other languages. Access limitations were also faced due to some libraries asking for fees to access the papers. Therefore, for future works, a study to expand on the SLR results and reveal the current usability models of mobile applications for the visually impaired to verify the SLR results is needed so that this work contributes positively to assessing difficulties and expanding the field of usability of mobile applications for users with visual impairments.

Conclusions

In recent years, the number of applications focused on people with visual impairments has grown, which has led to positive enhancements in those people’s lives, especially if they do not have people around to assist them. In this paper, the research papers focusing on usability for visually impaired users were analyzed and classified into seven themes: accessibility, daily activities, assistive devices, gestures, navigation, screen division layout, and audio guidance. We found that various research studies focus on accessibility of mobile applications to ensure that the same user experience is available to all users, regardless of their abilities. We found many studies that focus on how the design of the applications can assist in performing daily life activities like braille-based application studies and applications to enhance the independence of VI users. We also found papers that discuss the role of assistive devices like screen readers and wearable devices in solving challenges faced by VI users and thus improving their quality of life. We also found that some research papers discuss limited mobility of some gestures for VI users and investigated ways in which we can know what gestures are usable by people with visual impairments. We found many research papers that focus on improving navigation for VI users by incorporating different output modalities like sound and vibration. We also found various studies focusing on screen division layout. By dividing the screen and focusing on visual impairment-related issues while developing user interfaces, visually impaired users can easily locate the objects and items on the screens. Finally, we found papers that focus on audio guidance to improve usability. The proposed applications use voice-over and speech interactions to guide visually impaired users in performing different activities through their mobiles. Most of the researchers focused on usability in different applications and evaluated the usability issues of these applications with visually impaired participants. Some of the studies included sighted participants to compare the number and type of problems they faced. The usability evaluation was generally based on the following criteria: accessibility, efficiency, learnability, memorability, errors, safety, and satisfaction. Many of the studied applications show a good indication of these applications’ usability and follow the participants’ comments to ensure additional enhancements in usability. This paper aims to provide an overview of the developments on usability of mobile applications for people with visual impairments and use this overview to highlight potential future directions.

References selected for each theme.

Funding Statement

This research project was supported by a grant from the Research Center of the Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

The authors declare there are no competing interests.

Muna Al-Razgan , Sarah Almoaiqel , Nuha Alrajhi , Alyah Alhumegani , Abeer Alshehri , Bashayr Alnefaie , Raghad AlKhamiss and Shahad Rushdi conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Open access
• Published: 07 May 2013

Usability of mobile applications: literature review and rationale for a new usability model

• Rachel Harrison 1 ,
• Derek Flood 1 &
• David Duce 1  

Journal of Interaction Science volume  1 , Article number:  1 ( 2013 ) Cite this article

191k Accesses

347 Citations

3 Altmetric

Metrics details

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

Introduction

Advances in mobile technology have enabled a wide range of applications to be developed that can be used by people on the move. Developers sometimes overlook the fact that users will want to interact with such devices while on the move. Small screen sizes, limited connectivity, high power consumption rates and limited input modalities are just some of the issues that arise when designing for small, portable devices. One of the biggest issues is the context in which they are used. As these devices are designed to enable users to use them while mobile, the impact that the use of these devices has on the mobility of the user is a critical factor to the success or failure of the application.

Current research has demonstrated that cognitive overload can be an important aspect of usability [ 1 , 2 ]. It seems likely that mobile devices may be particularly sensitive to the effects of cognitive overload, due to their likely deployment in multiple task settings and limitations of size. This aspect of usability is often overlooked in existing usability models, which are outlined in the next section, as these models are designed for applications which are seldom used in a mobile context. Our PACMAD usability model for mobile applications, which we then introduce, incorporates cognitive load as this attribute directly impacts and may be impacted by the usability of an application.

A literature review, outlined in the following section, was conducted as validation of the PACMAD model. This literature review examined which attributes of usability, as defined in the PACMAD usability model, were used during the evaluation of mobile applications presented in a range of papers published between 2008 and 2010. Previous work by Kjeldskov & Graham [ 3 ] has looked at the research methods used in mobile HCI, but did not examine the particular attributes of usability incorporated in the PACMAD model. We also present the results of the literature review.

The impact of this work on future usability studies and what lessons other researchers should consider when performing usability evaluations on mobile applications are also discussed.

Background and literature review

Existing models of usability.

Nielsen [ 4 ] identified five attributes of usability:

  Efficiency : Resources expended in relation to the accuracy and completeness with which users achieve goals;

  Satisfaction : Freedom from discomfort, and positive attitudes towards the use of the product.

  Learnability : The system should be easy to learn so that the user can rapidly start getting work done with the system;

  Memorability : The system should be easy to remember so that the casual user is able to return to the system after some period of not having used it without having to learn everything all over again;

  Errors : The system should have a low error rate, so that users make few errors during the use of the system and that if they do make errors they can easily recover from them. Further, catastrophic errors must not occur.

In addition to this Nielsen defines Utility as the ability of a system to meet the needs of the user. He does not consider this to be part of usability but a separate attribute of a system. If a product fails to provide utility then it does not offer the features and functions required; the usability of the product becomes superfluous as it will not allow the user to achieve their goals. Likewise, the International Organization for Standardization (ISO) defined usability as the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [ 5 ]. This definition identifies 3 factors that should be considered when evaluating usability.

  User : Person who interacts with the product;

  Goal : Intended outcome;

  Context of use : Users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a product is used.

Each of the above factors may have an impact on the overall design of the product and in particular will affect how the user will interact with the system. In order to measure how usable a system is, the ISO standard outlines three measurable attributes:

  Effectiveness : Accuracy and completeness with which users achieve specified goals;

Unlike Nielsen’s model of usability, the ISO standard does not consider Learnability, Memorability and Errors to be attributes of a product’s usability although it could be argued that they are included implicitly within the definitions of Effectiveness, Efficiency and Satisfaction. For example, error rates can be argued to have a direct effect on efficiency.

Limitations for mobile applications

The models presented above were largely derived from traditional desktop applications. For example, Nielsen’s work was largely based on the design of telecoms systems, rather than computer software. The advent of mobile devices has presented new usability challenges that are difficult to model using traditional models of usability. Zhang and Adipat [ 6 ] highlighted a number of issues that have been introduced by the advent of mobile devices:

  Mobile Context : When using mobile applications the user is not tied to a single location. They may also be interacting with nearby people, objects and environmental elements which may distract their attention.

  Connectivity : Connectivity is often slow and unreliable on mobile devices. This will impact the performance of mobile applications that utilize these features.

  Small Screen Size : In order to provide portability mobile devices contain very limited screen size and so the amount of information that can be displayed is limited.

  Different Display Resolution : The resolution of mobile devices is reduced from that of desktop computers resulting in lower quality images.

  Limited Processing Capability and Power : In order to provide portability, mobile devices often contain less processing capability and power. This will limit the type of applications that are suitable for mobile devices.

  Data Entry Methods : The input methods available for mobile devices are different from those for desktop computers and require a certain level of proficiency. This problem increases the likelihood of erroneous input and decreases the rate of data entry.

From our review it is apparent that many existing models for usability do not consider mobility and its consequences, such as additional cognitive load. This complicates the job of the usability practitioner, who must consequently define their task model to explicitly include mobility. One might argue that the lack of reference to a particular context could be a strength of a usability model provided that the usability practitioner has the initiative and knows how to modify the model for a particular context.

The PACMAD usability model aims to address some of the shortcomings of existing usability models when applied to mobile applications. This model builds on existing theories of usability but is tailored specifically for applications that can be used on mobile devices. The PACMAD usability model is depicted in Figure  1 side by side with Nielsen’s and the ISO’s definition of usability. The PACMAD usability model incorporates the attributes of both the ISO standard and Nielsen’s model and also introduces the attribute of cognitive load which is of particular importance to mobile applications. The following section introduces the PACMAD usability model and describes in detail each of the attributes of usability mentioned below as well as the three usability factors that are part of this model: user, task and context.

Comparison of usability models.

The PACMAD usability model for mobile applications identifies three factors (User, Task and Context of use) that should be considered when designing mobile applications that are usable. Each of these factors will impact the final design of the interface for the mobile application. In addition to this the model also identifies seven attributes that can be used to define metrics to measure the usability of an application. The following section outlines each of these factors and attributes in more detail.

Factors of usability

The PACMAD usability model identifies three factors which can affect the overall usability of a mobile application: User , Task and Context of use . Existing usability models such as those proposed by the ISO [ 5 ] and Nielsen [ 4 ] also recognise these factors as being critical to the successful usability of an application. For mobile applications Context of use plays a critical role as an application may be used in multiple, very different contexts.

User It is important to consider the end user of an application during the development process. As mobile applications are usually designed to be small, the traditional input methods, such as a keyboard and mouse, are no longer practical. It is therefore necessary for application designers to look at alternative input methods. Some users may find it difficult to use some of these methods due to physical limitations. For example it has been shown [ 7 ] that some Tetraplegic users who have limited mobility in their upper extremities tend to have high error rates when using touch screens and this may cause unacceptable difficulties with certain (usually small) size targets.

Another factor that should be considered is the user’s previous experience. If a user is an expert at the chosen task then they are likely to favour shortcut keys to accomplish this task. On the other hand novice users may prefer an interface that is intuitive and easy to navigate and which allows them to discover what they need. This trade-off must be considered during the design of the application.

Task The word task refers here to the goal the user is trying to accomplish with the mobile application. During the development of applications, additional features can be added to an application in order to allow the user to accomplish more with the software. This extra functionality comes at the expense of usability as these additional features increase the complexity of the software and therefore the user’s original goal can become difficult to accomplish.

For example, consider a digital camera. If a user wants to take a photograph, they must first select between different modes (e.g. video, stills, action, playback, etc.) and then begin to line up the shot. This problem is further compounded if the user needs to take a photograph at night and needs to search through a number of menu items to locate and turn on a flashlight.

Context of use The word context refers here to the environment in which the user will use the application. We want to be able to view context separately from both the user and the task. Context not only refers to a physical location but also includes other features such as the user’s interaction with other people or objects (e.g. a motor vehicle) and other tasks the user may be trying to accomplish. Research has shown that using mobile applications while walking can slow down the walker’s average walking speed [ 8 ]. As mobile applications can be used while performing other tasks it is important to consider the impact of using the mobile application in the appropriate context.

Attributes of usability

The PACMAD usability model identifies 7 attributes which reflect the usability of an application: Effectiveness , Efficiency , Satisfaction , Learnability , Memorability , Errors and Cognitive load . Each of these attributes has an impact on the overall usability of the application and as such can be used to help assess the usability of the application.

Effectiveness Effectiveness is the ability of a user to complete a task in a specified context. Typically effectiveness is measured by evaluating whether or not participants can complete a set of specified tasks.

Efficiency Efficiency is the ability of the user to complete their task with speed and accuracy. This attribute reflects the productivity of a user while using the application. Efficiency can be measured in a number of ways, such as the time to complete a given task, or the number of keystrokes required to complete a given task.

Satisfaction Satisfaction is the perceived level of comfort and pleasantness afforded to the user through the use of the software. This is reflected in the attitudes of the user towards the software. This is usually measured subjectively and varies between individual users. Questionnaires and other qualitative techniques are typically used to measure a user’s attitudes towards a software application.

Learnability A recent survey of mobile application users [ 9 ] found that users will spend on average 5 minutes or less learning to use a mobile application. There are a large number of applications available on mobile platforms and so if users are unable to use an application they may simply select a different one. For this reason the PACMAD model includes the attribute Learnability as suggested by Nielsen.

Learnability is the ease with which a user can gain proficiency with an application. It typically reflects how long it takes a person to be able to use the application effectively. In order to measure Learnability, researchers may look at the performance of participants during a series of tasks, and measure how long it takes these participants to reach a pre-specified level of proficiency.

Memorability The survey also found that mobile applications are used on an infrequent basis and that participants used almost 50% of the applications only once a month [ 9 ]. Thus there may be a large period of inactivity between uses and so participants may not easily recall how to use the application. Consequently the PACMAD usability model includes the attribute of Memorability as also suggested by Nielsen.

Memorability is the ability of a user to retain how to use an application effectively. Software might not be used on a regular basis and sometimes may only be used sporadically. It is therefore necessary for users to remember how to use the software without the need to relearn it after a period of inactivity. Memorability can be measured by asking participants to perform a series of tasks after having become proficient with the use of the software and then asking them to perform similar tasks after a period of inactivity. A comparison can then be made between the two sets of results to determine how memorable the application was.

Errors The PACMAD usability model extends the description of Errors, first proposed by Nielsen, to include an evaluation of the errors that are made by participants while using mobile apps. This allows developers to identify the most troublesome areas for users and to improve these areas in subsequent iterations of development. This attribute is used to reflect how well the user can complete the desired tasks without errors. Nielsen [ 4 ] states that users should make few errors during the use of a system and that if they do make errors they should be able to easily recover from them. The error rate of users may be used to infer the simplicity of a system. The PACMAD usability model considers the nature of errors as well as the frequency with which they occur. By understanding the nature of these errors it is possible to prevent these errors from occurring in future versions of the application.

Cognitive load The main contribution of the PACMAD model is its inclusion of Cognitive Load as an attribute of usability. Unlike traditional desktop applications, users of mobile applications may be performing additional tasks, such as walking, while using the mobile device. For this reason it is important to consider the impact that using the mobile device will have on the performance of the user of these additional tasks. For example a user may wish to send a text message while walking. In this case the user’s walking speed will be reduced as they are concentrating on sending the message which is distracting them from walking.

Cognitive load refers to the amount of cognitive processing required by the user to use the application. In traditional usability studies a common assumption is that the user is performing only a single task and can therefore concentrate completely on that task. In a mobile context users will often be performing a second action in addition to using the mobile application [ 8 , 10 ]. For example a user may be using a stereo while simultaneously driving a car. In this scenario it is important that the cognitive load required by the mobile application, in this case the stereo, does not adversely impact the primary task.

While the user is using the application in a mobile context it will impact both the user’s ability to move and to operate the mobile application. Therefore it is important to consider both dimensions when studying the usability of mobile applications. One way this can be measured is through the NASA Task Load Index (TLX) [ 11 ]. This is a subjective workload assessment tool for measuring the cognitive workload placed on a user by the use of a system. In this paper we adopt a relatively simple view of cognitive load. For a more accurate assessment it may be preferable to adopt a more powerful multi-factorial approach [ 1 , 12 ] but this is beyond the scope of this paper.

Literature review

In order to evaluate the appropriateness and timeliness of the PACMAD usability model for mobile applications, a literature review was conducted to review current approaches and to determine the need for a comprehensive model that includes cognitive load. We focused on papers published between 2008 and 2010 which included an evaluation of the usability of a mobile application.

Performing the literature review

The first step in the literature review was to collect all of the publications from the identified sources. These sources were identified by searching the ACM digital library, IEEE digital library and Google Scholar. The search strings used during these searches were “ Mobile Application Evaluations ”, “ Usability of mobile applications ” and “ Mobile application usability evaluations ”. The following conferences and journals were identified as being the most relevant sources: the Mobile HCI conference (MobileHCI), the International Journal of Mobile Human Computer Interaction (IJMHCI), the ACM Transactions on Computer-Human Interaction (TOCHI), the International Journal of Human Computer Studies (IJHCS), the Personal and Ubiquitous Computing journal (PUC), and the International Journal of Human-Computer Interaction (IJHCI). We also considered the ACM Conference on Human Factors in Computing Systems (CHI) and the IEEE Transactions on Mobile Computing (IEEE TOMC). These sources were later discarded as very few papers (less than 5% of the total) were relevant.

The literature review was limited to the publications between the years 2008 and 2010 due to the emergence of smart phones during this time. Table  1 shows the number of publications that were examined from each source.

The sources presented above included a number of different types of publications (Full papers, short papers, doctoral consortium, editorials, etc.). We focused the study only on full or short research papers from peer reviewed sources. This approach was also adopted by Budgen et al. [ 13 ]. Table  2 shows the number of remaining publications by source.

The abstract of each of the remaining papers was examined to determine if the paper:

Conducted an evaluation of a mobile application/device;

Contained some software component with which the users interact;

Conducted an evaluation which was focused on the interaction with the application or device;

Publications which did not meet the above criteria were removed.

The following exclusion criteria were u sed to exclude papers:

Focused only on application development methodologies and techniques;

Contained only physical interaction without a software component;

Examined only social aspects of using mobile applications;

Did not consider mobile applications.

Each abstract was reviewed by the first two authors to determine if it should be included within the literature review. When a disagreement arose between the reviewers it was discussed until mutual agreement was reached. A small number of relevant publications were unavailable to the authors. Table  3 shows the number of papers included within the literature review by source.

Each of the remaining papers was examined by one reviewer (either the first or second author of this paper). The reviewer examined each paper in detail and identified for each one:

 The attribute of usability that could be measured through the collected metrics;

 The focus of the research presented.

 The type of study conducted;

To ensure the quality of the data extraction performed the first and second author independently reviewed a 10% sample and compared these results. When a disagreement arose it was discussed until an agreement was reached.

Twenty papers that were identified as being relevant did not contain any formal evaluations of the proposed technologies. The results presented below exclude these 20 papers. In addition to this some papers presented multiple studies. In these cases each study was considered independently and so the results based on the number of studies within the evaluated papers rather than the number of papers.

Limitations

This literature review is limited for a number of reasons. Firstly a small number of papers were unavailable to the researchers (8 out of 139 papers considered relevant). This unavailability of less than 6% of the papers probably does not have a large impact on the results presented. By omitting certain sources from the study a bias may have been introduced. We felt that the range of sources considered was a fair representation of the field of usability of mobile applications although some outlying studies may have been omitted due to limited resources. Our reviews of these sources led us to believe that the omitted papers were of borderline significance. Ethical approval for this research was given by Oxford Brookes University Research Ethics Committee.

Research questions

To evaluate the PACMAD usability model three Research Questions (RQ1 to RQ3) were established to determine how important each of the factors and attributes of usability are in the context of mobile applications.

RQ1: What attributes are used when considering the usability of mobile applications?

This research question was established to discover what attributes are typically used to analyse mobile applications and which metrics are associated with them. The answers to this question provide evidence and data for the PACMAD usability model.

RQ2: To what extent are the factors of usability considered in existing research?

In order to determine how research in mobile applications is evolving, RQ2 was established to examine the current research trends into mobile applications, with a particular focus on the factors that affect usability.

In addition to this we wanted to establish which research methods are most commonly used when evaluating mobile applications. For this reason, a third research question was established.

RQ3: What research methodologies are used to evaluate the usability of mobile applications?

There are many ways in which mobile applications can be evaluated including controlled studies, field studies, ethnography, experiments, case-studies, surveys, etc. This research question aims to identify the most common research methodologies used to evaluate mobile apps. The answers to this question will throw light on the maturity of the mobile app engineering field.

The above research questions were answered by examining the literature on mobile applications. The range of literature on the topic of mobile applications is so broad it was important to limit the literature review to the most relevant and recent publications and to limit the publication interval to papers published between 2008 and 2010.

Table  4 shows the percentage of studies that include metrics, such as time to complete a given task, which either directly or indirectly assesses the attributes of usability included within the PACMAD usability model. In some cases the studies evaluated multiple attributes of usability and therefore the results above present both the percentage and the number of studies in which each attribute was considered. These studies often do not explicitly cite usability or any usability related criteria, and so the metrics used for the papers’ analyses were used to discover the usability attributes considered. This lack of precision is probably due to a lack of agreement as to what constitutes usability and the fact that the attributes are not orthogonal. The three most common attributes, Effectiveness, Efficiency and Satisfaction, correspond to the attributes identified by the ISO’s standard for usability.

One of the reasons these attributes are so widely considered is their direct relationship to the technical capabilities of the system. Both Effectiveness and Efficiency are related to the design and implementation of the system and so are usually tested thoroughly. These attributes are also relatively easy to measure. In most cases the Effectiveness of the system is evaluated by monitoring whether a user can accomplish a pre-specified task. Efficiency can be measured by finding the time taken by the participant to complete this task. Questionnaires and structured interviews can be used to determine the Satisfaction of users towards the system. Approximately 22% of the papers reviewed evaluated all three of these attributes.

The focus on these attributes of usability implies that Learnability, Memorability, Errors, and Cognitive load, are considered to be of less importance than Effectiveness, Efficiency and Satisfaction. Learnability, Memorability, Errors, and Cognitive load are not easy to evaluate and this may be why their assessment is often overlooked. As technology matures designers have begun to consider usability earlier in the design process. This is reflected to a certain extent by technological changes away from command line towards GUI based interfaces.

The aspects of usability that were considered least often in the papers reviewed are Learnability and Memorability. There are numerous reasons for this. The nature of these attributes demands that they are evaluated over periods of time. To effectively measure Learnability, users’ progress needs to be checked at regular intervals or tracked over many completions of a task. In the papers reviewed, Learnability was usually measured indirectly by the changes in effectiveness or efficiency over many completions of a specified task.

Memorability was only measured subjectively in the papers reviewed. One way to objectively measure Memorability is to examine participants’ use of the system after a period of inactivity with the system. The practical problem of recruiting participants who are willing to return multiple times to participate in an evaluation is probably one of the reasons why this attribute is not often measured objectively.

What differentiates mobile applications from more traditional applications is the ability of the user to use the application while moving. In this context, the users’ attention is divided between the act of moving and using the application. About 26% of the studies considered cognitive load. Some of these studies used the change in performance of the user performing the primary task (which was usually walking or driving) as an indication of the cognitive load. Other studies used the NASA TLX [ 11 ] to subjectively measure cognitive load.

Table  5 shows the current research trends within mobile application research. It can be seen that the majority of work is focused on a task approximately 47% of the papers reviewed focus on allowing users to complete a specific task. The range of tasks considered is too broad to provide a detailed description and so we present here only some of the most dominant trends seen within the literature review.

The integration of cameras into mobile devices has enabled the emergence of a new class of application for mobile devices known as augmented reality. For example Bruns and Bimber [ 14 ] have developed an augmented reality application which allows users to take a photograph of an exhibit at an art gallery which allows the system to find additional information about the work of art. Similar systems have also been developed for Points of Interest (POIs) for tourists [ 15 ].

While using maps is a traditional way of navigating to a destination, mobile devices incorporating GPS (Global Positioning Satellite) technology have enabled researchers to investigate new ways of helping users to navigate. A number of systems [ 16 , 17 ] have proposed the use of tactile feedback to help guide users. Through the use of different vibration techniques the system informs users whether they should turn left, right or keep going straight. Another alternative to this is the use of sound. By altering the spatial balance and volume of a user’s music, Jones et al. [ 18 ] have developed a system for helping guide users to their destination.

One of the biggest limitations to mobile devices is the limited input modalities. Developers of apps do not have a large amount of space for physical buttons and therefore researchers are investigating other methods of interaction. This type of research accounts for approximately 29% of the studies reviewed.

The small screen size found on mobile applications has meant that only a small fraction of a document can be seen in detail. When mobile devices are used navigating between locations, this restriction can cause difficulty for users. In an effort to address this issue Burigat et al. [ 19 ] have developed a Zoomable User Interface with Overview (ZUIO). This interface allows a user to zoom into small sections of a document, such as a map, while displaying a small scale overview of the entire document so that the user can see where on the overall document they are. This type of system can also be used with large documents, such as web pages and images.

Audio interfaces [ 20 ] are a type of interface that is being investigated to assist drivers to use in-car systems. Traditional interfaces present information to users by visual means, but for drivers this distraction has safety critical implications. To address this issue audio inputs are common for in-vehicle systems. The low quality of voice recognition technology can limit its effectiveness within this context. Weinberg et al. [ 21 ] have shown that multiple push-to-talk buttons can improve the performance of users of such systems. Other types of interaction paradigms in these papers include touch screens [ 22 ], pressure based input [ 23 ], spatial awareness [ 24 ] and gestures [ 25 ]. As well as using these new input modalities a number of researchers are also looking at alternative output modes such as sound [ 26 ] and tactile feedback [ 27 ].

In addition to considering the specific tasks and input modalities, a small number of researchers are investigating ways to assist specific types of users, such as those suffering from physical or psychological disabilities, to complete common tasks. This type of research accounts for approximately 9% of the evaluated papers. Approximately 8% of the papers evaluated have focused on the context in which mobile applications are being used. The remaining 6% of studies are concerned with new development and evaluation methodologies for mobile applications. These include rapid prototyping tools for in-car systems, the effectiveness of expert evaluations and the use of heuristics for evaluating mobile haptic interfaces.

RQ3 was posed to investigate how usability evaluations are currently conducted. The literature review revealed that 7 of the papers evaluated did not contain any usability evaluations. Some of the remaining papers included multiple studies to evaluate different aspects of a technology or were conducted at different times during the development process. Table  6 shows the percentage of studies that were conducted using each research methodology.

By far the most dominant research methodology used in the examined studies was controlled experiments, accounting for approximately 59% of the studies. In a controlled experiment, all variables are held constant except the independent variable, which is manipulated by the experimenter. The dependant variable is the metric which is measured by the experimenter. In this way a cause and effect relationship may be investigated between the dependant and independent variables. Causality can be inferred from the covariation of the independent and dependent variables, temporal precedence of the cause as the manipulation of the independent variable and the elimination of confounding factors though control and internal validity tests.

Although the most common approach is the use of controlled experiments, other research methodologies were also used. A number of studies evaluated the use of new technologies through field studies. Field studies are conducted in a real world context, enabling evaluators to determine how users would use a technology outside of a controlled setting. These studies often revealed issues that would not be seen in a controlled setting.

For example a system designed by Kristoffersen and Bratteberg [ 28 ] to help travellers get to and from an airport by train without the use of paper tickets was deployed. This system used a credit card as a form of ticket for a journey to or from the airport. During the field study a number of usability issues were experienced by travellers. One user wanted to use a card to buy a ticket for himself and a companion; the system did not include this functionality as the developers of the system had assumed each user would have their own credit card and therefore designed the system to issue each ticket on a different credit card.

The evaluation also revealed issues relating to how the developers had implemented the different journey types, i.e. to and from the airport. When travelling to the airport users are required to swipe their credit card at the beginning and end of each journey, whereas when returning from the airport the user only needs to swipe their card when leaving the airport. One user found this out after he had swiped his card to terminate a journey from the airport, but was instead charged for a second ticket to the airport.

Although controlled experiments and field studies account for almost 90% of the studies, other strategies are also used. Surveys were used to better understand how the public reacted to mobile systems. Some of these studies were specific to a new technology or paradigm, [ 29 ] while others considered uses such as working while on the move [ 30 ]. In two cases (1% of the studies) archival research was used to investigate a particular phenomena relating to mobile technologies. A study conducted by Fehnert and Kosagowsky [ 31 ] used archival research to investigate the relationship between expert evaluations of user experience quality of mobile phones and subsequent usage figures. Lacroix et al. [ 32 ] used archival research to investigate the relationship between goal difficulty and performance within the context of an on-going activity intervention program.

In some cases it was found that no formal evaluation was conducted but instead the new technology presented in the paper was evaluated informally with colleagues of the developers. These evaluations typically contained a small number of participants and provide anecdotal evidence of a system’s usability.

The results obtained during the literature review reinforced the importance of cognitive load as an attribute of usability. It was found that almost 23% of the studies measured the cognitive load of the application under evaluation. These results show that current researchers in the area of mobile applications are beginning to recognise the importance of cognitive load in this domain and as such there is sufficient evidence for including it within the PACMAD model of usability.

The results also show that Memorability is not considered an important aspect of usability by many researchers. Only 2% of the studies evaluated Memorability. If an application is easy to learn then users may be willing to relearn how to use the application and therefore Memorability may indeed not be significant. On the other hand, some applications have a high learning curve and as such require a significant amount of time to learn. For these applications Memorability is an important attribute.

The trade-off between Learnability and Memorability is a consideration for application developers. Factors such as the task to be accomplished and the characteristics of the user should be considered when making this decision. The PACMAD model recommends that both factors should be considered although it also recognises that it may be adequate to evaluate only one of these factors depending on the application under evaluation. The literature review has also shown that the remaining attributes of usability are considered extensively by current research. Effectiveness, Efficiency and Satisfaction were included in over 50% of the studies. It was also found the Errors were evaluated in over 30% of these studies.

When considering the factors that can affect usability, it was found that the task is the most dominant factor being researched. Over 45% of the papers examined focused primarily on allowing a user to accomplish a task. When the interaction with an application is itself considered as a task this figure rises to approximately 75%. Context of use and the User were considered in less than 10% of the papers. Context of use can vary enormously and so should be considered an important factor of usability [ 5 , 33 ]. Our results indicate that context is not extensively researched and this suggests a gap in the literature.

It was revealing that some components of the PACMAD model occur only infrequently in the literature. As mentioned above Learnability and Memorability are rarely investigated, perhaps suggesting that researchers expected users to be able to learn to use apps without much difficulty., This finding could also be due to the difficulty of finding suitable subjects willing to undergo experiments on these attributes or the lack of standard research methods for these attributes. Effectiveness, Efficiency, Satisfaction and Errors were investigated more frequently, possibly because these attributes are widely recognised as important, and also possibly because research methods for investigating these attributes are well understood and documented. Almost a quarter of the studies investigated discussed Cognitive Load. It is surprising that this figure is not higher although this could again be due to the lack of a well-defined research methodology for investigating this attribute.

Conclusions

The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services that they provide. The small size of mobile devices has limited the ways in which users can interact with them. Issues such as the small screen size, poor connectivity and limited input modalities have an effect on the usability of mobile applications.

The prominent models of usability do not adequately capture the complexities of interacting with applications on a mobile platform. For this reason, this paper presents our PACMAD usability model which augments existing usability models within the context of mobile applications.

To prove the concept of this model a literature review has been conducted. This review has highlighted the extent to which the attributes of the PACMAD model are considered within the mobile application domain. It was found that each attribute was considered in at least 20% of studies, with the exception of Memorability. It is believed one reason for this may be the difficulty associated with evaluating Memorability.

The literature review has also revealed a number of novel interaction methods that are being researched at present, such as spatial awareness and pressure based input. These techniques are in their infancy but with time and more research they may eventually be adopted.

Appendix A: Papers used in the literature review

Apitz, G., F. Guimbretière, and S. Zhai, Foundations for designing and evaluating user interfaces based on the crossing paradigm. ACM Trans. Comput.-Hum. Interact., 2008. 17(2): p. 1–42.

Arning, K. and M. Ziefle, Ask and You Will Receive: Training Novice Adults to use a PDA in an Active Learning Environment. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(1): p. 21–47.

Arvanitis, T.N., et al., Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal Ubiquitous Comput., 2009. 13(3): p. 243–250.

Axtell, C., D. Hislop, and S. Whittaker, Mobile technologies in mobile spaces: Findings from the context of train travel. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 902–915.

Baber, C., et al., Mobile technology for crime scene examination. Int. J. Hum.-Comput. Stud., 2009. 67(5): p. 464–474.

Bardram, J.E., Activity-based computing for medical work in hospitals. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–36.

Bergman, J., J. Kauko, and J. Keränen, Hands on music: physical approach to interaction with digital music, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bergman, J. and J. Vainio, Interacting with the flow, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bertini, E., et al., Appropriating Heuristic Evaluation for Mobile Computing International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 20–41.

Böhmer, M. and G. Bauer, Exploiting the icon arrangement on mobile devices as information source for context-awareness, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bostr, F., et al., Capricorn - an intelligent user interface for mobile widgets, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Brewster, S.A. and M. Hughes, Pressure-based text entry for mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bruns, E. and O. Bimber, Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput., 2009. 13(2): p. 165–178.

Brush, A.J.B., et al., User experiences with activity-based navigation on mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Burigat, S., L. Chittaro, and S. Gabrielli, Navigation techniques for small-screen devices: An evaluation on maps and web pages. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 78–97.

Büring, T., J. Gerken, and H. Reiterer, Zoom interaction design for pen-operated portable devices. Int. J. Hum.-Comput. Stud., 2008. 66(8): p. 605–627.

Buttussi, F., et al., Using mobile devices to support communication between emergency medical responders and deaf people, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chen, N.Y., F. Guimbretière, and C.E. Löckenhoff, Relative role of merging and two-handed operation on command selection speed. Int. J. Hum.-Comput. Stud., 2008. 66(10): p. 729–740.

Chen, T., Y. Yesilada, and S. Harper, What input errors do you experience? Typing and pointing errors of mobile Web users. Int. J. Hum.-Comput. Stud., 2010. 68(3): p. 138–157.

Cherubini, M., et al., Text versus speech: a comparison of tagging input modalities for camera phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Chittaro, L. and A. Marassi, Supporting blind users in selecting from very long lists of items on mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chittaro, L. and D. Nadalutti, Presenting evacuation instructions on mobile devices by means of location-aware 3D virtual environments, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Clawson, J., et al., Mobiphos: a collocated-synchronous mobile photo sharing application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Cockburn, A. and C. Gutwin, A model of novice and expert navigation performance in constrained-input interfaces. ACM Trans. Comput.-Hum. Interact., 2010. 17(3): p. 1–38.

Cox, A.L., et al., Tlk or txt? Using voice input for SMS composition. Personal Ubiquitous Comput., 2008. 12(8): p. 567–588.

Crossan, A., et al., Instrumented Usability Analysis for Mobile Devices International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 1–19.

Cui, Y., et al., Linked internet UI: a mobile user interface optimized for social networking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Cummings, M.L., et al., Supporting intelligent and trustworthy maritime path planning decisions. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 616–626.

Dahl, Y. and D. Svan, A comparison of location and token-based interaction techniques for point-of-care access to medical information. Personal Ubiquitous Comput., 2008. 12(6): p. 459–478.

Dai, L., A. Sears, and R. Goldman, Shifting the focus from accuracy to recallability: A study of informal note-taking on mobile information technologies. ACM Trans. Comput.-Hum. Interact., 2009. 16(1): p. 1–46.

Decle, F. and M. Hachet, A study of direct versus planned 3D camera manipulation on touch-based mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Duh, H.B.-L., V.H.H. Chen, and C.B. Tan, Playing different games on different phones: an empirical study on mobile gaming, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Dunlop, M.D. and M.M. Masters, Investigating five key predictive text entry with combined distance and keystroke modelling. Personal Ubiquitous Comput., 2008. 12(8): p. 589–598.

Ecker, R., et al., pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Eslambolchilar, P. and R. Murray-Smith, Control centric approach in designing scrolling and zooming user interfaces. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 838–856.

Fehnert, B. and A. Kosagowsky, Measuring user experience: complementing qualitative and quantitative assessment, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Fickas, S., M. Sohlberg, and P.-F. Hung, Route-following assistance for travelers with cognitive impairments: A comparison of four prompt modes. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 876–888.

Froehlich, P., et al., Exploring the design space of Smart Horizons, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gellersen, H., et al., Supporting device discovery and spontaneous interaction with spatial references. Personal Ubiquitous Comput., 2009. 13(4): p. 255–264.

Ghiani, G., B. Leporini, and F. Patern, Vibrotactile feedback as an orientation aid for blind users of mobile guides, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gostner, R., E. Rukzio, and H. Gellersen, Usage of spatial information for selection of co-located devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Goussevskaia, O., M. Kuhn, and R. Wattenhofer, Exploring music collections on mobile devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Greaves, A. and E. Rukzio, Evaluation of picture browsing using a projector phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hachet, M., et al., Navidget for 3D interaction: Camera positioning and further uses. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 225–236.

Hall, M., E. Hoggan, and S. Brewster, T-Bars: towards tactile user interfaces for mobile touchscreens, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hang, A., E. Rukzio, and A. Greaves, Projector phone: a study of using mobile phones with integrated projector for interaction with maps, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hardy, R., et al., Mobile interaction with static and dynamic NFC-based displays, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Heikkinen, J., T. Olsson, and K. Väänänen-Vainio-Mattila, Expectations for user experience in haptic communication with mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Henze, N. and S. Boll, Evaluation of an off-screen visualization for magic lens and dynamic peephole interfaces, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Herbst, I., et al., TimeWarp: interactive time travel with a mobile mixed reality game, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hinze, A.M., C. Chang, and D.M. Nichols, Contextual queries express mobile information needs, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Hutter, H.-P., T. Müggler, and U. Jung, Augmented mobile tagging, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jones, M., et al., ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Comput., 2008. 12(7): p. 513–525.

Joshi, A., et al., Rangoli: a visual phonebook for low-literate users, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jumisko-Pyykk, S. and M.M. Hannuksela, Does context matter in quality evaluation of mobile television?, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kaasinen, E., User Acceptance of Mobile Services. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 79–97 pp.

Kaasinen, E., et al., User Experience of Mobile Internet: Analysis and Recommendations. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 4–23.

Kane, S.K., J.O. Wobbrock, and I.E. Smith, Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kang, N.E. and W.C. Yoon, Age- and experience-related user behavior differences in the use of complicated electronic devices. Int. J. Hum.-Comput. Stud., 2008. 66(6): p. 425–437.

Kanjo, E., et al., MobGeoSen: facilitating personal geosensor data collection and visualization using mobile phones. Personal Ubiquitous Comput., 2008. 12(8): p. 599–607.

Kawsar, F., E. Rukzio, and G. Kortuem, An explorative comparison of magic lens and personal projection for interacting with smart objects, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Keijzers, J., E.d. Ouden, and Y. Lu, Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kenteris, M., D. Gavalas, and D. Economou, An innovative mobile electronic tourist guide application. Personal Ubiquitous Comput., 2009. 13(2): p. 103–118.

Komninos, A. and M.D. Dunlop, A calendar based Internet content pre-caching agent for small computing devices. Personal Ubiquitous Comput., 2008. 12(7): p. 495–512.

Kratz, S., I. Brodien, and M. Rohs, Semi-automatic zooming for mobile map navigation, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Kray, C., et al., Bridging the gap between the Kodak and the Flickr generations: A novel interaction technique for collocated photo sharing. Int. J. Hum.-Comput. Stud., 2009. 67(12): p. 1060–1072.

Kristoffersen, S. and I. Bratteberg, Design ideas for IT in public spaces. Personal Ubiquitous Comput., 2010. 14(3): p. 271–286.

Lacroix, J., P. Saini, and R. Holmes, The relationship between goal difficulty and performance in the context of a physical activity intervention program, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lavie, T. and J. Meyer, Benefits and costs of adaptive user interfaces. Int. J. Hum.-Comput. Stud., 2010. 68(8): p. 508–524.

Lee, J., J. Forlizzi, and S.E. Hudson, Iterative design of MOVE: A situationally appropriate vehicle navigation system. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 198–215.

Liao, C., et al., Papiercraft: A gesture-based command system for interactive paper. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–27.

Lin, P.-C. and L.-W. Chien, The effects of gender differences on operational performance and satisfaction with car navigation systems. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 777–787.

Lindley, S.E., et al., Fixed in time and “time in motion”: mobility of vision through a SenseCam lens, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Liu, K. and R.A. Reimer, Social playlist: enabling touch points and enriching ongoing relationships through collaborative mobile music listening, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Liu, N., Y. Liu, and X. Wang, Data logging plus e-diary: towards an online evaluation approach of mobile service field trial, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Liu, Y. and K.-J. Räihä, RotaTxt: Chinese pinyin input with a rotator, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lucero, A., J. Keränen, and K. Hannu, Collaborative use of mobile phones for brainstorming, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Luff, P., et al., Swiping paper: the second hand, mundane artifacts, gesture and collaboration. Personal Ubiquitous Comput., 2010. 14(3): p. 287–299.

Mallat, N., et al., An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput., 2008. 12(1): p. 57–65.

McAdam, C., C. Pinkerton, and S.A. Brewster, Novel interfaces for digital cameras and camera phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

McDonald, D.W., et al., Proactive displays: Supporting awareness in fluid social environments. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–31.

McKnight, L. and B. Cassidy, Children’s Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 1–18.

Melto, A., et al., Evaluation of predictive text and speech inputs in a multimodal mobile route guidance application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Miyaki, T. and J. Rekimoto, GraspZoom: zooming and scrolling control model for single-handed mobile interaction, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Moustakas, K., et al., 3D content-based search using sketches. Personal Ubiquitous Comput., 2009. 13(1): p. 59–67.

Oakley, I. and J. Park, Motion marking menus: An eyes-free approach to motion input for handheld devices. Int. J. Hum.-Comput. Stud., 2009. 67(6): p. 515–532.

Oulasvirta, A., Designing mobile awareness cues, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Oulasvirta, A., S. Estlander, and A. Nurminen, Embodied interaction with a 3D versus 2D mobile map. Personal Ubiquitous Comput., 2009. 13(4): p. 303–320.

Ozok, A.A., et al., A Comparative Study Between Tablet and Laptop PCs: User Satisfaction and Preferences. International Journal of Human-Computer Interaction, 2008. 24(3): p. 329–352.

Park, Y.S., et al., Touch key design for target selection on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Peevers, G., G. Douglas, and M.A. Jack, A usability comparison of three alternative message formats for an SMS banking service. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 113–123.

Preuveneers, D. and Y. Berbers, Mobile phones assisting with health self-care: a diabetes case study, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Puikkonen, A., et al., Practices in creating videos with mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Reischach, F.v., et al., An evaluation of product review modalities for mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Reitmaier, T., N.J. Bidwell, and G. Marsden, Field testing mobile digital storytelling software in rural Kenya, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Robinson, S., P. Eslambolchilar, and M. Jones, Exploring casual point-and-tilt interactions for mobile geo-blogging. Personal and Ubiquitous Computing, 2010. 14(4): p. 363–379.

Rogers, Y., et al., Enhancing learning: a study of how mobile devices can facilitate sensemaking. Personal Ubiquitous Comput., 2010. 14(2): p. 111–124.

Rohs, M., et al., Impact of item density on the utility of visual context in magic lens interactions. Personal Ubiquitous Comput., 2009. 13(8): p. 633–646.

Sá, M.d. and L. Carriço, Lessons from early stages design of mobile applications, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sadeh, N., et al., Understanding and capturing people’s privacy policies in a mobile social networking application. Personal Ubiquitous Comput., 2009. 13(6): p. 401–412.

Salvucci, D.D., Rapid prototyping and evaluation of in-vehicle interfaces. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–33.

Salzmann, C., D. Gillet, and P. Mullhaupt, End-to-end adaptation scheme for ubiquitous remote experimentation. Personal Ubiquitous Comput., 2009. 13(3): p. 181–196.

Schildbach, B. and E. Rukzio, Investigating selection and reading performance on a mobile phone while walking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schmid, F., et al., Situated local and global orientation in mobile you-are-here maps, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schröder, S. and M. Ziefle, Making a completely icon-based menu in mobile devices to become true: a user-centered design approach for its development, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Scott, J., et al., RearType: text entry using keys on the back of a device, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Seongil, L., Mobile Internet Services from Consumers’ Perspectives. International Journal of Human-Computer Interaction, 2009. 25(5): p. 390–413.

Sharlin, E., et al., A tangible user interface for assessing cognitive mapping ability. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 269–278.

Sintoris, C., et al., MuseumScrabble: Design of a Mobile Game for Children’s Interaction with a Digitally Augmented Cultural Space. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 53–71.

Smets, N.J.J.M., et al., Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sodnik, J., et al., A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud., 2008. 66(5): p. 318–332.

Sørensen, C. and A. Al-Taitoon, Organisational usability of mobile computing-Volatility and control in mobile foreign exchange trading. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 916–929.

Stapel, J.C., Y.A.W.d. Kort, and W.A. IJsselsteijn, Sharing places: testing psychological effects of location cueing frequency and explicit vs. inferred closeness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Streefkerk, J.W., M.P.v. Esch-Bussemakers, and M.A. Neerincx, Field evaluation of a mobile location-based notification system for police officers, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Takayama, L. and C. Nass, Driver safety and information from afar: An experimental driving simulator study of wireless vs. in-car information services. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 173–184.

Takeuchi, Y. and M. Sugimoto, A user-adaptive city guide system with an unobtrusive navigation interface. Personal Ubiquitous Comput., 2009. 13(2): p. 119–132.

Tan, F.B. and J.P.C. Chou, The Relationship Between Mobile Service Quality, Perceived Technology Compatibility, and Users’ Perceived Playfulness in the Context of Mobile Information and Entertainment Services. International Journal of Human-Computer Interaction, 2008. 24(7): p. 649–671.

Taylor, C.A., N. Samuels, and J.A. Ramey, Always On: A Framework for Understanding Personal Mobile Web Motivations, Behaviors, and Contexts of Use. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 24–41.

Turunen, M., et al., User expectations and user experience with different modalities in a mobile phone controlled home entertainment system, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Vartiainen, E., Improving the User Experience of a Mobile Photo Gallery by Supporting Social Interaction International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 42–57.

Vuolle, M., et al., Developing a questionnaire for measuring mobile business service experience, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Weinberg, G., et al., Contextual push-to-talk: shortening voice dialogs to improve driving performance, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wilson, G., C. Stewart, and S.A. Brewster, Pressure-based menu selection for mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wobbrock, J.O., B.A. Myers, and H.H. Aung, The performance of hand postures in front- and back-of-device interaction for mobile computing. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 857–875.

Xiangshi, R. and Z. Xiaolei, The Optimal Size of Handwriting Character Input Boxes on PDAs. International Journal of Human-Computer Interaction, 2009. 25(8): p. 762–784.

Xu, S., et al., Development of a Dual-Modal Presentation of Texts for Small Screens. International Journal of Human-Computer Interaction, 2008. 24(8): p. 776–793.

Yong, G.J. and J.B. Suk, Development of the Conceptual Prototype for Haptic Interface on the Telematics System. International Journal of Human-Computer Interaction, 2010. 26(1): p. 22–52.

Yoo, J.-W., et al., Cocktail: Exploiting Bartenders’ Gestures for Mobile Interaction. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(3): p. 44–57.

Yoon, Y., et al., Context-aware photo selection for promoting photo consumption on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

You, Y., et al., Deploying and evaluating a mixed reality mobile treasure hunt: Snap2Play, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Yu, K., F. Tian, and K. Wang, Coupa: operation with pen linking on mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Authors’ note

This research is supported by Oxford Brookes University through the central research fund and in part by Lero - the Irish Software Engineering Research Centre ( http://www.lero.ie ) grant 10/CE/I1855.

Adams R: Decision and stress: cognition and e-accessibility in the information workplace. Springer Universal Access in the Information Society 2007, 5 (4):363–379. 10.1007/s10209-006-0061-9

Article   Google Scholar  

Adams R: Applying advanced concepts of cognitive overload and augmentation in practice; the future of overload. In Foundations of augmented cognition . 2nd edition. Edited by: Schmorrow D, Stanney KM, Reeves LM. Arlington, VA: Springer Berlin Heidelberg; 2006:223–229.

Google Scholar  

Kjeldskov J, Graham C: A review of mobile HCI research methods . Udine, Italy: 5th International Symposium, Mobile HCI 2003; 2003. September 8–11, 2003, Proceedings

Book   Google Scholar  

Nielsen J: Usability engineering. Morgan Kaufmann Pub 1994.

ISO 9241: Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs) International Standards Organisation, Geneva 1997.

Zhang D, Adipat B: Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction 2005, 18 (3):293–308. 10.1207/s15327590ijhc1803_3

Guerreiro TJV, Nicolau H, Jorge J, Gonçalves D Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. In Assessing mobile touch interfaces for tetraplegics . Lisbon, Portugal: ACM; 2010. 2010

Chapter   Google Scholar  

Schildbach B, Rukzio E Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Investigating selection and reading performance on a mobile phone while walking . Lisbon, Portugal: ACM; 2010. 2010

Flood D, Harrison R, Duce D, Iacob C: Evaluating Mobile Applications: A Spreadsheet Case Study. International Journal of Mobile Human Computer Interaction (IJMHCI) 2013, 4 (4):37–65. 10.4018/jmhci.2012100103

Salvucci DD: Predicting the effects of in-car interface use on driver performance: an integrated model approach. International Journal of Human-Computer Studies 2001, 55 (1):85–107. 10.1006/ijhc.2001.0472

Hart SG, Staveland LE: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Human mental workload 1988, 1 (3):139–183.

Flood D, Germanakos P, Harrison R, Mc Caffery F: Estimating cognitive overload in mobile applications for decision support within the medical domain . Wroclaw, Poland: 14th International conference on Enterprise Information Systems (ICEIS 2012); 2012.

Budgen D, Burn AJ, Brereton OP, Kitchenham BA, Pretorius R: (2010) Empirical evidence about the UML: a systematic literature review . Software: Practice and Experience; 2010.

Bruns E, Bimber O: Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput 2009, 13 (2):165–178. 10.1007/s00779-008-0194-3

Schinke T, Henze N, Boll S Proceedings of the 12th international conference on human computer interaction with mobile devices and services, September 07–10, 2010. In Visualization of off-screen objects in mobile augmented reality . Portugal: Lisbon; 2010.

Smets NJJM, Brake GM, Neerincx MA, Lindenberg J Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness . Amsterdam, The Netherlands: ACM; 2008.

Ghiani G, Leporini B, Patern F Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Vibrotactile feedback as an orientation aid for blind users of mobile guides . Amsterdam, The Netherlands: ACM; 2008.

Jones M, Jones S, Bradley G, Warren N, Bainbridge D, Holmes G: ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Computing 2008, 12 (7):513–525. 10.1007/s00779-007-0155-2

Burigat, S, Chittaro, L, Parlato, E, ACM In proceedings of the 10th international conference on Human computer interaction with mobile devices and services (pp. 147–156). Map, diagram, and web page navigation on mobile devices: the effectiveness of zoomable user interfaces with overviews 2008. September

Sodnik J, Dicke C, Tomaic S, Billinghurst M: A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud 2008, 66 (5):318–332. 10.1016/j.ijhcs.2007.11.001

Weinberg G, Harsham B, Forlines C, Medenica Z Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Contextual push-to-talk: shortening voice dialogs to improve driving performance . Lisbon, Portugal: ACM; 2010. 2010

Park YS, Han SH, Park J, Cho Y Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Touch key design for target selection on a mobile phone . Amsterdam, The Netherlands: ACM; 2008.

Brewster SA, Hughes M Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Pressure-based text entry for mobile devices . Bonn, Germany: ACM; 2009.

Oakley I, Park J: Motion marking menus: an eyes-free approach to motion input for handheld devices. Int J Hum.-Comput. Stud 2009, 67 (6):515–532. 10.1016/j.ijhcs.2009.02.002

Hall M, Hoggan E, Brewster S Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In T-Bars: towards tactile user interfaces for mobile touchscreens . Amsterdam, The Netherlands: ACM; 2008. 2008

McAdam C, Pinkerton C, Brewster SA Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Novel interfaces for digital cameras and camera phones . Lisbon, Portugal: ACM; 2010. 2010

Heikkinen J, Olsson T, Väänänen-Vainio-Mattila K Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Expectations for user experience in haptic communication with mobile devices . Bonn, Germany: ACM; 2009.

Kristoffersen S, Bratteberg I: Design ideas for IT in public spaces. Personal Ubiquitous Comput 2010, 14 (3):271–286. 10.1007/s00779-009-0255-2

Mallat N, Rossi M, Tuunainen VK, Oörni A: An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput 2008, 12 (1):57–65.

Axtell C, Hislop D, Whittaker S: Mobile technologies in mobile spaces: findings from the context of train travel. International Journal of Human Computer Studies 2008, 66 (12):902–915. 10.1016/j.ijhcs.2008.07.001

Fehnert B, Kosagowsky A Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In Measuring user experience: complementing qualitative and quantitative assessment . Amsterdam, The Netherlands: ACM; 2008.

Lacroix J, Saini P, Holmes R Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In The relationship between goal difficulty and performance in the context of a physical activity intervention program, . Amsterdam, The Netherlands: ACM; 2008.

Maguire M: Context of use within usability activities. International Journal of Human-Computer Studies 2001, 55 (4):453–483. 2001 10.1006/ijhc.2001.0486

Download references

Author information

Authors and affiliations.

Oxford Brookes University, Oxford, UK

Rachel Harrison, Derek Flood & David Duce

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rachel Harrison .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

DF performed the literature review, helped to propose the PACMAD model and drafted the manuscript. RH assisted the literature review, proposed the PACMAD model and drafted the limitations section. DAD helped to refine the conceptual framework and direct the research. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, rights and permissions.

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Harrison, R., Flood, D. & Duce, D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 1 , 1 (2013). https://doi.org/10.1186/2194-0827-1-1

Download citation

Received : 10 March 2013

Accepted : 10 March 2013

Published : 07 May 2013

DOI : https://doi.org/10.1186/2194-0827-1-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile Phone
• Mobile Device
• Cognitive Load
• Augmented Reality
• Usability Model

• Research article
• Open access
• Published: 10 July 2012

A Systematic Review of Healthcare Applications for Smartphones

• Abu Saleh Mohammad Mosa 1 ,
• Illhoi Yoo 1 , 2 &
• Lincoln Sheets 1 , 3  

BMC Medical Informatics and Decision Making volume  12 , Article number:  67 ( 2012 ) Cite this article

146k Accesses

704 Citations

102 Altmetric

Metrics details

Advanced mobile communications and portable computation are now combined in handheld devices called “smartphones”, which are also capable of running third-party software. The number of smartphone users is growing rapidly, including among healthcare professionals. The purpose of this study was to classify smartphone-based healthcare technologies as discussed in academic literature according to their functionalities, and summarize articles in each category.

In April 2011, MEDLINE was searched to identify articles that discussed the design, development, evaluation, or use of smartphone-based software for healthcare professionals, medical or nursing students, or patients. A total of 55 articles discussing 83 applications were selected for this study from 2,894 articles initially obtained from the MEDLINE searches.

A total of 83 applications were documented: 57 applications for healthcare professionals focusing on disease diagnosis (21), drug reference (6), medical calculators (8), literature search (6), clinical communication (3), Hospital Information System (HIS) client applications (4), medical training (2) and general healthcare applications (7); 11 applications for medical or nursing students focusing on medical education; and 15 applications for patients focusing on disease management with chronic illness (6), ENT-related (4), fall-related (3), and two other conditions (2). The disease diagnosis, drug reference, and medical calculator applications were reported as most useful by healthcare professionals and medical or nursing students.

Conclusions

Many medical applications for smartphones have been developed and widely used by health professionals and patients. The use of smartphones is getting more attention in healthcare day by day. Medical applications make smartphones useful tools in the practice of evidence-based medicine at the point of care, in addition to their use in mobile clinical communication. Also, smartphones can play a very important role in patient education, disease self-management, and remote monitoring of patients.

Peer Review reports

Recent years have seen an increased adoption of smartphones by healthcare professionals as well as the general public [ 1 – 6 ]. The smartphone is a new technology that combines mobile communication and computation in a handheld-sized device, facilitating mobile computing at the point of care. The main objective of this study is to classify the smartphone-based healthcare technologies in the literature according to their functionalities and summarize them in each category. We present a systematic literature review in this regard. To the best of our knowledge, this study is the first study for classifying and summarizing healthcare applications for smartphones in a systematic literature review format.

The healthcare system is highly mobile in nature, involving multiple clinical locations such as clinics, inpatient wards, outpatient services, emergency departments, operating theaters, intensive care units (ICUs), laboratories, etc. [ 7 – 10 ]. As such, working in the healthcare system requires extensive mobility of healthcare professionals as well as communication and collaboration among different individuals, including their colleagues and patients. Healthcare professionals mainly used pagers for mobile communication until the wide availability of cell phones in 1990s [ 11 ]. The advent of mobile Personal Digital Assistants (PDAs) during 1990s enabled healthcare professionals to organize their contacts and calendars electronically, adding another device in their pockets. The combined functionality of a pager, a cell phone and a PDA is now replaced by a single device called a “smartphone”, which is becoming very popular among healthcare professionals as well as the general public [ 12 ]. Further details on smartphones and their operating-system platforms are discussed in Appendix I. Table 1 in Appendix I illustrates an overview of OS features of smartphone platforms and Table 2 in Appendix I illustrates the support of common features by smartphone OS platforms with the availability of hardware in the device.

A systematic review summarizing 23 surveys on PDA usage by healthcare professionals (conducted in the U.S. (16 surveys), Canada (4 surveys), Australia (1 survey), both the U.S. and Puerto Rico (1 survey), and both the U.S. and Canada (1 survey) between 2000 and 2005) demonstrated that overall adoption rate varied between 45% and 85% in 2004–2005 [ 1 ]. The patterns of PDA usage reported by this study [ 1 ] were: (1) younger physicians (94%) were more likely than older physicians (84.5%) to use a PDA, and students and medical residents tended to be younger and were more likely to use a PDA; (2) no significant gender difference in PDA users was reported among physicians, internists or residents; (3) the biggest adopters of PDAs were family and general practitioners; (4) large-practice and hospital-based physicians were higher adopters of PDAs than office-based physicians; and (5) PDA use was more likely among urban physicians than rural physicians. According to research conducted by Manhattan Research in 2009 on the professional use of smartphones by physicians, about 64% of the physicians in the U.S. used smartphones in 2009 compared to only 30% in 2001. This report found a noticeable increase in smartphone adoption and predicted that 81% of physicians would use smartphone in the U.S. by 2012 [ 2 ].

During recent years, healthcare professionals have required access to many technologies at the point of care, such as: (1) Hospital Information Systems (HISs) including Electronic Health Record (EHR) or Electronic Medical Record (EMR) systems, Clinical Decision Support Systems (CDSSs), Picture Archiving and Communication Systems (PACSs), Laboratory Information Systems (LISs), etc.; (2) evidence-based resources such as PubMed and Up-to-Date; (3) clinical applications such as medical calculators, drug databases, and disease diagnosis applications; and (4) clinical communication such as voice calling, video conferencing, text messaging, and email messaging. Access to information systems or clinical applications in healthcare settings is mainly provided through stationary computers, which does not fully support the mobile nature of healthcare. In response, additional portable and wireless mobile information communication technologies (MICTs) such as Computers on Wheels (COWs) or Workstations on Wheels (WOWs) are used in some healthcare setup to further facilitate access to information technologies at the point of care [ 13 ]. The increased adoption of smartphones by healthcare professionals demonstrates the opportunity for improved clinical communication, and access to information systems and clinical tools at the point of care, or from anywhere at anytime [ 11 , 13 – 26 ]. Accordingly, many software applications have been produced for healthcare professionals in order to facilitate the practice of evidence-based medicine (EBM) at the point of care.

The main stakeholders in the healthcare process are healthcare consumers (patients). Consumer-oriented care, where patients are directly involved in the process of care, will greatly improve the healthcare process. Technology can play key roles in consumer-oriented healthcare (for example, making information accessible to consumers, integrating consumers’ preferences into HISs, remote monitoring, communication, etc.), which is studied in a branch of medical informatics called Consumer Health Informatics (CHI) [ 27 ]. The management of diseases with chronic conditions is very costly. For example, the report published in the 2011 National Diabetes Fact Sheet by the Centers for Disease Control and Prevention (CDCP) of the U.S. Department of Health and Human Services demonstrated that about 25.8 million people in the U.S. (8.3% of population) have diabetes and the estimated national cost (direct and indirect) of diabetes was 174 million dollars in 2007 [ 28 ]. Self-management and remote monitoring of patients are becoming viable solutions for management of diseases with chronic conditions, and smartphones are playing very important role [ 29 – 35 ]. Clinician-led patient education in disease prevention and management through smartphones and text-messaging is convenient and effective [ 36 , 37 ]. A study performed by comScore demonstrated an increase of 27.7 million smartphone users (from 49.1 million at the end of May 2010 to 76.8 million at the end of May 2011) in the U.S. [ 4 , 5 ]. We discuss 6 major operating systems (OS) for smartphones and their current market share and worldwide market share forecasts for 2015, and compare their features in Table 1 and Table 2 in Appendix I.

Data Sources

In this study, we present a systematic literature review of healthcare applications for smartphones following the PRISMA statement for systematic reviews [ 38 ]. MEDLINE citations were searched in April 2011, using the PubMed search engine, for articles that discuss the design, development, evaluation, or use of smartphone software applications to be used by healthcare professionals or patients. The MeSH vocabulary, which is used to index MEDLINE articles, does not contain “smartphone”. However, since smartphones are the successors of PDAs and handheld computers, the MeSH term "computers, handheld" was used in MEDLINE to index the articles with a focus on smartphones. In this study, the search terms used for eligible articles were: (1) "computers, handheld"[MeSH Terms], or ("computer$"[All Fields] and "handheld"[All Fields]), or "handheld computer$"[All Fields]; or (2) "smartphone$"[All Fields], or "smart-phone$"[All Fields], or "smart phone$"[All Fields], or "iPhone"[All Fields], or "Android"[All Fields], or "blackberry"[All Fields], or "black berry"[All Fields], or "Windows Mobile"[All Fields] or "Windows Phone"[All Fields]. The search terms only focused on the terms synonymously to the term “smartphone.” The search criteria did not include any limitation on publication date, and the earliest eligible article was published in 2003. The reference lists of included articles were also searched systematically.

Inclusion and Exclusion Criteria

This study mainly focused on the functionality of software for smartphones within the scope of healthcare. As such, the inclusion criteria for the articles were the design, development, evaluation or use of smartphone-based applications for healthcare. As the search terms only focused on the terms synonymously related to the term “smartphone” and included the MeSH term "computers, handheld"[MeSH Terms], the search result included many articles that have no focus on application programs of smartphones but used smartphones for other purposes, or used handheld computer devices than smartphones or their predecessors, PDAs. Those articles were excluded from this study. This study also excluded articles that were not published in English. In short, we adopted a recall-focused retrieval strategy not to miss relevant documents in the MEDLINE. Irrelevant documents by the strategy are excluded manually.

Study Selection and Data Extraction

The titles and abstracts of the identified citations were read to screen the articles based on the selection criteria described in the previous section. The remaining articles were read in full text to extract information from each article. The extracted information is presented in tables (Tables 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12) including smartphone application names, supported operating system platforms, descriptions and functionalities of the applications. In addition, we accessed the websites of the applications to get the latest release information (last access in June 2011).

The flow diagram of identifying eligible articles for this study is shown in Figure 1 . The literature searches resulted in a total of 2,894 articles, which were then initially screened based on the titles and abstracts. The inclusion and exclusion criteria, described in the previous section, were followed in the screening process resulting in the exclusion of 2,780 articles. The remaining 114 articles were then reviewed in full text, and an additional 59 articles were excluded because they did not primarily discuss smartphone applications but used smartphones for other purposes. One article was excluded because it was not written in English. The resulting 55 articles, discussing 83 smartphone-based healthcare applications, met the eligibility criteria. The earliest eligible articles were published in 2003, and 24 of the 55 articles were published in 2010 through April 2011.

Trial Flow Diagram. This figure presents the trial flow diagram of identifying eligible articles for this study. A total of 2,894 articles were returned from the literature searches. Initially, a total of 2,780 articles were screened based on their titles and abstracts satisfying the inclusion and exclusion criteria. An additional 59 articles were excluded after full text review of 114 articles. Finally, 55 articles discussing 83 smartphone-based healthcare applications met the eligibility criteria. The earliest eligible articles were published in 2003, and 24 of the 55 articles were published in 2010 through April 2011.

The applications were grouped by the targeted user of the applications, as divided into three groups: (1) healthcare professionals, (2) medical or nursing students, and (3) patients.

Application for Healthcare Professionals

There are many smartphone-based applications for healthcare professionals. In this study, a total of 57 applications for healthcare professionals were identified from 43 eligible articles. These applications were grouped into 7 categories based on functional similarity: disease diagnosis, drug reference, medical calculators, literature search, clinical communication, HIS clients, and medical training. Some applications did not fall into any of those categories and were discussed in the section titled “general healthcare applications”.

Disease Diagnosis Applications

Disease diagnosis applications were designed to access diagnosis and treatment information in a few taps on a smartphone. A total of 21 disease diagnosis applications were discussed in 16 articles. Table 3 in Appendix II provides detailed information about the 21 applications. Of these, eight articles mainly focused on the overview and uses of applications [ 11 , 17 , 23 , 26 , 39 – 42 ], four articles published surveys on the use of the applications [ 16 , 43 – 45 ], two articles compared the applications in terms of treatment recommendations [ 46 , 47 ], and two articles investigated the application of smartphones to diagnosis and treatment [ 48 , 49 ].

Handheld versions of printed medical references for disease diagnosis were available on smartphones, providing information on infectious diseases, pathogens, diagnosis, treatment, medications, differential diagnosis etc. There are eight of these applications including Johns Hopkins Antibiotic Guide (JHABx), 5-Minute Clinical Consult (5MCC), 5-Minute Infectious Diseases Consult (5MIDC), Sanford Guide to Antimicrobial Therapy (SG), ePocrates ID, Infectious Disease Notes (ID Notes), Pocket Medicine Infectious Diseases (PMID), and IDdx. These applications also provide internal links for easy navigation and searching.

A 2004 study evaluated SG, JHABx, 5MIDC, 5MCC, PMID, and ePocrates ID for treatment recommendations on 202 cases and reported that five of the applications provided treatment recommendations in more than 95% cases. Specifically, SG and ePocrates ID provided treatment recommendations in every case, JHABx in 99% of cases, 5MCC in 97%, 5MID in 95%, and PMID in 52% [ 46 ]. The treatment recommendations of four applications, ePocrates ID, JHABx, 2002 SG, and ID Notes, were compared with current practice guidelines by Miller et al. (2003) [ 47 ], who concluded that JHABx is highly preferable for its inclusion of more details, accurate treatment and diagnosis information, and an automatic update process. A survey study reported that nurse practitioners ranked medical text and reference books as the second most useful category of PDA applications, and 14% of responders specifically mentioned 5MCC as very useful [ 43 ].

UpToDate is an evidence-based clinical smartphone tool that provides the most recent clinical evidence and includes more than 9,000 physicians’ topics, 5,000 drug topics, practice change updates, etc. This application is very useful in the practice of EBM at the bedside [ 40 ] and is very useful for the integration of test results with clinical information [ 17 ]. In a community hospital, internal medicine residents reported UpToDate as their most commonly used evidence-based resource and about 50% of them reported using it for general medical and scientific information as well as specific questions on patient management [ 45 ]. Phua & Lim (2008) [ 16 ] conducted a survey on the use of evidence-based resources by residents in a tertiary care hospital in Singapore with an institutional subscription to UpToDate. They found that only 69.4% of the residents were aware of their institutional subscription, so the actual use of UpToDate was low (56.7%). However, most of the users (93.4%) found it useful and would recommend UpToDate to their colleagues, and for about three-fifth of them using UpToDate led to changes in patient-management decisions [ 16 ]. One of our authors (LS), a practicing physician, has observed that his colleagues in U.S. private and academic practice consult UpToDate far more often than any other single resource at the point of care, and that it seldom fails to return fast, focused, and practical results.

There were six applications providing common laboratory test information, including reference values and interpretation, causes for abnormal (increased or decreased) values, and laboratory unit conversions. These were Palm LabDX, Normal Lab Values, Lab Unit Converter, Labs 360 o , Davis’s Laboratory and Diagnostic Tests, and Pocket Guide to Diagnostic Tests [ 17 , 26 , 44 ]. Palm LabDX was evaluated by third-year medical students in their internal medicine clerkships, mostly during patient care activities, and found to be useful [ 44 ]. Lippi & Plebani (2011) [ 26 ] emphasized the formal review process of the laboratory applications in strict compliance with laboratory medicine professional bodies such as the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC).

Smartphones can also be applied in the process of diagnosis and treatment using software application. A simple smartphone application for eye-care professionals is a visual acuity test. For example, EyeChart is an iPhone application that includes the Snellen eye chart to measure visual acuity [ 41 ]. A similar application is EyePhone, which includes a distance E-test, near visual acuity test, color test, Amsler grid, and pupil gauge test [ 42 ]. Our physician author, who frequently practices in low-resource clinical settings, has found that having convenient access to a Snellen chart is indispensable although its use is limited to patients that are literate in the Roman alphabet. The DizzyFIX application guides clinicians in the Epley Maneuver, a series of precise head and body positions that is the primary treatment of Benign Paroxysmal Positional Vertigo [ 41 ]. Mezzana et al. (2011) [ 49 ] used the Video Laser Level application in oculoplastic surgery by superimposing two digital horizontal lines onto a real image to show canthal ligament misalignment. Joundi et al. (2011) [ 48 ] measured tremor frequency using the iPhone iSeismometer application and found that it matched the more sophisticated and expensive EMG analysis.

Smartphone-based disease diagnosis applications are useful evidence-based resources at the bedside. These applications can also help clinicians in identifying appropriate laboratory tests based on symptoms, decreasing unnecessary tests and reducing cost of care. Radiology tests, for example, are very expensive and unnecessary tests are undesirable; but clinicians can use the “eRoentgen Radiology Dx” application on their smartphones to identify the most appropriate radiology exams for patients, reducing the cost of care and improving patient safety [ 23 ]. Other applications use clinical algorithms to help clinicians understand and apply principles of disease diagnosis. The flowcharts included in 5MCC and Pocket Guide to Diagnostic Tests, for example, can help physicians see at a glance diagnostic possibilities that they may have overlooked.

Drug Reference Applications

A total of six drug reference applications were discussed in the eight articles. Table 4 in Appendix II provides detailed information about the drug reference applications. Six of these articles mainly focused on the overview and use of the applications [ 11 , 39 , 40 , 50 – 52 ], and two articles published surveys on use of the applications [ 43 , 53 ]. The six applications are: Skyscape’s RxDrugs, Epocrates, Medscape, SafeMed Pocket, FDA drugs, and DrugDoses.net. The drug reference applications generally include the names of drugs, their indications, dosages, pharmacology, drug-drug interactions, contraindications, cost, and identifying characteristics. Epocrates was cited as the most commonly accessed drug-reference application [ 11 , 50 , 51 ]. Epocrates and Skyscape’s RxDrugs, another drug reference that our physician author has seen widely used in the U.S., can check multiple drug-drug interaction at the same time [ 11 ]. FDA Drugs, which includes package inserts or official labels of FDA-approved prescription and over-the-counter drugs, shares this functionality and also searches by active ingredients [ 39 ]. SafeMed Pocket, which was designed to be used in Sweden and lists drugs that are licensed for sale in that country, is integrated with a clinical decision support system (CDSS) for use by geriatiric homecare nurses to warn of drug-drug interactions, therapeutic duplications, and dosages that are unsuitable for elderly people [ 52 ].

Smartphone-based drug reference applications can be useful evidence-based resources at the point of care, as Richardson & Burdette (2003) [ 40 ] show in their example of using Epocarates in the practice of evidence-based medicine during hospital rounds. A survey of PDA usage by nurse practitioners reported that drug reference applications were the most useful of all applications, and that 92% of the PDA users surveyed used Epocrates [ 43 ]. Another survey of PDA usage by undergraduate medical students at the University of Alberta, Canada reported that 77.4% of students used drug reference applications, making them the most commonly used medical application in that population [ 53 ].

Medical Calculator Applications

A medical calculator or clinical calculator is a software program for calculating various clinical scores and indices such as body mass index (BMI), body surface area (BSA), coronary heart disease risk, individual drug dosing, etc. Usually calculation of clinical scores or indices involves complex formulas using several input parameters. Medical calculators typically provide a user interface to enter parameters and calculate scores using a standard formula. Users do not need to use or even know the actual formula for calculating a clinical score or index. For example, body mass index or BMI (also known as the Quetelet Index for its creator, Belgium statistician Adolphe Quetelet, 1796–1874), is the most commonly used measure of obesity internationally [ 54 ]. Our physician author has found that the w e i g h t k g ( h e i g h t ( m ) ) 2 or w e i g h t l b × 703 ( h e i g h t ( i n ) ) 2 formula for calculating BMI, while simple to remember, is surprisingly time consuming and error-prone in the time-pressured environment of a busy clinic. The user only needs to enter a patient’s weight and height in a typical medical calculator, however, to calculate the BMI quickly and confidently.

Initially, medical calculator software programs were available on personal computers. Later, online versions of some calculators were accessible through the Internet. However, physicians were often unable to use this software at the point of care due to a lack of computer access. Now medical calculators are available for several smartphone platforms. Table 5 in Appendix II presents eight smartphone-based medical calculator applications that were discussed in 10 articles [ 11 , 16 , 26 , 39 , 41 – 43 , 47 , 55 , 56 ]. These eight applications are: Epocrates MedMath, MedCalc, Medical Calculator, Calculate, Archimedes, uBurn Lite, Softforce’s Antibiotic Dosage Calculator, and Paeds ED. The most commonly used medical calculators are MedMath and MedCalc [ 11 , 43 ]. The calculation of drug doses for pediatric patients is very crucial during medical emergencies, and the “Paeds ED” application calculates drug dosages for children based on their age in years [ 41 ]. Drug dosages for patients with renal failure can be calculated by Softforce’s Antibiotic Dosage Calculator. The uBurn app helps surgeons to calculate the percentage of total body surface area affected in adult burn victims [ 56 ].

Literature Search Applications

Literature search applications for healthcare professionals facilitate searching biomedical literature databases such as PubMed/MEDLINE, Essie, etc. to find and display medical reference information. The experience of our physician author is that these resources are seldom useful at the point of care and seldom used by clinicians in that setting. However, our review found six literature search applications discussed in 15 articles [ 41 , 42 , 45 , 50 , 51 , 55 – 64 ]. Table 6 in Appendix II presents the functions of these six applications, PubSearch, PubMed on Tap, MEDLINE Database on Tap (MD on Tap or MDoT), askMEDLINE, PICO, and Disease Associations.

The free PubSearch [ 51 ] and fee-base PubMed on Tap [ 41 , 42 , 56 ] are two commercial applications for the iOS platform that facilitate PubMed/MEDLINE search from iPhone. These two applications are client applications for PubMed/MEDLINE. The National Library of Medicine (NLM) provided MD on Tap (MDoT) to help healthcare professionals find answers to clinical questions and access medical reference information at the point of care through three search engines: PubMed, Essie, and Google [ 58 ]. In some articles [ 57 , 58 ], the NLM’s MDoT project was referred as “PubMed on Tap” since it was initially named as “PubMed on Tap”, which is different than the “PubMed on Tap” for iPhone mentioned at the beginning of this paragraph. MDoT is a client–server application in which the software client (available for Palm OS and Windows Mobile platform) sends the user’s plain-text query to the intermediate server (called MD on Tap server), which formats the query into appropriate search terms for the selected search engine (Essie, PubMed/MEDLINE, or Google) and returns the search results to the client application [ 58 ]. The search result is then displayed by grouping the articles into several clusters. The performance of MDoT in answering clinical question varies with the selection of search engine. Studies showed that Essie performed better than PubMed/MEDLINE [ 59 , 64 ]. However, later studies showed that medical residents using PubMed/MEDLINE on MDoT during daily rounds answered most of their clinical questions (68% and 86% in two separate studies) when queries consisted of three or more medical terms [ 63 , 64 ].

The MDoT client application required improvement in navigational and functional characteristics such as incorporating visual cues to indicate a visited citation [ 57 ]. The requirement of maintaining a high performance server for hundreds of concurrent users is a disadvantage for MDoT compared to client-only literature search applications like PubSearch or PubMed on Tap, since the search engines for these latter are already deployed in high performance servers and maintained by the providers. MDoT’s advantage is that the user interface was implemented independently while the processing power was delegated to the server and all transactions are stored in a local database on the server, facilitating research on users’ queries from mobile devices [ 58 , 64 ]. In 2007, the development and support for MDoT client application was stopped, introducing compatibility issues with future operating system versions [ 65 ].

Search results mainly depend on the query’s precision, so experienced users use specialized vocabulary such as MeSH to find relevant citations in PubMed/MEDLINE. The “PubMed for Handhelds” site ( http://pubmedhh.nlm.nih.gov/ ) provides access to the PICO (Patient, Intervention, Comparison and Outcome), askMEDLINE, and Disease Associations (DA) search engines, developed by the NLM to facilitate literature search without knowing MeSH. PICO facilitates well-formatted search that includes four text fields: problem, intervention, compare to, and outcome [ 45 , 60 , 62 ]. The askMEDLINE search engine was developed from PICO to allow natural language query [ 45 , 60 – 62 ]. DA is a search interface for case reports and review of reported cases in PubMed/MEDLINE [ 45 ].

Clinical Communication Applications

Smartphones support several means of communication including voice calling, video calling, text messaging, email messaging, multimedia (text, image, and video) messaging, and conferencing through the cellular phone service provider. Besides standard communications, clinical communication applications are designed to simplify communication among clinicians within a hospital. A total of five articles discussing clinical communication using smartphones were included in this study [ 12 , 20 , 51 , 66 , 67 ]. Among them two articles discussed three smartphone-based communication applications [ 20 , 51 ], and three articles demonstrated the impact and improvement in clinical communication using smartphones [ 12 , 66 , 67 ].

Table 7 in Appendix II shows the functionality of three clinical communication applications that were discussed in two articles [ 20 , 51 ]. These three applications are Voalté One, Amcom Mobile Connect, and mVisum; all require the installation of proprietary communication servers. Voalté One combines phone calls, text messaging, and alarm alerts into one device. Amcom Mobile Connect is a messaging and alerting application that separates critical messages from less important one. mVisum is a specialized application for cardiology communications that receives monitor data, alarms, ECGs, and lab results on smartphones.

The use of mobile phone communication in critical care environments facilitates relaying important information quickly, reducing the risk of medical errors [ 66 ]. Nurses may need to go through a complex process to find the responsible physician for a patient using traditional numeric paging. In Toronto General Hospital, the use of smartphones simplifies reaching the responsible physician. Nurses can send email messages indicating priority and requesting feedback if necessary, or can call directly in emergency cases [ 12 ]. However, mobile communication may impact informal group discussions among workers in the common areas that support some important activities such as managing schedule changes, and discussions related to professional feedback and quality control. This impact in a surgical department was found to be negative and additional socio-technical mechanisms may be required to overcome this [ 67 ].

HIS Client Applications

Client applications for Hospital Information Systems (HISs), such as electronic health records (EHR), electronic medical records (EMR), and picture archiving and communication systems (PACS), provide the flexibility of accessing patient information securely from anywhere at any time. A total of five articles discussed the use of smartphones to access patients’ clinical information [ 20 , 22 , 23 , 51 , 68 ]. Table 8 in Appendix II provides detailed information about HIS client applications for smartphones. These applications provide some of the functionality of their PC counterparts. OsiriX Mobile is the client application for OsiriX PACS, which processes and displays images using the DICOM standard for digital image storage [ 22 , 23 , 51 , 68 ]. MEDITECH, a client application for MEDITECH EMR, and PatientKeeper Mobile Clinical Results, a client application for PatientKeeper EMR, provide access to patients’ clinical information from a hospital EMR including lab results, medication lists, clinical notes, problem lists, vital signs, and allergies [ 20 ]. AirStrip OB is designed for obstetricians to access their EMR’s real-time and historical waveforms, fetal strips and maternal contraction patterns.

Medical Training Applications

Smartphones are also used for medical training and continuing medical education (CME). CME provides training in the most current evidence-based medical practice [ 69 ]. A total of four articles discussed the use of smartphones in medical training [ 25 , 69 – 71 ]. Among them, one article discussed smartphone-based HIV/AIDS CME materials [ 25 ], another article evaluated print and smartphone-based CME for men’s sexual health, and the other two articles discussed two smartphone-based software applications for medical training. These two applications are listed in Table 9 in Appendix II: iCPR and iResus. Both of the applications are designed for the iOS platform and available for free. iCPR is a self-directed Cardiopulmonary Resuscitation (CPR) training application that is based on both American Heart Association and European Resuscitation Council guidelines. This application measures the chest compression rate and gives audiovisual feedback, improving the performance of chest compression by helping the user to achieve the correct chest compression rate [ 71 ]. The United Kingdom’s resuscitation guidelines including adult and pediatric algorithms are visualized in an interactive format in the iResus application. The use of iResus significantly improved the performance of certified advanced life support clinicians [ 70 ]. Both iCPR and iResus were evaluated in simulated clinical scenarios and further studies are required in real clinical settings [ 70 , 71 ].

Smartphone-based HIV/AIDS CME materials, including 3D learning scenarios simulating interactive clinical cases, were provided to the clinicians in urban and peri-urban clinics in Peru. This software, while not commercially available, gave flexibility to the mobile health care workers in accessing learning content in a resource-limited setting [ 25 ]. However, another study [ 69 ] showed no significant difference in evidence-based clinical choices between users of printed vs. electronic CME modules on men’s sexual health.

General Healthcare Applications

Seven smartphone-based applications were categorized as general healthcare applications, as they do not fit any of the categories discussed in the previous sections. These seven applications are HCSIT, Borboleta, LIFe-reader, Multimedia Paging Based Clinical Alarm, Outbreaks Near Me, H1N1 Swine Flu Update, and WISER. These applications were discussed in six articles [ 39 , 52 , 72 – 75 ]. Table 10 in Appendix II shows detailed information about these applications.

The Handheld Computer Smoking Intervention Tool (HCSIT) assists clinicians with smoking cessation counseling and improved physicians’ comfort level significantly in counseling patients about smoking cessation [ 72 ]. A real-time clinical alarm monitoring system was developed by van Ettinger et al. (2010) [ 75 ] to monitor intensive care unit patients by smartphone, displaying the alarms for an entire unit or one bed, with the vital signs at the moment of the alarm color-coded by severity. Outbreak Near Me provides real-time disease outbreak information and H1N1 Swine Flu Update provides a news feed for Swine Flu (H1N1) outbreaks [ 39 ]. H1N1 Swine Flu Update is now discontinued since the H1N1 pandemic has abated. WISER is an application for Emergency Medical Service Specialists that identifies chemical and biological hazards on the basis of symptoms and signs from NLM’s Hazardous Substances Data Bank (HSDB), and radiological and biological substance reports [ 74 ].

In homecare, nurses visit patients’ homes and may be isolated from the healthcare center. Borboleta, a smartphone-based mobile telehealth system, was developed in Brazil for nurses in primary healthcare to use during patient homecare visits. The nurses can complete the patient registration and schedule a visit on their smartphones instead of using paper and pencil, and the data is centrally stored in a web server. In addition, the nurses can access patient data, caregiver information, socioeconomic data, visit history, disease history, and medication history during homecare visits [ 73 ]. LIFe-reader, a CDSS system for nurses in geriatric homecare in Sweden, was implemented to help nurses to obtain patients medication profiles by scanning the European Article Number (EAN) on the drug package. As described in the section tittled “Drug Reference Applications”, it can also check for inappropriate drugs, drug-drug interactions, and therapeutic duplication. LIFe-reader reduces the drug-related risks of falling and drug-related admissions to hospitals [ 52 ].

Applications for Medical and Nursing Students

There are many smartphone-based applications containing primarily as educational material for medical or nursing students. Seven articles discuss a total of eleven applications [ 11 , 17 , 23 , 41 , 47 , 51 , 56 ]. Table 11 in Appendix II lists the eleven applications and provides detailed information for each. They are I-Surgery Notebook, Eponyms, Netter’s Atlas of Human Anatomy, Netter’s Anatomy Flash Cards, Blausen Ear Atlas, Oxford Handbook of Clinical Specialties, Dissection, Cranial Nerves, iSilo, Mobipocket Reader, and Instant ECG.

The anatomy tools for students are very useful, and our physician author found the flash-card application to be a particularly good way to study during spare moments in medical school when books were not available. Netter’s Atlas of Human Anatomy contains more than 532 colored anatomic illustrations that are mainly designed for educational purpose [ 23 , 56 ] and its Netter’s Anatomy Flash Cards version contains 300 interactive flash cards [ 51 , 56 ]. Dissection is an anatomy tool that displays dissection of the human head and neck [ 41 ]. A set of ear-related video animations including cochlear implants, ear pressure, ear tubes, hearing loss, hearing tests, and otitis media are available in the Blausen Ear Atlas [ 41 ]. Cranial Nerves is a learning tool that includes interactive visualization along with textual information about cranial nerves and the skull base, based on high-resolution CT scans [ 41 ]. Instant ECG is a basic ECG tutorial application that includes ECG electrophysiology, myocardial action potential, associated waveforms, and intervals and segments [ 23 ].

The details of eponymous signs and diseases are available on smartphone through the Eponyms application [ 51 ]. Students as well as surgeons, surgical interns, and residents can use I-Surgery Notebook on their iPhone or Android phone during their surgical sessions to log surgical cases including procedure, pre-operative and post-operative diagnosis, list of involved surgeons, and type of anesthesia used [ 51 ]. The printed version of the Oxford handbook of Clinical Specialties, which includes 12 books, is available as handheld version on smartphones [ 41 ].

Several other medical books are available as electronic version through electronic book reader application on smartphone such as iSilo and Mobipocket Reader. iSilo is a fee-based program that stores text in a highly compressed format and facilitates text search within a document or set of documents [ 17 , 47 ]. Mobipocket Reader is available for free and includes a library of all eBooks stored in local media, with the ability to annotate, highlight, or bookmark any part of the eBook, and lookup any word in several dictionaries [ 11 ].

Smartphone-based healthcare applications for clinicians, discussed in the section titled “Application for Healthcare Professionals”, can also be used by medical or nursing students for educational purposes. For example, Cibulka & Crane-Wider (2010) [ 76 ] designed teaching strategies for nursing students using smartphones that includeed a clinical consult guide, a prescribing reference, and a differential diagnosis tool from Skyscape or Epocrates. The students used the software packages during their classes to access clinically relevant information and found very useful. In a systematic review by Kho et al. (2006) [ 77 ], medical calculators and drug reference applications were also found to be very useful in medical education.

Applications for Patients

Fourteen articles discuss fifteen smartphone-based patient oriented applications [ 23 , 29 , 32 , 35 , 41 , 78 – 86 ]. Table 12 in Appendix II shows detailed information about these applications. There are six applications for management of chronic conditions: Diabeo, Cardiomobile, Pulmonary Rehabilitation, PAL Calculator, Asthma Peak Flow Monitoring, and eCAALYX; four ENT-related applications including Hearing Check, uHear, iTinnitus, and Sleep Aid; three applications for human fall detection including Fall Detector by Hansen et al. (2005) [ 78 ], Fall Detector by Zhang et al. (2006) [ 80 ], and iFall; and two others: Purdue Momentary Assessment Tool, and Mayo Clinic Meditation.

Chronic disease management applications provide expert feedback to patients based on their input. The Diabeo system helps diabetic patients by calculating bolus insulin dose based on carbohydrate intake, pre-meal blood glucose, and anticipated physical activity reported; and automatically adjusts carbohydrate ratio and basal insulin. The patient data is sent from a smartphone through a General Packet Radio Service (GPRS) connection to a medical staff computer and stored there to facilitate further tele-consultation. The Diabeo telemedicine system reduces the cost of care and improves metabolic control for diabetes patients [ 32 ].

Cardiomobile is a real-time remote monitoring system for exercise-based cardiac rehabilitation that is comprised of a heart and activity monitor, single lead ECG, GPS receiver, and programmed smartphone (i-Mate SP3, Dubai). The GPS receiver and monitor connect to a smartphone via Bluetooth. The smartphone sends ECG rate, walking speed, heart rate, elapsed distance, and patient location to a secure server for real-time monitoring by a qualified exercise scientist. The system supports cardiac patients who are unable to access hospital-based rehabilitation due to excessive travel time or lack of available programs in their area [ 35 ].

Pulmonary Rehabilitation is an application for chronic obstructive pulmonary disease (COPD) rehabilitation and self-management, developed for smartphones by Marshall et al. (2008) [ 29 ], using a Bluetooth pulse oximeter to measure the heart rate during exercise. Patients can follow the step-by-step exercise instructions on their smartphones displaying patient heart rate, exercise time remaining in seconds, and feedback color (green: normal physical condition, amber: normal condition but near acceptable limits, red: dangerous physical condition). No assessment or clinical evaluation of this application was reported [ 29 ].

The measurement of physical activity level (PAL) is important in many chronic diseases. The PAL Calculator application uses smartphone questionnaires to calculate PAL. Study results demonstrated a high accuracy from this method in comparison to reference values [ 82 ].

Ryan et al. (2005) [ 79 ] described an observational study on asthma peak-flow monitoring using an electronic peak-flow meter connected to a smartphone. The application sends peak-flow readings through the GPRS network to a secure server, and receives asthma trend analysis feedback from the server. The study demonstrated a satisfactory primary outcome measure [ 79 ]. eCAALYX is a remote monitoring system for elderly patients with multiple chronic conditions presented by Boulos et al. (2011) [ 83 ] that receives data from wearable health sensors in a smart garment , transmits data to the monitoring server, and identifies higher-level information such as tachycardia and signs of respiratory infections based on established medical knowledge. Users can see current medical reports on their smartphones based on sensor data, perform new measurements, and communicate with caregivers through the application [ 83 ].

Hearing Check and uHear are two free hearing loss self-assessment tests for iPhone. Hearing Check was developed by the UK Royal National Institute for Deaf People (RNID), and calls a landline number to receive a free hearing check [ 86 ]. uHear includes three assessment tests: hearing sensitivity, speech in noise, and a questionnaire about common listening situations [ 41 ]. In addition to its use as a self-testing device, our physician author has used uHear in low-resource clinical environments where complete audiometry was not available. iTinnitus is a sound-therapy application for patient with tinnitus, which records tinnitus by frequency in Hertz and keeps track of the tinnitus in a daily diary graph. iTinnitus supports full masking therapy with a sound played at a frequency around the same frequency as the patient’s tinnitus [ 41 ]. Sleep Aid is a sleep apnea management application that records snoring, generates a graph of the snoring, plays the recorded snoring sound, and provides information about sleep apnea [ 41 ].

Three human fall detection applications were prototyped using smartphones. Hansen et al. (2005) [ 78 ] used a separate wearable tri-axial accelerometer and camera phone to build a fall detection system. The accelerometer processes the data locally that identifies a fall and send the data to the phone during a suspected fall. The phone then requests vocal or keypad response from the user, and if there is no response it automatically makes an emergency call and sends data and video from the phone’s camera to the emergency system [ 78 ]. The user may forget to wear the sensor, however, which remains a major problem with fall detection systems based on a wearable accelerometer. Zhang et al. (2006) [ 80 ] embedded the tri-axial accelerometer in the cell phone to overcome the problem. In cases of possible falls, the data is sent to the server for further analysis. This has been shown to be an effective method for fall detection [ 80 ]. Sposaro and Tyson (2009) [ 81 ] demonstrated the cost-effectiveness of iFall, which utilizes the inbuilt tri-axial accelerometer of an Android phone, processes the data on the phone, and prompts the user in cases of possible falls. If the fall is confirmed by the user not responding to the application, iFall makes an emergency call [ 81 ].

The Purdue Momentary Assessment Tool (PMAT) is a human behavior monitoring tool using smartphones to facilitate event-driven study design [ 87 ]. PMAT was successfully used to monitor substance use and symptom expression in schizophrenia patients [ 84 , 85 ]. Mayo Clinic Meditation helps patients practice meditation with a short training video introducing the key concepts of meditation, and 15-minute and 5-minute meditation programs [ 23 ].

Application Distribution

In this study, a total of 83 applications were discussed. Among them, 57 applications were designed for clinicians, 11 applications were designed for medical or nursing students, and 15 applications were designed to be used by the patients. Figure 2 presents the distribution of these applications for the major smartphone OS platforms. 74 applications are developed for at least one OS platform and the remaining 9 applications can be accessed either on a web-enabled or java-enabled smartphone. None of the six OS platforms (discussed in Appendix I) support all of these 74 applications. iOS is the most popular platform for healthcare smartphone applications because 57 out of 74 applications are developed for iOS.

Number of Healthcare Applications per OS Platform Discussed in this Study. This figure presents the distribution of the smartphone-based healthcare applications that are discussed in this study for all of the six major OS platforms. The distribution is describes in two categories: the first breakdown is according to the intended users, that is, healthcare professionals, medical and nursing students, and patients; and the second breakdown is according to their latest release or update date, that is, recent (latest update in 2011), contemporary (latest update during 2009 to 2010), legacy (latest update on or before 2008), or prototype (not released yet for real use).

Figure 2 also presents a breakdown of the number of applications in two categories: (1) intended users (healthcare professionals, medical and nursing students, and patients), and (2) their latest release or update date (recent (latest update in 2011), contemporary (latest update during 2009 to 2010), legacy (latest update on or before 2008), or prototype (not released yet for real use)). Most of the applications for each platform are intended for healthcare professionals. For example, 74% of the iOS applications and 71% of the Android applications are for healthcare professionals. The percentage of applications for medical and nursing students for each OS platform is below 25%. The Windows platform has the highest number of applications (12.9%) for patients and BlackBerry has no application for patients. At least half of the applications have an updated or new version release in 2011. The BlackBerry has the highest ratio (80%) of recently released application followed by 71.4% for Android and 59.6% for iOS. The iOS has the highest number of applications but a large portion (38.6%) of the applications was released in 2009 and 2010. The Palm OS has the least number of total applications but has the highest ratio of legacy (25%) applications.

Symbian OS has a very low coverage of healthcare applications because only three applications are developed for the OS. Other five OS platforms (i.e. iOS, Android, Palm, Windows, and BlackBerry) have extensive coverage of healthcare applications because at least one application is available for all of these five OS platforms in each of the broader category. In the following, the coverage of healthcare applications for these five OS platforms is discussed.

In “Disease Diagnosis” category, we discussed a total of nine medical reference applications (i.e. JHABx, 5MCC, 5MIDC, SG, ePocrates ID, ID Notes, PMID, and UpToDate) in which at least four of them are available for all of these five OS platforms. We also discussed six lab reference applications (i.e. Palm Lab DX, Normal Lab Values, Lab Unit Converter, Labs 360 0 , Davis’s Laboratory and Diagnostic Tests, and Pocket Guide to Diagnostic Tests) in the same category in which at least three of them are available for all of these five OS platforms. There are six other useful “Disease Diagnosis” applications (including Eroentgen Radiology Dx, iSeismometer, Video Laser Level, EyeChart, EyePhone, and DizzyFix) that are available for iOS only except iSeismometer that is also available for Windows Phone.

For the six “Drug Reference” applications, at least two of them are available for all of these five OS platforms. Among the nine “Medical Calculator” applications, three specialist medical calculators (i.e. uBurn Lite, Softforce’s Antibiotic Dosage Calculator, and Paeds ED) are available for iOS only and at least two out of six other general purpose medical calculators are available for all of these five OS platforms. The “Literature Search” applications are mainly developed for iOS, Palm and Windows Phone platforms. However, three applications for literature search can be accessed from any web-enabled smartphone through their native internet browser. Accordingly, the “Literature Search” applications are available virtually for any OS platforms. The “Clinical Communication” and “HIS Client” applications are not available for all OS platforms. These applications require installation of proprietary servers and the vendors of these servers produce the smartphone applications based on their clients’ requirements. Currently, clinical communication and HIS client applications are mainly available for iOS, Android, and BlackBerry. The “Medical Training” applications are available for iOS only. The “General Healthcare” applications, “Medical Education” applications, and applications for patients do not serve a common purpose in their category. In order for these applications to be widely used in practice, they should be available for all OS platforms. Every medical education application except “MobiPocket Reader” is available for iOS. Not a single application in the category of “Application for Patients” is available for two different OS platforms.

Using multiple smartphone devices for use of several healthcare applications is not practical for all three user groups including healthcare professionals, medical and nursing students, and patients. We believe every healthcare application should be developed for at least the top 2 to 3 OS platforms. We can observe that there is low coverage in the following categories: specialist medical calculator, clinical communication, HIS client, medical training, general healthcare, medical education and applications for patients. The future development should focus on increasing the coverage of applications for all OS platforms in the above mentioned areas.

Smartphones access clinical applications, evidence-based resources, and advanced mobile communication in one handheld-sized device at the point of care. Their mobility enables health professionals to use them in a clinical setting for patient care [ 88 ]. This study presents smartphone-based healthcare applications that were discussed in the literature. The applications were categorized based on target users: clinicians, medical or nursing students, or patients. The functionalities of the applications and supported smartphone platforms were discussed and presented in tabular format.

Studies show wide adoption of smartphones by healthcare professionals during recent years [ 1 , 2 ]. Smartphones are becoming popular for clinical use among clinicians, and medical and nursing students [ 43 , 53 , 55 , 77 , 89 – 91 ]. PDAs, the predecessors of smartphones, have been recognized in a systematic review article as useful in physicians’ practices for rapid response, error prevention, and data management and accessibility [ 92 ]. Real-time clinical information at the point of care is very important in the practice of EBM, since clinicians may not seek answers to clinical questions after completion of a clinical encounter [ 93 , 94 ]. Drug reference applications, medical textbooks and references for disease diagnosis, and medical calculator applications were reported as the most useful by clinicians and medical and nursing students [ 43 , 53 , 77 , 89 , 91 , 95 ]. Single applications may not provide all required information, requiring use of combinations of applications [ 40 ].

Medical reference applications, such as the Sanford Guide, Johns Hopkins Antibiotic Guide, 5-Minute Infectious Diseases Consult, 5-Minute Clinical Consult, and ePocrates ID, provide treatment recommendations for most cases [ 46 , 47 ]. UpToDate is a useful evidence-based clinical information tool at the point of care, but requires an institutional or personal annual subscription [ 16 , 17 , 40 , 45 ]. The drug-drug interaction-checking feature of drug reference applications like Epocrates and Skyscape’s RxDrugs is a very useful evidence-based resource at the point of care [ 11 , 40 ]. Medical calculators help clinicians calculate various clinical scores and indices, and MedMath and MedCalc were reported as the most commonly used among eight medical calculators listed in Table 5 [ 11 , 43 ].

The literature search applications listed in Table 5 facilitate PubMed/MEDLINE search from smartphones. The use of a specialized medical vocabulary (such as MeSH) in search queries is very effective for finding relevant citations. Usually, queries consisting of three or more medical terms can retrieve relevant documents to answer clinical questions successfully [ 63 , 64 ]. However, learning effective search strategies using a specialized vocabulary could be impractical for clinicians and the general public [ 96 , 97 ]. Therefore, the NLM developed smartphone-based applications like askMEDLINE, PICO, and Disease Associations for non-expert clinical information seekers [ 45 , 60 – 62 ]. The presentation of search results for clinical use on the small screens of smartphones is challenging. The visualization of the search result by grouping or clustering retrieved documents, with summary information for the group, was found to be useful for effective and intuitive navigation [ 24 , 58 , 98 – 100 ].

The use of smartphones for mobile clinical communication facilitates various means of communication among clinicians, such as text messages, email messages, voice, video, and images, and reduces communication delay [ 12 , 20 , 66 ]. However, mobile communications in clinical settings may have a negative impact on informal interaction among workers and require socio-technical mechanisms to overcome this [ 67 ]. Bedside access, or access at anytime from anywhere, to patients’ clinical information from Hospital Information Systems (HISs) can be facilitated through smartphones [ 20 , 22 , 23 , 51 , 68 ]. Also, electronic capture of patient data into HIS within resource-limited settings in developing countries has become feasible, using smartphone-based applications [ 101 ]. However, HIS client applications must comply with national legislation on privacy and security, such as the U.S. Health Information Portability and Accountability Act [ 102 ], the European Union’s regulation 95/46/EC on processing of personal data [ 103 ] or the UK National Health Service’s privacy and confidentiality issues [ 104 ], etc. [ 33 , 51 , 55 , 105 , 106 ].

Medical training applications on smartphones make medical guidelines or CME materials accessible from anywhere, including resource-limited settings [ 25 , 69 – 71 ]. Current disease outbreak updates, real-time ICU patient monitoring, and electronic mobile homecare have also become feasible as a result of smartphone technology [ 39 , 52 , 73 , 75 ]. Handheld versions of medical books, journals like BMJ and JACC, interactive anatomy tools, medical calculators, medical references, and drug references on smartphones provide mobile learning opportunities for medical and nursing students [ 11 , 17 , 23 , 41 , 47 , 51 , 56 , 77 , 107 – 109 ]. Accordingly, these applications may be included in medical or nursing curriculum [ 76 , 110 ].

There are many advantages of using smartphone-based healthcare applications in medical practice. For example, they allow for advanced mobile clinical communications using multimedia functions and provide access to various clinical resources at the point of care such as up-to-date evidence-based clinical resources, medical formula calculator, drug reference and interaction checking, etc. In addition, they can provide secure remote access to real-time patient monitoring system and EMR systems for better patient care. Hospitals should encourage healthcare professionals to use smartphone-based healthcare applications, and financially support them to keep the applications up-to-date with regular update.

Smartphone-based patient oriented applications deliver healthcare services for patients with chronic conditions, which is the purpose of mobile health or mHealth [ 29 , 32 , 35 , 79 , 82 , 83 ]. The mHealth component of eHealth delivers medical and healthcare services through mobile devices [ 111 ]. The World Health Organization (WHO) has recently defined mHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices” [ 112 ]. The wide adoption of high-functionality smartphones by the general public highlights the increased demand for better mHealth services through smartphones [ 3 – 6 , 86 ]. Mobile telemedicine services with video capability have become viable by using smartphones [ 30 , 33 ]. However, while the advancement in smartphone-based mHealth services may be seen in developed countries, developing countries have yet to get its potential benefits [ 42 ]. It is anticipated that about 500 million smartphone users around the world will use mHealth services by 2015 [ 113 ]. Since anyone can implement applications for smartphones, healthcare applications need to be controlled and validated through appropriate organizations such as the U.S. Food and Drug Administration (FDA), the Australian Therapeutic Goods Administration (TGA), the International Federation of Clinical Chemistry (IFCC), etc. [ 19 , 26 , 114 ]. For example, Abroms et al. (2011) [ 115 ] examined 47 iPhone applications for smoking cessation that were found to have low adherence to clinical practice guidelines.

An interesting feature of smartphone devices is Bluetooth, which is a technology for short-distance wireless data transmission. Nowadays, many medical devices (such as glucose meters, thermometers, etc.) have this functionality. In order to standardize the interoperability between medical devices using Bluetooth, the medical working group of the Bluetooth Special Interest Group provided the specification for Bluetooth Health Device Profile (HDP) [ 116 ]. The smartphones should support built-in Bluetooth HDP for standard Bluetooth communication with medical devices. This will enable the smartphone applications to work with medical devices from different vendors. Currently, only the Android platform supports built-in Bluetooth HDP (see Appendix I).

The challenges of smartphone-based healthcare include limited battery life, small screen size, potentially erroneous data input, computer viruses including spyware, magnetic interference with medical devices, potentially inefficient patient-physician interactions, loss or theft, and breaches of data privacy and security. Data input is much slower and erroneous on smartphones using a stylus [ 117 ]. Antivirus software must be used to protect devices from computer viruses and spyware, and must be updated regularly. The use of smartphones in hospital environments may have a small risk of electromagnetic interference with medical devices and they should be used with caution in certain areas. Studies suggest that using mobile devices in a normal way beyond a one-meter range of medical devices is safe [ 66 , 118 – 121 ]. The interaction between patients and physicians may be hampered by the use of a PDA during the patient encounter, but explaining the reason for using a PDA to the patient was found to have a positive effect on patient-physician interactions and communications [ 122 ]. The privacy and security concerns of storing or communicating patient data with smartphones should be addressed cautiously. These security features of smartphones, while not available for all devices, may be useful: data backup, encryption of stored patient data, remote wiping to destroy all data on a device in case of loss or theft, and securely encrypted wireless data transmission over WiFi [ 123 – 126 ].

The information contained in healthcare applications must be accurate. In general, application users must agree with the terms and conditions of use of applications to use the applications, and the users are mainly liable for utilizing the information in the applications. As a result, incorrect or outdated information from healthcare applications may lead to medico-legal consequences and users instead of software companies are responsible for them. This problem may affect many healthcare applications including disease diagnosis, drug reference, and medical calculator applications. The peer reviewed applications (such as JHABx guide) are more reliable than non-peer reviewed applications [ 47 ]. There are a few articles that discuss the accuracy of some selected applications [ 47 , 127 ]. Our future work is to provide guidelines for developing and using smartphone-based healthcare applications in medical practices. The focus would be on medico-legal and ethical issues regarding use of smartphone-based healthcare applications.

Limitations

Many smartphone-based medical applications are available in online application stores (e.g., Apple’s App Store). However, most of them have not been discussed in the medical literature. Those healthcare applications were not included in this study. We would like to emphasize that our goal was to systematically review articles in the academic literature discussing smartphone-based healthcare applications.

In this study, we discussed many smartphone-based healthcare applications from the literature. These applications were grouped according to targeted users (i.e., clinicians, medical and nursing students, and patients). These applications are not intended to replace desktop applications, but to add to existing technologies for better healthcare. The functionalities of the applications are growing day by day and new functionalities are available with every major release. The work of healthcare professionals is very mobile in nature. Smartphones enable for advanced mobile communication between health professionals, makes medical formula calculations available anywhere anytime, and provides access to evidence-based medical resources including disease diagnosis guides, drug references, literature search, and continuing medical education materials at the point of care. In addition, smartphones enable health professionals to access to EMR systems from anywhere thus facilitating remote consultation and telemedicine. Moreover, performing simple medical exams such as visual acuity test is also viable using a smartphone. The wide adoption of smartphones by the general public emphasizes the opportunity of better mHealth and mobile telemedicine services through patient oriented applications, for example, patient education, disease self-management, and remote monitoring of patients.

Most of the applications discussed in the study are standalone applications. There is an immense need for developing guidelines for standardizing smartphone-based healthcare applications so that the applications are used together seamlessly for specific purposes and are integrated with HISs such as EMR and patient monitoring systems to maximize the power of mobile applications. This will enable healthcare professionals to use the applications in a more meaningful way for better patient care.

Smartphone-based applications are getting more attention in healthcare day by day; all of the 55 articles included in this systematic review were published after 2003 and 24 of these articles were published recently between January 2010 and April 2011. The full potential of smartphones has yet to be exploited. We believe that this study provides a better understanding and greater insight into the effectiveness of the smartphone-based healthcare applications in improving patient care and reducing healthcare expenses.

Appendix I: Smartphone Platform Overview

Smartphones.

Smartphones are essentially cell phones with advanced connectivity and computing capability. There is no standard definition of smartphone found in the industry. In the annual report published in 2010 by the U.S. Federal Communications Commission (FCC), smartphones are defined as mobile devices with cell-phone capability having “an HTML browser that allows easy access to the full, open Internet; an operating system that provides a standardized interface and platform for application developers; …a larger screen size than a traditional handset…and touch screens and/or a QWERTY keypad” [ 128 ]. Another report submitted to the FCC by the Telecommunication Industry Association (TIA) in 2010 defined a smartphone as “a mobile device that offers the most advanced compu-ting ability and connectivity available today….having intelligence similar to personal computers while offering the capabilities of a mobile phone…running robust operating systems (OS) software that provides platforms for…third-party applications….having more processing power and memory…multiple connections, such as WiFi and Bluetooth, multimedia applications, such as photos, music, and video, and GPS functions” [ 129 ].

The first commercial cell phone was released publicly in 1983, 10 years after a cell phone prototype was first publicly demonstrated in April 3, 1973 with a call placed by Martin Cooper, the general manager of Communications Systems division at Motorola [ 130 ]. Since then, the development of cell phone technology has advanced very quickly due to the huge demand. The advancement of cell phone technology includes more powerful microprocessors, larger screen, and longer battery life. In a parallel trend, handheld-sized computers or Personal Digital Assistants (PDAs) emerged in the early 1990’s with the advancement of computer technology. Most recently, cell-phone capability has been introduced in PDAs, reducing the need to carry two separate devices, and producing the so-called smartphone. Thus, smartphones are the convergent technology of cell phones and PDAs [ 131 ]. In addition to phone services (voice calling, text and multimedia messaging, etc.), the common functionality of smartphones includes e-mails, calendars, contact lists, task lists, camera and video capabilities, and Internet access [ 132 ]. Smartphones are also generally equipped with Bluetooth, WiFi and USB connectivity.

Smartphone Platforms

Smartphones are usually based on specially designed operating system (OS) platforms for mobile computing and phone services. These OS platforms are now capable of running third-party applications including medical and healthcare applications, which is a great advantage of smartphones [ 26 ]. There are six major smartphone OS platforms: Symbian OS, Palm OS, Windows Phone, BlackBerry, iOS, and Android. Figure 3 illustrates three-month market-share averages for different smartphone OS platforms in the U.S. from February 2010 to May 2011 [ 3 – 5 ]. Figures 4 and 5 illustrate worldwide market share forecasts of smartphone platforms through 2015, published during the first quarter of 2011 by IDC [ 133 ] and Gartner [ 134 ] respectively.

Smartphone U.S. Market Share Feb-2010 to May-2011 [ 3 – 5 ] . This figure presents the market share of five major smartphone platforms (i.e. Palm Web OS, Windows Phone, BlackBerry, iOS, Android) in the United States during the period of 2010 – 2011. In the middle of 2011, Android has become the leader in smartphone market share while all other platforms have shown decreasing trend except iOS. The market share of iOS was almost consistent during this period.

Smartphone Worldwide Market Share Forecast 2015 [ 133 ] . This histogram presents the worldwide market share forecast data from IDC for six smartphone OS platforms in 2015. Android is predicted to be the global market leader in smartphones acquiring almost half of the market share by 2015.

Smartphone Worldwide Market Share Forecast 2010–2015 [ 134 ] . This figure presents the market share forecast data from Gartner for five smartphone OS platforms up to 2015. Android is predicted to be the leader in smartphone market by 2015 acquiring almost half of the market share. Symbian OS will lose almost all of the market share since its vendor Nokia announced in February, 2011 to shift from Symbian OS to Windows Phone 7 [ 153 ], thus global market share for Windows Phone is forecast to gain by 2015, placing the platform in second position.

An overview of OS features of smartphone platforms is illustrated in Table 1. All of the platforms provide standard applications such as organizers, contact lists, e-mail, Web browsers, photo galleries etc. The multitasking feature allows multiple applications to run concurrently. The notification system bar displays system status (e.g. battery, network, etc.) and notification (e.g. received text messages, email messages, etc.) while the user works on an application window, and the toolbar facilitates control buttons (e.g. maximize, minimize, close) for applications. In some smartphones, the home screen is customizable to allow the user to add or remove application shortcuts in the home screen, and also allows the user to drop widgets onto the home screen. The app folder facilitates storing applications in a special system-defined folder, and all recently used applications are displayed in recent apps . Smartphones now support text selection, copying to the clipboard, and pasting anywhere. The search box may combine searching of the Web and the device internally together, which is called universal search . The Adobe Flash support is very attractive to the user as it allows all the flash based resources (e.g. games, videos etc.) from the Web. HTTP or RTSP live streaming facilitates real-time audio or video streaming. Data security can be enhanced by storage area encryption , remote wiping (during loss or theft), and WiFi security for secure wireless data transmission. Some platforms also support accessibility features for the disabled [ 129 , 135 – 138 ] and multiple languages.

Table 2 illustrates the support of common features by smartphone OS platforms with the availability of hardware in the device. These are touch screens, multi-touch user interfaces, virtual keyboards, external keyboards, cameras, video recording, voice commands, tethering (Internet-connection sharing with other devices using cable, Bluetooth, WiFi, etc.), multi-core processor support, accelerometers, gyroscopes, screen re-orientation, Global Positioning System (GPS) features, Universal Serial Bus (USB) connections, and wireless connections (Bluetooth, WiFi, 3G/4G).

Symbian OS was developed by Symbian Ltd. and subsequently acquired by Nokia. The most recent release of this platform is Version 3 as of June 2011. This platform is prominent globally but not in the U.S. [ 6 , 134 ]. Symbian OS had nearly 40% of global market share in 2010 but was forecast to be below 1% by 2015 (Figures 4 and 5 ), which may be due to Nokia’s February 2011 announcement to shift from Symbian OS to Windows Phone 7 [ 133 , 134 , 139 ]. Symbian OS supports almost every feature listed in Tables 1 and 2 except recent apps, universal search (it supports internal search only), remote wiping, video calling (third-party software may be available), multi-core processor support, and gyroscopes. Stored-data encryption and accelerometers are available on selected devices only.

Palm Web OS

Palm Web OS (Version 2.3 as of June 2011) is the successor of Palm OS, which was introduced in January 2009 by Palm and acquired by Hewlett Packard (HP) in February 2011. Palm has a small market share in the U.S. with a decreasing trend during 2010–2011 (See Figure 3 ). Palm supports almost all the features listed in Tables 1 and 2 except recent apps, app folders, widgets, storage-area encryption, video calling, voice commands, and gyroscopes. Palm provides multi-core processor support but this is currently available in touch-pad devices only, not in smartphone devices.

Windows Phone

Windows Phone (Version 7 as of June 2011) is the successor of the Windows CE and Windows Mobile platforms developed by Microsoft. During 2010–2011, Windows Phone lost about 9.3% of U.S. market share within a 15-month period (Figure 3 ); however, global market share is forecast to gain by 2015, placing the platform in second position (Figures 4 and 5 ). Nokia announced in February 2011 a switch from Symbian OS to Windows Phone [ 139 ]. The multitasking support in Windows Phone is restricted to allowing third-party applications to run limited actions in the background. Unsupported features are recent apps, app folders (folders are arranged in hubs), support for external keyboard, video calling, tethering, and multi-core processor support. Windows phones provide a hardware button to facilitate universal search.

BlackBerry (Version 6 as of June 2011) was developed by Research In Motion (RIM) of Canada and is very prominent in the U.S. BlackBerry lost 17.4% of U.S. market share in a 15-month period during 2010–2011 (Figure 3 ), but its global market share was forecast to decrease little before 2015 (Figures 4 and 5 ). BlackBerry Version 6 is only available in newly released BlackBerry devices, and devices with old platforms cannot be upgraded to version 6. This platform does not support recent apps, widgets, external keyboards, video calling, tethering, or multi-core processor support. The remote wiping feature is available through the BlackBerry Protect application, which is available free of cost.

iOS (Version 5 as of November 2011) was developed by Apple for their iPhone (the only smartphone based on iOS), iPod, and iPad. Its 2010–2011 U.S. market share was consistently around 25%, placing it in second position at the end of May 2011 (Figure 3 ). The global market share forecast is also consistent through 2015 with some fluctuations, remaining in third position by 2015 (Figures 4 and 5 ). This is the only platform among the six platforms discussed in this article that does not support the popular Adobe Flash Player . There is also no support for installing widgets, and multi-core processor support is available only on the iPad. The video calling functionality is available using the FaceTime application (developed by Apple) in iPhone 4, iPod, and iPad. This platform is very prominent for its user interface and multi-touch gesture functionality. Unlike other platforms, notifications are not displayed in the system bar and the user needs to slide down from the top to access notifications. The iPhone is only smartphone device available based on iOS, and no future plans have been announced to release others.

Android (Version 4.0 as of November 2011) is an open-source platform that was initially developed by Android and later purchased by Google. This platform is becoming prominent in the U.S. as well as globally [ 6 , 133 , 134 ]. Android U.S. market share increased from 9% to 38.1% within a 15-month period during 2010–2011, placing this platform in the top position (Figure 3 ). Android is predicted to be the global market leader in smartphones, acquiring nearly 50% market share by 2015 (Figures 4 and 5 ). This platform supports all the features listed in Tables 1 and 2, but some features (storage-area encryption, video calling, and multi-core processor support) are available in selected smartphone devices only. The system bar provides a software navigation button in addition to system status and notifications. Of the six major OS platforms, only Android 4.0 has built-in support for connecting to Bluetooth Health Device Profile (HDP) devices [ 140 ].

Smartphone Applications

The total number of applications in application stores (especially in the Apple’s App Store (iOS) and Google’s Android Market ) is growing very fast (Figure 6 ) [ 141 – 156 ]. According to the latest “Apple Press Info” on Apples’ App Store, there are more than 425,000 applications as of July, 2011 [ 141 ]. The Google’s Android Market has more than 352,800 applications as of November, 2011 according to the recent update from Distimo [ 152 ]. The total number of applications for other four OS platforms is very low. As of March 2011, a total of 29,920 applications are available in the Ovi store (Symbian OS), 26,771 in the BlackBerry App World, 11,731 in the MarketPlace (Windows Phone), and 6,363 in the Palm App Catalog [ 151 ]. Overall, the iOS leads in the number of applications, but its growth rate is much slower than Android’s (Figure 6 ). As seen in Figures 3 , 4 and 5 , Android is currently leading both of the U.S. and Global market share, and the growth rate of its application stores is almost proportional to the increase in market share during the period of 2010 – 2011. Android also leads in the total number of free applications, and has become an increasingly popular competitor of iOS (for iPhone) [ 151 , 157 ]. The market share of iOS is almost consistent during the period of 2010 – 2011 though its application store size increased (Refer to Figures 3 , 4 and 5 for details).

Number of Applications in Apple’s App Store and Google’s Android Market (July’08 – Nov’11) [ 141 – 156 ] . This figure presents the growth rate of two major smartphone application stores: Apples’s App Store and Google’s Android Market, during the period of July, 2008 to November 2011. Both of the stores are growing very fast. According to the most recent updates, the total number of applications in Apple’s App Store is more than 425,000 as of July, 2011 [ 141 ] and in Android Market is more than 352,800 as of November, 2011 [ 152 ]. Overall, the Apple’s App Store is currently leading in terms of the application store size; however, the growth rate is much slower than Android Market.

Appendix II: Healthcare Applications for Smartphones

In this study, a total of 83 smartphone-based healthcare applications were discussed. These applications were grouped by the targeted user of the applications, as divided into three groups: (1) 57 applications for healthcare professionals, (2) 11 applications for medical or nursing students, and (3) 15 applications for patients. The functionalities of the applications and supported smartphone platforms were discussed and presented in tabular format. Tables 3, 4, 5, 6, 7, 8, 9 and 10 presents a total of 57 applications for healthcare professionals; Table 3: 21 disease diagnosis applications, Table 4: 6 drug reference applications, Table 5: 8 medical calculator applications, Table 6: 6 literature search applications, Table 7: 3 clinical communication applications, Table 8: 4 HIS client applications, Table 9: 2 medical training applications, and Table 10: 7 general healthcare applications. The applications for medical and nursing students and patients are listed in Tables 11 and 12 respectively.

Garritty C, El Emam K: Who’s using PDAs? Estimates of PDA use by health care providers: a systematic review of surveys. Journal of medical Internet research. 2006, 8: e7-10.2196/jmir.8.2.e7.

PubMed   PubMed Central   Google Scholar  

Physician smartphone adoption rate to reach 81% in 2012. 2012, [ http://manhattanresearch.com/News-and-Events/Press-Releases/physician-smartphones-2012 ]

comScore Reports. 2010, [ http://www.comscore.com/Press_Events/Press_Releases/2011/1/comScore_Reports_November_2010_U.S._Mobile_Subscriber_Market_Share ], November U.S. Mobile Subscriber Market Share

comScore Reports. 2010, [ http://www.comscore.com/Press_Events/Press_Releases/2010/7/comScore_Reports_May_2010_U.S._Mobile_Subscriber_Market_Share ], May U.S. Mobile Subscriber Market Share

comScore Reports. 2011, [ http://www.comscore.com/Press_Events/Press_Releases/2011/7/comScore_Reports_May_2011_U.S._Mobile_Subscriber_Market_Share ], May U.S. Mobile Subscriber Market Share

US Mobile Year in Review. [ http://www2.comscore.com/l/1552/ileYearinReview2010Webinar-pdf/QOZR3 ]

Ammenwerth E, Buchauer A, Bludau B, Haux R: Mobile information and communication tools in the hospital. International journal of medical informatics. 2000, 57: 21-40. 10.1016/S1386-5056(99)00056-8.

CAS   PubMed   Google Scholar  

Banitsas KA, Georgiadis P, Tachakra S, Cavouras D: Engineering in Medicine and Biology Society, 2004. 2004, San Franscisco, CA: IEEE: IEMBS’04. 26th Annual International Conference of the IEEE, 3105-3108. 2nd

Google Scholar  

Bardram JE: Activity-based computing: support for mobility and collaboration in ubiquitous computing. Personal and Ubiquitous Computing. 2005, 9: 312-322. 10.1007/s00779-004-0335-2.

Bardram JE, Bossen C: Mobility Work: The Spatial Dimension of Collaboration at a Hospital. Computer Supported Cooperative Work (CSCW). 2005, 14: 131-160. 10.1007/s10606-005-0989-y.

Burdette SD, Herchline TE, Oehler R: Practicing medicine in a technological age: using smartphones in clinical practice. Clin Infect Dis. 2008, 47: 117-122. 10.1086/588788.

PubMed   Google Scholar  

Wu RC, Morra D, Quan S, Lai S, Zanjani S, Abrams H, Rossos PG: The use of smartphones for clinical communication on internal medicine wards. J Hosp Med. 2010, 5: 553-559. 10.1002/jhm.775.

Junglas I, Abraham C, Ives B: Mobile technology at the frontlines of patient care: Understanding fit and human drives in utilization decisions and performance. Decis Support Syst. 2009, 46: 634-647. 10.1016/j.dss.2008.11.012.

Sherry J, Salvador T: Running and grimacing: the struggle for balance in mobile work. Wireless world: social and interactional aspects of the mobile age. Edited by: Brown B, Green N, Harper R. 2001, New York, NY: Springer-Verlag, 108-120.

Lindquist AM, Johansson PE, Petersson GI, Saveman BI, Nilsson GC: The use of the Personal Digital Assistant (PDA) among personnel and students in health care: a review. Journal of medical Internet research. 2008, 10: e31-10.2196/jmir.1038.

Phua J, Lim TK: How residents and interns utilise and perceive the personal digital assistant and UpToDate. BMC medical education. 2008, 8: 39-10.1186/1472-6920-8-39.

Serdar MA, Turan M, Cihan M: Rapid access to information resources in clinical biochemistry: medical applications of Personal Digital Assistants (PDA). Clinical and experimental medicine. 2008, 8: 117-122. 10.1007/s10238-008-0166-y.

New Tool in the MD’s Bag: A Smartphone. [ http://www.washingtonpost.com/wp-dyn/content/article/2009/05/18/AR2009051802234.html ]

Knowledge on call: Physicians increasingly are discovering smartphones serve a purpose beyond being a convenient communication gadget. [ http://www.ama-assn.org/amednews/2009/01/05/plus/bisa0105.pdf ]

Gamble K: Beyond phones. With the proper infrastructure, smartphones can help improve clinician satisfaction and increase EMR use. Healthcare informatics: the business magazine for information and communication systems. 2009, 26: 23-24.

M B: Proceedings of Ume°a’s 14th Student Conference in Computing Science. Opportunities and Challenges when Applying Mobile Technology in Health Care. 2010, UME°A, SWEDEN: UME°A UNIVERSITY, 1-12.

Choudhri AF, Radvany MG: Initial Experience with a Handheld Device Digital Imaging and Communications in Medicine Viewer: OsiriX Mobile on the iPhone. Journal of Digital Imaging. 2010, 24: 184-189.

PubMed Central   Google Scholar  

Sarasohn-Kahn J: How Smartphones Are Changing Health Care for Consumers and Providers. 2010, Oakland, CA: California HealthCare Foundation, http://posterous.com/getfile/files.posterous.com/mobihealth/Pqvz1Ff7GcGuEwfNYj1WFMiuBfqUe0h2NY7fqAisAPu0okpu0fmuCGAytEI0/HowSmartPhonesChngHealthcare.pdf ,

Millán M, Muñoz A, de la Villa M, Maña MJ: A Biomedical Information Retrieval System based on Clustering for Mobile Devices. Procesamiento del lenguaje natural. 2010, 255-258.

Zolfo M, Iglesias D, Kiyan C, Echevarria J, Fucay L, Llacsahuanga E, de Waard I, Suarez V, Castillo Llaque W, Lynen L: Mobile learning for HIV/AIDS healthcare worker training in resource-limited settings. AIDS Res Ther. 2010, 7: 35-10.1186/1742-6405-7-35.

Lippi G, Plebani M: Laboratory applications for smartphones: Risk or opportunity?. Clin Biochem. 2011, 44: 273-10.1016/j.clinbiochem.2010.12.016.

Eysenbach G: Consumer health informatics. British Medical Journal. 2000, 320: 1713-10.1136/bmj.320.7251.1713.

CAS   PubMed   PubMed Central   Google Scholar  

U.S. Department of Health and Human Services, Centers for Disease Control and Prevention: National diabetes fact sheet: national estimates and general information on diabetes and prediabetes in the United States. 2011, Atlanta, GA: Department of Health and Human Services, Centers for Disease Control and Prevention, [ http://www.cdc.gov/diabetes/pubs/pdf/ndfs_2011.pdf ]

Marshall A, Medvedev O, Antonov A: Use of a smartphone for improved self-management of pulmonary rehabilitation. International journal of telemedicine and applications. 2008

Hong S, Kim S, Kim J, Lim D, Jung S, Kim D, Yoo S: 31st Annual International Conference of the IEEE EMBS. Portable emergency telemedicine system over wireless broadband and 3G networks. 2009, Minneapolis, Minnesota, USA: IEEE, 1250-1253.

Krishna S, Boren SA, Balas EA: Healthcare via cell phones: a systematic review. Telemedicine and e-Health. 2009, 15: 231-240. 10.1089/tmj.2008.0099.

Charpentier G, Benhamou P-Y, Dardari D, Clergeot A, Franc S, Schaepelynck-Belicar P, Catargi B, Melki V, Chaillous L, Farret A, Bosson J-L, Penfornis A: The Diabeo Software Enabling Individualized Insulin Dose Adjustments Combined With Telemedicine Support Improves HbA1c in Poorly Controlled Type 1 Diabetic Patients: A 6-month, randomized, open-label, parallel-group, multicenter trial (TeleDiab 1 Study). Diabetes Care. 2011, 34: 533-539. 10.2337/dc10-1259.

Demaerschalk BM: Telemedicine or Telephone Consultation in Patients with Acute Stroke. Current neurology and neuroscience reports. 2011, 11: 42-51. 10.1007/s11910-010-0147-x.

Liang X, Wang Q, Yang X, Cao J, Chen J, Mo X, Huang J, Wang L, Gu D: Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabet Med. 2011, 28: 455-463.

Worringham C, Rojek A, Stewart I, Miranda JJ: Development and Feasibility of a Smartphone. ECG and GPS Based System for Remotely Monitoring Exercise in Cardiac Rehabilitation. PloS one. 2011, 6: e14669-10.1371/journal.pone.0014669.

Brock TP, Smith SR: Using digital videos displayed on personal digital assistants (PDAs) to enhance patient education in clinical settings. International journal of medical informatics. 2007, 76: 829-835. 10.1016/j.ijmedinf.2006.09.024.

Cole-Lewis H: Text messaging as a tool for behavior change in disease prevention and management. Epidemiologic reviews. 2010, 32: 56-69. 10.1093/epirev/mxq004.

Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ (Clinical research ed.). 2009, 339: b2535-10.1136/bmj.b2535.

Oehler RL, Smith K, Toney JF: Infectious diseases resources for the iPhone. Clin Infect Dis. 2010, 50: 1268-1274. 10.1086/651602.

Richardson WS, Burdette SD: Practice corner: taking evidence in hand. Evidence Based Medicine. 2003, 8: 4-10.1136/ebm.8.1.4.

Pope L, Silva P, Almeyda R: i-Phone applications for the modern day otolaryngologist. Clin Otolaryngol. 2010, 35: 350-354. 10.1111/j.1749-4486.2010.02170.x.

Busis N: Mobile Phones to Improve the Practice of Neurology. Neurol Clin. 2010, 28: 395-410. 10.1016/j.ncl.2009.11.001.

Stroud SD, Smith CA, Erkel EA: Personal digital assistant use by nurse practitioners: a descriptive study. Journal of the American Academy of Nurse Practitioners. 2009, 21: 31-38. 10.1111/j.1745-7599.2008.00368.x.

Schreiber WE, Busser JR, Huebsch S: A portable laboratory test reference for handheld computers: evaluation on an internal medicine clerkship. Am J Clin Pathol. 2008, 129: 439-444. 10.1309/5044VKTB6MMMNVGM.

León SA, Fontelo P, Green L, Ackerman M, Liu F: Evidence-based medicine among internal medicine residents in a community hospital program using smart phones. BMC Medical Informatics and Decision Making. 2007, 7: 5-10.1186/1472-6947-7-5.

Burdette SD, Herchline TE, Richardson WS: Killing bugs at the bedside: a prospective hospital survey of how frequently personal digital assistants provide expert recommendations in the treatment of infectious diseases. Ann Clin Microbiol Antimicrob. 2004, 3: 22-10.1186/1476-0711-3-22.

Miller SM, Beattie MM, Butt AA: Personal digital assistant infectious diseases applications for health care professionals. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2003, 36: 1018-1029. 10.1086/368198.

CAS   Google Scholar  

Joundi RA, Brittain JS, Jenkinson N, Green AL, Aziz T: Rapid tremor frequency assessment with the iPhone accelerometer. Parkinsonism & Related Disorders. 2011, 17: 288-290. 10.1016/j.parkreldis.2011.01.001.

Mezzana P, Scarinci F, Marabottini N: Augmented reality in oculoplastic surgery: first iPhone application. Plast Reconstr Surg. 2011, 127: 57e-58e. 10.1097/PRS.0b013e31820632eb.

Lapinsky SE: Mobile computing in critical care. Journal of critical care. 2007, 22: 41-44. 10.1016/j.jcrc.2006.12.007.

Dala-Ali BM, Lloyd MA, Al-Abed Y: The uses of the iPhone for surgeons. Surgeon. 2011, 9: 44-48. 10.1016/j.surge.2010.07.014.

Johansson PE, Petersson GI, Nilsson GC: Personal digital assistant with a barcode reader–A medical decision support system for nurses in home care. International journal of medical informatics. 2010, 79: 232-242. 10.1016/j.ijmedinf.2010.01.004.

Chatterley T, Chojecki D: Personal digital assistant usage among undergraduate medical students: exploring trends, barriers, and the advent of smartphones. Journal of the Medical Library Association. 2010, 98: 157-160. 10.3163/1536-5050.98.2.008.

Eknoyan G: Adolphe Quetelet (1796–1874)–the average man and indices of obesity. Nephrology, dialysis, transplantation: official publication of the European Dialysis and Transplant Association - European Renal Association. 2008, 23: 47-51.

Baumgart DC: Personal digital assistants in health care: experienced clinicians in the palm of your hand?. Lancet. 2005, 366: 1210-1222. 10.1016/S0140-6736(05)67484-3.

Hunter T, Hardwicke J: The smart phone: An indispensable tool for the plastic surgeon?. Journal of Plastic, Reconstructive & Aesthetic Surgery. 2010, 63: e426-e427. 10.1016/j.bjps.2009.11.010.

Alexander G, Hauser S, Steely K, Ford G, Demner-Fushman D: A usability study of the PubMed on Tap user interface for PDAs. Studies in health technology and informatics. 2004, 107: 1411-1415.

Hauser SE, Demner-Fushman D, Ford G, Thoma GR: PubMed on Tap: discovering design principles for online information delivery to handheld computers. Studies in health technology and informatics. 2004, 107: 1430-1433.

Sutton VR, Hauser SE: Preliminary comparison of the Essie and PubMed search engines for answering clinical questions using MD on Tap, a PDA-based program for accessing biomedical literature. AMIA Annu Symp Proc. 2005, 2005: 1128-

Fontelo P, Liu F, Ackerman M: askMEDLINE: a free-text, natural language query tool for MEDLINE/PubMed. BMC Medical Informatics and Decision Making. 2005, 5: 5-10.1186/1472-6947-5-5.

Fontelo P, Liu F, Ackerman M: MeSH Speller + askMEDLINE: auto-completes MeSH terms then searches MEDLINE/PubMed via free-text, natural language queries. AMIA Annu Symp Proc. 2005, 2005: 957-

Fontelo P, Liu F, Ackerman M, Schardt CM, Keitz SA: askMEDLINE: a report on a year-long experience. AMIA 2006 Symposium. 2006, 2005: 923-

Demner-Fushman D, Hauser SE, Humphrey SM, Ford GM, Jacobs JL, Thoma GR: AMIA Annual Symposium Proceedings. MEDLINE as a source of just-in-time answers to clinical questions. 2006, American Medical Informatics Association, 190-194.

Hauser SE, Demner-Fushman D, Jacobs JL, Humphrey SM, Ford G, Thoma GR: Using wireless handheld computers to seek information at the point of care: an evaluation by clinicians. Journal of the American Medical Informatics Association. 2007, 14: 807-10.1197/jamia.M2424.

MEDLINE database on Tap (MDOT). [ http://mdot.nlm.nih.gov/proj/mdot/mdot.php ]

Soto RG, Chu LF, Goldman JM, Rampil IJ, Ruskin KJ: Communication in critical care environments: mobile telephones improve patient care. Anesth Analg. 2006, 102: 535-541. 10.1213/01.ane.0000194506.79408.79.

Hasvold PE, Scholl J: Disrupted rhythms and mobile ICT in a surgical department. International Journal of Medical Informatics. 2011, 80: e72-e84. 10.1016/j.ijmedinf.2011.01.006.

Volonté F, Robert JH, Ratib O, Triponez F: A lung segmentectomy performed with 3D reconstruction images available on the operating table with an iPad. Interact Cardiovasc Thorac Surg. 2011, 12: 1066-1068. 10.1510/icvts.2010.261073.

Broderick GA, Abdolrasulnia M: Men’s sexual health: evaluating the effectiveness of print- and PDA-based CME. The journal of sexual medicine. 2009, 6: 2417-2424. 10.1111/j.1743-6109.2009.01270.x.

Low D, Clark N, Soar J, Padkin A, Stoneham A, Perkins GD, Nolan J: A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency. Anaesthesia. 2011, 66: 255-262. 10.1111/j.1365-2044.2011.06649.x.

Semeraro F, Taggi F, Tammaro G, Imbriaco G, Marchetti L, Cerchiari EL: iCPR: a new application of high-quality cardiopulmonary resuscitation training. Resuscitation. 2011, 82: 436-441. 10.1016/j.resuscitation.2010.11.023.

Strayer SM, Rollins LK, Martindale JR: A handheld computer smoking intervention tool and its effects on physician smoking cessation counseling. The Journal of the American Board of Family Medicine. 2006, 19: 350-10.3122/jabfm.19.4.350.

Correia R, Kon F, Kon R: Proceedings of the 2008 ACM symposium on Applied computing. A mobile telehealth system for primary homecare. 2008, Fortaleza, Ceará, Brazil: Borboleta, 1343-1347.

Focosi D: Smartphone utilities for infectious diseases specialists. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2008, 47: 1234-1235.

van Ettinger M, Lipton J, Nelwan S, van Dam T, van der Putten N: Computing in Cardiology 2010. Multimedia paging for clinical alarms on mobile platforms. 2010, Belfast, U: IEEE, 57-60.

Cibulka NJ, Crane-Wider L: Introducing Personal Digital Assistants to Enhance Nursing Education in Undergraduate and Graduate Nursing Programs. J Nurs Educ. 2010, 50: 115-118.

Kho A, Henderson LE, Dressler DD, Kripalani S: Use of handheld computers in medical education. A systematic review. Journal of general internal medicine. 2006, 21: 531-537. 10.1111/j.1525-1497.2006.00444.x.

Hansen TR, Eklund JM, Sprinkle J, Bajcsy R, Sastry S: European Medicine, Biology and Engineering Conference. Using smart sensors and a camera phone to detect and verify the fall of elderly persons. 2005, Citeseer: Prague, Czech Republic

Ryan D, Cobern W, Wheeler J, Price D, Tarassenko L: Mobile phone technology in the management of asthma. Journal of telemedicine and telecare. 2005, 11: 43-46. 10.1258/1357633054461714.

Zhang T, Wang J, Liu P, Hou J: Fall detection by embedding an accelerometer in cellphone and using KFD algorithm. International Journal of Computer Science and Network Security: IJCSNS. 2006, 6: 277-284.

Sposaro F, Tyson G: 31st Annual International Conference of the IEEE EMBS. iFall: An android application for fall monitoring and response. 2009, Minneapolis, Minnesota: IEEE; 2009, 6122-6119.

Bexelius C, Löf M, Sandin S, Lagerros YT, Forsum E, Litton JE: Measures of physical activity using cell phones: validation using criterion methods. Journal of Medical Internet Research. 2010, 12: 1-15. 10.2196/jmir.1337.

Boulos MNK, Wheeler S, Tavares C, Jones R: How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomedical engineering online. 2011, 10: 24-10.1186/1475-925X-10-24.

Csernansky JG, Smith MJ: Thought, Feeling, and Action in Real Time–Monitoring of Drug Use in Schizophrenia. Am J Psychiatry. 2011, 168: 120-10.1176/appi.ajp.2010.10111601.

Swendsen J, Ben-Zeev D, Granholm E: Real-time electronic ambulatory monitoring of substance use and symptom expression in schizophrenia. Am J Psychiatry. 2011, 168: 202-209. 10.1176/appi.ajp.2010.10030463.

Ly K: MHealth: better health through your smartphone. Community practitioner: the journal of the Community Practitioners’ & Health Visitors’ Association. 2011, 84: 16-17.

Constructing EMA Studies with PMAT: The Purdue Momentary Assessment Tool User’s Manual. [ http://www.ruf.rice.edu/~dbeal/pmatusermanual.pdf ]

Romana DM, Nada P, Andre K: Group differences in physician responses to handheld presentation of clinical evidence: a verbal protocol analysis. BMC Medical Informatics and Decision Making. 2007, 7: 22-10.1186/1472-6947-7-22.

Patel PD, Greenberg RB, Hughes Miller K, Carter MB, Ziegler CH: Assessing Medical Students’, Residents’, and the Public’s Perceptions of the Uses of Personal Digital Assistants. Medical education online. 2008, 13: 9-

Di Pietro T, Coburn G, Dharamshi N, Doran D, Mylopoulos J, Kushniruk A, Nagle L, Sidani S, Tourangeau A, Laurie-Shaw B, Lefebre N, Reid-Haughian C, Carryer J, McArthur G: What nurses want: diffusion of an innovation. Journal of nursing care quality. 2008, 23: 140-146. 10.1097/01.NCQ.0000313763.87019.0a.

Faulk J: Intensive care nurses’ interest in clinical personal digital assistants. Critical Care Nurse. 2009, 29: 58-64.

Prgomet M, Georgiou A, Westbrook JI: The impact of mobile handheld technology on hospital physicians’ work practices and patient care: a systematic review. Journal of the American Medical Informatics Association. 2009, 16: 792-801. 10.1197/jamia.M3215.

Sackett DL, Straus SE: Finding and applying evidence during clinical rounds: the “evidence cart”. JAMA: the journal of the American Medical Association. 1998, 280: 1336-1338. 10.1001/jama.280.15.1336.

Ramos K, Linscheid R, Schafer S: Real-time information-seeking behavior of residency physicians. Family medicine. 2003, 35: 257-260.

Tempelhof MW: Personal digital assistants: a review of current and potential utilization among medical residents. Teaching and learning in medicine. 2009, 21: 100-104. 10.1080/10401330902791321.

Jacsó P: Natural language searching. Online Information review. 2004, 28: 75-79.

Plovnick RM, Zeng QT: Reformulation of consumer health queries with professional terminology: a pilot study. Journal of medical Internet research. 2004, 6: e27-10.2196/jmir.6.3.e27.

Demner-Fushman D, Hauser SE, Ford G, Thoma GR: Organizing literature information for clinical decision support. Studies in health technology and informatics. 2004, 107: 602-606.

Cortizo JC, Gachet D, de Buenaga M, Maña M: Second International Workshop on User-Centric Technologies and applications (MADRINET’08). Extending PubMed on Tap by means of MultiDocument Summarization. 2008, Spain: Salamanca

de Buenaga M, Gachet D, Maña MJ, de la Villa M, Mata J: SIGIR 2008 Workshop on Mobile Information Retrieval. Clustering and Summarizing Medical Documents to Improve Mobile Retrieval. 2008, Singapore, 54-57.

Maokola W, Willey BA, Shirima K, Chemba M, Armstrong Schellenberg JRM, Mshinda H, Alonso P, Tanner M, Schellenberg D: Enhancing the routine health information system in rural southern Tanzania: successes, challenges and lessons learned. Tropical medicine & international health. 2011, 16: 721-30. 10.1111/j.1365-3156.2011.02751.x.

Health Information Portability and Accountability Act (HIPAA). [ http://www.hhs.gov/ocr/hipaa/ ]

Directive 95/46/EC of the European Parliament and of the Council of 24. 1995, [ http://eur-lex.europa.eu/smartapi/cgi/sga_doc?smartapi!celexplus!prod!DocNumber&lg=en&type_doc=Directive&an_doc=95&nu_doc=46 ], October on the protection of individuals with regard to the processing of personal data and on the free movement of such data

The NHS Constitution. [ http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_113645.pdf ]

Bønes E, Hasvold P, Henriksen E, Strandenæs T: Risk analysis of information security in a mobile instant messaging and presence system for healthcare. International journal of medical informatics. 2007, 76: 677-687. 10.1016/j.ijmedinf.2006.06.002.

Tice AD: Gram stains and smartphones. Clin Infect Dis. 2011, 52: 278-9. 10.1093/cid/ciq131.

FDA may regulate iPhone Health Apps. [ http://mobihealthnews.com/474/fda-may-regulate-iphone-health-apps/ ]

Mashman W: The iPad in cardiology: tool or toy? JACC. Cardiovascular interventions. 2011, 4: 258-259. 10.1016/j.jcin.2011.01.001.

Payne D, Godlee F: The BMJ is on the iPad. BMJ. 2011, 342: 184-

Luanrattana R, Win KT, Fulcher J: Use of personal digital assistants (PDAs) in medical education. International Symposium on Computer-Based Medical Systems (CBMS’07). Edited by: Twentieth IEEE. 2007, Maribor, Slovenia: IEEE Computer Society, 307-312.

What is mHealth?. [ http://www.mhealthalliance.org/about/frequently-asked-questions ]

mHealth: New horizons for health through mobile technologies. [ http://www.who.int/goe/publications/goe_mhealth_web.pdf ]

Mobile Health Market Report 2010–2015: The Impact of Smartphone Applications on the Mobile Health Industry. http://www.research2guidance.com/shop/index.php/mhealth-report ,

Russell TG, Jones AF: Implications of regulatory requirements for smartphones, gaming consoles and other devices. Journal of physiotherapy. 2011, 57: 5-7. 10.1016/S1836-9553(11)70001-7.

Abroms LC, Padmanabhan N, Thaweethai L, Phillips T: iPhone Apps for Smoking Cessation: A Content Analysis. American Journal of Preventive Medicine. 2011, 40: 279-285. 10.1016/j.amepre.2010.10.032.

Health Device Profile. [ http://www.bluetooth.org/DocMan/handlers/DownloadDoc.ashx?doc_id=119999 ]

Haller G, Haller DM, Courvoisier DS, Lovis C: Handheld vs. laptop computers for electronic data collection in clinical research: a crossover randomized trial. Journal of the American Medical Informatics Association. 2009, 16: 651-10.1197/jamia.M3041.

Shaw CI, Kacmarek RM, Hampton RL, Riggi V, El Masry A, Cooper JB, Hurford WE: Cellular phone interference with the operation of mechanical ventilators. Crit Care Med. 2004, 32: 928-931. 10.1097/01.CCM.0000120061.01431.DB.

Lapinsky SE, Easty AC: Electromagnetic interference in critical care. Journal of critical care. 2006, 21: 267-270. 10.1016/j.jcrc.2006.03.010.

Mayo Clinic Proceedings. Tri JL, Severson RP, Hyberger LK, Hayes DL. 2007, Mayo Clinic, 282-285. 82nd

van Lieshout EJ, van der Veer SN, Hensbroek R, Korevaar JC, Vroom MB, Schultz MJ: Interference by new-generation mobile phones on critical care medical equipment. 2007, London, England: Critical care, R98-11th

McCord G, Pendleton BF, Schrop SL, Weiss L, Stockton L, Hamrich LM: Assessing the impact on patient-physician interaction when physicians use personal digital assistants: a Northeastern Ohio Network (NEON) study. Journal of the American Board of Family Medicine: JABFM. 2009, 22: 353-359. 10.3122/jabfm.2009.04.080056.

Wi-Fi® enabled BlackBerry® smartphones: WLAN Support and Security Features. http://us.blackberry.com/ataglance/networks/WiFiCellularWhitepaper.pdf ,

iPhone in Business: Security Overview. [ http://images.apple.com/iphone/business/docs/iPhone_Security.pdf ]

Palm webOS Security Overview for Enterprise. [ http://www.hpwebos.com/us/assets/pdfs/business/Palm_WhitePaper_Security.pdf ]

Device Administration. [ http://developer.android.com/guide/topics/admin/device-admin.html ]

Barrons R: Evaluation of personal digital assistant software for drug interactions. American journal of health-system pharmacy: AJHP: official journal of the American Society of Health-System Pharmacists. 2004, 61: 380-385.

Annual Report and Analysis of Competitive Market Conditions With Respect to Mobile Wireless, Including Commercial Mobile Services. [ http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-10-81A1.pdf ]

Accessible Mobile Phone Options for People who are Blind Deaf-blind, or Have Low Vision. [ http://www.tiaonline.org/gov_affairs/fcc_filings/documents/Low%20Vision%20PN_TIA%20Comments_Final_09%2013%2010.pdf ]

Father of the cell phone: Marty Cooper, the pioneer of mobile telephony, has spent his entire career pushing wireless communications to new heights. [ http://www.economist.com/node/13725793?story_id=13725793 ]

Bertman J: Tech 101: a new generation of smartphones. MDNG Neurology. 2009, 11: 26-

Hoyt RE, Cruz RW: Chapter 11: Mobile Technology. In Medical Informatics. Practical Guide for Healthcare and Information Technology Professionals. Edited by: Hoyt RE. 2010, Lulu Enterprises Inc, 199-217. 4th

IDC Forecasts Worldwide Smartphone Market to Grow by Nearly 50% in. 2011, [ http://www.idc.com/getdoc.jsp?containerId=prUS22762811 ]

Gartner Says Android to Command Nearly Half of Worldwide Smartphone Operating System Market by Year-End. 2012, [ http://www.gartner.com/it/page.jsp?id=1622614 ]

Designing for Accessibility. [ http://developer.android.com/guide/practices/design/accessibility.html ]

iOS Accessibility. [ http://developer.apple.com/technologies/ios/accessibility.html ]

Making mobile devices accessible for all. [ http://www.nokiaaccessibility.com/ ]

Using accessibility devices with Treo 700p smartphones (TTY, TDD, VCO, HCO, and more). [ http://kb.hpwebos.com/wps/portal/kb2/common/article/18100_en.html ]

Nokia’s new strategy: Windows Phone 7. [ http://money.cnn.com/2011/02/11/technology/nokia_microsoft/index.htm ]

Android 4.0 Platform Highlights. [ http://developer.android.com/sdk/android-4.0-highlights.html ]

Apple’s App Store Downloads Top 15 Billion. [ http://www.apple.com/pr/library/2011/07/07Apples-App-Store-Downloads-Top-15-Billion.html ]

Apple’s App Store Downloads Top 10 Billion. [ http://www.apple.com/pr/library/2011/01/22Apples-App-Store-Downloads-Top-10-Billion.html ]

iPod + iTunes Timeline. [ http://www.apple.com/pr/products/ipodhistory/ ]

Apple Presents iPhone 4. [ http://www.apple.com/pr/library/2010/06/07Apple-Presents-iPhone-4.html ]

Apple Announces Over 100,000 Apps Now Available on the App Store. [ http://www.apple.com/pr/library/2009/11/04Apple-Announces-Over-100-000-Apps-Now-Available-on-the-App-Store.html ]

Apple’s App Store Downloads Top Two Billion. [ http://www.apple.com/pr/library/2009/09/28Apples-App-Store-Downloads-Top-Two-Billion.html ]

Apple’s App Store Downloads Top 1.5 Billion in First Yea. [ http://www.apple.com/pr/library/2009/07/14Apples-App-Store-Downloads-Top-1-5-Billion-in-First-Year.html ]

Apple’s Revolutionary App Store Downloads Top One Billion in Just Nine Months. [ http://www.apple.com/pr/library/2009/04/24Apples-Revolutionary-App-Store-Downloads-Top-One-Billion-in-Just-Nine-Months.html ]

App Store Downloads Top 100 Million Worldwide. [ http://www.apple.com/pr/library/2008/09/09App-Store-Downloads-Top-100-Million-Worldwide.html ]

Android Market Insights. 2011, [ http://www.research2guidance.com/shop/index.php/android-market-insights-september-2011 ], September

Distimo: The battle for the most content and the emerging tablet market. [ http://www.distimo.com/publications/ ]

Google Android Market. [ http://www.distimo.com/appstores/stores/view/19-Google_Android_Market ]

Android Market Needs More Filters, T-Mobile Says. [ http://www.pcworld.com/article/161410/android_market_needs_more_filters_tmobile_says.html ]

Introducing the T-Mobile G2 with Google — the First Smartphone Delivering 4G Speeds on T-Mobile’s Super-Fast HSPA+ Network. [ http://www.htc.com/us/press/introducing-the-t-mobile-g2-with-google--the-first-smartphone-delivering-4g-speeds-on-t-mobiles-super-fast-hspa-network/19 ]

Google Offers New Model for Consumers to Buy a Mobile Phone. [ http://investor.google.com/releases/2010/0105.html ]

Android Market: 250,000+ apps; Over 6 billion downloads. [ http://telematicsnews.info/2011/07/15/android-market-250000-apps-over-6-billion-downloads_jl2151/ ]

Amin K, Fu B, Lim J: Re: iPhone applications for the modern-day Otolaryngologist. Clin Otolaryngol. 2011, 36: 90-91.

Symbian C++. [ http://www.developer.nokia.com/Develop/Other_Technologies/Symbian_C ]

HP webOS. [ http://www.hpwebos.com/us/products/software/webos/index.html ]

HP webOS 2.0 Implementation Guide. [ http://www.hpwebos.com/us/assets/pdfs/business/HP_webOS_2.0_Implementation_Guide_3-8-2011.pdf ]

Windows Phone: Discover. [ http://www.microsoft.com/windowsphone/en-us/features/default.aspx ]

BlackBerry 6. [ http://us.blackberry.com/developers/blackberry6/ ]

BlackBerry 6. [ http://us.blackberry.com/apps-software/blackberry6/ ]

iOS Features. [ http://developer.apple.com/technologies/ios/features.html ]

iOS 4: The world’s most advanced mobile operating system. [ http://www.apple.com/iphone/ios4/ ]

iOS 5. [ http://www.apple.com/ios/ ]

Android 3.1 Platform Highlights. [ http://developer.android.com/sdk/android-3.1-highlights.html ]

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6947/12/67/prepub

Download references

Author information

Authors and affiliations.

University of Missouri Informatics Institute (MUII), 241 Engineering Building West, Columbia, MO, 65211, USA

Abu Saleh Mohammad Mosa, Illhoi Yoo & Lincoln Sheets

Health Management and Informatics (HMI) Department, University of Missouri School of Medicine, CS&E Bldg. DC006.00, Columbia, MO, 65212, USA

Department of Family and Community Medicine, University of Missouri School of Medicine, M226 Medical Sciences Building, DC032.00, Columbia, MO, 65212, USA

Lincoln Sheets

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Illhoi Yoo .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

ASMM formulated the study design, performed literature search, screened and reviewed the articles satisfying the eligibility criteria, collected data from each eligible article and drafted the manuscript. IY participated in the study design and helped draft the manuscript. LS helped draft the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, authors’ original file for figure 4, authors’ original file for figure 5, authors’ original file for figure 6, rights and permissions.

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Mosa, A.S.M., Yoo, I. & Sheets, L. A Systematic Review of Healthcare Applications for Smartphones. BMC Med Inform Decis Mak 12 , 67 (2012). https://doi.org/10.1186/1472-6947-12-67

Download citation

Received : 20 December 2011

Accepted : 14 June 2012

Published : 10 July 2012

DOI : https://doi.org/10.1186/1472-6947-12-67

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continue Medical Education
• Nursing Student
• Hospital Information System
• Client Application

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Mobile Health Applications in Weight Management: A Systematic Literature Review

Affiliations.

• 1 School of Social Sciences, Education and Social Work, Queen's University of Belfast, Belfast, Northern Ireland. Electronic address: [email protected].
• 2 Imec-SMIT, Vrije Universiteit Brussel, Brussels, Belgium.
• PMID: 31003801
• DOI: 10.1016/j.amepre.2018.12.005

Context: Weight management is an effective strategy for controlling chronic disease and maintaining physical health, and research on this topic has risen dramatically over the past four decades. The present systematic literature review aimed to identify existing evidence on the efficacy of mobile health technology in facilitating weight management behaviors, such as healthy food consumption and physical activity.

Evidence acquisition: A systematic search was conducted in Ovid MEDLINE and Ovid PsycINFO databases with the aim to identify studies published in peer-reviewed journal articles between 2012 and 2017.

Evidence synthesis: A total of 39 studies were analyzed in spring 2018 and are presented here in terms of participant characteristics, effective technology components, additional treatments, impact on health-related behaviors, and treatment efficacy. Indicators of study quality and social validity are also provided.

Conclusions: Mobile health apps are widely considered as satisfactory, easy to use, and helpful in the pursuit of weight loss goals by patients. The potential of mobile health apps in facilitating weight loss lies in their ability to increase treatment adherence through strategies such as self-monitoring. These findings indicate that satisfactory treatment adherence and consequent weight loss and maintenance are achieved in the presence of high levels of engagement with a mobile health app. The research quality assessment of RCTs reveals a great need for following international standards both when conducting and reporting research.

Copyright © 2019 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.

PubMed Disclaimer

Similar articles

• The use of mobile apps to improve nutrition outcomes: A systematic literature review. DiFilippo KN, Huang WH, Andrade JE, Chapman-Novakofski KM. DiFilippo KN, et al. J Telemed Telecare. 2015 Jul;21(5):243-53. doi: 10.1177/1357633X15572203. Epub 2015 Feb 12. J Telemed Telecare. 2015. PMID: 25680388 Review.
• A Comparison of Physical Activity Mobile Apps With and Without Existing Web-Based Social Networking Platforms: Systematic Review. Petersen JM, Prichard I, Kemps E. Petersen JM, et al. J Med Internet Res. 2019 Aug 16;21(8):e12687. doi: 10.2196/12687. J Med Internet Res. 2019. PMID: 31420956 Free PMC article.
• Understanding Time Series Patterns of Weight and Meal History Reports in Mobile Weight Loss Intervention Programs: Data-Driven Analysis. Kim J, Kam HJ, Kim Y, Lee Y, Lee JH. Kim J, et al. J Med Internet Res. 2020 Aug 11;22(8):e17521. doi: 10.2196/17521. J Med Internet Res. 2020. PMID: 32780028 Free PMC article.
• Can Mobile Phone Apps Influence People's Health Behavior Change? An Evidence Review. Zhao J, Freeman B, Li M. Zhao J, et al. J Med Internet Res. 2016 Oct 31;18(11):e287. doi: 10.2196/jmir.5692. J Med Internet Res. 2016. PMID: 27806926 Free PMC article. Review.
• Evidence-based strategies in weight-loss mobile apps. Pagoto S, Schneider K, Jojic M, DeBiasse M, Mann D. Pagoto S, et al. Am J Prev Med. 2013 Nov;45(5):576-82. doi: 10.1016/j.amepre.2013.04.025. Am J Prev Med. 2013. PMID: 24139770
• App Engagement as a Predictor of Weight Loss in Blended-Care Interventions: Retrospective Observational Study Using Large-Scale Real-World Data. Lehmann M, Jones L, Schirmann F. Lehmann M, et al. J Med Internet Res. 2024 Jun 7;26:e45469. doi: 10.2196/45469. J Med Internet Res. 2024. PMID: 38848556 Free PMC article.
• The effectiveness of mobile app usage in facilitating weight loss: An observational study. Huntriss R, Salimgaraev R, Nikogosov D, Powell J, Varady KA. Huntriss R, et al. Obes Sci Pract. 2024 May 13;10(3):e757. doi: 10.1002/osp4.757. eCollection 2024 Jun. Obes Sci Pract. 2024. PMID: 38745944 Free PMC article.
• MyTrack+: Human-centered design of an mHealth app to support long-term weight loss maintenance. Chen YP, Woodward J, Shankar MN, Bista D, Ugwoaba U, Brockmann A, Ross KM, Ruiz J, Anthony L. Chen YP, et al. Front Digit Health. 2024 Apr 22;6:1334058. doi: 10.3389/fdgth.2024.1334058. eCollection 2024. Front Digit Health. 2024. PMID: 38711677 Free PMC article.
• Clinical Service Incorporating Mobile Technology on Weight Loss in Patients With Metabolic Dysfunction-Associated Steatotic Liver Disease: A Translation From Research Trial. Ang SM, Lim SL, Dan YY, Chan YH, Yap QV, Chen J. Ang SM, et al. Endocrinol Diabetes Metab. 2024 May;7(3):e00485. doi: 10.1002/edm2.485. Endocrinol Diabetes Metab. 2024. PMID: 38685702 Free PMC article.
• Barriers and facilitators to parents' engagement with and perceived impact of a childhood obesity app: A mixed-methods study. Milne-Ives M, Rahman E, Bradwell H, Baines R, Boey T, Potter A, Lawrence W, Helena van Velthoven M, Meinert E. Milne-Ives M, et al. PLOS Digit Health. 2024 Mar 27;3(3):e0000481. doi: 10.1371/journal.pdig.0000481. eCollection 2024 Mar. PLOS Digit Health. 2024. PMID: 38536852 Free PMC article.

Publication types

• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• ClinicalKey
• Elsevier Science
• MedlinePlus Consumer Health Information
• MedlinePlus Health Information

Miscellaneous

• NCI CPTAC Assay Portal

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Aims & Scopes
• Editorial Board
• Young Academic Editors
• Subscription
• Current Issue
• Most Downloaded
• Advance Search
• Submission & Review
• Guide for Authors
• Publishing Process
• Publishing Ethics
• Peer View Process
• Article Publish Charge
• Submission Template

Systematic literature review on the applications, impacts, and public perceptions of autonomous vehicles in road transportation system

• Isaac Oyeyemi Olayode , 
• Bo Du , 
• Alessandro Severino , 
• Tiziana Campisi , 
• Frimpong Justice Alex

Export File

You can copy and paste references from this page.

Computational Models for In-Vehicle User Interface Design: A Systematic Literature Review

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, supplemental material, index terms.

General and reference

Document types

Surveys and overviews

Human-centered computing

Human computer interaction (HCI)

HCI design and evaluation methods

User models

User studies

HCI theory, concepts and models

Recommendations

User-interface tools: introduction and survey.

An overview is given of user-interface development systems (UIDS). Systems are classified by how they let the programmer specify the interfaces, and examples of each type are given. The three types are language-based, graphical, and automatic creation ...

User interface design and verification for semi-autonomous driving

This paper presents a project in its early stages of development, in which we propose a solution to the problem of human interaction with autonomous vehicles. We have devised a method for design of a user interface that displays sufficient and crucial ...

In-Vehicle Transitional Interfaces: User Journey in Virtual Reality

In this poster, we investigate novel user-centered methods and techniques for cross-virtuality user journeys for designing in-vehicle experiences. We argue that rapid prototyping of automotive human-machine interfaces (HMIs) can be achieved with mixed ...

Information

Published in.

• SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Association for Computing Machinery

New York, NY, United States

Publication History

Check for updates, author tags.

• automated driving
• cognitive modeling
• computational modeling
• literature review
• user interfaces
• Research-article
• Refereed limited

Funding Sources

• Federal Ministry of Education and Research of Germany and by Sächsische Staatsministerium für Wissenschaft, Kultur und Tourismus in the programme Center of Excellence for AI-research ðCenter for Scalable Data Analytics and Artificial Intelligence Dresden/Leipzig

Acceptance Rates

Upcoming conference, contributors, other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Online First
• Invasive Neisseria meningitidis subtype C in gay, bisexual and other men who have sex with men: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Lucy Rabuszko 1 ,
• Sarah Stuart-George 2 ,
• http://orcid.org/0009-0006-6348-8121 Callum Chessell 2 ,
• http://orcid.org/0000-0001-5764-5860 Colin Fitzpatrick 3 ,
• Deborah Williams 3 ,
• http://orcid.org/0000-0003-0955-6307 Daniel Richardson 2 , 3
• 1 Department of Primary Care and Public Health , Brighton and Sussex Medical School , Brighton , UK
• 2 Sexual Health & HIV Medicine , Brighton and Sussex Medical School , Brighton , UK
• 3 Sexual Health & HIV , University Hospitals Sussex NHS Foundation Trust , Brighton , UK
• Correspondence to Professor Daniel Richardson; daniel.richardson7{at}nhs.net

Introduction Outbreaks of invasive Neisseria meningitidis subtype C in networks of gay, bisexual and other men who have sex with men (MSM) have been reported. We aimed to explore any factors seen in MSM with invasive N.meningitidis subtype C.

Method We searched three bibliographical databases for manuscripts written in English exploring at least one factor seen in MSM with invasive N. meningitidis subtype C published up to May 2024. Following an initial search, removal of duplicates and abstract review, two authors independently reviewed full-text manuscripts and performed a risk of bias assessment using the Joanna Briggs Institute toolkit. Narrative data were synthesised to generate themes.

Results 16 manuscripts were included in this review from the USA (n=10), Germany (n=2), France (n=2), Canada (n=1) and Italy (n=1) and consisted of nine case series, four cross-sectional studies, two case reports and one case–control study published between 2003 and 2024 involving 236 MSM with invasive N. meningitidis subtype C, of which at least 64 died. We have highlighted some demographic (African-American or Hispanic identity in the USA, living with HIV), behavioural (kissing, sharing drinks, visiting sex-on-premises venues, visiting gay-oriented venues, using websites/mobile phone apps to meet sexual partners, recreational drug use, multiple and non-regular sexual partners) and infection (previous Chlamydia trachomatis, Treponema pallidum, Neisseria gonorrhoeae , Mpox) factors in MSM with invasive N. meningitidis subtype C.

Conclusion These data serve as an important resource to inform and target future public health strategies and outbreak control measures for the prevention of invasive N. meningitidis subtype C in MSM.

PROSPERO registration number CRD42024543551.

• NEISSERIA MENINGITIDIS
• Homosexuality, Male

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

https://doi.org/10.1136/sextrans-2024-056269

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Handling editor Stefano Rusconi

Contributors DR came up with the study concept and designed the study protocol, LR conducted the data search, LR, SS DR independently reviewed the manuscripts for eligibility, LR and DR independently conducted the risk of bias assessment, LR and DR conducted the data synthesis and LR, SS, CC, CF, DW and DR all contributed to the final manuscript. DR is acting as the guarantor.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

A systematic literature review on the usability of mobile applications for visually impaired users

Interacting with mobile applications can often be challenging for people with visual impairments due to the poor usability of some mobile applications. The goal of this paper is to provide an overview of the developments on usability of mobile applications for people with visual impairments based on recent advances in research and application development. This overview is important to guide decision-making for researchers and provide a synthesis of available evidence and indicate in which direction it is worthwhile to prompt further research. We performed a systematic literature review on the usability of mobile applications for people with visual impairments. A deep analysis following the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) guidelines was performed to produce a set of relevant papers in the field. We first identified 932 papers published within the last six years. After screening the papers and employing a snowballing technique, we identified 60 studies that were then classified into seven themes: accessibility, daily activities, assistive devices, navigation, screen division layout, and audio guidance. The studies were then analyzed to answer the proposed research questions in order to illustrate the different trends, themes, and evaluation results of various mobile applications developed in the last six years. Using this overview as a foundation, future directions for research in the field of usability for the visually impaired (UVI) are highlighted.

Introduction

The era of mobile devices and applications has begun. With the widespread use of mobile applications, designers and developers need to consider all types of users and develop applications for their different needs. One notable group of users is people with visual impairments. According to the World Health Organization, there are approximately 285 million people with visual impairments worldwide ( World Health Organization, 2020 ). This is a huge number to keep in mind while developing new mobile applications.

People with visual impairments have urged more attention from the tech community to provide them with the assistive technologies they need ( Khan & Khusro, 2021 ). Small tasks that we do daily, such as picking out outfits or even moving from one room to another, could be challenging for such individuals. Thus, leveraging technology to assist with such tasks can be life changing. Besides, increasing the usability of applications and developing dedicated ones tailored to their needs is essential. The usability of an application refers to its efficiency in terms of the time and effort required to perform a task, its effectiveness in performing said tasks, and its users’ satisfaction ( Ferreira et al., 2020 ). Researchers have been studying this field intensively and proposing different solutions to improve the usability of applications for people with visual impairments.

This paper provides a systematic literature review (SLR) on the usability of mobile applications for people with visual impairments. The study aims to find discussions of usability issues related to people with visual impairments in recent studies and how they were solved using mobile applications. By reviewing published works from the last six years, this SLR aims to update readers on the newest trends, limitations of current research, and future directions in the research field of usability for the visually impaired (UVI).

This SLR can be of great benefit to researchers aiming to become involved in UVI research and could provide the basis for new work to be developed, consequently improving the quality of life for the visually impaired. This review differs from previous review studies ( i.e.,   Khan & Khusro, 2021 ) because we classified the studies into themes in order to better evaluate and synthesize the studies and provide clear directions for future work. The following themes were chosen based on the issues addressed in the reviewed papers: “Assistive Devices,” “Navigation,” “Accessibility,” “Daily Activities,” “Audio Guidance,” and “Gestures.” Figure 1 illustrates the percentage of papers classified in each theme.

Figure 1: Percentages of classification themes.

The remainder of this paper is organized as follows: the next section specifies the methodology, following this, the results section illustrates the results of the data collection, the discussion section consists of the research questions with their answers and the limitations and potential directions for future work, and the final section summarizes this paper’s main findings and contribution.

Survey Methodology

This systematic literature review used the Meta-Analyses (PRISMA, 2009) guidelines to produce a set of relevant papers in the field. This SLR was undertaken to address the research questions described below. A deep analysis was performed based on a group of studies; the most relevant studies were documented, and the research questions were addressed.

A. Research questions

The research questions addressed by this study are presented in Table 1 with descriptions and the motivations behind them.

B. Search strategy

This review analysed and synthesised studies on usability for the visually impaired from a user perspective following a systematic approach. As proposed by Tanfield, Denyer & Smart (2003) , the study followed a three-stage approach to ensure that the findings were both reliable and valid. These stages were planning the review, conducting the review by analysing papers, and reporting emerging themes and recommendations. These stages will be discussed further in the following section.

1. Planning stage

The planning stage of this review included defining data sources and the search string protocol as well as inclusion and exclusion criteria.

Data sources.

We aimed to use two types of data sources: digital libraries and search engines. The search process was manually conducted by searching through databases. The selected databases and digital libraries are as follows:

ACM Library

IEEE Xplore

ScienceDirect

SpringerLink

ISI Web of Knowledge

The selected search engines were as follows:

DBLP (Computer Science Bibliography Website)

Google Scholar

Microsoft Academic

Search string.

The above databases were initially searched using the following keyword protocol: (“Usability” AND (”visual impaired” OR ”visually impaired” OR “blind” OR “impairment”) AND “mobile”). However, in order to generate a more powerful search string, the Network Analysis Interface for Literature Studies (NAILS) project was used. NAILS is an automated tool for literature analysis. Its main function is to perform statistical and social network analysis (SNA) on citation data ( Knutas et al., 2015 ). In this study, it was used to check the most important work in the relevant fields as shown in Fig. 2 .

NAILS produced a report displaying the most important authors, publications, and keywords and listed the references cited most often in the analysed papers ( Knutas et al., 2015 ) . The new search string was generated after using the NAILS project as follows: (“Usability” OR “usability model” OR “usability dimension” OR “Usability evaluation model” OR “Usability evaluation dimension”) AND (“mobile” OR “Smartphone”) AND (“Visually impaired” OR “Visual impairment” OR “Blind” OR “Low vision” OR “Blindness”).

Figure 2: NAILS output sample.

Figure 3: Number of papers per database.

Inclusion and exclusion criteria..

To be included in this systematic review, each study had to meet the following screening criteria:

The study must have been published between 2015 and 2020.

The study must be relevant to the main topic (Usability of Mobile Applications for Visually Impaired Users).

The study must be a full-length paper.

The study must be written in English because any to consider any other languages, the research team will need to use the keywords of this language in this topic and deal with search engines using that language to extract all studies related to our topic to form an SLR with a comprehensive view of the selected languages. Therefore, the research team preferred to focus on studies in English to narrow the scope of this SLR.

A research study was excluded if it did not meet one or more items of the criteria.

2. Conducting stage

The conducting stage of the review involved a systematic search based on relevant search terms. This consisted of three substages: exporting citations, importing citations into Mendeley, and importing citations into Rayyan.

Exporting citations.

First, in exporting the citations and conducting the search through the mentioned databases, a total of 932 studies were found. The numbers are illustrated in Fig. 3 below. The highest number of papers was found in Google Scholar, followed by Scopus, ISI Web of Knowledge, ScienceDirect, IEEE Xplore, Microsoft Academic, and DBLP and ACM Library with two studies each. Finally, SpringerLink did not have any studies that met the inclusion criteria.

The chance of encountering duplicate studies was determined to be high. Therefore, importing citations into Mendeley was necessary in order to eliminate the duplicates.

Figure 4: Search stages.

Importing citations into mendeley..

Mendeley is an open-source reference and citation manager. It can highlight paragraphs and sentences, and it can also list automatic references on the end page. Introducing the use of Mendeley is also expected to avoid duplicates in academic writing, especially for systematic literature reviews ( Basri & Patak, 2015 ). Hence, in the next step, the 932 studies were imported into Mendeley, and each study’s title and abstract were screened independently for eligibility. A total of 187 duplicate studies were excluded. 745 total studies remained after the first elimination process. The search stages are shown in Fig. 4 below.

Importing citations into rayyan.

Rayyan QCRI is a free web and mobile application that helps expedite the initial screening of both abstracts and titles through a semi-automated process while incorporating a high level of usability. Its main benefit is to speed up the most tedious part of the systematic literature review process: selecting studies for inclusion in the review ( Ouzzani et al., 2016 ). Therefore, for the last step, another import was done using Rayyan to check for duplications a final time. Using Rayyan, a total of 124 duplicate studies were found, resulting in a total of 621 studies. Using Rayyan, a two-step filtration was conducted to guarantee that the papers have met the inclusion criteria of this SLR. After filtering based on the abstracts, 564 papers did not meet the inclusion criteria. At this stage, 57 studies remained. The second step of filtration eliminated 11 more studies by reading the full papers; two studies were not written in the English language, and nine were inaccessible.

Snowballing.

Snowballing is an emerging technique used to conduct systematic literature reviews that are considered both efficient and reliable using simple procedures. The procedure for snowballing consisted of three phases in each cycle. The first phase is refining the start set, the second phase is backward snowballing, and the third is forward snowballing. The first step, forming the start set, is basically identifying relevant papers that can have a high potential of satisfying the criteria and research question. Backward snowballing was conducted using the reference list to identify new papers to include. It shall start by going through the reference list and excluding papers that do not fulfill the basic criteria; the rest that fulfil criteria shall be added to the SLR. Forward snowballing refers to identifying new papers based on those papers that cited the paper being examined ( Juneja & Kaur, 2019 ). Hence, in order to be sure that we concluded all related studies after we got the 46 papers, a snowballing step was essential. Forward and backward snowballing were conducted. Each of the 46 studies was examined by checking their references to take a look at any possible addition of sources and examining all papers that cited this study. The snowballing activity added some 38 studies, but after full reading, it became 33 that matched the inclusion criteria. A total of 79 studies were identified through this process.

Quality assessment.

A systematic literature review’s quality is determined by the content of the papers included in the review. As a result, it is important to evaluate the papers carefully ( Zhou et al., 2015 ). Many influential scales exist in the software engineering field for evaluating the validity of individual primary studies and grading the overall intensity of the body of proof. Hence, we adapted the comprehensive guidelines specified by Kitchenhand and Charters ( Keele, 2007 ), and the quasi-gold standard (QGS) ( Keele, 2007 ) was used to establish the quest technique, where a robust search strategy for enhancing the validity and reliability of a SLR’s search process is devised using the QGS. By applying this technique, our quality assessment questions were focused and aligned with the research questions mentioned earlier.

In our last step, we had to verify the papers’ eligibility; we conducted a quality check for each of the 79 studies. For quality assessment, we considered whether the paper answered the following questions:

QA1: Is the research aim clearly stated in the research?

QA2: Does the research contain a usability dimension or techniques for mobile applications for people with visual impairments?

QA3: Is there an existing issue with mobile applications for people with visual impairments that the author is trying to solve?

QA4: Is the research focused on mobile application solutions?

After discussing the quality assessment questions and attempting to find an answer in each paper, we agreed to score each study per question. If the study answers a question, it will be given 2 points; if it only partially answers a question, it will be given 1 point; and if there is no answer for a given question in the study, it will have 0 points.

The next step was to calculate the weight of each study. If the total weight was higher or equal to four points, the paper was accepted in the SLR; if not, the paper was discarded since it did not reach the desired quality level. Figure 5 below illustrates the quality assessment process. After applying the quality assessment, 39 papers were rejected since they received less than four points, which resulted in a final tally of 60 papers.

Figure 5: Quality assessment process.

To summarize, this review was conducted according to the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) ( Liberati et al., 2009 ). The PRISMA diagram shown in Fig. 6 illustrates all systematic literature processes used in this study.

Figure 6: PRISMA flow diagram.

3. analysing stage.

All researchers involved in this SLR collected the data. The papers were distributed equally between them, and each researcher read each paper completely to determine its topic, extract the paper’s limitations and future work, write a quick summary about it, and record this information in an Excel spreadsheet.

All researchers worked intensively on this systematic literature review. After completing the previously mentioned steps, the papers were divided among all the researchers. Then, each researcher read their assigned papers completely and then classified them into themes according to the topic they covered. The researchers held several meetings to discuss and specify those themes. The themes were identified by the researchers based on the issues addressed in the reviewed papers. In the end, the researchers resulted in seven themes, as shown in Fig. 7 below. The references selected for each theme can be found in the Table A1 . Afterwards, each researcher was assigned one theme to summarize its studies and report the results. In this section, we review the results.

Figure 7: Results of the SLR.

A. accessibility.

Of a total of 60 studies, 10 focused on issues of accessibility. Accessibility is concerned with whether all users are able to have equivalent user experiences, regardless of abilities. Six studies, Darvishy, Hutter & Frei (2019) , Morris et al. (2016) , Qureshi & Hooi-Ten Wong (2020) , Khan, Khusro & Alam (2018) , Paiva et al. (2020) , and Pereda, Murillo & Paz (2020) , gave suggestions for increasing accessibility, ( Darvishy, Hutter & Frei, 2019 ; Morris et al., 2016 ), gave some suggestions for making mobile map applications and Twitter accessible to visually impaired users, and ( Qureshi & Hooi-Ten Wong, 2020 ; Khan, Khusro & Alam, 2018 ) focused on user interfaces and provided accessibility suggestions suitable for blind people. Paiva et al. (2020) and Pereda, Murillo & Paz (2020) proposed a set of heuristics to evaluate the accessibility of mobile applications. Two studies, Khowaja et al. (2019) and Carvalho et al. (2018) , focused on evaluating usability and accessibility issues on some mobile applications, comparing them, and identifying the number and types of problems that visually impaired users faced. Aqle, Khowaja & Al-Thani (2020) proposed a new web search interface designed for visually impaired users. One study, McKay (2017) , focused on accessibility challenges by applying usability tests on a hybrid mobile app with some visually impaired university students.

B. Assistive devices

People with visual impairments have an essential need for assistive technology since they face many challenges when performing activities in daily life. Out of the 60 studies reviewed, 13 were related to assistive technology. The studies Smaradottir, Martinez & Håland (2017) , Skulimowski et al. (2019) , Barbosa, Hayes & Wang, (2016) , Rosner & Perlman (2018) , Csapó et al. (2015) , Khan & Khusro (2020) , Sonth & Kallimani (2017) , Kim et al. (2016) , Vashistha et al. (2015) ; Kameswaran et al. (2020) , Griffin-Shirley et al. (2017) , and Rahman, Anam & Yeasin (2017) were related to screen readers (voiceovers). On the other hand, Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) were related to proposing an assistant device for the visually impaired. Of the studies related to screening readers, Sonth & Kallimani, (2017) , Vashistha et al. (2015) , Khan & Khusro (2020) Lewis et al. (2016) cited challenges faced by visually impaired users. Barbosa, Hayes & Wang (2016) , Kim et al. (2016) , Rahman, Anam & Yeasin (2017) suggested new applications, while Smaradottir, Martinez & Håland (2017) , Rosner & Perlman (2018) , Csapó et al. (2015) and Griffin-Shirley et al. (2017) evaluated current existing work. The studies Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) proposed using wearable devices to improve the quality of life for people with visual impairments.

C. Daily activities

In recent years, people with visual impairments have used mobile applications to increase their independence in their daily activities and learning, especially those based on the braille method. We divide the daily activity section into braille-based applications and applications designed to enhance the independence of the visually impaired. Four studies, Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , Araújo et al. (2016) and Gokhale et al. (2017) , implemented and evaluated the usability of mobile phone applications that use braille to help visually impaired people in their daily lives. Seven studies, Vitiello et al. (2018) , Kunaratana-Angkul, Wu & Shin-Renn (2020) , Ghidini et al. (2016) , Madrigal-Cadavid et al. (2019) , Marques, Carriço & Guerreiro (2015) , Oliveira et al. (2018) and Rodrigues et al. (2015) , focused on building applications that enhance the independence and autonomy of people with visual impairments in their daily life activities.

D. Screen division layout

People with visual impairments encounter various challenges in identifying and locating non-visual items on touch screen interfaces like phones and tablets. Incidents of accidentally touching a screen element and frequently following an incorrect pattern in attempting to access objects and screen artifacts hinder blind people from performing typical activities on smartphones ( Khusro et al., 2019 ). In this review, 9 out of 60 studies discuss screen division layout: ( Khusro et al., 2019 ; Khan & Khusro, 2019 ; Grussenmeyer & Folmer, 2017 ; Palani et al., 2018 ; Leporini & Palmucci, 2018 ) discuss touch screen (smartwatch tablets, mobile phones, and tablet) usability among people with visual impairments, while ( Cho & Kim, 2017 ; Alnfiai & Sampalli, 2016 ; Niazi et al., 2016 ; Alnfiai & Sampalli, 2019 ) concern text entry methods that increase the usability of apps among visually impaired people. Khusro et al. (2019) provides a novel contribution to the literature regarding considerations that can be used as guidelines for designing a user-friendly and semantically enriched user interface for blind people. An experiment in Cho & Kim (2017) was conducted comparing the two-button mobile interface usability with the one-finger method and voiceover. Leporini & Palmucci (2018) gathered information on the interaction challenges faced by visually impaired people when answering questions on a mobile touch-screen device, investigated possible solutions to overcome the accessibility and usability challenges.

E. Gestures

In total, 3 of 60 studies discuss gestures in usability. Alnfiai & Sampalli (2017) compared the performance of BrailleEnter, a gesture based input method to the Swift Braille keyboard, a method that requires finding the location of six buttons representing braille dot, while Buzzi et al. (2017) and Smaradottir, Martinez & Haland (2017) provide an analysis of gesture performance on touch screens among visually impaired people.

F. Audio guidance

People with visual impairment primarily depend on audio guidance forms in their daily lives; accordingly, audio feedback helps guide them in their interaction with mobile applications.

Four studies discussed the use of audio guidance in different contexts: one in navigation ( Gintner et al., 2017 ), one in games ( Ara’ujo et al., 2017 ), one in reading ( Sabab & Ashmafee, 2016 ), and one in videos ( Façanha et al., 2016 ). These studies were developed and evaluated based on usability and accessibility of the audio guidance for people with visual impairments and aimed to utilize mobile applications to increase the enjoyment and independence of such individuals.

G. Navigation

Navigation is a common issue that visually impaired people face. Indoor navigation is widely discussed in the literature. Nair et al. (2020) , Al-Khalifa & Al-Razgan (2016) and De Borba Campos et al. (2015) discuss how we can develop indoor navigation applications for visually impaired people. Outdoor navigation is also common in the literature, as seen in Darvishy et al. (2020) , Hossain, Qaiduzzaman & Rahman (2020) , Long et al. (2016) , Prerana et al. (2019) and Bandukda et al. (2020) . For example, in Darvishy et al. (2020) , Touch Explorer, an accessible digital map application, was presented to alleviate many of the problems faced by people with visual impairments while using highly visually oriented digital maps. Primarily, it focused on using non-visual output modalities like voice output, everyday sound, and vibration feedback. Issues with navigation applications were also presented in Maly et al. (2015) . Kameswaran et al. (2020) discussed commonly used technologies in navigation applications for blind people and highlighted the importance of using complementary technologies to convey information through different modalities to enhance the navigation experience. Interactive sonification of images for navigation has also been shown in Skulimowski et al. (2019) .

In this section, the research questions are addressed in detail to clearly achieve the research objective. Also, a detailed overview of each theme will be mentioned below.

Answers to the research questions

This section will answer the research question proposed:

RQ1: What existing UVI issues did authors try to solve with mobile devices?

Mobile applications can help people with visual impairments in their daily activities, such as navigation and writing. Additionally, mobile devices may be used for entertainment purposes. However, people with visual impairments face various difficulties while performing text entry operations, text selection, and text manipulation on mobile applications ( Niazi et al., 2016 ). Thus, the authors of the studies tried to increase touch screens’ usability by producing prototypes or simple systems and doing usability testing to understand the UX of people with visual impairments.

RQ2: What is the role of mobile devices in solving those issues?

Mobile phones are widely used in modern society, especially among users with visual impairments; they are considered the most helpful tool for blind users to communicate with people worldwide ( Smaradottir, Martinez & Håland, 2017 ). In addition, the technology of touch screen assistive technology enables speech interaction between blind people and mobile devices and permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities perform actions or interact with systems ( Niazi et al., 2016 ).

RQ3: What are the publication trends on the usability of mobile applications among the visually impaired?

As shown in Fig. 8 below, research into mobile applications’ usability for the visually impaired has increased in the last five years, with a slight dip in 2018. Looking at the most frequent themes, we find that “Assistive Devices” peaked in 2017, while “Navigation” and “Accessibility” increased significantly in 2020. On the other hand, we see that the prevalence of “Daily Activities” stayed stable throughout the research years. The term “Audio Guidance” appeared in 2016 and 2017 and has not appeared in the last three years. “Gestures” also appeared only in 2017. “Screen Layout Division” was present in the literature in the last five years and increased in 2019 but did not appear in 2020.

Figure 8: Publication trends over time.

Rq4: what are the current research limitations and future research directions regarding usability among the visually impaired.

We divide the answer to this question into two sections: first, we will discuss limitations; then, we will discuss future work for each proposed theme.

A. Limitations

Studies on the usability of mobile applications for visually impaired users in the literature have various limitations, and most of them were common among the studies. These limitations were divided into two groups. The first group concerns proposed applications; for example, Rahman, Anam & Yeasin (2017) , Oliveira et al. (2018) and Madrigal-Cadavid et al. (2019) faced issues regarding camera applications in mobile devices due to the considerable effort needed for its usage and being heavily dependent on the availability of the internet. The other group of studies, Rodrigues et al. (2015) , Leporini & Palmucci (2018) , Alnfiai & Sampalli (2016) , and Ara’ujo et al. (2017) , have shown limitations in visually impaired users’ inability to comprehend a graphical user interface. Alnfiai & Sampalli (2017) and Alnfiai & Sampalli (2019) evaluated new braille input methods and found that the traditional braille keyboard, where knowing the exact position of letters QWERTY is required, is limited in terms of usability compared to the new input methods. Most studies faced difficulties regarding the sample size and the fact that many of the participants were not actually blind or visually impaired but only blindfolded. This likely led to less accurate results, as blind or visually impaired people can provide more useful feedback as they experience different issues on a daily basis and are more ideal for this type of study. So, the need for a good sample of participants who actually have this disability is clear to allow for better evaluation results and more feedback and recommendations for future research.

B. Future work

A commonly discussed future work in the chosen literature is to increase the sample sizes of people with visual impairment and focus on various ages and geographical areas to generalize the studies. Table 2 summarizes suggestions for future work according to each theme. Those future directions could inspire new research in the field.

RQ5: What is the focus of research on usability for visually impaired people, and what are the research outcomes in the studies reviewed?

There are a total of 60 outcomes in this research. Of these, 40 involve suggestions to improve usability of mobile applications; four of them address problems that are faced by visually impaired people that reduce usability. Additionally, 16 of the outcomes are assessments of the usability of the prototype or model. Two of the results are recommendations to improve usability. Finally, the last two outcomes are hardware solutions that may help the visually impaired perform their daily activities. Figure 9 illustrates these numbers.

Figure 9: Outcomes of studies.

Overview of the reviewed studies.

In the following subsections, we summarize all the selected studies based on the classified theme: accessibility, assistive devices, daily activities, screen division layout, gestures, audio guidance, and navigation. The essence of the studies will be determined, and their significance in the field will be explored.

For designers dealing with mobile applications, it is critical to determine and fix accessibility issues in the application before it is delivered to the users ( Khowaja et al., 2019 ). Accessibility refers to giving the users the same user experience regardless of ability. In Khowaja et al. (2019) and Carvalho et al. (2018) , the researchers focused on comparing the levels of accessibility and usability in different applications. They had a group of visually impaired users and a group of sighted users test out the applications to compare the number and type of problems they faced and determine which applications contained the most violations. Because people with visual impairments cannot be ignored in the development of mobile applications, many researchers have sought solutions for guaranteeing accessibility. For example, in Qureshi & Hooi-Ten Wong (2020) , the study contributed to producing a new, effective design for mobile applications based on the suggestions of people with visual impairments and with the help of two expert mobile application developers. In Khan, Khusro & Alam (2018) , an adaptive user interface model for visually impaired people was proposed and evaluated in an empirical study with 63 visually impaired people. In Aqle, Khowaja & Al-Thani (2020) , the researchers proposed a new web search interface for users with visual impairments that is based on discovering concepts through formal concept analysis (FCA). Users interact with the interface to collect concepts, which are then used as keywords to narrow the search results and target the web pages containing the desired information with minimal effort and time. The usability of the proposed search interface (InteractSE) was evaluated by experts in the field of HCI and accessibility, with a set of heuristics by Nielsen and a set of WCAG 2.0 guidelines.

In Darvishy, Hutter & Frei (2019) , the researchers proposed a solution for making mobile map applications accessible for people with blindness or visual impairment. They suggested replacing forests in the map with green color and birds’ sound, replacing water with blue color and water sounds, replacing streets with grey color and vibration, and replacing buildings with yellow color and pronouncing the name of the building. The prototype showed that it was possible to explore a simple map through vibrations, sounds, and speech.

In Morris et al. (2016) the researchers utilized a multi-faceted technique to investigate how and why visually impaired individuals use Twitter and the difficulties they face in doing so. They noted that Twitter had become more image-heavy over time and that picture-based tweets are largely inaccessible to people with visual impairments. The researchers then made several suggestions for how Twitter could be amended to continue to be usable for people with visual impairments.

The researchers in Paiva et al. (2020) focused on how to evaluate proposed methods for ensuring the accessibility and usability of mobile applications. Their checklist, Acc-MobileCheck, contains 47 items that correspond to issues related to comprehension (C), operation (O), perception (P), and adaptation (A) in mobile interface interaction. To validate Acc-MobileCheck, it was reviewed by five experts and three developers and determined to be effective. In Pereda, Murillo & Paz (2020) , the authors also suggest a set of heuristics to evaluate the accessibility of mobile e-commerce applications for visually impaired people. Finally, McKay (2017) conducted an accessibility test for hybrid mobile apps and found that students with blindness faced many barriers to access based on how they used hybrid mobile applications. While hybrid apps can allow for increased time for marketing, this comes at the cost of app accessibility for people with disabilities.

A significant number of people with visual impairments use state-of-the-art software to perform tasks in their daily lives. These technologies are made up of electronic devices equipped with sensors and processors that can make intelligent decisions.

One of the most important and challenging tasks in developing such technologies is to create a user interface that is appropriate for the sensorimotor capabilities of users with blindness ( Csapó et al., 2015 ). Several new hardware tools have proposed to improve the quality of life for people with visual impairments. Three tools were presented in this SLR: a smart stick that can notify the user of any obstacle, helping them to perform tasks easily and efficiently ( Bharatia, Ambawane & Rane, 2019 ), and an eye that can allow users to detect colors (medical evaluation is still required) ( Lewis et al., 2016 ).

The purpose of the study in Griffin-Shirley et al. (2017) was to understand how people with blindness use smartphone applications as assistive technology and how they perceive them in terms of accessibility and usability. An online survey with 259 participants was conducted, and most of the participants rated the applications as useful and accessible and were satisfied with them.

The researchers in Rahman, Anam & Yeasin (2017) designed and implemented EmoAssist, which is a smartphone application that assists with natural dyadic conversations and aims to promote user satisfaction by providing options for accessing non-verbal communication that predicts behavioural expressions and contains interactive dimensions to provide valid feedback. The usability of this application was evaluated in a study with ten people with blindness where several tools were applied in the application. The study participants found that the usability of EmoAssist was good, and it was an effective assistive solution.

This theme contains two main categories: braille-based application studies and applications to enhance the independence of VIU. Both are summarized below.

1- Braille-based applications

Braille is still the most popular method for assisting people with visual impairments in reading and studying, and most educational mobile phone applications are limited to sighted people. Recently, however, some researchers have developed assistive education applications for students with visual impairments, especially those in developing countries. For example, in India, the number of children with visual impairments is around 15 million, and only 5% receive an education ( Gokhale et al., 2017 ). Three of the braille studies focused on education: ( Nahar, Sulaiman & Jaafar, 2020 ; Nahar, Jaafar & Sulaiman, 2019 , and Araújo et al., 2016 ). These studies all used smartphone touchscreens and action gestures to gain input from the student, and then output was provided in the form of audio feedback. In Nahar, Sulaiman & Jaafar (2020) , vibrational feedback was added to guide the users. The participants in Nahar, Sulaiman & Jaafar (2020) ; Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) included students with blindness of visual impairment and their teachers. The authors in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) evaluated the usability of their applications following the same criteria (efficiency, learnability, memorability, errors, and satisfaction). The results showed that in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) , the applications met the required usability criteria. The authors in Gokhale et al. (2017) presented a braille-based solution to help people with visual impairments call and save contacts. A braille keypad on the smartphone touchscreen was used to gain input from the user, which was then converted into haptic and auditory feedback to let the user know what action was taken. The usability of this application was considered before it was designed. The participants’ responses were positive because this kind of user-centric design simplifies navigation and learning processes.

2- Applications to enhance the independence of people with visual impairments

The authors in the studies explored in this section focused on building applications that enhance independence and autonomy in daily life activities for users with visual impairments.

In Vitiello et al. (2018) , the authors presented their mobile application, an assistive solution for visually impaired users called “Crania”, which uses machine learning techniques to help users with visual impairments get dressed by recognizing the colour and texture of their clothing and suggesting suitable combinations. The system provides feedback through voice synthesis. The participants in the study were adults and elderly people, some of whom were completely blind and the rest of whom had partial sight. After testing for usability, all the participants with blindness agreed that using the application was better than their original method, and half of the participants with partial sight said the same thing. At the end of the study, the application was determined to be accessible and easy to use.

In Kunaratana-Angkul, Wu & Shin-Renn (2020) , an application which allows elderly people to measure low vision status at home through their smartphones instead of visiting hospitals was tested, and most of the participants considered it to be untrustworthy because the medical information was insufficient. Even when participants were able to learn how to use the application, most of them were still confused while using it and needed further instruction.

In Ghidini et al. (2016) , the authors studied the habits of people with visual impairments when using their smartphones in order to develop an electronic calendar with different interaction formats, such as voice commands, touch, and vibration interaction. The authors presented the lessons learned and categorized them based on usability heuristics such as feedback, design, user freedom and control, and recognition instead of remembering.

In Madrigal-Cadavid et al. (2019) , the authors developed a drug information application for people with visual impairments to help them access the labels of medications. The application was developed based on a user-centered design process. By conducting a usability test, the authors recognized some usability issues for people with visual impairments, such as difficulty in locating the bar code. Given this, a new version will include a search function that is based on pictures. The application is searched by capturing the bar code or text or giving voice commands that allow the user to access medication information. The participants were people with visual impairments, and most of them required assistance using medications before using the application. This application will enhance independence for people with visual impairments in terms of using medications.

In Marques, Carriço & Guerreiro (2015) , an authentication method is proposed for users with visual impairments that allows them to protect their passwords. It is not secure when blind or visually impaired users spell out their passwords or enter the numbers in front of others, and the proposed solution allows the users to enter their password with one hand by tapping the screen. The blind participants in this study demonstrated that this authentication method is usable and supports their security needs.

In Oliveira et al. (2018) , the author noted that people with visual impairments face challenges in reading, thus he proposed an application called LeR otulos. This application was developed and evaluated for the Android operating system and recognizes text from photos taken by the mobile camera and converts them into an audio description. The prototype was designed to follow the guidelines and recommendations of usability and accessibility. The requirements of the application are defined based on the following usability goals: the steps are easy for the user to remember; the application is efficient, safe, useful, and accessible; and user satisfaction is achieved.

Interacting with talkback audio devices is still difficult for people with blindness, and it is unclear how much benefit they provide to people with visual impairments in their daily activities. The author in Rodrigues et al. (2015) investigates the smartphone adoption process of blind users by conducting experiments, observations, and weekly interviews. An eight-week study was conducted with five visually impaired participants using Samsung and an enabled talkback 2 screen reader. Focusing on understanding the experiences of people with visual impairments when using touchscreen smartphones revealed accessibility and usability issues. The results showed that the participants have difficulties using smartphones because they fear that they cannot use them properly, and that impacts their ability to communicate with family. However, they appreciate the benefits of using smartphones in their daily activities, and they have the ability to use them.

People with visual impairments encounter various challenges identifying and locating non-visual items on touch screen interfaces, such as phones and tablets. Various specifications for developing a user interface for people with visual impairments must be met, such as having touch screen division to enable people with blindness to easily and comfortably locate objects and items that are non-visual on the screen ( Khusro et al., 2019 ). Article ( Khusro et al., 2019 ) highlighted the importance of aspects of the usability analysis, such as screen partitioning, to meet specific usability requirements, including orientation, consistency, operation, time consumption, and navigation complexity when users want to locate objects on their touchscreen. The authors of Khan & Khusro (2019) describe the improvements that people with blindness have experienced in using the smartphone while performing their daily tasks. This information was determined through an empirical study with 41 people with blindness who explained their user and interaction experiences operating a smartphone.

The authors in Palani et al. (2018) provide design guidelines governing the accurate display of haptically perceived graphical materials. Determining the usability parameters and the various cognitive abilities required for optimum and accurate use of device interfaces is crucial. Also the authors of Grussenmeyer & Folmer (2017) highlight the importance of usability and accessibility of smartphones and touch screens for people with visual impairments. The primary focus in Leporini & Palmucci (2018) is on interactive tasks used to finish exercises and to answer questionnaires or quizzes. These tools are used for evaluation tests or in games. When using gestures and screen readers to interact on a mobile device, difficulties may arise ( Leporini & Palmucci, 2018 ), The study has various objectives, including gathering information on the difficulties encountered by people with blindness during interactions with mobile touch screen devices to answer questions and investigating practicable solutions to solve the detected accessibility and usability issues. A mobile app with an educational game was used to apply the proposed approach. Moreover, in Alnfiai & Sampalli (2016) and Niazi et al. (2016) , an analysis of the single-tap braille keyboard created to help people with no or low vision while using touch screen smartphones was conducted. The technology used in Alnfiai & Sampalli (2016) was the talkback service, which provides the user with verbal feedback from the application, allowing users with blindness to key in characters according to braille patterns. To evaluate single tap braille, it was compared to the commonly used QWERTY keyboard. In Niazi et al. (2016) , it was found that participants adapted quickly to single-tap Braille and were able to type on the touch screen within 15 to 20 min of being introduced to this system. The main advantage of single tap braille is that it allows users with blindness to enter letters based on braille coding, which they are already familiar with. The average error rate is lower using single-tap Braille than it is on the QWERTY keyboard. The authors of Niazi et al. (2016) found that minimal typing errors were made using the proposed keypad, which made it an easier option for people with blindness ( Niazi et al., 2016 ). In Cho & Kim (2017) , the authors describe new text entry methods for the braille system including a left touch and a double touch scheme that form a two-button interface for braille input so that people with visual impairments are able to type textual characters without having to move their fingers to locate the target buttons.

One of the main problems affecting the visually impaired is limited mobility for some gestures. We need to know what gestures are usable by people with visual impairments. Moreover, the technology of assistive touchscreen-enabled speech interaction between blind people and mobile devices permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities to perform actions or interact with systems. Smaradottir, Martinez & Haland (2017) analyses a voiceover screen reader used in Apple Inc.’s products. An assessment of this assistive technology was conducted with six visually impaired test participants. The main objectives were to pinpoint the difficulties related to the performance of gestures applicable in screen interactions and to analyze the system’s response to the gestures. In this study, a user evaluation was completed in three phases. The first phase entailed training users regarding different hand gestures, the second phase was carried out in a usability laboratory where participants were familiarized with technological devices, and the third phase required participants to solve different tasks. In Knutas et al. (2015) , the vital feature of the system is that it enables the user to interactively select a 3D scene region for sonification by merely touching the phone screen. It uses three different modes to increase usability. Alnfiai & Sampalli (2017) explained a study done to compare the use of two data input methods to evaluate their efficiency with completely blind participants who had prior knowledge of braille. The comparison was made between the braille enter input method that uses gestures and the swift braille keyboard, which necessitates finding six buttons representing braille dots. Blind people typically prefer rounded shapes to angular ones when performing complex gestures, as they experience difficulties performing straight gestures with right angles. Participants highlighted that they experienced difficulties particularly with gestures that have steep or right angles. In Buzzi et al. (2017) , 36 visually impaired participants were selected and split into two groups of low-vision and blind people. They examined their touch-based gesture preferences in terms of the number of strokes, multitouch, and shape angles. For this reason, a wireless system was created to record sample gestures from various participants simultaneously while monitoring the capture process.

People with visual impairment typically cannot travel without guidance due to the inaccuracy of current navigation systems in describing roads and especially sidewalks. Thus, the author of Gintner et al. (2017) aims to design a system to guide people with visual impairments based on geographical features and addresses them through a user interface that converts text to audio using a built-in voiceover engine (Apple iOS). The system was evaluated positively in terms of accessibility and usability as tested in a qualitative study involving six participants with visual impairment.

Based on challenges faced by visually impaired game developers, Ara’ujo et al. (2017) provides guidance for developers to provide accessibility in digital games by using audio guidance for players with visual impairments. The interactions of the player can be conveyed through audio and other basic mobile device components with criteria focused on the game level and speed adjustments, high contrast interfaces, accessible menus, and friendly design. Without braille, people with visual impairments cannot read, but braille is expensive and takes effort, and so it is important to propose technology to facilitate reading for them. In Sabab & Ashmafee (2016) , the author proposed developing a mobile application called “Blind Reader” that reads an audio document and allows the user to interact with the application to gain knowledge. This application was evaluated with 11 participants, and the participants were satisfied with the application. Videos are an important form of digital media, and unfortunately people with visual impairment cannot access these videos. Therefore, Façanha et al. (2016) aims to discover sound synthesis techniques to maximize and accelerate the production of audio descriptions with low-cost phonetic description tools. This tool has been evaluated based on usability with eight people and resulted in a high acceptance rate among users.

1- Indoor navigation

Visually impaired people face critical problems when navigating from one place to another. Whether indoors or outdoors, they tend to stay in one place to avoid the risk of injury or seek the help of a sighted person before moving ( Al-Khalifa & Al-Razgan, 2016 ). Thus, aid in navigation is essential for those individuals. In Nair et al. (2020) , Nair developed an application called ASSIST, which leverages Bluetooth low energy (BLE) beacons and augmented reality (AR) to help visually impaired people move around cluttered indoor places ( e.g. , subways) and provide the needed safe guidance, just like having a sighted person lead the way. In the subway example, the beacons will be distributed across the halls of the subway and the application will detect them. Sensors and cameras attached to the individual will detect their exact location and send the data to the application. The application will then give a sequence of audio feedback explaining how to move around the place to reach a specific point ( e.g. , “in 50 ft turn right”, “now turn left”, “you will reach the destination in 20 steps”). The application also has an interface for sighted and low-vision users that shows the next steps and instructions. A usability study was conducted to test different aspects of the proposed solution. The majority of the participants agreed that they could easily reach a specified location using the application without the help of a sighted person. A survey conducted to give suggestions from the participants for future improvements showed that most participants wanted to attach their phones to their bodies and for the application to consider the different walking speeds of users. They were happy with the audio and vibration feedback that was given before each step or turn they had to take.

In Al-Khalifa & Al-Razgan (2016) , the main purpose of the study was to provide an Arabic-language application for guidance inside buildings using Google Glass and an associated mobile application. First, the building plan must be set by a sighted person who configures the different locations needed. Ebsar will ask the map builder to mark each interesting location with a QR code and generate a room number, and the required steps and turns are tracked using the mobile device’s built-in compass and accelerometer features. All of these are recorded in the application for the use of a visually impaired individual, and at the end, a full map is generated for the building. After setting the building map, a user can navigate inside the building with the help of Ebsar, paired with Google Glass, for input and output purposes. The efficiency, effectiveness, and levels of user satisfaction with this solution were evaluated. The results showed that the errors made were few, indicating that Ebsar is highly effective. The time consumed in performing tasks ranged from medium to low depending on the task; this can be improved later. Interviews with participants indicated the application’s ease of use. De Borba Campos et al. (2015) shows an application simulating a museum map for people with visual impairments. It discusses whether mental maps and interactive games can be used by people with visual impairments to recognize the space around them. After multiple usability evaluation sessions, the mobile application showed high efficiency among participants in understanding the museum’s map without repeating the visitation. The authors make a few suggestions based on feedback from the participants regarding enhancing usability, including using audio cues, adding contextual help to realise the activities carried around in a space, and focusing on audio feedback instead of graphics.

2- Outdoor navigation

Outdoor navigation is also commonly discussed in the literature. In Darvishy et al. (2020) , Touch Explorer was presented to alleviate many of the problems faced by visually impaired people in navigation by developing a non-visual mobile digital map. The application relies on three major methods of communication with the user: voice output, vibration feedback, and everyday sounds. The prototype was developed using simple abstract visuals and mostly relies on voice for explanation of the content. Usability tests show the great impact the prototype had on the understanding of the elements of the map. Few suggestions were given by the participants to increase usability, including GPS localization to locate the user on the map, a scale element for measuring the distance between two map elements, and an address search function.

In Hossain, Qaiduzzaman & Rahman (2020) , a navigation application called Sightless Helper was developed to provide a safe navigation method for people with visual impairments. It relies on footstep counting and GPS location to provide the needed guidance. It can also ensure safe navigation by detect objects and unsafe areas and can detect unusual shaking of the user and alert an emergency contact about the problem. The user interaction categories are voice recognition, touchpad, buttons, and shaking sensors. After multiple evaluations, the application was found to be useful in different scenarios and was considered usable by people with visual impairments. The authors in Long et al. (2016) propose an application that uses both updates from users and information about the real world to help visually impaired people navigate outdoor settings. After interviews with participants, some design goals were set, including the ability to tag an obstacle on the map, check the weather, and provide an emergency service. The application was evaluated and was found to be of great benefit; users made few errors and found it easy to use. In Prerana et al. (2019) , a mobile application called STAVI was presented to help visually impaired people navigate from a source to a destination safely and avoid issues of re-routing. The application depends on voice commands and voice output. The application also has additional features, such as calling, messages, and emergency help. The authors in Bandukda et al. (2020) helped people with visual impairments explore parks and natural spaces using a framework called PLACES. Different interviews and surveys were conducted to identify the issues visually impaired people face when they want to do any leisure activity. These were considered in the development of the framework, and some design directions were presented, such as the use of audio to share an experience.

3- General issues

The authors in Maly et al. (2015) discuss implementing an evaluation model to assess the usability of a navigation application and to understand the issues of communication with mobile applications that people with visual impairments face. The evaluation tool was designed using a client–server architecture and was applied to test the usability of an existing navigation application. The tool was successful in capturing many issues related to navigation and user behavior, especially the issue of different timing between the actual voice instruction and the position of the user. The authors in Kameswaran et al. (2020) conducted a study to find out which navigation technologies blind people can use and to understand the complementarity between navigation technologies and their impact on navigation for visually impaired users. The results of the study show that visually impaired people use both assistive technologies and those designed for non-visually impaired users. Improving voice agents in navigation applications was discussed as a design implication for the visually impaired. In Skulimowski et al. (2019) , the authors show how interactive sonification can be used in simple travel aids for the blind. It uses depth images and a histogram called U-depth, which is simple auditory representations for blind users. The vital feature of this system is that it enables the user to interactively select a 3D scene region for sonification by touching the phone screen. This sonic representation of 3D scenes allows users to identify the environment’s general appearance and determine objects’ distance. The prototype structure was tested by three blind individuals who successfully performed the indoor task. Among the test scenes used included walking along an empty corridor, walking along a corridor with obstacles, and locating an opening between obstacles. However, the results showed that it took a long time for the testers to locate narrow spaces between obstacles.

RQ6: What evaluation methods were used in the studies on usability for visually impaired people that were reviewed?

The most prevalent methods to evaluate the usability of applications were surveys and interviews. These were used to determine the usability of the proposed solutions and obtain feedback and suggestions regarding additional features needed to enhance the usability from the participants’ points of view. Focus groups were also used extensively in the literature. Many of the participants selected were blindfolded and were not actually blind or visually impaired. Moreover, the samples selected for the evaluation methods mentioned above considered the age factor depending on the study’s needs.

Limitation and future work

The limitations of this paper are mainly related to the methodology followed. Focusing on just eight online databases and restricting the search with the previously specified keywords and string may have limited the number of search results. Additionally, a large number of papers were excluded because they were written in other languages. Access limitations were also faced due to some libraries asking for fees to access the papers. Therefore, for future works, a study to expand on the SLR results and reveal the current usability models of mobile applications for the visually impaired to verify the SLR results is needed so that this work contributes positively to assessing difficulties and expanding the field of usability of mobile applications for users with visual impairments.

Conclusions

In recent years, the number of applications focused on people with visual impairments has grown, which has led to positive enhancements in those people’s lives, especially if they do not have people around to assist them. In this paper, the research papers focusing on usability for visually impaired users were analyzed and classified into seven themes: accessibility, daily activities, assistive devices, gestures, navigation, screen division layout, and audio guidance. We found that various research studies focus on accessibility of mobile applications to ensure that the same user experience is available to all users, regardless of their abilities. We found many studies that focus on how the design of the applications can assist in performing daily life activities like braille-based application studies and applications to enhance the independence of VI users. We also found papers that discuss the role of assistive devices like screen readers and wearable devices in solving challenges faced by VI users and thus improving their quality of life. We also found that some research papers discuss limited mobility of some gestures for VI users and investigated ways in which we can know what gestures are usable by people with visual impairments. We found many research papers that focus on improving navigation for VI users by incorporating different output modalities like sound and vibration. We also found various studies focusing on screen division layout. By dividing the screen and focusing on visual impairment-related issues while developing user interfaces, visually impaired users can easily locate the objects and items on the screens. Finally, we found papers that focus on audio guidance to improve usability. The proposed applications use voice-over and speech interactions to guide visually impaired users in performing different activities through their mobiles. Most of the researchers focused on usability in different applications and evaluated the usability issues of these applications with visually impaired participants. Some of the studies included sighted participants to compare the number and type of problems they faced. The usability evaluation was generally based on the following criteria: accessibility, efficiency, learnability, memorability, errors, safety, and satisfaction. Many of the studied applications show a good indication of these applications’ usability and follow the participants’ comments to ensure additional enhancements in usability. This paper aims to provide an overview of the developments on usability of mobile applications for people with visual impairments and use this overview to highlight potential future directions.

Download article

Report a problem.

Common use cases Typos, corrections needed, missing information, abuse, etc

Our promise PeerJ promises to address all issues as quickly and professionally as possible. We thank you in advance for your patience and understanding.

Typo Missing or incorrect metadata Quality: PDF, figure, table, or data quality Download issues Abusive behavior Research misconduct Other issue not listed above

Follow this publication for updates

You can also choose to receive updates via daily or weekly email digests. If you are following multiple publications then we will send you no more than one email per day or week based on your preferences.

Note: You are now also subscribed to the subject areas of this publication and will receive updates in the daily or weekly email digests if turned on. You can add specific subject areas through your profile settings.

Change notification settings or unfollow

Loading ...

Usage since published - updated daily

Top referrals unique visitors

Share this publication, articles citing this paper.