Healthc Inform Res Search


Healthc Inform Res > Volume 24(4); 2018 > Article
Yang, Ryu, Han, Oh, and Choi: Ecological Momentary Assessment Using Smartphone-Based Mobile Application for Affect and Stress Assessment



This study aimed to describe the process of utilizing a mobile application for ecological momentary assessment (EMA) to collect data on stress and mood in daily life setting.


A mobile application for the Android operating system was developed and installed with a set of questions regarding momentary mood and stress into a smartphone of a participant. The application sets alarms at semi-random intervals in 60-minute blocks, four times a day for 7 days. After obtaining all momentary affect and stress, the questions to assess the usability of the mobile EMA application were also administered.


The data were collected from 97 police officers working in Gyeonggi Province of South Korea. The mean completion rate was 60.0% ranging from 3.5% to 100%. The means of positive and negative affect were 18.34 of 28 and 19.09 of 63. The mean stress was 17.92 of 40. Participants responded that the mobile application correctly measured their affect (4.34 ± 0.83) and stress (4.48 ± 0.62) of 5-point Likert scale.


Our study investigated the process of utilizing a mobile application to assess momentary affect and stress at repeated times. We found challenges regarding adherence to the research protocol, such as completion and delay of answering after alarm notification. Despite this inherent issue of adherence to the research protocol, the EMA still has advantages of reducing recall bias and assessing the actual moment of interest at multiple time points that improves ecological validity.

I. Introduction

Ecological momentary assessments (EMA) or experience sampling methods (ESM) are a design for longitudinal repeated measurements and started with paper diaries combined with electronic wristwatches or pagers [1]. These methods have been receiving high interests from researchers from various scientific background since they are considered to reduce the recall bias and improve accuracy by capturing momentary changes in real-life [2,3]. As technology has advanced, personal digital assistants and smartphone applications have been utilized to collect the data using EMA or ESM [4]. An early study used a palm-top computer to assess cigarette smoking cessation and relapse [5]. In this case, EMA repeatedly measured a behavior, mood, and activities as factors to track down the quitting process over time by electronic diaries five times per day for several weeks. In 2008, the first application developed as a software downloaded on a mobile device and utilized [6]. Since then, EMA using smartphones has increased rapidly and expected to grow continuously.
EMA has a number of advantages. It reduces recall bias and prevents aggregation by obtaining information at the actual moment of interest at multiple time points, and improves ecological validity by measuring the variables in natural and real-life environments [2]. Thus, EMA achieves increased accuracy and sensitivity in assessing changes in psychological properties [4]. EMA is considered a suitable method for observing momentary changes in affect and stress and understanding the psychological changes by yielding abundant and accumulated data from the repeated assessments in various situations [7,8,9].
Smartphone applications are broadly utilized to collect data used for EMA; this is attributable to the developed logistics and convenience of carrying smartphones. Researchers have developed their own applications or also utilized some commercial applications to conduct EMA through mobile or smartphones [10,11,12,13,14]. This case report describes the process of utilizing an EMA application to collect data on affect and stress, including the momentary activity, place, and persons whom the subject is with at the moment. Additionally, we also addressed issues during implementation of the EMA application.

II. Case Descriptions

We will describe the specifics of the EMA application and a case of preliminary implementation of utilizing the application.

1. Structure of Mobile EMA Service

To conduct an EMA study, researchers prepare survey questions and plan the scheduling of data collection points in advance. Then an administrator registers survey questions in advance in an EMA portal web application and establishes the schedule for survey questions. Furthermore, a participation schedule is set up separately for each participant (subscription). According to the schedules determined previously, the scheduler on the service uses Firebase Cloud Messaging (FCM) and Access Point Names (APNs) to automatically send alarms regarding responses from a participant. In some cases, an alarm can be sent out several times for just one scheduled question. To encourage participants to respond actively during the subscription period, gift-icons can be sent as an incentive. As for the gift-icons, various goods can be provided through a separate company, and depending on the case, interoperating with other companies is also possible.
Various response data collected in this process can be analyzed by using a data analysis tool. The mobile EMA service can extract the data in comma-separated values (CSV) format from such various data, thereby providing users with opportunities to analyze such data by using a tool familiar to them or any specialized tool. Furthermore, a stable service can be provided through cloud networking, and if necessary, it can be easily scaled up/scaled out. The structure of the mobile EMA service is presented in Figure 1.

2. Flow of Mobile EMA Service Process

The process of the mobile EMA service is summarized as follows. After an administrator, usually a researcher prepares the survey questions and gets ready to distribute them to participants. The administrator sends the application URL to a participant. The participant downloads the application to her/his smartphone and sends a request to be registered to the administration. Then the administrator allows the participant to register, and the study participant registers with a username and password. The administrator sets up subscription dates and time schedules for each participant. According to the date and time determined, a push notification is sent to the corresponding participant, and the participant receiving the push notification can answer the survey questions immediately, which is an advantage in comparison to the conventional paper survey method. When the participant completes the survey, she/he submits the responses to the administrator. If the survey questions are not answered on time, the researcher can send a push alarm separately to the corresponding participant as a reminder to participate in the survey. The data sent to the administrator are stored on a cloud server and exported in CSV or Excel format for further analysis (Figure 2).
In the case of using the mobile EMA service, the subscription dates can be set differently for each participant, and the push notifications can be defined according to the corresponding subscription dates. Therefore, the researcher can obtain the participants' momentary data, for example, momentary affect and stress that the participant experiences at a given moment.

3. Features and Advantages of Mobile EMA Service

The conventional paper survey method required as much efforts as the number of survey participants in terms of organizing the question items and collecting the data. Questions on paper entail a reliability issue for the responses of participants (e.g., responses are all composed at the same time, response is not composed on time, and participants may omit a question).
Meanwhile, in mobile EMA design, researchers can easily create the corresponding survey items, compose the desired questions using various formats and setup an explanation for each question. This will help greatly in saving resources and time, thereby helping the researchers perform activities more efficiently. When comparing digitalized survey method to the conventional paper survey method, a participant can manage and respond to the survey questions by using mobile devices easily. The administrator can receive the survey responses from each participant without laboring to transfer the data collected on papers to Excel one by one. Furthermore, the response results can be obtained as raw data in an Excel file. When multiple participants are targeted, the registration and management of participants are easy in comparison to the paper survey method. When a response request signal is received through a smartphone, a participant responds immediately by recording data, such as current activity, environment, affect, stress, and so forth. The administrator can distribute survey questions with a minimal effort since functions such as sending alarm notifications for response requests to each participants are automated (Figure 3).

4. Utilizing EMA Application for Data Collection

A convenient sample was drawn from police officers who were recruited from the police stations in Gyeonggi Province of the Republic of Korea in 2018. The study was approved by the Institutional Review Board of Yonsei University (No. Y-2018-0035). Our research team installed a mobile application into the participants' smartphones, and participants were requested to answer a set of questions of a total of 25 items assessing their momentary affect (13 items; 9 for positive affect and 4 for negative affect) and stress (8 items), including their shift of work, what they do, whom they are with, and where they are at the moment of responding to the questions. Most participants spent 2 to 3 minutes to answer those questions. The momentary affect and stress questions [15,16] were translated into Korean language using translations and back-translations and obtained content validity from a group of experts in the related fields. We also measured the participants' perception of the data quality and technical challenges in using this mobile application in terms of usability and these items were derived from the previous study utilizing EMA and modified for this study [17].
We asked multiple repeated responses, four times a day (08:00 to 9:00, 13:00 to 14:00, 17:00 to 16:00, and 21:00 to 22:00) for seven consecutive days, which yielded 28 data points per person. The application sets alarms at semirandom interval with 60-minute blocks. For example, if the notification alarm for a morning survey is sent at 8:30 to the participant, which is randomly set by the EMA application, she or he is supposed to respond by 9:30, within 60 minutes.
The participants were 97 police officers, and the mean age was 37.7 ± 10.8 and ranged from 22 to 60 years old. Ninety-two persons (94.8%) were males. About two-thirds (66.0%) were college graduates or higher. The mean completion rate was 60.0%, ranging from 3.5% to 100% per person. For affect, the means of positive and negative affect were 18.34 ± 4.57 of total score of 28 and 19.09 ± 9.75 of total score of 63, respectively. The mean of momentary stress was 7.92 ± 5.76 of total score of 40 (Table 1).
We also assessed the usability of the mobile EMA application from the participants. Eighty participants responded that the mobile application correctly measured their affect (4.34 ± 0.83) and stress (4.48 ± 0.62) on a 5-point Likert scale. They also felt that the mobile application was easy to use (4.05 ± 1.04), and they were comfortable to answer the questions in the application (4.11 ± 0.89) installed on their smartphones (Table 2).

III. Discussion

This case report described the process of utilizing a mobile application and methodological strategies to assess affect and stress of from police officers by using EMA methods. While we distributed the application and collected the data from the participants, we identified some issues. It was difficult for participants to adhere to the protocol because EMA methods usually require the collection of multiple data points for a relatively long period of time from the participants. Therefore, the burden of participating can be an issue. We provided approximately $18 (20,000 KRW) as an incentive—a small gift when obtaining informed consents from the participant to encourage participation and beverage gifticons on completion of the EMA survey. Although the mean completion rate was 60.0%, which is comparable to the rates of other studies of EMA measuring affect or stress (50% to 82%) [9,12,13], it would be better to provide the incentive first, in the middle, and the end of the EMA data collection to maintain a high completion rate. The medium completion rate may be related to the job characteristics of police officers, who sometimes work at the site.
Even though we found some limitations in utilizing the mobile application, our participants responded positively regarding their perception that the mobile application correctly measured their momentary status, and easiness to operate this mobile application, and to answer the questions in the application on the acceptability questions. We believe that this case report can provide a helpful reference for researchers who plan to utilize EMA design to measuring psychological and physical aspects in the daily lives of study participants. Based on the EMA findings, researchers should consider the implementation of intervention to the participants for further research.


This research was supported by Basic Science Research Program through the National Foundation of Korea (NRF) funded by the Ministry of Education (No. 2017R1D-1A1B03030706).


Conflict of Interest: No potential conflict of interest relevant to this article was reported


1. Stone AA, Shiffman S. Ecological momentary assessment (EMA) in behavioral medicine. Annu Behav Med 1994;16(3):199-202.
2. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008;4:1-32. PMID: 18509902.
3. Bolger N, Laurenceau JP. Intensive longitudinal methods: an introduction to diary and experience sampling research. New York (NY): Guilford Press; 2013.

4. Doherty ST, Lemieux CJ, Canally C. Tracking human activity and well-being in natural environments using wearable sensors and experience sampling. Soc Sci Med 2014;106:83-92. PMID: 24549253.
5. Shiffman S. Dynamic influences on smoking relapse process. J Pers 2005;73(6):1715-1748. PMID: 16274451.
6. Donker T, Petrie K, Proudfoot J, Clarke J, Birch MR, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15(11):e247. PMID: 24240579.
7. Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J. Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med 2009;39(9):1533-1547. PMID: 19215626.
8. Schwartz JE, Stone AA. Strategies for analyzing ecological momentary assessment data. Health Psychol 1998;17(1):6-16. PMID: 9459065.
9. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009;21(4):463-475. PMID: 19947781.
crossref pmid
10. Band R, Barrowclough C, Emsley R, Machin M, Wearden AJ. Significant other behavioural responses and patient chronic fatigue syndrome symptom fluctuations in the context of daily life: An experience sampling study. Br J Health Psychol 2016;21(3):499-514. PMID: 26700742.
11. Clasen PC, Fisher AJ, Beevers CG. Mood-reactive self-esteem and depression vulnerability: person-specific symptom dynamics via smart phone assessment. PLoS One 2015;10(7):e0129774. PMID: 26131724.
12. Cook PF, Schmiege SJ, Bradley-Springer L, Starr W, Carrington JM. Motivation as a mechanism for daily experiences' effects on HIV medication adherence. J Assoc Nurses AIDS Care 2018;29(3):383-393. PMID: 28967490.
13. Faherty LJ, Hantsoo L, Appleby D, Sammel MD, Bennett IM, Wiebe DJ. Movement patterns in women at risk for perinatal depression: use of a mood-monitoring mobile application in pregnancy. J Am Med Inform Assoc 2017;24(4):746-753. PMID: 28339686.
crossref pmid pmc
14. Fuller-Tyszkiewicz M, Karvounis T, Pemberton R, Hartley-Clark L, Richardson B. Determinants of depressive mood states in everyday life: an experience sampling study. Motiv Emot 2017;41(4):510-521.
15. Verhagen SJ, Berben JA, Leue C, Marsman A, Delespaul PA, van Os J, et al. Demonstrating the reliability of transdiagnostic mHealth Routine Outcome Monitoring in mental health services using experience sampling technology. PLoS One 2017;12(10):e0186294. PMID: 29023568.
crossref pmid pmc
16. Powell DJH, Liossi C, Schlotz W, Moss-Morris R. Tracking daily fatigue fluctuations in multiple sclerosis: ecological momentary assessment provides unique insights. J Behav Med 2017;40(5):772-783. PMID: 28281106.
17. Jones KK, Zenk SN, McDonald A, Corte C. Experiences of African-American women with smartphone-based ecological momentary assessment. Public Health Nurs 2016;33(4):371-380. PMID: 26530781.
Figure 1

Structure of mobile ecological momentary assessment (EMA) service. FCM: Firebase Cloud Messaging, APNs: Access Point Names, API: application programming interface.

Figure 2

Flow of mobile ecological momentary assessment (EMA) service.

Figure 3

Screen examples of mobile EMA application.

Table 1

Completion rate and momentary measurements variables (n = 97)


Values are presented as mean ± standard deviation (range).

Table 2

Usability of EMA using the mobile application (n = 80)


Values are presented as mean ± standard deviation.

EMA: ecological momentary assessment.

Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 8 Crossref
  • 5   Scopus
  • 753 View
  • 6 Download
Related articles in Healthc Inform Res


Browse all articles >

Editorial Office
1618 Kyungheegung Achim Bldg 3, 34, Sajik-ro 8-gil, Jongno-gu, Seoul 03174, Korea
Tel: +82-2-733-7637, +82-2-734-7637    E-mail:                

Copyright © 2020 by Korean Society of Medical Informatics. All rights reserved.

Developed in M2community

Close layer
prev next