I. Introduction
As the paradigm of healthcare has evolved from the diagnosis and treatment of disease to its prevention, various groups have rallied around the World Health Organization (WHO) to direct greater attention and effort to health promotion. Obesity is recognized as a prominent healthcare issue in the 20th century due to it being a risk factor for various diseases, such as hypertension, cardiovascular disease, stroke, and hyperlipidemia; therefore, the management of obesity is crucial to disease prevention and health promotion [1].
Many nations worldwide have seen a rapid increase in the prevalence of obesity. According to the Korea National Health and Nutritional Examination, the prevalence of obesity in Korea was 23.3% in 2011, an increase of 0.8% from 2010 and the highest it has been since obesity-related statistics were first collected [2].
To manage and prevent obesity, the United States, Germany, and other developed countries have developed (and are encouraging the use of) scientific-evidence-based clinical practice guidelines that can be used as guidelines for prevention and management by the individual or the community. To control obesity, individuals need interventions that can help them to continuously control their weight [3] rather than short-term education or intervention programs focused on specific factors. Long-term weight management must occur through making a habit of a proper diet and regular exercise. To achieve this, intervention needs to be provided continuously, unrestricted by temporal or spatial constraints. Nursing interventions using information technology, such as the Internet, have been shown to be effective in terms of accessibility, equity, and cost, and without temporal or spatial constraints [4].
With the widespread introduction of handheld wireless devices, such as mini PCs, PDAs, and smartphones, the healthcare sector has embraced mobile healthcare systems that utilize such mobile devices [5]. Smartphone applications (hereafter 'apps') have already been developed and put into general use in obesity management [6], but they were not developed based the available evidence-based clinical practice guidelines. Jeon et al. [7] reported that only 3 of 148 obesity-management apps developed in South Korea provided good-quality health information. The provision of low-quality health information on the Web or in apps can be potentially harmful to users, so it is important to provide users with high-quality health information [8]. Furthermore, the available apps were developed for healthcare providers rather than for the users who are the targets of obesity management, and they have not been evaluated for usability, so user feedback on the quality of such systems, the information they provide, and their user interfaces have not been taken into account.
With this background, this study aimed to develop an obesity-management app based on evidence-based clinical practice guidelines and to evaluate the underlying algorithms with domain experts, the user interface with informatics experts, and the usability with actual users.
II. Methods
The study was conducted in the order of analysis-design-implementation-evaluation according to the standard lifecycle of system development. The stages of the study and the content and outcome of each stage are shown in Figure 1.
1. System Analysis
The system analysis phase involved defining the requirements of the obesity-management app. We identified knowledge requirements for obesity management, and functional requirements for system implementation.
1) Knowledge requirements
We used published clinical practice guidelines for obesity management and related papers as knowledge sources. Clinical practice guidelines and related papers were searched by applying the keywords 'obesity management' and 'obesity treatment' to the National Guideline Clearinghouse (NGC) and National Institute for Health and Clinical Excellence (NICE) search engines. We limited included publications to those dealing with obesity in adult populations written in English or Korean and published by national research institutes or professional organizations.
Since it was impossible to include all contents from the clinical practice guidelines and relevant literature in the obesity-management app developed in this study, we selected the contents that were to be included in the app by consulting with four experts in obesity management, medical informatics, and nursing informatics [9,10].
2) Obesity-management algorithms
We refined and structured the extracted knowledge into the form of an algorithm in which the rules, the flow of decision-making, and the actions could be expressed clearly. For example, the guideline of the National Heart, Lung, and Blood Institute (NHLBI) [11] states that the target weight should be achieved by losing less than 10% of the current weight. This knowledge is expressed in an explicit form (rules and actions) for a computer to understand, such as 'IF targetWeight < currentWeight × 0.9, THEN Display recommended optimal target weight'.
This study referred to the obesity-management algorithms proposed by the NHLBI when developing the main algorithm for obesity management.
3) Functional requirements
A system's functional requirements can be obtained by surveying experts, whereas the target users of the app developed in this study are members of the general public who wish to control their weight. Since members of the general public lack obesity-management knowledge, we extracted functional requirements from obesity-management algorithms instead of surveying the general public. The extracted functional requirements were expressed by a use case diagram and activity diagrams.
2. System Design
1) Database design
A data dictionary defining data objects (the smallest content units in the obesity management app) and their types and possible values were defined, and then the database was designed using the nursing processes of assessment, diagnosis, intervention, and evaluation as a framework.
3. System Implementation
The obesity-management app was developed using the Android SDK Platform 4.0.3, the Java Development Kit (JDK) 1.7.0, and the Eclipse open-source Java development program.
4. System Evaluation
During the system evaluation stage, the appropriateness of the obesity-management algorithms was evaluated by comparing scenario-based recommendations. The user interface was evaluated using mobile heuristics evaluation tools, and the usability was evaluated using the Post-Study System Usability Questionnaire (PSSUQ) [12], which reflects the cognitive characteristics of the users.
1) Knowledge evaluation
The knowledge evaluation of the obesity-management app-that is, whether the obesity-management algorithms based on clinical practice guidelines are appropriate-was carried out by comparing the recommendations made by the app with the recommendations that three nurses working in research centers of obesity management and participating in the study would have made if they were to follow clinical guidelines using scenarios. Scenarios were developed that included all possible decision nodes in the algorithm.
If the nurses' recommendations were identical to the app's recommendations down to the specifics, 1 point was awarded. If the recommendation groups were identical (e.g., if both the recommendations belonged to the same recommendation group of weight loss, weight maintenance, or weight-loss cessation) but the specific recommendations differed, 0 points were awarded. If the recommendation groups were not identical, 1 point was subtracted. The proficiency and efficiency scores of each scenario were then calculated according to the following formulae [13]:
where proficiency is an index quantifying the appropriateness of the recommendations, and efficiency is an index quantifying the rate of consensus of the recommendations.
2) Heuristics evaluation
The heuristics evaluation was conducted with five experts who had masters or higher degrees using mobile heuristics evaluation tools consisting of eights items as proposed by Bertini et al. [14]. Evaluators were asked to describe the problem for each heuristics evaluation item and to rate the severity of each problem with the scenarios provided. If a problem received more than 4 points for severity, or if two or more evaluators proposed modifications, the system was modified accordingly.
3) Usability evaluation
The usability evaluation was performed by 10 healthy adults who were selected to use the obesity-management app developed in this study. They were asked to answer a PSSUQ survey developed by Lewis [12], which is currently in its third revision and consists of 16 questions in 3 domains.
The usability evaluation score was calculated as the mean score for all questions and the three domains of system usability, information quality, and interface quality. Any questions to which the users gave a score of 1 (i.e., 'strongly disagree') were used as criteria for further modification.
III. Results
1. System Analysis
1) Knowledge requirements
The knowledge sources that were selected in accordance with the search criteria comprised five clinical practice guidelines-published by the NICE, American Medical Association [15], US National Institutes of Health, NHLBI [16], Korean Society for the Study of Obesity [17], and Korean Endocrine Society [18]-as well as seven relevant papers on obesity management.
We excluded drug or surgical interventions based on suggestions made by the experts. In total, 131 knowledge statements were extracted from these sources and categorized according to the nursing processes of assessment (n = 46), diagnosis (n = 1), intervention (n = 76), and evaluation (n = 8).
2) Obesity-management algorithms
We developed five algorithms in total, comprising a main algorithm (Figure 2) and four subalgorithms for obesity diagnosis, goal-setting, intervention selection, and cause-offailure analysis. These five algorithms contain 25 rules and 29 actions.
3) Functional requirements
The use case diagram included 3 actors and 11 use cases (functions). The relationships between actors and use cases are presented in Figure 3. Three activity diagrams were constructed to describe the processes of assessment, diagnosis, intervention, and evaluation. Figure 4 shows an example activity diagram of the assessment process during a client's initial visit.
2. System Design
1) Database
The data dictionary defined the following 2,354 entities: 31 input data, 22 diet and exercise diary data, 14 output data, 1,497 food lists, and 790 exercise lists. The database consists of 11 tables and 58 fields (Figure 5). Seven of these tables are static tables: 1) a 'Member' table for identifying the users; 2) an 'Information' table for users' demographic and clinical information; 3) a 'Diet' table for recording food intake; 4) a 'Food' table for the calories in each food item; 5) an 'Activity' table for recording exercise; 6) an 'Exercise' table for calories consumed for each exercise; and 7) a 'Diagnosis' table for diagnoses and recommendations. The rest of the tables are view tables, which are dynamic and contain data whose values are computed and modified within the system according to the input values. In each table, users' cell phone numbers are used as the primary key.
2) User interface
The user interface designed in this study consists of the following 12 interface categories (25 for the specifics): login, register, input initial data, main page, input diet diary data, input exercise diary data, view entered data, set goal, diagnosis, intervention, evaluation, and others (e.g., a pop-up window showing an alarm for restarting the obesity management).
The specific form of the initial data input page presented to the user varies with the user's input data, such as demographic information, obesity-related information, and willingness to lose weight (Figure 6).
3. System implementation
The 10 evaluators evaluated the heuristics and usability of the newly developed app during October and November 2012. The system was modified during November and December 2012 to reflect the findings of their heuristics and usability evaluation.
4. System Evaluation
1) Knowledge evaluation
Table 1 lists the proficiency and efficiency values of the obesity-management algorithm for each evaluator. The proficiency scores exceeded 80% for all three evaluators, while the efficiency scores also exceeded 80% for all of the evaluators except evaluator A.
2) Heuristics evaluation
In total, 57 suggestions were made by the 5 experts in the heuristics evaluation. Heuristic 2 ('match between system and the real world') had the most suggestions (n = 17), while heuristics 2 and 3 ('consistency and mapping') had the largest number of suggestions that were rated 4 for severity (n = 4 for each) (Table 2).
Ten suggestions (2 that were redundant) had a severity score exceeding 4, and 3 suggestions proposed by more than 2 evaluators (13 in total) were selected for modification.
3) Usability evaluation
The mean and standard deviation values of the scores for the 16 questions of the usability evaluation survey are presented in Table 3. The scores in the 3 higher-level categories of these 16 questions-system utility, information quality, and interface quality-are presented in Table 4.
The highest-rated item of the 16 items was 'the organization of information on the system screens was clear', with 4.0 out of 5 possible points, which was followed by 6 items each receiving 3.30 points. Of the higher-level categories, system usefulness and information quality received the highest score (20.9 for each), and interface quality received the lowest score (10.1).
We asked open-ended questions to elicit suggestions other than the items presented in the evaluation survey, which produced the following responses:
1. Allow the users to move to the information input screen after the user enters his or her e-mail address.
2. The message "what is your target weight in kg?" is ambiguous as to whether it refers to the amount of weight to lose (e.g., -4 kg) or the desired weight after the weight loss (e.g., 75 kg), and so needs to be edited for clarity.
3. Allow the user to return to the information input screen even after the customized recommendations have been presented.
4. Add descriptions for special terms, such as low HDL, hyperlipidemia, and impaired glucose tolerance.
The above four suggestions were included as a modification list along with the suggestions proposed by the experts during the heuristics evaluation stage.
IV. Discussion
1. System Analysis
We extracted the knowledge necessary for obesity management from obesity-management clinical practice guidelines and obesity-related papers, and then applied it to our system. Web- or app-based obesity management has the benefit of being unencumbered by temporal or spatial constraints [6]. However, drug or surgical interventions can only be provided by physicians and therefore require a physical visit to the doctor's office or another type of healthcare institution. We therefore excluded drug and surgical interventions from this study. For the same reason, Jeon and Kim [4] only included diet, exercise, and behavior modification in their Web-based coaching study of the obesity management of hypertensive patients.
In the present study, only self-monitoring was chosen among the behavioral therapy techniques presented by the NHLBI [12]. Other techniques, such as stress management, stimulus control, cognitive restructuring, and social support were not included because of the limitations of smartphones and the time span of the study. Breton et al. [6] found that only 7 of 204 weight-loss apps provided means to support or interact with others. Social networking has become a key feature of the online experience [19], so future studies should investigate the implementation of social support for obesity management.
2. System Design
We extracted knowledge from the obesity-management clinical practice guidelines, specified the rules and actions for decision-making, and then refined the concepts to express them as algorithms. These algorithms were designed only to assist the understanding of programmers or domain experts [20]; therefore, additional coding is necessary for the actual system development and implementation. Future studies should investigate approaches that would allow a domain expert to mark up content using knowledge expression tools into the form of computer-interpretable guidelines [11,20,21].
3. System Implementation
The app development environment was limited to Android in this study, so further investigations involving other operating systems are required, such as the development and evaluation of an iOS app.
4. System Evaluation
Knowledge of the obesity-management app was evaluated by computing the proficiency and efficiency of the obesitymanagement algorithms of the three evaluators. The proficiency score was 88.2% indicating that the evaluators were in general agreement with the system at the recommendation group level. However, the efficiency score (69.0%) was much lower than that of proficiency, indicating that the evaluators did not agree with the system at the specific recommendation level as much as the recommendation group level. Since evaluator A did not have as clear an understanding of how the flow of the algorithm works as the other two evaluators had due to a lack of system development experience, she was able to suggest an appropriate recommendation group but not a specific recommendation.
An open-ended mobile heuristics evaluation tool was used in this study, which resulted in the evaluators documenting the same problems using different heuristics items and giving 0 points for severity to items that the experts had already pointed out as being problematic. This could be due to the heuristics items being ambiguous and the method of scoring severity being unclear. To address these problems, items can be presented in a checklist, as in the heuristics evaluation tool developed by Pierotti [22], and specific criteria can be used to assign points to each category.
The present study was subject to a few limitations. First, we only chose self-monitoring behavioral therapy techniques suggested by the NHLBI. Future investigations need to consider how to make other behavioral therapy interventions, such as stress management, stimulus control, cognitive restructuring, and social support, available on smartphones.
Second, we did not make full use of the technological capabilities of smartphones, such as their embedded sensors. The obesity-management app developed in this study only allows data to be entered manually, and we could not evaluate the reliability of user-entered information. Future developments could include providing appropriate obesity-management interventions based on comparisons between the objective information collected using the embedded sensors in smartphones and self-reported subjective information [23].
Third, we were not able to evaluate the effectiveness of the developed obesity-management app in this study. The true utility of the app therefore needs to be verified by conducting evaluations of the health outcomes of real users-that is, obese users-after they have used the app to manage their obesity.
Fourth, the obesity-management app was developed in the present study for adult Android smartphone users only. Thus, apps for other age group users and users of other smartphones, such as iOS, should be developed in the future.
Mobile healthcare is becoming an active component of the healthcare industry due to the ever-increasing usage of smartphones and other wireless devices, but most of the available obesity-management apps provide low-quality information [7,8]. Despite the abovementioned limitations, the app developed in the present study has some strengths. First, the app might help obese users to manage their obesity since it has been developed based on high-quality health information extracted from published clinical practice guidelines.
Second, the developed app is aimed at consumers who want to manage their weight on their own, so it can strengthen self-management by allowing users to monitor their own diets and exercise records as well as set their own weight-loss goals.
Third, we modified the system based on suggestions made by real users during the usability evaluation stage, which resulted in the production of a user-centered system. Usability refers to the effectiveness of the interaction between humans and computer systems, and it can be specified in terms of how well potential users can perform and master tasks on the system [24]. A system's usability can also be measured empirically in terms of its effciency, error rate, and user satisfaction [25]. Since our evaluations focused mainly on user satisfaction, user evaluations involving other factors are recommended.
PDF Links
PubReader
XML Download
Full text via DOI
Full text via PMC
Download Citation
