Healthc Inform Res Search


Healthc Inform Res > Volume 22(3); 2016 > Article
Kim and An: New Evaluation Vector through the Stanford Mobile Inquiry-Based Learning Environment (SMILE) for Participatory Action Research



This article reviews an evaluation vector model driven from a participatory action research leveraging a collective inquiry system named SMILE (Stanford Mobile Inquiry-based Learning Environment).


SMILE has been implemented in a diverse set of collective inquiry generation and analysis scenarios including community health care-specific professional development sessions and community-based participatory action research projects. In each scenario, participants are given opportunities to construct inquiries around physical and emotional health-related phenomena in their own community.


Participants formulated inquiries as well as potential clinical treatments and hypothetical scenarios to address health concerns or clarify misunderstandings or misdiagnoses often found in their community practices. From medical universities to rural village health promotion organizations, all participatory inquiries and potential solutions can be collected and analyzed. The inquiry and solution sets represent an evaluation vector which helps educators better understand community health issues at a much deeper level.


SMILE helps collect problems that are most important and central to their community health concerns. The evaluation vector, consisting participatory and collective inquiries and potential solutions, helps the researchers assess the participants' level of understanding on issues around health concerns and practices while helping the community adequately formulate follow-up action plans. The method used in SMILE requires much further enhancement with machine learning and advanced data visualization.

I. Introduction

Community- and organization-based research studies, which are conducted by directly involving local participants or indigenous researchers mainly for transforming social inequalities, have been described as action research (AR), participatory research (PR), participatory action research (PAR), or community based participatory research (CBPR) [1]. Although these terms are often interchangeably used, what connects them all is the final outcome of such efforts. PAR is usually carried out more for the sake of action or transformation as an outcome rather than the sake of conducting research. Action facilitates community engagement, encourages culturally responsive strategies, and focuses on rendering better health services as well as adequately addressing health disparities the participating communities may be facing [2].
PAR can help a group or community and even historically disempowered or marginalized communities to set their own goals and equip themselves with the strategies of change to influence policy makers who may be governing the communities [2,3]. PAR was often used in rural areas [4] or developing countries for needs assessments [4], planning and evaluating health services [3]. Recently, PAR has been used in healthcare research, which is based on reflection, data collection, and action to improve health and reduce health inequities through involving community members and consumers who seek to be active and more powerful agents over the practices of local or regional clinics and the knowledge of surveillance system as a part of efforts to improve their own communal health [5].
Practitioners in healthcare have applied PAR for clinical setting-based projects [3] to explore health related topics affecting particular populations within a community [2,6], for example, diabetes and obesity [9], multiple sclerosis and other chronic illnesses [8,9], and HIV/AIDS prevention [6,10,11,12]. Such method also has been employed to address communal issues of diverse populations (i.e., culturally and ethnically) [11,12].
In the process of PAR, participants meet in peer group meetings periodically for a certain period of self-reflection and action planning (i.e., so-called, stepping stone approach) or interchanging with members from different peer groups together (i.e., often called, mixed sessions) [6]. These discussions allow the participants to examine similarities, differences, awareness of health problems and well-being, respect for others, and positive solutions among the community participants [6]. Oftentimes, it becomes somewhat complex for practitioners to translate and integrate theory-based knowledge into evidence-based practice effectively. Therefore, an ontological inquiry process has been considered to enhance possibilities for action in a way-of-being [13] and other types of modern communication technologies. For example, the increasing penetration rate of mobile phones and the Internet use have made tremendous improvements in numerous aspects of PAR to date. Utilizing mobile devices to access health related information and service by the public is not just a trend but a new practice of innovation [14], which is so-called mobile health (mHealth) [15]. At the same time, such mobile ecosystem helps researchers gather new kinds of data that were otherwise impossible.
Stanford Mobile Inquiry-based Learning Environment (SMILE) is one of advanced communication technologies leveraging crowd-sourcing for collective inquiries. In the field of public health, it has been used for connecting communities of health practitioners or indigenous members of communities to exchange inquiries and solutions to cause a paradigm shift within the participating community by enabling participants to become active agents in their own education, practice, and empowerment process [16,17].
Due to the infancy stage of the use of social media for healthcare and education, uncertainty is abundant among researchers who have employed traditional research methods. However, due to its pervasiveness, immediacy, and wide-outreach capability, the need to leverage such important technology is increasingly critical in healthcare [15].
Therefore, the purpose of this article is to review how interactive technologies such as SMILE can be leveraged to enhance PAR-type projects and how such intervention may work to help active participants create, solve, evaluate, present, share, and reflect communal inquiries and action plans central to the highly idiosyncratic nature of local communities.

II. SMILE in Public Health

SMILE has been implemented in a diverse set of collective inquiry generation and analysis scenarios including community-based healthcare-specific professional development sessions and community-based PAR projects [17]. In each scenario, participants are given opportunities to construct inquiries around physical and emotional health phenomena in their own community. Participants formulated not only inquiries but also potential clinical treatments and hypothetical scenarios to address health concerns or clarify misunderstandings or misdiagnoses, which are often found in rural community or clinical practices. In the end, the participating community can collectively generate, solve, evaluate, and discuss questions and possible solutions in real-time [18]. In the long run, the questions become sharable with a much larger community at the global level as well. Unlike traditional prescribed surveys, SMILE helps gather questions and solutions that are considered more important to the local community. Analyzing the evaluation vector created from community inquiries, driven from collectively formulated questions and solutions, helps researchers better understand: interests, phenomena, malpractices, and the misunderstanding of health issues and treatments at a much deeper level.

III. Why SMILE Integrated PAR?

It is important to recognize the rationale of leveraging SMILE as a new evaluative vector model in PAR because SMILE replaces conventional surveys that are often used in public healthcare relevant studies. Although its convenience outweighs the reliability or validity of research, the self-reporting survey method has been known to have numerous limitations. For example, the major limitation of the survey method is that it relies heavily on self-responded indicators, often associated with intentional deception or conformity to the societal norms, affecting the validity of the collected data [19,20]. Also, pre-formulated survey questions from external researchers are often predefining "known" problems in the view of the researcher while limited answer choices are habitually leading to limited responses the researchers may be seeking [21]. Therefore, the survey method makes it hard to distinguish between realities and biases [22,23]. More importantly, with the survey method, a researcher may not be told about critical conversations and so may never know how the outcome is accurately demonstrated in the data [24].
SMILE enables the participants to formulate inquiries and potential solutions that are most important and known to participants themselves. Also, SMILE leverages the collective construction of inquiries, leading to more reliable and accurate depiction of the reality as a whole. By empowering the participants to co-evaluate and reflect their own phenomena, the external researcher's role is simply facilitating, not intervening.
The collective inquiry process enables the participants to think beyond simple recalls of health-relevant facts while pervading critical thinking on knowledge and practices that are local and specific to their own community. Here are a couple of examples of lessons learned from questions collected through SMILE sessions.
  • Distinguish facts and opinions of the community as a whole (e.g., hypertension is from taking too much salt).

  • Verify sources of facts collected (e.g., vector control is the most useful method to prevent Chagas disease in Latin America).

  • Analyze cause/effect relationships reflected in collective inquiries (e.g., rash is one of symptoms caused by Zika virus).

  • Determine faulty generalizations common to the local region (e.g., doctors say sugar is worse than salt for patients with diabetes).

  • Avoid oversimplification, marketing propaganda, distortion, deception, or stereotyping (e.g., bacitracin ointment products cure most conjunctivitis).

IV. What is the Technological Function of SMILE?

SMILE is mobile interaction management server software. The SMILE server software is designed to create a highly interactive learning environment that promotes critical thinking opportunities (e.g., creation of inquiries, presentation of questions, analysis of peer-generated questions, evaluation of individual participants and overall inquiry exchange process, etc.); engages participants in inquiry-based learning sessions at a global scale; generates transparent real-time analytics; and enables a low-cost mobile wireless interaction environment for extremely underserved regions where access to Internet and electricity may not be reliable or available.
A SMILE Plug computer (Figure 1) contains the SMILE server software, KIWIX (mobile downloadable version of Wikipedia,, Khan Academy Lite (i.e.,, various open education resources including open education textbooks, and different simulation and experiment programs (i.e., current contents listed at
Recently, the SMILE server has become accessible in the cloud at The SMILE server software enables participants to generate questions in both public and private groups for all participants to share, solve, evaluate, and discuss questions. The overall SMILE software and features are constantly evolving and enhanced to support participant learning in various levels and scenarios.

V. How Does SMILE Session Work?

Based on the SMILE activity model (Figure 2), a typical SMILE session involves a cyclical model where participants are guided to investigate an inquiry topic and access reference materials to formulate a high quality question (i.e., ultimately triggering higher order thinking) [25].
With SMILE, participants can explore healthcare topics that are most important and interesting to themselves. For example, when participants make multiple-choice questions, they also need to list answer choices they believe correct. While generating questions, participants can capture images or graphs from reference books, take pictures of physical environments (e.g., symptoms, sources of contaminants, etc.) around their community, and record audios or videos of patient testimonies.
Once questions are generated, the mobile-based SMILE management system aggregates all inquiries coming from participants and redistributes them for the participants to reflect, respond, evaluate, and verify. When participants are responding to peer-generated inquiries, they get to cover many topics in numerous forms and shapes that they usually do not experience otherwise.
After responding to the inquiries, participants are asked to present their questions and explain the rationale behind their logic, background, critical components, etc. Facilitators also see if there is any deficiency in the structure of the question, relevancy of the question, and appropriate use of the supporting media for their questions.
In addition, participants rank the peer-generated questions based on the perceived level of question quality. If they believe their peers' questions are not mere simple recall questions and simple opinions, they give a higher rating. In fact, if they find their peers' questions trigger critical thinking, they give the highest rating. The highly ranked questions are often the best questions; therefore, session facilitators can use them as learning stimuli for further discussions [26].
As a motivational tool for the SMILE session, incentives and rewards are crucial. An organizing facilitator, therefore, may have participants compete in teams and award the winning team (i.e., receiving the highest rating on team inquiries or solving the most number of questions correctly, etc.). Activities can turn into a highly stimulating game atmosphere; therefore, utilizing SMILE could make the community inquiry processes engaging and research-oriented [18]. Table 1 shows various modes of SMILE sessions.
Overall, the result section gives a quick overview on the number of questions generated, questions answered, the percentage of correctly answering questions (i.e., perceived to be 'correct' by the participants), average rating given to peer questions, and average rating received by peers (Figure 3). Each of the section can be sorted in both ascending and descending orders. Also, the results can be presented either by the list of member or by the list of questions.

VI. Scaffolding Structure in SMILE

Helping participants generate high quality inquiries requires ample and persistent guidance, modeling, sample question sets, evaluation rubrics, and practice. Therefore, some, yet still limited scaffolding features are available in SMILE and these are constantly updated.
  • Rubric enables organizers to define 5 different levels of question quality. The rubrics are available for participants to reference when assigning 1 to 5 stars to peer questions being evaluated.

  • Prompter enables organizers to assign a list of key phrases or words to be incorporated in questions. This forces the participants to formulate questions by incorporating one or more key phrases for the system to accept. For example, if the organizer makes words 'What' and 'If', all participants in a particular activity must incorporate 'What' and 'If' in his or her question. Otherwise, the system will reject the question.

  • Hide identity feature, organizers have an option to make the participant's identity visible or invisible. With hidden identity, participants may feel less intimidated to share questions that might be innovative, yet radical or awkward with a given topic or within the context of discussions.

VII. SMILE Study Cases

SMILE has reached over 25 countries and mostly in underserved communities in developing countries. Since SMILE is leveraging mobile technology, participants can pick up a conventional mobile phone and participate in SMILE activities within a few minutes. The use of conventional mobile phones makes the model simple to adopt and less threatening for participants [18].
For example, a participant in Argentina is using her mobile phone to take photos of sanitation issues in her community to formulate a public inquiry of conditions susceptible to spread diseases including Chagas [27]. Also, a medical practitioner in Argentina is incorporating medical images in his inquiry to engage in a SMILE session regarding symptoms and treatments of Chagas disease [27]. A public health major university student in Colombia is creating questions to share with his community regarding Chagas disease using his mobile phones [28]. These connect to the concept of mHealth in public health.
For a SMILE session in Korea, a group of medical students are generating a collective inquiry regarding HIV/AIDS [29]. In this, facilitators are playing a key role during the session as they are enjoying the participatory inquiry as a valuable teaching and learning method. Since SMILE leverages mobile technology, the collective inquiry takes place regardless of time and location. A central cloud server ( collects all inquiries and sessions from around the world [30]. People who are interested in a particular topic (e.g., for example, 'health'), they can search the keyword and also create their own questions, respond to existing questions, or comment on questions and answers (Figure 4).
As shown in Figure 5, a list of sample questions with the keyword search of 'health' in a public health session is derived from a rural region of Colombia [31]. Examining the questions and answer choices generated by participants help researchers uncover important insights. Some of them may include: (1) What the common health concerns may be in the region, (2) What symptoms they believe are associated with a particular illness, (3) What treatments they believe may be available for a particular disease, (4) What practices they believe have been successful in their community, (5) How participants identified flaws or misunderstanding from peer inquiries or solution choices, (6) How local communities believe the workable solutions may be politically correct, (7) How the community members agree on inquiries or actionable solutions as most appropriate, (8) How some of the inquiries or solution choices change over time, (9) How the local community members construct their own inquiry evaluation rubrics, and (10) How they evaluate and rank potential solution plans collectively.

VIII. Conclusion

PAR was often used in rural areas or developing countries for analyzing community healthcare needs or planning and evaluating health services [3]. Recently, PAR has been used in healthcare research, which is based on reflection, data collection, and action to improve health and reduce health inequities. In such efforts, community members, as active agents, collectively analyze and improve local services [5] or enhance their knowledge. With the advancement of information and communication technologies, there are unprecedented number of opportunities to enhance various aspects and processes of PAR at multiple levels. One immediate area of enhancement in PAR is to leverage an intervention using information and communication technologies such as SMILE.
Unlike traditional survey methods used in PAR, SMILE collects problems and potential solutions that are most important and central to the condition of their local community. Collective inquiries, along with hypothetical scenarios with healthcare phenomena, policies, practices, or action plans based on divergent thinking ideas, provide an insightful evaluation vector more appropriate for the participating community. This article demonstrated the applicability of SMILE in public health by introducing a few different cases of SMILE in the developing countries, which have not been discussed in previous literature.
It is the hope that researchers and educators involved in PAR for public health and education leverage interventions such as SMILE to design data collection strategies, which are more contextualized and meaningful to local communities, and analyze such data in ways that can leverage analysis models backed by machine learning and augmented intelligence. In doing so, new insights can be uncovered; therefore, creative healthcare and education plans may be developed to bring about transformation, which is long overdue in rural communities of the developing regions.


This work was partially supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2014S1A5B8044097). The authors are very thankful to the editorial board of Healthcare Informatics Research.


Conflict of Interest: No potential conflict of interest relevant to this article was reported.


1. Lykes MB, Hershberg RM. Participatory action research and feminisms: Social inequalities and transformative praxis. In: Hesse-Biber SN, editors. Handbook of feminist research: theory and praxis. 2nd ed. Thousand Oaks (CA): Sage Publications; 2012. p. 331-367.

2. Fournier B, Mill J, Kipp W, Walusimbi M. Discovering voice: a participatory action research study with nurses in Uganda. Int J Qual Methods 2007;6(1):1-19.
3. Minkler M, Wallerstein N. Community based participatory research for health. San Francisco (CA): Jossey-Bass; 2003.

4. De Koning K, Martin M. Participatory research in health: issues and experiences. London: Zed Books; 1996.

5. Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health 2006;60(10):854-857. PMID: 16973531.
6. Welbourne A. Gender, participation and HIV: a positive force for change. In: Guijt I, Shah MK, editors. The myth of community: gender issues in participatory development. London: Intermediate Technology Publications; 1998. p. 131-140.

7. Harvey I, Schulz A, Israel B, Sand S, Myrie D, Lockett M, et al. The Healthy Connections project: a community-based participatory research project involving women at risk for diabetes and hypertension. Prog Community Health Partnersh 2009;3(4):287-300. PMID: 20097990.
8. Allen D, Hutchinson T. Using PAR or abusing its good name? The challenges and surprises of PhotoVoice and PAR in a study of chronic illness. Int J Qual Methods 2009;8(2):115-128.
9. Kralik D, Koch T, Eastwood S. The salience of the body: transition in sexual self-identity for women living with multiple sclerosis. J Adv Nurs 2003;42(1):11-20. PMID: 12641807.
10. Welbourne A. Participatory approaches to HIV/AIDS programmes. PLA Notes 1995;23:57-80.

11. Rhodes SD, Eng E, Hergenrather KC, Remnitz IM, Arceo R, Montano J, et al. Exploring Latino men's HIV risk using community-based participatory research. Am J Health Behav 2007;31(2):146-158. PMID: 17269905.
12. Coker-Appiah D. The CEDAW convention and harmful practices against women: the work of the CEDAW Committee. New York (NY): United Nations; 2009.

13. Doane GH, Varcoe C. Family nursing as relational inquiry developing health promoting practice. Philadelphia (PA): Lippincott Williams & Wilkins; 2004.

14. Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med 2009;360(15):1477-1479. PMID: 19321856.
15. Fernandez-Luque L, Bau T. Health and social media: perfect storm of information. Healthc Inform Res 2015;21(2):67-73. PMID: 25995958.
16. Buckner E, Kim P. Integrating technology and pedagogy for inquiry-based learning: the Stanford Mobile Inquiry-based Learning Environment (SMILE). Prospects 2014;44(1):99-118.
17. UNESCO Information Services Section. ICT in education [Internet]. Paris: UNESCO; c2016. cited at 2016 Jul 1. Available from:

18. SMILE UNESCO presentation by Paul Kim [Internet]. [place unknown]: SlideShare; c2016. cited at 2016 Jul 1. Available from:

19. De Cooman R, De Gieter S, Pepermans R, Du Bois C, Caers R, Jegers M. Freshmen in nursing: job motives and work values of a new generation. J Nurs Manag 2008;16(1):56-64. PMID: 18211336.
20. Wong LP, Tan HM, Low WY, Ng CJ. Traditional and complementary medicine (T/CM) in the treatment of erection difficulties–experience from the Asian Men's Attitudes to Life Events and Sexuality (MALES) study. J Mens Health 2008;5(4):356-365.
21. Fry L, Santos YE, Zhang Y. Health information use: preliminary results from a systematic review. Proc Assoc Inf Sci Technol 2015;52(1):1-3.
22. Pepe MV, Sanders DW, Symons CW. Sexual behaviors of university freshmen and the implications for sexuality educators. J Sex Educ Ther 1993;19(1):20-30.
23. Ouzounian M, Hassan A, Teng CJ, Tang GH, Vanderby SA, Latham TB, et al. The cardiac surgery workforce: a survey of recent graduates of Canadian training programs. Ann Thorac Surg 2010;90(2):460-466. PMID: 20667330.
24. Jordon M, Lanham HJ, Anderson RA, McDaniel RR Jr. Implications of complex adaptive systems theory for interpreting research about health care organizations. J Eval Clin Pract 2010;16(1):228-231. PMID: 20367840.
25. Kim P. Future education ecosystem. Paris: UNESCO; 2011.

26. Kim P. It's time to flip the classroom [Internet]. [place unknown]: wise; 2016. cited at 2016 Jul 1. Available from:

27. Kim P. Inquiry based public health education in Buenos Aires, Argentina. Stanford (CA): SMILE; 2012.

28. Kim P. Inquiry based public health education in La Guajira, Columbia. Stanford (CA): SMILE; 2013.

29. Kim P. Inquiry based medical education in Chungbuk National University, Korea. Stanford (CA): SMILE; 2013.

30. SMILE (Stanford Mobile Inquiry-based Learning Environment) - medical education [Internet]. [place unknown]: SlideShare; c2016. cited at 2016 Jul 1. Available from:

31. Kim P. Inquiry based public health education in La Guajira, Columbia. Stanford (CA): SMILE; 2016.

Figure 1

Stanford Mobile Inquiry-based Learning Environment (SMILE) Plug computer designed by Stanford University Graduate School of Education and manufactured by Marvell Semiconductor and GlobalScale.

Figure 2

Stanford Mobile Inquiry-based Learning Environment (SMILE) activity model.

Figure 3

Sample Stanford Mobile Inquiry-based Learning Environment (SMILE) result page.

Figure 4

Central cloud server ( collecting all questions from around the world.

Figure 5

Questions derived from a participatory action research case in Colombia.

Table 1

Modes of Stanford Mobile Inquiry-based Learning Environment (SMILE) sessions

Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 1 Crossref
  • 1   Scopus
  • 1,118 View
  • 2 Download
Related articles in Healthc Inform Res


Browse all articles >

Editorial Office
1618 Kyungheegung Achim Bldg 3, 34, Sajik-ro 8-gil, Jongno-gu, Seoul 03174, Korea
Tel: +82-2-733-7637, +82-2-734-7637    E-mail:                

Copyright © 2021 by Korean Society of Medical Informatics. All rights reserved.

Developed in M2community

Close layer
prev next