Trends in Health Information Technology Safety: From Technology-Induced Errors to Current Approaches for Ensuring Technology Safety

Article information

Healthc Inform Res. 2013;19(2):69-78
Publication date (electronic) : 2013 June 30
doi : https://doi.org/10.4258/hir.2013.19.2.69
School of Health Information Science, University of Victoria, Victoria, BC, Canada.
Corresponding Author: Elizabeth Borycki, RN, PhD. School of Health Information Science, University of Victoria, 3800 Finnerty Road, Victoria, BC V8P 5C2, Canada. Tel: +1-250-472-5432, Fax: +1-250-472-4751, emb@uvic.ca
Received 2013 June 03; Revised 2013 June 19; Accepted 2013 June 20.

Abstract

Objectives

Health information technology (HIT) research findings suggested that new healthcare technologies could reduce some types of medical errors while at the same time introducing classes of medical errors (i.e., technology-induced errors). Technology-induced errors have their origins in HIT, and/or HIT contribute to their occurrence. The objective of this paper is to review current trends in the published literature on HIT safety.

Methods

A review and synthesis of the medical and life sciences literature focusing on the area of technology-induced error was conducted.

Results

There were four main trends in the literature on technology-induced error. The following areas were addressed in the literature: definitions of technology-induced errors; models, frameworks and evidence for understanding how technology-induced errors occur; a discussion of monitoring; and methods for preventing and learning about technology-induced errors.

Conclusions

The literature focusing on technology-induced errors continues to grow. Research has focused on the defining what an error is, models and frameworks used to understand these new types of errors, monitoring of such errors and methods that can be used to prevent these errors. More research will be needed to better understand and mitigate these types of errors.

I. Introduction

In the early 1990s we saw an international trend arise where a number of publications from countries such as the United States, Canada, the United Kingdom, and Australia identified medical errors as a significant public health and healthcare issue as they could lead to adverse events [1-4]. For example, Australian researchers reviewed 14,000 medical records from 28 hospitals in several Australian states and found that 16.6% of hospital admissions were linked to an adverse event [1]. Canadian researchers in their study of the incidence of adverse events in hospitals found that 7.5% of hospital admissions (i.e., about 185,000) led to an adverse event [2]. In the United States, a study of 51 randomly selected New York state hospitals revealed 3.7% of hospitalizations were associated with an adverse event with 27% of these adverse events arising "from negligence". Findings from a similar study conducted in British hospitals revealed an overall adverse event rate of 11.7% [4]. Internationally, the research suggested that adverse events across countries with similar healthcare systems can range from 7.5% to 16.6%. More importantly, costs to the healthcare system and the personal costs to patients, families and health professionals were significant [1-4].

Researchers, policy makers and administrators from around the world agreed that medical errors leading to adverse events were a significant health systems issue that needed to be addressed. They also identified that traditional approaches to preventing medical errors were not effective and that new and innovative ways needed to be developed in order to address these errors. This work culminated in the publication of the text "To Err Is Human: Building a Safer Health System" in 1999 [5] and "Patient Safety: Achieving a New Standard of Care" in 2004 [6]. In these landmark works several recommendations were made by experts in the healthcare field, among them a need to implement specific types of health information technologies (e.g., computerized physician order entry, decision support systems) in order to reduce medical error rates [5,6]. Health information technologies (HIT) such as computerized physician order entry were identified as an innovative intervention that could be applied to and could significantly reduce medical error rates [5-7]. Since then, there have been several studies that have documented the ability of decision support systems [6,7], pharmacy information systems [8], electronic medication administration systems [8] and clinical documentation systems to reduce medical error rates [9]. However, in 2005 research began to emerge suggesting that HIT could introduce new types of medical errors (i.e., technology-induced errors) [10-14].

HIT research findings suggested that new healthcare technologies could reduce some types of medical errors while at the same time introducing classes of medical errors (i.e., technology-induced errors). Technology-induced errors have: their origins in HIT, and/or HIT contribute to their occurrence [10-13]. Since 2005 this research literature has grown significantly. Several trends have emerged. In this paper we outline some of these trends as researchers and policy makers attempt to understand the nature of this emerging problem, the prevalence of these types of errors and the ways in which such errors are modeled, prevented and understood in an effort to improve HIT safety. The researcher begins by defining technology-induced errors.

II. Four Main Trends in the Literature on Technology-Induced Errors

1. Definitions of Technology-Induced Errors

Technology induced errors are medical errors that arise from the: design and development of a technology; implementation and customization of a technology; and interactions between the operation of a new technology and the new work processes that arise from the technology's use [10]. Technology-induced errors typically manifest themselves in the complex context of use of HIT in real clinical settings and so they may be difficult to detect using traditional software testing methods. Technology-induced errors include slips and mistakes. Slips have been defined as errors "which at some point the user notices and corrects", for example, an incorrect medication entry the user amends. Mistakes are errors which are not observed or corrected by a user [12]. The user enters a medication dose thinking it is correct and a default auto-populates the field with another dose [11-15]. As another example, more than one patient record is opened on the computer screen and the user (e.g., physician) inadvertently enters patient data in the wrong record after being called away from the computer to deal with a patient issue [11]. These errors also include information transfer errors that users may not be aware of and are occurring on the back end of a system [14,15]. In one situational context a technology-induced error may lead to no harm (as in the case of a slip). Here, a health professional corrects the error or prevents the error from propagating throughout the system. In another context, harm, disability, and death may occur (as in the case of a mistake). Technology-induced errors involving slips and mistakes should be treated as important as they offer an opportunity to address HIT related issues and the opportunity to prevent future technology-induced errors [12,13].

Some researchers have referred to technology-induced errors as unintended consequences [16,17] while others have characterized these types of errors as e-iatrogenic events [18]. It must be noted that there are differences in the definitions of "technology-induced error", "unintended consequence", and "e-iatrogenic" event. These differences are significant and important to better understand. Unintended consequences "arise from the use of a technology" and include all those consequences that "lack purposeful action or causation" [16,17]. The definition is more global and takes into account the positive, neutral, and harmful consequences of introducing a new technology. This is important as we also need to understand how technology use can improve healthcare as well as detract from healthcare work in a global sense [16,17]. As Kuziemsky et al. [19] identify, unintended consequences arising from the use of a technology can harm and/or support patients, health professionals and healthcare system activities. These "unintended consequences" may also have unexpected, positive, benefits for patients and health professionals and we need to take advantage of them [19].

The term e-iatrogenic event has also been used by researchers. E-iatrogenic events focus on "patient harms caused in part by the application of HIT" [18]. The focus of this definition is on "patient harm". As outlined earlier, other researchers have shown that some technology-induced errors may lead to slips or in some cases death, disability or injury [12]. These researchers have also found that health professionals are also able to identify examples of HIT interface design and related workflows where there have been "slips" and correct potential errors [12]. This suggests there is a need to use a broad definition of technology-induced errors, one that includes "mistakes" (i.e., when an error leads to death, disability and/or injury), and "slips" sometimes referred to as near misses (i.e., when a technology-induced error is caught by a health professional before it occurs) [12]. All of these definitions are important. Unintended consequences take into account the positive and negative outcomes of introducing HIT [16,17]. E-iatrogenic events acknowledge the importance of attending to technologies that may cause harm [18]. Technology-induced errors focus on the occurrence of both slips and mistakes involving technology. Both slips and mistakes offer organizations the ability to identify opportunities to learn from potential and actual errors with a focus upon improving HIT across the systems development lifecycle [12,13].

2. Models, Frameworks and Evidence of Technology-Induced Errors

There is general agreement among researchers and policymakers that technology-induced errors arise from a number of sources [10-16,18-20]. Researchers have suggested human factors, sociotechnical, organizational and/or software factors may have a role in technology induced errors [21]. There is widespread agreement among researchers and policy makers that technology-induced errors are complex and require careful investigation to fully understand how they occur. There is consensus among researchers and policy makers that we have only begun to investigate the causes of these errors and a science of HIT safety will need to be developed and extended in the upcoming years [17,20]. Researchers have also only recently begun to understand extent and nature of this problem from a software engineering, human factors, sociotechnical and organizational perspective [16,17,21,22].

As outlined earlier in this paper, research about technology-induced error in HIT began to emerge in the early 2000s with the first seminal papers being published in the international literature [11,12,22,23]. Today, there is agreement among researchers and policy makers that errors can have their origins in HIT [11,12,17,20-22,24]. In addition to this, researchers acknowledge that there are safe and unsafe HIT, recognizing that most HIT fall on a continuum somewhere between those systems that are safe and those that are unsafe (i.e., HIT has both safe and unsafe software features and functions) [25].

Researchers have developed models and frameworks to better understand the sources of technology-induced error. Some of these frameworks focus on aspects of interface design and workflow [21]. Others take a whole systems approach to identifying the possible origins and contributing factors that lead to technology-induced errors from government policies, through to technology vendors and implementing organizations (i.e., regional healthcare organizations) and the users themselves [10]. Researchers are only beginning to uncover the possible sources of technology-induced error. As the number of Electronic Health Record, healthcare software applications, and healthcare device implementations continues to grow exponentially along with the number of technologies that exchange information between technology software and hardware in healthcare. Models and frameworks published in the literature are beginning to inform our thinking as researchers, policy makers and HIT practitioners' [20]. They have helped us to identify potential causes of technology-induced errors and have focused our attention on our practices and processes (i.e., where selection and procurement) are concerned so we can identify safer HIT, and identify methods for unraveling the complexities of understanding where a technology-induced error comes from [20,26]. More importantly, researchers are now testing these models and frameworks to develop representative models of where errors may occur in a healthcare system in order to address these errors and prevent their future occurrence [21,27]. Empirically validated models add to our knowledge and understanding of technology-induced errors and provide strong support for improving the safety of HIT.

Some models and frameworks have suggested and research has found that the source of technology-induced errors rests with the technology itself [14,15]. Errors can arise from interface design and workflows emerging from the technology [11-13,28,29]. Research identifies technology-induced errors have their origins in requirements specification, design, development, programming, customization, implementation, and maintenance of HIT (including software upgrades, software maintenance activities as well as interactions between software and devices that are being integrated with systems) [10]. Future research aims need to include identifying both safe and unsafe aspects of HIT. There is a need to ensure future HIT falls into the part of the HIT quality continuum where systems are highly safe (rather than partially safe or unsafe) [25].

Other research has identified organizational-technology and sociotechnical interactions may lead to technology-induced errors. Extending beyond human factors considerations, socio-technical issues have emerged as adding to our understanding and knowledge of technology-induced errors [17,22]. For example, work by Aarts [22] suggests that healthcare is a complex sociotechnical system. Poor HIT design and implementation may lead to a failure to implement a HIT as well as lead to technology-induced errors, when there is poor integration with the sociotechnical system of a healthcare organization [22]. More recently, Elkin [30] and Aarts [22] have suggested context is important when considering HIT influences on healthcare processes and outcomes. Country, healthcare system, regional (e.g., provincial, territorial, state, and county), and local organizational (e.g., health maintenance organization, regional health authority) policies, procedures and processes may influence technology-induced error rates [10].

Research by Han et al. [31] and Del Beccaro et al. [32] illustrate this best. Two different studies in the same country involving the same commercial electronic record system led to differing results, when patient mortality was considered. Some have suggested organizational decision making influenced the outcomes of these two different implementations. Others have suggested that the customization of the technology to a specific patient population may have influenced the outcomes of these studies [33,34]. Still others have suggested that research design needs to be considered [33]. Even so there is a general consensus that we must learn from HIT implementations so that we can prevent technology-induced errors [22].

A growing body of work by Magrabi et al. [14,15] suggests there are country level influences. Technology-induced error rates and harms differ among countries. In reviewing technology-induced errors by searching two differing adverse events databases-one in Australia and one in the United States, the researchers noted similarities in the types of errors that were identified in both countries. Yet, there were differences in reported harms and deaths arising from technology-induced errors. Here, country context of use and customization of HIT to a local country healthcare system may influence the occurrence rates of errors [14,16]. There is a need for healthcare researchers, administrators and policy makers to understand the implications of country context upon technology-induced errors and a need to exchange this information at an international level.

As HIT vendors increase their global reach and sell products to countries with varied healthcare systems, there will be a need for multi-site, within country studies and multi-site studies across countries to fully understand the effects of country context on technology-induced errors. In summary there is growing recognition that context of use should be considered where technology-induced errors are concerned. Country, local, and healthcare system influences may lead to technology-induced errors. Acknowledging contextual influences is a key to ensuring strong task-technology or sociotechnical fit and reducing the likelihood of a technology-induced error.

3. Monitoring and Reporting Technology-Induced Errors

There is great debate in the research and the HIT community regarding the prevalence of technology-induced errors. Some researchers have suggested that HIT contributes very little to the overall rate of medical errors [14,15]. For example, according to Magrabi et al. [15], only 0.1% of all reports in the United States Food and Drug Administration Manufacturer and User Facility Device Experience (MAUDE) involved HIT [15]. In another publication by this group (this time examining reports from a voluntary incident database) from Australia, the researchers reported 0.2% of all incidents involved computers [14].

Other research suggests that technology-induced error incident rates are much higher. Koppel et al. [11] research identified that 75% of house staff (i.e., physicians) who participated in their study noted the presence of several differing types of error risks (e.g., "pharmacy inventory displays mistaken for dosage guidelines", "ignored antibiotic renewal notices placed on paper charts") as having occurred "weekly" and in some cases "more often". In another more recent study, Samaranayake et al. [35] reported on findings from a study of medication incidents in a large tertiary care hospital that was highly automated (from 2006-2010). The researchers found that 17.1% of all medication incidents involved technologies such as "computerized prescription order entry", "bar-coded patient identification labels", "infusion pumps", and "computer-aided dispensing labels" [35].

Highly automated hospitals may have higher rates of technology-induced errors. As the introduction of electronic records in physician offices, hospitals and other organizations is in its infancy perhaps we are only seeing the beginning of reports on these types of errors: "the tip of the iceberg" [11,35]. Given this research in involving hospitals that are highly automated, there is a need to recognize that technology-induced errors may be more prevalent than we think and that underreporting by health and HIT professionals may be occurring. There may also be a number of reasons for the under-reporting of technology-induced errors by this group, for example, a belief among some health and HIT professionals that all HIT is fully tested and safe, a lack of knowledge of what constitutes a technology-induced error [12], and a lack of knowledge about what to report, when to report it and where to report the occurrence of such events. Such a lack of knowledge, where technology-induced errors are concerned suggests, the need to develop formalized educational programs aimed at these professions in order to generate awareness of this new types of errors and to educate health and HIT professionals about where to report such errors so that their rates of occurrence can be effectively monitored and critical errors (i.e., those involving harm and death) can be addressed. An onus needs to be placed on regional, provincial, and national governments and organizations responsible for technology safety, and to provide consistent information that is easily understood by health professionals. These organizations should also provide a single site to report such errors so that information about all technology-induced errors is effectively obtained (e.g., as confusion may arise if health professionals are asked to report on incidents to their local healthcare, provincial/statewide, and a national organization) at the same time.

In summary underreporting of errors is an issue and will continue to be a concern until the health and HIT professional community are sufficiently aware of this problem and its potential sequelae for patients (i.e., disability and death). Early research using data from incident monitoring systems suggests there is a need to collect data about technology-induced errors [22] as research has confirmed technology-induced errors can lead to patient harm, death, and disability [14,15]. In addition to this, health and HIT professionals (when aware of technology-induced errors) will need to make an effort to report them. Samaranayake's work [35] suggests, we are only beginning to fully document technology-induced errors. As we implement electronic records and other types of healthcare software, there will be new opportunities and new types of technology-induced errors that will emerge. More investment will be needed to educate health and HIT professionals so there is full disclosure and understanding to technology-induced errors in terms of capturing information about the context, conditions and origins of the technology-induced errors. Such research involving monitoring systems will improve the quality and safety of our HIT in the long term.

4. Preventing and Learning More about Technology-Induced Errors: Towards Safer HIT

With the understanding that technology-induced errors can arise from many sources and the belief that they are underreported researchers in several countries have attempted to identify methods that can be used to better understand where technology-induced errors come from. A better understanding of technology-induced errors will lead to improvements in HIT software design, development, implementation, monitoring and maintenance. According to Borycki and Keay [34], these methods have been used by researchers to identify and diagnose technology-induced errors prior to and following HIT implementation as well as after a technology-induced error has occurred. Methods can be proactive (i.e., the methodology can be used to identify potential causes of technology-induced error prior to their occurring) or they can be reactive (i.e., occurring after an error has taken place in an effort to prevent future errors and stimulate organizational learning). Some methods that can be used proactively and reactively to study HIT safety are given in Table 1, while Table 2 provides descriptions of these methods.

Table 1

Methods used to proactively and reactively respond to technology-induced error

Table 2

Methods used for dealing with technology-induced error

III. Methods to Reduce Technology-Induced Errors

1. Proactive Methods

Proactive methods can be used before a HIT is implemented in order to predict and prevent errors before they occur in real settings. These methods include: heuristic evaluation, cognitive walkthrough, usability testing, and clinical simulations (Table 2). The benefits associated with using these methods are significant. Technology-induced errors can be prevented before they have a chance to occur if they can be predicted and rectified before widespread system release. The costs associated with addressing these errors are significantly lower as changes can be made to the HIT before implementation (to be discussed in further detail in the section entitled Use of Methods as a Cost Reduction Strategy). The methods also lead to the improvement of the overall quality and safety of software over time-a critical and important consideration for vendors and regional health authorities implementing these technologies.

2. Reactive Methods

Reactive methods can be applied after a system has been implemented when there is a need to determine the cause(s) of technology-induced errors. For example, ethnographic approaches (i.e., traditional and rapid ethnographic assessment) have been used to successfully identify sources of reported technology-induced error. Traditional and rapid ethnographic assessment (REA) employ interviews, focus groups and observation to identify potential sources of technology-induced errors shortly after HIT implementation. REA differs from traditional ethnography in that REA employs techniques that increase the speed of data collection (i.e., through the use of teams who include insiders who are familiar with the culture of the organization or group being studied). As well, REA takes a more problem-based, focused approach to answering questions. Researchers have demonstrated these techniques, when conduced with health professionals can be used to effectively catalogue health and HIT professionals' experiences with HIT, specifically their concerns about potential sources of technology-induced error.

Once a technology-induced error has occurred there are a number of techniques and methods that can be applied to determine the factors that contributed to an error. They include methods inspired by case study literature such as the use of computer log data, expert reviews of the HIT (including paper processes) involved in the error as well as interviews with those health and HIT professionals who were involved in the error. Combinations of these methods have been effectively used to determine the processes, HIT and other organizational factors that can contribute to a technology-induced error [47]. Future research may involve the application of proactive approaches after a technology-induced error to better understand the individual patient, health professional, HIT, organizational and healthcare systems interactions that led to errors.

Approaches such as clinical simulation and usability testing, which can be considered to be proactive methods, may also be applied after an error has occurred. Methods such as heuristic evaluation, cognitive walkthrough, RAE, and ethnography provide insights into the safe and unsafe aspects of a HIT that may have led to a technology-induced error. These insights can be followed by usability testing and clinical simulations. Here, investigators can use these techniques to determine if the error can be reproduced (with similar individuals using the HIT in similar circumstances and a similar setting). Such work has provided investigators with insights into how a technology-induced error transpires as the context and events that led to the technology-induced error are mimicked. As a result, there is the opportunity to gain fuller and more robust insights into the dynamics of the technology-induced error [12].

Each of these methods support healthcare organizations (e.g., vendors, regional health authorities) that are involved in developing, testing, purchasing or customizing the HIT to determine its safety prior to and after implementation. These methods are oriented towards identifying potential technology-induced errors prior to implementation and learning from technology-induced errors. The methods allow vendors and healthcare organizations (e.g., physician offices, regional health authorities) to "prune off" potential technology-induced errors before they occur and to prevent any future occurrence of such errors. Such work is essential as HIT need to be moved along the HIT quality continuum from unsafe systems to safe systems. Ongoing work in this area will ensure unsafe HIT features and functions are identified and their safety is improved [25,34].

3. Use of Methods as a Cost Reduction Strategy

More recently, these methods have been applied to understanding the cost implications of technology-induced errors. If a technology-induced error occurs, there is a need to correct it. Costs associated with addressing a HIT software issue are significant after a technology has been implemented. For vendors the costs arise from re-work by software developers and programmers. For vendors and organizations that have implemented HIT-costs are associated with software fixes, customization fixes, costs of re-implementation, and costs associated with training staff to use the software after the technological fixes have been made [13,23,34]. According to the software engineering literature, costs are logarithmic in nature (i.e., costs increase 10 times as time increases); therefore, the earlier the software issue is identified in the software development lifecycle, the lower the cost of fixing it [48].

According to Martin and McClure [49] the relative costs of fixing a software issue (such as a technology-induced error) are 3% in the requirements phase, 3% in the specification phase, 5% in the design phase, 7% in the coding phase, 15% in testing, and 67% in the operations and maintenance phase. These costs are substantively lower than if the software issue was detected prior to systems implementation and addressed before the HIT has been released for wide spread use [23,48]. In an extension of this work Baylis et al. [46], linked technology-induced errors to healthcare system costs (i.e., what would be the cost of the technology-induced error to the healthcare system if it lead to harm). Here, Baylis et al. [46] determined that the costs of identifying and correcting a technology-induced error prior to HIT release would lead to healthcare systems savings (eliminating the cost of providing care to patients who had been harmed). In this workproactive testing on a system was conducted prior to its widespread release. The researchers identified several potential technology-induced errors during HIT testing. They determined the costs of addressing the technology-induced error prior to the release of the HIT to the healthcare system costs associated with treating patients who were harmed by the technology-induced errors. Their findings revealed that the costs of correcting the technology-induced error prior to systems release were substantively lower than the cost of treating a patient who had been harmed [46].

IV. Conclusion

The safety of HIT has become an important issue. A variety of technology-induced errors have been reported in the literature and in national monitoring systems. In response to increasing reports of the occurrence of technology-induced errors in healthcare, the Institute of Medicine has issued a report recommending greater attention and action be paid to preventing, detecting and reporting such errors. In this paper the author has defined technology-induced error and has discussed efforts underway to prevent, detect, and report on the occurrence of such error. The author has discussed the economic reasons for eliminating technology-induced errors early in the software development lifecycle to reduce costs associated with addressing these types of errors from a software and human cost perspective. As discussed in the paper there are a number of methods that can be applied to reduce technology-induced errors in HIT. These methods can be proactive and reactive. Furthermore work in development and refinement of both types of methods is needed. It is expected that the focus on technology-induced errors will become increasingly more important as HIT becomes more widely used and the public is increasingly affected by these types of errors.

Notes

No potential conflict of interest relevant to this article was reported.

References

1. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in australian health care study. Med J Aust 1995;163(9):458–471. 7476634.
2. Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004;170(11):1678–1686. 15159366.
3. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(6):370–376. 1987460.
4. Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322(7285):517–519. 11230064.
5. Kohn LT, Corrigan JM, Donaldson MS. Institute of Medicine. To err is human: building a safer health system Washington (DC): National Academy Press; 2000.
6. Committee on Data Standards for Patient Safety. Institute of Medicine. Patient safety: achieving a new standard for care Washington (DC): National Academies Press; 2004.
7. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163(12):1409–1416. 12824090.
8. Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ 2000;320(7237):788–791. 10720369.
9. Menke JA, Broner CW, Campbell DY, McKissick MY, Edwards-Beckett JA. Computerized clinical documentation system in the pediatric intensive care unit. BMC Med Inform Decis Mak 2001;1:3. 11604105.
10. Borycki EM, Kushniruk AW, Keay L, Kuo A. A framework for diagnosing and identifying where technology-induced errors come from. Stud Health Technol Inform 2009;148:181–187. 19745249.
11. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293(10):1197–1203. 15755942.
12. Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform 2005;74(7-8):519–526. 16043081.
13. Borycki E, Kushniruk A. Identifying and preventing technology-induced error using simulations: application of usability engineering techniques. Healthc Q 2005;8 Spec No:99–105. 16334081.
14. Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc 2010;17(6):663–670. 20962128.
15. Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc 2012;19(1):45–53. 21903979.
16. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14(4):415–423. 17460127.
17. Borycki EM, Kushniruk AW. Towards an integrative cognitive-socio-technical approach in health informatics: analyzing technology-induced error involving health information systems to improve patient safety. Open Med Inform J 2010;4:181–187. 21594010.
18. Weiner JP, Kfuri T, Chan K, Fowles JB. "e-Iatrogenesis": the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc 2007;14(3):387–388. 17329719.
19. Kuziemsky CE, Borycki E, Nohr C, Cummings E. The nature of unintended benefits in health information systems. Stud Health Technol Inform 2012;180:896–900. 22874322.
20. Institute of Medicine. Health IT and patient safety: building safer systems for better care Washington (DC): National Academies Press; 2012.
21. Borycki EM, Kushniruk AW, Bellwood P, Brender J. Technology-induced errors. The current use of frameworks and models from the biomedical and life sciences literatures. Methods Inf Med 2012;51(2):95–103. 22101488.
22. Aarts J. Towards Safe Electronic Health Records: a socio-technical perspective and the need for incident reporting. Health Policy Tech 2012;1(1):8–15.
23. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc 2005;12(4):377–382. 15802485.
24. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006;13(5):547–556. 16799128.
25. Borycki EM, Househ MS, Kushniruk AW, Nohr C, Takeda H. Empowering patients: making health information and systems safer for patients and the public. Contribution of the IMIA health informatics for patient safety working group. Yearb Med Inform 2012;7(1):56–64. 22890342.
26. Kushniruk A, Beuscart-Zephir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthc Q 2010;13 Spec No:53–58. 20959731.
27. Singh H, Ash JS, Sittig DF. Safety Assurance Factors for Electronic Health Record Resilience (SAFER): study protocol. BMC Med Inform Decis Mak 2013;13:46. 23587208.
28. Kuwata S, Kushniruk A, Borycki E, Watanabe H. Using simulation methods to analyze and predict changes in workflow and potential problems in the use of a barcoding medication order entry system. AMIA Annu Symp Proc 2006;2006:994. 17238613.
29. Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008;15(4):408–423. 18436903.
30. Elkin PL. Human factors engineering in HI: so what? who cares? and what's in it for you? Healthc Inform Res 2012;18(4):237–241. 23346473.
31. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506–1512. 16322178.
32. Del Beccaro MA, Jeffries HE, Eisenberg MA, Harry ED. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 2006;118(1):290–295. 16818577.
33. Ammenwerth E, Talmon J, Ash JS, Bates DW, Beuscart-Zephir MC, Duhamel A, et al. Impact of CPOE on mortality rates-contradictory findings, important messages. Methods Inf Med 2006;45(6):586–593. 17149499.
34. Borycki E, Keay E. Methods to assess the safety of health information systems. Healthc Q 2010;13 Spec No:47–52. 20959730.
35. Samaranayake NR, Cheung ST, Chui WC, Cheung BM. Technology-related medication errors in a tertiary hospital: a 5-year analysis of reported medication incidents. Int J Med Inform 2012;81(12):828–833. 23022632.
36. Kaner C, Falk JL, Nguyen HQ. Testing computer software 2nd edth ed. New York (NY): Wiley; 1999.
37. Carvalho CJ, Borycki EM, Kushniruk AW. Using heuristic evaluations to assess the safety of health information systems. Stud Health Technol Inform 2009;143:297–301. 19380951.
38. Usability Body of Knowledge. Cognitive walkthrough [Internet] Bloomingdale (IL): User Experience Professionals Association; 2010. cited at 2013 May 25. Available from: http://www.usabilitybok.org/cognitivewalkthrough.
39. Nielsen J. Heuristic evaluation [Internet] Fremont (CA): Nielsen Norman Group; 2013. cited at 2013 May 25. Available from http://www.nngroup.com/topic/heuristic-evaluation/.
40. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004;37(1):56–76. 15016386.
41. Borycki EM, Kushniruk AW, Kuwata S, Kannry J. Engineering the Electronic Health Record for safety: a multilevel video-based approach to diagnosing and preventing technology-induced error arising from usability problems. Stud Health Technol Inform 2011;166:197–205. 21685625.
42. Borycki EM, Kushniruk AW, Kuwata S, Kannry J. Use of simulation in the study of clinician workflow. AMIA Annu Symp Proc 2006;2006:61–65. 17238303.
43. Borycki E, Kushniruk A, Carvalho C. A methodology for validating safety heuristics using clinical simulations: identifying and preventing possible technology-induced errors related to using health information systems. Comput Math Methods Med 2013;2013:526419. 23606902.
44. Borycki EM, Kushniruk A, Keay E, Nicoll J, Anderson J, Anderson M. Toward an integrated simulation approach for predicting and preventing technology-induced errors in healthcare: implications for healthcare decision-makers. Healthc Q 2009;12 Spec No Patient:90–96. 19667784.
45. Anderson JG, Aydin CE. Evaluating the organizational impact of health care information systems New York (NY): Springer; 2005.
46. Baylis TB, Kushniruk AW, Borycki EM. Low-cost rapid usability testing for health information systems: is it worth the effort? Stud Health Technol Inform 2012;180:363–367. 22874213.
47. Ash JS, Sittig DF, McMullen CK, Guappone K, Dykstra R, Carpenter J. A rapid assessment process for clinical informatics interventions. AMIA Annu Symp Proc 2008;2008:26–30. 18999075.
48. Patton R. Sofware testing Indianapolis (IN): SAMS; 2001.
49. Martin J, McClure CL. Software maintenance: the problem and its solutions Englewood Cliffs (NJ): Prentice-Hall; 1983.

Article information Continued

Table 1

Methods used to proactively and reactively respond to technology-induced error

Table 1

Table 2

Methods used for dealing with technology-induced error

Table 2