Handbook of Evaluation Methods for Health Informatics

"Show-me-the-money." This is Tom Cruise's well-known catch phrase from the 1996 film Jerry Maguire. All of us want to see evidence as to whether something is worth investing in or not. I remember one public health informatician who was my boss at the time of my post-doc fellowship telling the fellows: "If you figure out how to evaluate health information technology (HIT) well enough, you'll never have to worry about your career in the future." At the time I had to wonder if that could be true. Even now, I certainly do not know. My boss probably said so because we are living in an outcome-oriented world and we expect that the evaluation of HIT will be able to demonstrate how well HIT works in order to justify the investment of money, time, and resources. Hence, knowledge and skill in the field of HIT evaluation is one competency that a health informatician must have.

Evaluation has been considered a difficult task owing to the complexity of the environment in which information systems are implemented and the busy schedule related to working with patients, especially in a hospital setting [1]. On top of that, we often need to know the baseline data, in order to compare it with the results after the deployment of a new HIT system for evaluation. This requires thoughtful planning ahead; if not, it is often too late to perform evaluation once the IT system is implemented.

HIT evaluation is defined as "the act of measuring or assessing the properties of a health information system (in terms of planning, development, implementation, or operation), the result of which informs a decision to be made concerning that system in a specific context" [2]. As the title, Handbook of Evaluation Methods for Health Informatics, suggests, this is not a textbook but a reference book [3]. Handbooks are usually designed to be easily consulted and to provide quick answers in a certain field [4].

The author claimed the purpose of this book was to illustrate options for finding appropriate tools and to support the user in accomplishing an assessment study [3]. The book consists of three parts. Part I introduces underpinning conceptual bases of evaluation and terminology. Part II is the main body of this reference book, focusing on methods and techniques of evaluation. The author shows more than thirty evaluation methods. The author uses graphical icons to represent the phases of the system lifecycle, the reliability of the methods, the degree of difficulty, and the resource requirements for each method in order to help readers easily identify methods based on their needs. Each method is presented under the common structure, as follows: 1) areas of application; 2) description; 3) assumptions for application; 4) perspectives; 5) frame of reference for interpretation; 6) perils and pitfalls; 7) advice and comments; and 8) reference list. Readers may find a number of these methods unfamiliar; however, the author does a fairly good job providing readers with basic information so that readers can go out to find further information on any particular method. As explained in the note to readers, the author excludes economic and statistical methods, which require specialized understanding. Other than that, this book touches a majority of the evaluation methods we can find in the literature.

Finally, part III consists of a review of the perils and pitfalls for experimental investigations with cases from the relevant literature. The author mentions that the intention of Part III is to give readers the ability to carry out an evaluation with a judgment of a method's validity. Thus, this part might more suitable for readers with some evaluation experience, who can appreciate the author's efforts to pinpoint the threats and shortcomings of the methods.

Six years have passed since book was published, so readers should search in recent publications if they want to know more. This book might not provide sufficient information on healthcare issues to someone expecting a look at very current issues. Thus, accompanied with this book, I also suggest "Evaluation Methods in Biomedical Informatics [1]", a well known textbook in the HIT evaluation.

Overall, Handbook of Evaluation Methods for Health Informatics is interesting and informational enough for readers in both academia and industry, anyone who is eager to learn more in the field of HIT evaluation. I hope that readers will take advantage of this book as a handy reference work in the field of evaluation. It will be a valuable addition to your bookshelf.