Monday, February 1, 2016

60 Years of Informatics: In the Context of Data Science

Like many academic health science universities, my institution has undertaken a planning process around data science. In the process of figuring how to merge our various data-related silos, we tried to look at what other universities were doing. One high-profile effort has been launched at the University of Michigan, and the formation of their program and those of others inspired a statistician, David Donoho, to look at data science from the purview of his field 50 years after famed statistician John Turkey had called for reformulation of the discipline into a science of learning from data. Donoho’s resulting paper [1] motivated me to look at data science from the purview of my field, biomedical and health informatics.

Statistics has of course been around for centuries, although this author drew from an event 50 years ago, a lecture by George Tukey. The informatics field has not been in existence for as many centuries, but one summary of its history by Fourman credits the origin of the term to Philip Dreyfus in 1962 [2]. However, the Wikipedia entry for informatics attributes the term to a German computer scientist Karl Steinbuch in 1956. Fourman also notes that the heaviest use of the term informatics comes from its attachment to various biomedical and health terms [2].

If the informatics field is indeed 60 years old, I have been working in it for about half of its existence, since I started my National Library of Medicine (NLM) medical informatics fellowship in 1987. I have certainly devoted a part of my career to raising awareness of the term informatics, making the case for it as a discipline [3]. Clearly the discipline has become recognized, with many academic departments, mostly in health science universities, and a new physician subspecialty devoted to it [4].

And now comes data science. What are we in informatics to make of this new field? Is it the same as informatics? If not, how does it differ? I have written about this before.

Donoho’s paper does offer some interesting insights [1]. I get a kick out of one tongue-in-cheek definition he gives of a data scientist, whom he defines as a “person who is better at statistics than any software engineer and better at software engineering than any statistician.” Perhaps we could substitute informatician for software engineer, i.e., a data scientist is someone who is better at statistics than any informatician and is better at informatics than any statistician?

Donoho does later provide a more serious definition of data science, which is that it is “the science of learning from data; it studies the methods involved in the analysis and processing of data and proposes technology to improve methods in an evidence-based manner.” He goes on to further note, “the scope and impact of this science will expand enormously in coming decades as scientific data and data about science itself become ubiquitously available.”

Donoho goes on to note six key aspects (he calls them “divisions” of “greater data science”) that I believe further serve to define the work of the field:
  • Data Exploration and Preparation
  • Data Representation and Transformation
  • Computing with Data
  • Data Modeling
  • Data Visualization and Presentation
  • Science about Data Science
Clearly data is important to informatics. But is it everything? We can being to answer this question by thinking about the activities of informatics where data, at least not “Big Data,” is not central. While I suppose it could be argued that all applications of informatics make use of some amount of data, there are aspects of those applications where data is not the central element. Consider the many complaints that have emerged around the adoption of electronic health records, such as poor usability, impeding of workflow, and even concerns around patient safety [5]. Academic health science leaders can lead the charge in use of data but must do so in the context of a framework that protects the rights of patients, clinicians, and others [6].

Like many informaticians, I do remain enthusiastic for the prospect of the growing quantity of data to advance our understanding of human health and disease, and how to treat the latter better. But I also have some caveats. I have concerns that some data scientists read too much into correlations and associations, especially in the face of so much medical data capture being imprecise, our lack of adoption of standards, and its inaccessibility when not structured well (which can lead us to try to “unscramble eggs”).

It is clear that informatics cannot ignore data science, but our field must also be among the leaders in determining its proper place and usage, especially in health-related areas. We must recognize the overlap as well as appreciate the areas where informatics can be synergistic with data science.


1. Donoho, D (2015). 50 years of Data Science. Princeton NJ, Tukey Centennial Workshop.
2. Fourman, M (2002). Informatics. In International Encyclopedia of Information and Library Science, 2nd Edition. J. Feather and P. Sturges. London, England, Routledge: 237-244.
3. Hersh, W (2009). A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making. 9: 24.
4. Detmer, DE and Shortliffe, EH (2014). Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association. 311: 2067-2068.
5. Rosenbaum, L (2015). Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 373: 1585-1588.
6. Koster, J, Stewart, E, et al. (2016). Health care transformation: a strategy rooted in data and analytics. Academic Medicine. Epub ahead of print.

Monday, January 25, 2016

Biomedical Data Science Needs Measures of Information Density and Value

I wrote recently that one of my concerns for data science is the Big Data over-emphasis on one of its four Vs, namely volume. Since then, I was emailing with Dr. Shaun Grannis and other colleagues from the Indiana Health Information Exchange (IHIE). I asked them about size of their data for near 6 billion clinical observations from the 17 million patients in their system. I was somewhat surprised to hear that the structured data only takes up 26 terabytes. I joked that I almost have that much disk storage lying around my office and home. That is a huge amount of data, but some in data science seem to imply that data sizes that do not seem to start with at least “peta-” are somehow not real data science.

Of course, imaging and other binary data add much more to the size of the IHIE data, as will the intermediate products of various processing that are carried out when doing analysis. But it is clear that the information “density” or “value” contained in that 26 terabytes is probably much higher than a comparable amount of binary (e.g., imaging, genome, etc.) data. This leads me to wonder whether we should be thinking about how we might measure the density or value of different types of biomedical and health information, especially if we are talking about the Vs of Big Data.

The measurement of information is decades old. Its origin is attributed to Shannon and Weaver from a seminal publication in 1949 [1]. They defined information as the number of forms a message could take. As such, a coin flip has 2 bits of information (heads or tails), a single die has 6 bits, and a letter in the English language has 26 bits. This measure is of course simplistic in that it assumes the value of each form in the message is equal. For this reason, others such as Bar Hillel and Carnap began adding semantics (meaning) that, among other things, allowed differing values for each form [2].

We can certainly think of plenty of biomedical examples where the number of different forms that data can take yields widely divergent value of the information. For example, the human genome contains 3 billion nucleotide pairs, each of which can take 4 forms. Uncompressed, and not accounting for the fact a large proportion is identical across all humans [3], this genome by Shannon and Weaver’s measure would have 12 billion bits of information. The real picture of human genomic variation is more complex (such as through copy number variations), and the point is that there is less information density in the huge amount of data in a genome than in, say, a short clinical fact, such as a physical exam finding or a diagnosis.

By the same token, images also have different information density than clinical facts. This is especially so as the resolution of digital images continues to increase. There is certainly value in higher-resolution images, but there are also diminishing returns in terms of the information value. Doubling or quintupling or any other increase of pixels or their depth will create more information as measured by Shannon and Weaver’s formula but not necessarily provide more value of that information.

Even clinical data may have diminishing returns based on its size. Some interesting work from OHSU faculty Nicole Weiskopf and colleagues demonstrates an obvious finding but one that has numerous implications for secondary use of clinical data, which is that sicker patients have more data in the electronic health record (EHR) [4-5]. The importance of this is that sicker patients may be “oversampled” in clinical data sets and thus skew secondary analysis by over-representing patients who have received more healthcare.

There are a number of implications for increasing volumes of data that we must take into consideration, especially when using such data for purposes for which it was not collected. This is probably true for any Big Data endeavor, where the data may be biased by the frequency and depth of its measuring. The EHR in particular is not a continuous sampling of a patient’s course, but rather represents periods of sampling that course. With the EHR there is also the challenge that different individual clinicians collect and enter data differently.

Another implication of data volumes is its impact on statistical significance testing. This is one form of what many criticize in science as “p-hacking,” where researchers modify the presentation of their data in order to achieve a certain value for the p statistic that measures the likelihood that differences are not due to chance [6]. Most researchers are well aware that their samples must be of sufficient size in order to achieve the statistical power to attain a significant difference. However, on the flip side, it is very easy to obtain a p value that shows small, perhaps meaningless, differences are statistically significant when one has very large quantities of data.

The bottom line is that as we think about using data science, certainly in biomedicine and health, and the development of information systems to store and analyze it, we must consider the value of information. Just because data is big does not mean it is more important than when data is small. Data science needs to focus on all types and sizes of data.


1. Shannon, CE and Weaver, W (1949). The Mathematical Theory of Communication. Urbana, IL, University of Illinois Press.
2/ Bar-Hillel, Y and Carnap, R (1953). Semantic information. British Journal for the Philosophy of Science. 4: 147-157.
3. Abecasis, GR, Auton, A, et al. (2012). An integrated map of genetic variation from 1,092 human genomes. Nature. 491: 56-65.
4. Weiskopf, NG, Rusanov, A, et al. (2013). Sick patients have more data: the non-random completeness of electronic health records. AMIA Annual Symposium Proceedings 2013, Washington, DC. 1472-1477.
5. Rusanov, A, Weiskopf, NG, et al. (2014). Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research. BMC Medical Informatics & Decision Making. 14: 51.
6. Head, ML, Holman, L, et al. (2015). The extent and consequences of p-hacking in science. PLoS Biology. 13: e1002106.

Monday, January 18, 2016

Meaningful Use Ending? Yes and No

The health information technology (HIT) world was shaken to its core last week by Andy Slavitt, Acting Administrator of the Centers for Medicare & Medicaid Services (CMS), who announced that the CMS Meaningful Use (MU) program was over. More precisely, he stated, “The Meaningful Use program as it has existed, will now be effectively over and replaced with something better.” He tweeted this as well, and it was retweeted by Dr. Karen DeSalvo, Director of the Office of the National Coordinator for Health IT (ONC). A transcript of his comments were posted in The CMS Blog. The health IT media was quick to pick up on his remarks.

Does this mean that eligible professionals and hospitals will no longer need to meet MU criteria to get CMS incentive dollars or avoid penalties. Sort of. Months ago, in one of the few bipartisan moves of the current Congress, passed the Medicare Access and CHIP Reauthorization Act (MACRA) legislation [1]. This legislation is best known as the “doc fix” because it fixed the problem of the old sustainable growth rate (SGR) formula of Medicare that threatened to substantially decrease physician reimbursement under Medicare.

Another part of the MACRA legislation addressed some criticisms of the various Medicare incentive programs, such as their multiple number, i.e., not only MU, but also the Physician Quality Reporting System (PQRS) and the Value-Based Payment Modifier (VM). These will all be rolled into a single Merit-based Incentive Payments (MIPS) program, which will start to assess penalties in a graduated way, from 4% in 2019 up to 9% in 2022. After 2019, CMS will also provide another pathway to incentives via Alternative Payment Models (APMs), such as through accountable care organizations (ACOs).

The MIPS program will consist of four categories of measures (divided among percentages): quality (30%), resource use (30%) , clinical practice improvement activities (15%), and MU of certified EHR technology (25%). The details of these are under development by CMS, but it is clear that within MIPS, MU will be part of what eligible hospitals and eligible professionals will need to achieve to qualify for Medicare incentives and avoid penalties. As of right now, that includes MU, including Stage 3 [2]. What any new approach will look like going forward is not known. Stay tuned!

Some suggestions for improving MU going forward from John Halamka speaking on behalf of about 30 healthcare delivery organizations and Peter Basch and Thomson Kuhn speaking on their own but as leaders from the American College of Physicians.

Both Basch and Kuhn as well as Halamka deem MU Stage 1 a success in terms of achieving widespread adoption, but note it is time to move beyond the functional-use measures of MU. They call for Stage 3 as it is currently planned to be abandoned and also note how the highly persecutive approach stifles innovation by clinicians and boxes in the work of EHE developers. Basch and Kuhn go farther in terms of making recommendations. They call for a reconfiguration of MU within MIPS, with the elimination of functional-use measure thresholds (e.g., 80% of patients with problem lists or use of five clinical decision support rules), judicious use of non-threshold functional-use measures, practical interoperability that allows the delivery of high-quality care, more flexible engagement with patients, more innovative approaches to participating in and measuring quality initiatives. A final call they make is for continuing medical education within the domains of health IT so that physicians (and others) can learn how to deliver the best care using IT.

When writing about the present situation of the HITECH Act, I often harken back to what I wrote when it was first unveiled: "This is a defining moment for the informatics field. Never before has such money and attention been lavished on it. HITECH provides a clear challenge for the field to 'get it right.' It will be interesting to look back on this time in the years ahead and see what worked and did not work. Whatever does happen, it is clear that informatics lives in a HITECH world now." It is now time to move on from HITECH and MU to a more sustaining health IT that meets the needs of the healthcare system going forward.


1. Doherty, RB (2015). Goodbye, sustainable growth rate—hello, merit-based incentive payment system. Annals of Internal Medicine. 163: 138-139.
2. O'Neill, T (2015). Primer: EHR Stage 3 Meaningful Use Requirements. Washington, DC, American Action Forum.

Wednesday, January 13, 2016

A Tale of Childbirth in Two Countries and Some Teachable Moments

As the father of two healthy adult children who were not born in hospitals, as well as being married to a certified nurse midwife (CNM), I take great interest in childbirth, especially its evidence-based aspects. Both of my children were born in out-of-hospital birth centers, where I was the only physician present, and was in no way there in a physician capacity. Both were delivered by CNMs. I always marvel at how pleasant the experience was relative to the eight deliveries I performed as a medical student in an academic hospital in the 1980s.

Two recent studies provide a number of teachable moments concerning evidence-based care. Both studies asked roughly the same question, looking at the risk of perinatal mortality for planned in-hospital vs. out-of-hospital births. One study from Ontario, Canada found no differences in neonatal mortality for planned out-of-hospital births by midwives compared to planned in-hospital births [1]. But another study from Oregon found that there was a higher risk of neonatal mortality in planned out-of-hospital births (3.9 deaths compared to 1.8 deaths per 1000 deliveries) [2]. This study also found a much higher rate of cesarean section (C-section) for in-hospital births (28.1% versus 6.2%). Caesarian sections are associated with a variety of short and longer term patient complications.

What are the teachable moments? Certainly one is that out-of-hospital birth is very safe, especially in a healthcare system when care among midwives and physicians is highly coordinated, as occurs in Canada and most other developed countries. That is not the case in the United States, and it is likely that some complications in Oregon were a result of that lack of coordination. But even in the United States, at least in Oregon, out-of-hospital birth is relatively safe.

Another teachable moment concerns relative versus absolute risk. When stated as a relative risk ratio, the difference in mortality in Oregon was 2.2-fold higher. But neonatal mortality is an extremely rare event. As such, the absolute risk difference between in-hospital and out-of-hospital birth was only 2.1 deaths per 1000 deliveries. This was accompanied by a relative risk ratio for C-section that was 4.5-fold higher and which also had a substantially higher absolute risk difference of 21.9% of all deliveries.

An additional teachable moment concerns some important issues around data. This study was made possible because several years ago, the state of Oregon added a question to birth certificates that asked all women who had an in-hospital delivery, “Did you go into labor planning to deliver at home or at a freestanding birthing center?” This enabled the researchers to determine planned vs. unplanned out-of-hospital births and thus made this study possible. Data does not magically just appear; we have to determine what we want to collect, make plans to collect it, and determine its completeness and validity.

These studies also raise the question of whether the difference in neonatal mortality in Oregon could be ameliorated by better care coordination. The Oregon study also raises the question of whether moving deliveries from hospitals to other settings would reduce the C-section rate. Hopefully these and other questions will be answerable in the future. I certainly hope it may lead to more families being able to experience the pleasant deliveries I was able to have with my children.


1. Hutton, EK, Cappelletti, A, et al. (2015). Outcomes associated with planned place of birth among women with low-risk pregnancies. Canadian Medical Association Journal. Epub ahead of print.
2. Snowden, JM, Tilden, EL, et al. (2015). Planned out-of-hospital birth and birth outcomes. New England Journal of Medicine. 373: 2642-2653.

Saturday, January 9, 2016

My Changing Relationship With Photography

I imagine that like many people, my relationship with photography has changed over the years. In my current life, I do like to capture many aspects of my life with pictures. But my approach to photography has changed over the years.

I was not always a big-time photographer. While I certainly have plenty of pictures of my children as well as major events in my life, I did very little to capture much more beyond that in my earlier adulthood.

That all changed with the advent of digital photography. Even with the early digital cameras that took pictures that were (compared to today) of poorer quality and had extremely limited memory (so you could not store many pictures on the camera), I took to electronic photos. No doubt the convenience and instant gratification of seeing the results right away and no longer having to send film for developing played a huge role.

Despite my proclivity for digital photography, I have never invested in any high-end cameras. I have definitely preferred the convenience of point-and-shoot cameras that were quick to turn on, easy to take pictures, and simple to transfer them to a computer.

Along the way I also for the most part stopped printing pictures. While I do occasionally have a reason to print a photo, for the most part I view my pictures on my computer or mobile devices.

In 2014 I made the plunge into a somewhat higher-end camera, a Sony Alpha 6000. These “mirrorless” cameras supposedly approach digital single-lens reflex (DSLR) cameras in quality, with a more compact size (though not able to fit in a pocket). This particular camera also gave me, for the first time ever, a changeable (zoom) lens. It is definitely a nice camera and takes great pictures, especially when a zoom lens is preferable or lighting conditions are suboptimal.

About the same time, however, I had upgraded my smartphone to an iPhone 6. The newer cameras on these modern smartphones also take excellent pictures, especially in decent light and when not requiring any sort of zoom. Of course a major convenience is that they can be carried in a pocket (and also serve as phone, Internet access device, music player, and more). Another critical benefit of smartphone cameras is the convenience of posting photos on Facebook, which is something I do frequently. I can also carry it when I go running if there is a need for having a camera.

I now take the overwhelming majority of my pictures with my smartphone. The quality of most of them is exemplary, and when married with the convenience of being part of a device that easily fits in my pocket, my Sony camera is reserved mostly for special occasions. Those occasions are still important, so I will likely never part with a camera better than that on my smartphone, although who knows what technology for the latter may develop in the future to change even that.

Thursday, December 31, 2015

Annual Reflections at the End of 2015

As regular readers of this blog know, I traditionally end each year with a posting reflecting back on the past year. While this year has been another great success for myself and our informatics program at Oregon Health & Science University (OHSU), it has been somewhat of a transitional year for the informatics field. Many of the new and exciting initiatives in the informatics field from recent years are no longer novel, with some now settling into “midlife” and others being called out for retirement.

One program settling into midlife, although being called out for retirement by many, has been the Health Information Technology for Economic and Clinical Health (HITECH) Act. The launching of this blog, and indeed the catapult to much larger visibility of the informatics field, owes a great deal to HITECH. There is no question that HITECH has succeeded on some levels, at least in terms of increasing electronic health record (EHR) adoption, as I have noted before. A recent report from the Commonwealth Fund confirms what statistics from the Office of the National Coordinator for Health IT (ONC) show: the US is no longer a world laggard in health IT and is in some ways a global leader [1].

But there is no question that not all with HITECH has gone well. Despite the widespread adoption of EHRs, they are still very imperfect [2]. At best, they impede clinician workflow and at worst, they cause some of the safety problems they have been touted to rectify. And one vision has clearly not been achieved, which is interoperable data across systems, even those from the same vendor [3]. Going forward, the informatics field must provide leadership to guide the best use of EHRs and related systems, which is spelled out excellently in the AMIA EHR-2020 Task Force white paper [4].

Another interesting happening, perhaps related to health IT achieving midlife, is that the quantity of health IT blogging seems to be tapering off. In this blog for example, I had fewer posts this year than any since the first year I started the blog. The same is true for a number of other well-known health IT bloggers, such as Keith Boone and John Halamka. I do not view this as necessarily a bad thing, but perhaps just an indicator that some of the formerly novel aspects of informatics are reaching maturity, and there is less to say on a day-to-day basis.

Also a continuing happening this year was the continued growth of data science, and confusion as to its relationship to informatics. Informaticians are not the only ones expressing confusion where they belong in this new field; statisticians are feeling the same [5]. Nonetheless, there is no question that data and learning from it will drive many scientific fields going forward.

I would also like to call out some other year-end posts from some other bloggers, namely John Halamka, for recapping 2015 overall plus adding some focus on security and looking ahead to 2016, and the folks at HISTalk, who have a comprehensive list of 2015 top stories and 2016 predictions.

On a personal and program level, this year had a number of achievements. I was honored to be bestowed the HIMSS Physician IT Leadership Award. I was also awarded a new grant to update the ONC Health IT Curriculum. On a program level, the OHSU Department of Medical Informatics & Clinical Epidemiology (DMICE) launched its new clinical informatics fellowship and continued its mutli-faceted success in its major missions of research and education.

Looking ahead to 2016, there are plenty of new projects and other activities to keep myself and our department busy. It will be interesting to see how HITECH fares and how the critical need for data interoperability evolves. And of course, new opportunities will emerge for myself and DMICE, many of which cannot even be foreseen now.


1. Osborn, R, Moulds, D, et al. (2015). Primary care physicians in ten countries report challenges caring for patients with complex health needs. Health Affairs. 34: 2104-2112.
2. Rosenbaum, L (2015). Transitional chaos or enduring harm? The EHR and the disruption of medicine. New England Journal of Medicine. 373: 1585-1588.
3. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services.
4. Payne, TH, Corley, S, et al. (2015). Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. Journal of the American Medical Informatics Association. 22: 1102-1110.
5. Donoho, D (2015). 50 years of Data Science. Princeton NJ, Tukey Centennial Workshop.

Wednesday, December 30, 2015

Volume is Only One of the Four "V"s of Big Data, Especially for the Right Data

One widely accepted definition of Big Data is that it entails four “V”s: volume, velocity, variety, and veracity. In other words, Big Data is defined by there being a great deal of it (volume), coming at us rapidly and continuously (velocity), taking many different forms and types (variety), and originating from trustworthy sources (veracity). Among some people, however, there seems to be more focus on one of the Vs above all others, namely volume. I suppose that is not surprising, given that the adjective qualifying the noun head in Big Data is one that describes size.

However, as I and others have written over the years, there are many aspects of data that are just as important as its quantity. Even worse, I have heard many people imply in their statements about data science that you cannot do real data science without massive amounts of data, in turn requiring massive amounts of storage capacity and computer power (also costing much money).

Make no mistake, we do need to consider the volume aspects of data when discussing data science. But we must not lose in the discussion what we hope to accomplish with the data, which one writer refers to as the fifth V of Big Data, namely value [1]. Sometimes value emanates from harnessing the size of a data set, but other times the veracity or variety take on more importance.

I have written about the importance of value as well, noting that meaningless correlations with large amounts of data do not really mean much of anything, and that data scientists must also understand basic research principles, such as causality. So yes, let us prepare for a future where we leverage Big Data to improve health, biomedicine, and other important societal needs, but we also need to remember that we do not always need massive amounts of data, especially that whose veracity we may not know, to derive other value. Perhaps akin to the “rights” of clinical decision support [2], the best data science is more about having access to the right data using the right amount of data at the right time.


1. Marr, B (2015). Why only one of the 5 Vs of big data really matters. IBM Big Data & Analytics Hub.
2. Osheroff, JA, Teich, JM, et al. (2012). Improving Outcomes with Clinical Decision Support: An Implementer's Guide, Second Edition. Chicago, IL, Healthcare Information Management Systems Society.

Monday, December 21, 2015

New NIH Biosketch Allows Better Documentation of Contributions to Science

One of the most important documents for a US-based biomedical researcher is the National Institutes of Health (NIH) Biosketch. This short document summarizes the accomplishments of a scientist apply for an NIH grant, listing his or her job positions, educational history, a summary of key publications, and a listing of current grant funding. The NIH Biosketch is also often used as a summary of one’s larger curriculum vitae (CV).

NIH has tweaked the Biosketch over the years, and the most recent update provides an excellent approach that allows researchers to not just summarize their most prominent publications, but also to give a statement about them in the context of the individual's contributions to science. For each contribution, he or she can provide up to four key publications for each. I enjoyed the exercise of updating my Biosketch to the new form, and thought it would be worthwhile to reproduce the scientific contributions and key publications here.

1. My initial research focused on the development and implementation of information retrieval (IR, also called search) systems in biomedicine and health. I experimented with concept-based approaches to indexing and retrieval of knowledge-based information. Subsequently, I found that methods for evaluation systems were inadequate, and developed an interest in new approaches to evaluation. My interests in search have also evolved with the emergence of new content for retrieval, such as medical images and electronic health record data, especially textual notes in the latter.
  • Hersh WR, Greenes RA, SAPHIRE: an information retrieval system featuring concept matching, automatic indexing, probabilistic retrieval, and hierarchical relationships, Computers and Biomedical Research, 1990, 23: 410-425.
  • Hersh WR, Crabtree MK, Hickam DH, Sacherek L, Friedman CP, Tidmarsh P, Moesbaek C, Kraemer D, Factors associated with success for searching MEDLINE and applying evidence to answer clinical questions, Journal of the American Medical Informatics Association, 2002, 9: 283-293. PMC344588.
  • Hersh W, Kalpathy-Cramer J, Müller H, The ImageCLEFmed medical image retrieval task test collection, Journal of Digital Imaging, 2009, 22: 648-655.
  • Hersh W, Voorhees E, TREC Genomics special issue overview, Information Retrieval, 2009, 12: 1-15.
2. My interest work in IR has converged with another interest in the secondary use of clinical (especially electronic health record) data. I have made contributions not only in attempting to leverage such data, but also addressing caveats and recommendations for its use.
  • Voorhees E, Hersh W, Overview of the TREC 2012 Medical Records Track, The 21st Text Retrieval Conference - TREC 2012.
  • Edinger T, Cohen AM, Bedrick S, Ambert K, Hersh W, Barriers to retrieving patient information from electronic health record data: failure analysis from the TREC Medical Records Track, Proceedings of the AMIA 2012 Annual Symposium, 2012, 180-188, PMC3540501.
  • Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH, Caveats for the use of operational electronic health record data in comparative effectiveness research, Medical Care, 2013, 51(Suppl 3): S30-S37. PMC3748381.
  • Hersh WR, Cimino JJ, Payne PR, Embi PJ, Logan JR, Weiner MG, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Saltz JH, Recommendations for the use of operational electronic health record data in comparative effectiveness research, eGEMs (Generating Evidence & Methods to improve patient outcomes), 2013, 1:14.
3. I have also made contributions in conducting systematic reviews of evaluative research of informatics technologies. These reviews can be challenging because many evaluations use weak evaluation methodologies, in part because these technologies are tools rather than typical medical tests or treatments.
  • Hersh WR, Hickam DH, How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review, Journal of the American Medical Association, 1998, 280: 1347-1352.
  • Hersh WR, Hickam DH, Severance SM, Dana TL, Krages KP, Helfand M, Diagnosis, access, and outcomes: update of a systematic review on telemedicine services, Journal of Telemedicine and Telecare, 2006, 12(Supp 2): 3-31.
  • Stanfill MH, Williams M, Fenton SH, Jenders R, Hersh W, A systematic review of automated clinical coding and classification systems, Journal of the American Medical Informatics Association, 2010, 17: 646-651. PMC3000748.
  • Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS, Outcomes from health information exchange: systematic review and future research needs, JMIR Medical Informatics, 2015, 3(4): e39.
4. Being the leader of a major biomedical informatics educational program, I have also carried out research characterizing the informatics professional workforce. My study on the need for health IT professionals played a role in workforce development being a component of the Health Information Technology for Clinical and Economic Health (HITECH) Act of the American Recovery and Reinvestment Act (ARRA).
  • Hersh W, Who are the informaticians? What we know and should know, Journal of the American Medical Informatics Association, 2006, 13: 166-170. PMC1447543.
  • Hersh W, Wright A, What workforce is needed to implement the health information technology agenda? Analysis from the HIMSS Analytics™ Database, Proceedings of the AMIA 2008 Annual Symposium, 2008, 303-307. PMC2656033.
  • Hersh W, The health information technology workforce: estimations of demands and a framework for requirements, Applied Clinical Informatics, 2010, 1: 197-212. PMC3632279.
  • Hersh WR, Margolis A, Quirós F, Otero P, Building a health informatics workforce in developing countries, Health Affairs, 2010, 29: 274-277.
5. Also as a result of being an educational leader, I have carried out evaluation of educational programs in informatics, including those using distance learning technologies.
  • Hersh W, Williamson J, Educating 10,000 informaticians by 2010: the AMIA 10x10 program, International Journal of Medical Informatics, 2007, 76: 377-382.
  • Hersh WR, A stimulus to define informatics and health information technology, BMC Medical Informatics and Decision Making, 2009, 9: 24.
  • Otero P, Hersh W, Luna D, Quirós F, Translation, implementation and evaluation of a medical informatics distance-learning course for Latin America, Methods of Information in Medicine, 2010, 49: 310-315.
  • Mohan V, Abbott P, Acteson S, Berner ES, Devlin C, Hammond WE, Kukafka R, Hersh W, Design and evaluation of the ONC health information technology curriculum, Journal of the American Medical Informatics Association, 2014, 21: 509-516.
NIH now also allows scientists to create a complete list of published work in the MyBibliography section of the NCBI Web site.

Tuesday, December 15, 2015

The Evidence Base for Health Information Exchange

One of my major projects over the last couple years has been a systematic review of the research that has been conducted on health information exchange (HIE). I wrote about this project when it first started and when our protocol for the review was posted for public comment. The report was funded by the Evidence-Based Practice Centers Program of the Agency for Healthcare Research and Quality (AHRQ). While the review itself has been done for several months, we have been finalizing the report and publications derived from it since then. I am pleased to report that both the complete report [1] plus a paper reporting on the outcomes from studies of HIE [2] have now been published. There will be some additional papers on other aspects of the report as well as a book chapter summarizing the report to be published next year [3].

This report has certainly given me the opportunity to reflect over the last couple years of the state of HIE and the interoperability required to support it. The major finding of the report echoes findings of a similar couple of systematic reviews I led on the topic of telemedicine published in 2001 [4] and 2006 [5], which is that the breadth and quality of the research are limited. There is no question that performing research on HIE is difficult. After all, HIE is not a test or a treatment, but rather a tool that facilitates other aspects of healthcare. Nonetheless, the research base for HIE is limited, and should be improved if we want to discern it benefits and optimal use. The paper provides our recommendations for improving research on HIE outcomes going forward [2].

Our report also gives us an opportunity to think about some of the larger issues around the current role and future directions of HIE. If I had to lament about HIE, I would say that it is an unfortunate requirement at this time for us to need so many different organizations (135 according to the last eHI annual survey of them [6]) devoted to HIE. In the ideal world, there would be no need for HIE organizations, but instead, there would be sufficient interoperability of systems, along with rules and regulations, to allow information to flow seamlessly between appropriate parts of the healthcare system. For example, a physician in his or her office could seamlessly transmit a consultation, receive laboratory results or a discharge summary, or notify a public health department of a reportable event without requiring an HIE entity to facilitate those activities. The information transmitted would be formatted into some standardized form and sent securely to an authenticated site, all facilitated by standard protocols used by the entire industry.

Hopefully the new emphasis of ONC on interoperability [7] and the underlying standards required [8] will facilitate more seamless HIE. While many have argued that the criteria for meaningful use should have placed more emphasis on secure and standardized information exchange rather than specific EHR functionality, such as clinical decision support or specific quality measures, that is all now proverbial water under the bridge. I am certain everyone agrees that we need to focus on seamlessly interoperable health IT going forward. I also hope in the process that robust research is carried out, not only to assess the value of HIE but also determine the best ways to implement it.

An interesting side note to this report is an episode related to another systematic review on HIE that was published in late 2014 [9]. One of our competitors for the contract that was awarded by AHRQ to our institution went out and found another source for funding to carry out a review. Not only did they perform a review that was reduced in scope from our review (it omitted public health and any type of HIE outside the US), but they were also able to bypass all of the processes that AHRQ has to insure the systematic reviews it funds have stakeholder engagement, public comment, and broad peer review. As such, the other group was able to complete their review well in advance of ours and get it published in a very high profile journal, Annals of Internal Medicine. That journal publishes a good number of AHRQ-funded systematic reviews, but understandably did not want to publish ours after they had already published another systematic review on the topic of HIE. While I have no problems with science being competitive in terms of accolades going to the first to publish, I do find it disappointing that another group basically duplicated our review and short-circuited the usual processes of AHRQ.


1. Hersh W, Totten A, Eden K, Devine B, Gorman P, Kassakian S, Woods SS, Daeges M, Pappas M, McDonagh MS. Health Information Exchange. Evidence Report/Technology Assessment No. 220. AHRQ Publication No. 15(16)-E002-EF. Rockville, MD: Agency for Healthcare Research and Quality; 2015.
2. Hersh, WR, Totten, AM, et al. (2015). Outcomes from health information exchange: systematic review and future research needs. JMIR Medical Informatics, 3(4):e39.
3. Hersh, WR, Totten, AM, et al. (2016). The Evidence Base for Health Information Exchange. In Health Information Exchange: Navigating and Managing a Network of Health Information Systems. B. Dixon. Amsterdam, Netherlands, Elsevier, in press.
4. Hersh, WR, Helfand, M, et al. (2001). Clinical outcomes resulting from telemedicine interventions: a systematic review. BMC Medical Informatics and Decision Making. 1: 5.
5. Hersh, WR, Hickam, DH, et al. (2006). Diagnosis, access, and outcomes: update of a systematic review on telemedicine services. Journal of Telemedicine & Telecare. 12(Supp 2): 3-31.
6. Anonymous (2014). 2014 eHI Data Exchange Survey Key Findings. Washington, DC, eHealth Initiative,
7. Anonymous (2015). Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap version 1.0 (Roadmap). Washington, DC, Department of Health and Human Services.
8. Anonymous (2015). Draft 2016 Interoperability Standards Advisory. Washington, DC, Department of Health and Human Services.
9. Rudin, RS, Motala, A, et al. (2014). Usage and effect of health information exchange: a systematic review. Annals of Internal Medicine. 161: 803-811.

Sunday, November 15, 2015

Milestones for the AMIA Annual Symposium

One of my favorite events each year is the American Medical Informatics Association (AMIA) Annual Symposium. It is an excellent meeting that brings together presentation of new research, up-to-date summaries of progress in the field, and great social events.

This year marks the 30th consecutive year of my attending the Annual Symposium (which in its early days was called the Symposium on Computer Applications in Medical Care, or SCAMC). The first time I attended the event was in 1986, when I was in my last year of my internal medicine residency. The following year I started my National Library of Medicine (NLM) Postdoctoral Fellowship, the first support of a long line of NLM funding that has supported my career.

A number of AMIA meetings have been particularly memorable over the years. In 1996, I was elected to the American College of Medical Informatics (ACMI), an honorific society that recognizes expertise in biomedical Informatics. In 2005, the AMIA Symposium featured the first in-person session for the end first offering of the 10x10 (“ten by ten”) course, which would go on to become a major AMIA program to provide introductory education in informatics and also reached a new milestone described below. In 2008, I was awarded the Donald A.B. Lindberg Award for Innovation in Informatics. The meeting in 2012 was another special one, as I served as the Scientific Program Chair.

Another important aspect of the meeting for me is the showcase of scientific research and other work emanating from our biomedical and health informatics program at Oregon Health & Science University (OHSU). This year is no exception, and one thing we are particularly proud of this year is OHSU students comprising two of the four finalists for the AMIA 2015 Student Design Challenge. This year’s Student Design Challenge focus is, The Human Side of Big Data – Facilitating Human-Data Interaction. (Postscript: One of the two OHSU finalist teams took first place, the second consecutive year that an OHSU team won the Student Design Challenge!)

Below is a picture of me at this year’s Annual Symposium, enjoying myself as always, especially with this year’s plethora of ribbons everyone is attaching to their name badge.

Another milestone being achieved at this year’s AMIA Annual Symposium is the surpassing of 2000 who have completed the OHSU offering of the AMIA 10x10 course. While the exact number of people completing the course will not be final until a week or two after the meeting, there will be at least 2034 people who have completed the OHSU 10x10 course since its inaugural offering in 2005. We may not have achieved “ten thousand by 2010,” but I am reminded constantly by those who taken the course that it provided an excellent path into the informatics field. My original course has been the flagship course for the program, making up 74% of those who have completed any 10x10 course. The figure below shows the distribution of those completing the different courses of the 10x10 program.

Also of note is that 312 of those completing the OHSU 10x10 offering have done so via so-called “i10x10” offerings based in other countries, which have included Singapore, Saudi Arabia, and Israel. The OHSU 10x10 course also inspired the offering of the course in Spanish from colleagues at Hospital Italiano de Buenos Aires Argentina, starting initially as a direct translation into Spanish but taking on a more Latin American focus over the years.

Because of the interdisciplinary nature of the informatics, I deal with many different academic fields, from medicine to computer science. But there is no question that my heart and home are in informatics. Each year, the AMIA Annual Symposium reaffirms that this is my discipline, and my closest professional colleagues are also among my best friends.

Friday, October 30, 2015

Use Cases for Data Science at Academic Health Science Centers

Like many academic health science centers, my institution is undergoing a planning process to determine our strategy for data science. I have expressed my concern about the (lack of?) differences between data science and biomedical and health informatics, but the former term seems to be carrying the day. I consider it a personal mission to ensure that the long learned history of biomedical and health informatics is not lost in our rush to embrace this seemingly new data science.

One of my major contributions to our process has been to delineate a set of use cases for data science in academic health science centers. These institutions are distinct from organizations that are predominantly devoted to healthcare delivery, and tend to have small or non-existent research and education missions, and general universities, which may not have healthcare delivery activities integrated with their research and educational missions.

I have broken down my use cases into the three general missions that most academic health science centers have. I only present these at a high level, and there is obviously a much greater depth of detail that could be described for each. But these are the big-picture use cases that in my view drive data science in academic health science centers.

Use cases for the clinical mission of academic health science centers include:
  • Clinical decision support – improve clinical practice via predictive analytics and other uses of patient data, including precision medicine as it works its way into clinical practice
  • Quality measurement and improvement – use data to measure and improve quality of care delivered, especially as healthcare shifts to new value-based models of care
  • Business intelligence – apply data to improve business and financial operations of healthcare delivery
  • Patient engagement – patients upload and interact with data related to their care
  • Public health surveillance – use data for early detection and intervention in public health threats (natural and manmade)
Research is also critically important for academic health science centers, and here are some broad use cases, with many important variations on these themes:
  • Prospective studies – improve data capture and analysis for clinical trials and related studies
  • Retrospective studies – enhance ability to use data already collected
  • Basic science research – studies in the "omics," imaging, and other areas that lead to health-related applications
  • "Third science" research – advancing the science of healthcare delivery, the third science of healthcare (after basic and clinical sciences)
  • Data science and informatics research – advance the theory and practice of data science and biomedical and health informatics
Education is also a vital mission for academic health science centers, not only to train users and managers of data but also professionals and researchers who implement and advance the science:
  • Training for data users and managers, clinicians, and others – allowing those who implement programs applying data science to be more savvy in doing so
  • Education for data science and informatics professionals – master's-level education as well as the new clinical informatics fellowships for physicians
  • Advanced education for data science and informatics researchers – doctoral-level education to advance the science of this work
Data science is indeed unique in academic health science centers. These use cases demonstrate how it spans across all of their missions. The success of initiatives such as ours are likely to depend upon the integration all of three.

Tuesday, October 27, 2015

Meeting My Doppelgänger (Googlegänger)

One of my teachable moments in information retrieval (IR) is about uncommon words tending to be the most discriminating and leading to the best results in searching. I am hardly the first person to come up with this idea, as IR research pioneer Gerald Salton demonstrated its value and published about it in the 1970s [1]. I do, however, provide a modern example of it, which is demonstrated by searching (or Googling) on my name. My last name, in particular, is spelled in a somewhat unusual manner, as most people spell it Hirsh, Hersch, or Hirsch. Combined with my presence on the Web, with many links to my major pages (another teachable moment about the Google PageRank algorithm [2]!), I have never had to pay anyone for search engine optimization (SEO), and Googling “Bill Hersh” or “William Hersh” lists most of my key pages right at the top of the search output.

Early in the days of Google, I discovered another William Hersh, who was also in academia. I also noted him in PubMed (MEDLINE) author searches on our name (hersh w). I knew he was a Chemistry professor at Queens College in New York. Apparently over this nearly two decades, he knew of me as well. We both contemplated reaching out to each other, but neither of us ever did.

About a month ago, I received an email from a colleague of his at Queens College that was sent to my Gmail account by mistake. I replied to the email, telling the sender it was sent to me in error and probably meant for his co-worker. He sent my message to Bill, who reached out to me to apologize for the error. This started a conversation, with each of us describing how long we knew about each other. (He was even once invited to serve on an NIH review panel when they thought they were inviting me!). He also told me that a former student of his from years ago, upon finding me, told him that “that my googleganger is my doppelgänger.” (I have to admit I had to go Google the word doppelgänger to be certain of its meaning.)

We both noted we were academic graybeards, and after some discussion found out that our grandfathers emigrated from different cities in Poland (his Czestochowa and mine Lodz). In addition, like many people with last name Hersh, our grandfathers Anglicized their last names from Hershkowitz. They also both experienced anti-semitism in Eastern Europe, part of their motivation for emigrating to the US.

I also told Bill that I was going to be in New York City in late October, and we set a day and time to meet for lunch. We had that meeting yesterday, and it was enjoyable to trade stories of our somewhat common ancestry, our careers, and our families. My family got a kick out of my telling them that Bill too drives a Toyota Prius. Here is a picture of us together:

It was indeed fun to find Bill, meet him, and reflect on how our meeting was made possible by the Web and IR (my field of research). I do hope to keep in touch with him and meet him again.


1. Salton, G, Yang, CS, et al. (1975). A theory of term importance in automatic text analysis. Journal of the American Society for Information Science. 26: 33-44.
2. Brin, S and Page, L (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems. 30: 107-117.

Friday, October 9, 2015

A Huge Week for Health IT/Informatics

This past week was a busy week in the health IT/informatics world, as the US government released a flurry of rules and documents around health IT. As I tell my students, it is great to be living in this ever-changing part of the history of our field.

Probably making the most news was the release of the rules for Stage 3 of the EHR Incentive (also known as "Meaningful Use") Program by the Centers for Medicare & Medicaid Services (CMS). The Meaningful Use Program has taken its share of lumps in the last year or two, with the challenges providers have had in meeting its Stage 2 criteria and how it has consumed bandwidth that might be put toward other innovation by the healthcare system as well as the vendors. CMS has seem to have gotten the message somewhat, and the new criteria do dial back some on the requirements.

With the new rule, Stage 2 will be modified significantly. Some acute relief will be provided in the form of reduced requirements, from the necessity of reporting only 90 days (as opposed to a full year) of annual reporting to modification of the "view, download, and transmit" (VDT) requirement from five percent of an EP's patient panel to one single patient and reducing the secure messaging requirement from five percent to just being required to have the capability.

Also changed in Stage 2 itself, now called Modified Stage 2, which will be in effect from 2015-2017. The number of objectives is reduced to ten for eligible professionals (EPs) and nine for eligible hospitals (EHs), with each having one or more measures. The objectives are:
  1. Protect Patient Health Information
  2. Clinical Decision Support (CDS)
  3. Computerized Provider Order Entry (CPOE)
  4. Electronic Prescribing
  5. Health Information Exchange
  6. Patient Specific Education
  7. Medication Reconciliation
  8. Patient Electronic Access
  9. Secure Electronic Messaging (EPs only)
  10. Public Health Reporting
Starting in 2018, Stage 3 will become active, with the same objectives as above but with some more rigorous criteria for some of the measures. There is, however, one qualification to Stage 3, which is the opening of a comment period for how it could be changed to align with the new value-based care rules for Medicare. With the addition of calls for Stage 3 to be delayed or outright abandoned, it is not clear what it will ultimately look like.

The full rule is available, as is a brief summary. As always, my preference is for a detailed overview that provides enough detail for the informed reader, somewhere in between the minimally informative short summary and the exhaustive detail of the entire review, which CMS has also provided. (Note to standards developers! I prefer this approach for documentation of standards as well, eschewing both the superficial overviews as well as hundreds-of-pages implementation guides.)

Always a companion to the release of rules for the EHR Incentive Program is the release of the Health Information Technology Certification Criteria by the Office of the National Coordinator for Health IT (ONC). However, as noted by ONC, going forward the EHR Incentive Program will be decoupled from Health IT Certification. The EHR Incentive Program will still required use of certified products, but certification will also be used for other health IT functionality. As with CMS, a short summary and the detailed rule are provided, with the only interim document at this time being a Powerpoint deck that was used in the Webinar ONC presented to describe the new criteria.

The week’s activities did not, however, stop with release of the meaningful use and certification rules. ONC also released a final version of its Federal Health IT Strategic Plan for 2015-2020.
The stated mission of the plan is to "improve the health and well-being of individuals and communities through the use of technology and health information that is accessible when and where it matters most.” This will be achieved through four goals:
  1. Advance Person-centered health and self-management
  2. Transform health care delivery and community health
  3. Foster research, scientific knowledge and innovation
  4. Enhance the nation’s health IT infrastructure
One of the objectives of the fourth goal is to implement the Shared Nationwide Interoperability Roadmap, which was also released by ONC this week in its final Version 1.0 form. The roadmap was accompanied by an updated version of ONC's 2016 Interoperability Standards Advisory, which provides an exhaustive list of the best available standards and links to their implementation specifications. These releases were described in a blog post by ONC Director, Dr. Karen DeSalvo.

As if this week’s activities were not enough, last week was another major milestone, with the switchover to ICD-10-CM by hospitals physician offices, and others who bill in the healthcare system. Eerily similar to Y2K a decade and a half ago, there were very few reports of problems, presumably because the community was well-prepared. Of course, only time will tell, particularly if providers start having claims denied because of faulty coding.

Another recent event pertinent to all of the above occurred the week before, when I presented the inaugural Clinical Informatics Grand Rounds at OHSU. The Grand Rounds series will be part of our normal Thursday Conference Series, and I usually kick off the series each academic year. This year I chose to talk on the topic, HITECH and Meaningful Use: Results from the Grand Experiment and Future Directions. My talk (video and slides available) was built around a proclamation I made in this blog on January 24, 2010, in a posting entitled, Informatics Now Lives in a HITECH World:
"This is a defining moment for the informatics field. Never before has such money and attention been lavished on it. HITECH provides a clear challenge for the field to 'get it right. It will be interesting to look back on this time in the years ahead and see what worked and did not work. Whatever does happen, it is clear that informatics lives in a HITECH world now." Going forward, it will continue to be interesting to pause and reflect.

Thursday, October 1, 2015

Accolades for DMICE

As regular readers of this blog know, I periodically devote postings in this blog to accolades, usually for myself but sometimes for others. I would like to devote this posting to accolades for the many students and faculty in the Oregon Health & Science University (OHSU) Department of Medical Informatics & Clinical Epidemiology (DMICE). More details are provided in the recently published edition of our department newsletter. (Past newsletters are also available.

There are many accolades to point out in the newsletter (starting on page in parentheses):
  • Our new Accreditation Council for Graduate Medical Education (ACGME)-accredited clinical informatics fellowship has launched with its first two fellows (1)
  • Thirty-seven individuals graduated with degrees and certificates in biomedical informatics from OHSU in the 2015 academic year (2)
  • Faculty member Nicole Weiskopf, PhD receiving a Catalyst award from the Oregon Clinical and Translational Research Institute (OCTRI) (3)
  • Numerous faculty and students participating in OHSU Research Week in May, 2015 (4)
  • A surprise party celebrating 25 years at OHSU for myself and fellow faculty Mark Heland, MD (10)
  • New OHSU School of Medicine leadership appointments for Paul Gorman, MD and Heidi Nelson, MD (15)
One final accolade is particularly noteworthy to call out. We were recently informed that OHSU informatics students will be finalists in the 2015 AMIA Student Design Challenge. The theme of this year’s competition is, The Human Side of Big Data – Facilitating Human-Data Interaction. A number of student groups from around the country submitted entries to the competition, and four finalists were recently selected to present at the AMIA Annual Symposium in November. Two of those finalist groups consist of OHSU students:
  • Ashley Choi, Benjamin Cordier, Prerna Das, PhD, and Jason Li, MS will present on, “Take a Breather: Empowering Adherence & Patient Centered Research through Interactive Data Visualization, Social Engagement, & Gamification in Patients with Sleep Apnea.”
  • Michelle Hribar, PhD, L. Nelson Sanchez-Pinto, MD, Kate Fultz Hollis, MS, Gene Ren, and Deborah Woodcock, MBA will present on, “Learning from the Data: Exploring a Hepatocellular Carcinoma Registry Using Visual Analytics to Improve Multidisciplinary Clinical Decision- Making.”
I am delighted that our students were successful enough to get this far, and I hope that one of them emerges as the winner, as a group of OHSU informatics students did in last year’s event.

Monday, September 28, 2015

After 2018: The Future of Clinical Informatics Board Certification

I have noted in the past that I receive a steady stream of email from physicians asking about their eligibility for board certification in the clinical informatics subspecialty. I have created several posts that allow me to point them to a general answer, the most recent one of which was last year. My most prominent advice has always been for those who can get certified to do so prior to the end of the “grandfathering” period if they can.

I have also voiced my concerns about the whole process. This is not because I believe that board certification in clinical informatics is not a good thing. I do believe it provides an excellent professional recognition of the work physicians do in informatics.

But the process is problematic on several fronts. First, by choosing to have clinical informatics as a subspecialty of all specialties, we require all who are certified to maintain a primary medical specialty. Given that most medical specialties now have time-limited certifications, this can create a challenging situation for those who work predominantly in informatics. It also rules out certification for those who do not have a primary medical specialty, either because their certification lapsed, or they never pursued it in the first place. I know of plenty of highly capable physician-informaticians who are not eligible for board certification.

A second major problem concerns the title of this posting, which is what will happen in 2018. According to current rules, the grandfathering period will end at this time, and the only pathway to board certification will be a two-year on-site Accreditation Council for Graduate Medical Education (ACGME)-accredited fellowship. While such a fellowship (such as the one we have launched at Oregon Health & Science University) will serve as excellent training for a career in clinical informatics, I am not convinced it should be the only pathway by which one can become board-eligible. This is especially the case for the significant numbers of physicians who gravitate into informatics well into their careers and way beyond the end of their formal training. For a physician who has established a career and/or a family, it is unimaginable that he or she could give that up to return to the salary, relocation, and time commitment of a fellowship. This is also inconsistent with work in the 21st century, where professionals, especially in knowledge fields like medicine and informatics, transition to new career activities along the way. Requiring a two-year, on-the-ground experience to become a clinical informatics professional is a relic of 20th-century approaches to training, where you did all of your education before jumping into the workforce, never to return for more.

Our online graduate program at OHSU has shown there are other pathways to successful careers in clinical informatics. We have a track record of many of our physician graduates being hired into clinical informatics positions, including the coveted Chief Medical Informatics Officer (CMIO) role. We have had about 40 graduates successfully pass the board exam, and I am not aware of a single person who failed it. We have also demonstrated that online educational programs can not only provide knowledge, but also practical real-world experiences.

The problem of after 2018 is illustrated explicitly by three “case studies,” two of individuals who have emailed and another who is a current student in an educational program. Let’s look at their cases.

The first emailed to me, “I am planning to take the exam to become board certified. I have a valid medical license, recently matched into an internal medicine residency. I hold a master’s degree in biomedical informatics, and have experience in clinical informatics. I have a senior colleague who recently graduated from an internal medicine residency. He is planning to apply for the clinical informatics board exam, but the application requires that we should have at least three years of experience, which could not be practically possible given the hectic residency schedule.” This individual has more informatics training and experience than his senior colleague, who will be able to become certified during the grandfathering period, but he himself will come up against 2018.

The second emailed to me, “I am a current second-year resident in internal Medicine and I was hoping to get some advice regarding my path in clinical informatics. When I graduate from residency in June of 2017, I had planned on working as a hospitalist while completing OHSU’s online masters in clinical informatics.  My anticipated date of completion for which would be June of 2019. I had hoped to sit for the clinical informatics board certification at that time. Unfortunately because of my family, I will not be living in a city that I would be able to participate in one of the in-person fellowship programs. … If I am not eligible for grand-fathering, how important do you think it would be to be board-certified in clinical informatics vs. holding a master’s?”

The final individual is currently in an MD/PhD program. She will finish her dual degrees next year, in 2016. But she then will need to complete a residency in some specialty, which will end well after 2018. Despite having a PhD in biomedical informatics, this individual will not be eligible to sit for the board exam under the current rules.

I worry that the success of the clinical informatics subspecialty will be compromised by the post-2018 requirement of an ACGME-accredited fellowship. Clearly these fellowships are one of many possible pathways to obtain excellent training in clinical informatics. But having the fellowship be the only pathway to board certification may prohibit many highly capable physicians from achieving their full potential in clinical informatics. I do hope that more enlightened leaders within the American Board of Preventive Medicine (ABPM), ACGME, and other organizations will recognize these problems and provide additional pathways for physicians to train and become successful in clinical informatics.

Monday, August 31, 2015

Information is Different Now That You’re a Doctor: Introduction to Clinical Informatics

This blog posting is a reading assignment for Oregon Health & Science University (OHSU) medical students who will be attending a session I am leading in the Fundamentals block of their curriculum that introduces them to medicine and medical school. My goal for the session is to provide a high-level overview of the information-related issues and challenges they will deal with as physicians and in process introducing them to the field of clinical informatics.

Even though many medical students and physicians do not acknowledge it, information has always been a major focus of clinical practice [1]. Physicians have always spent a great deal of time with information, as evidenced by studies that describe time use of physicians. Even in the era before widespread use of computers in medical practice, physicians spent more of their time in “indirect” care of patients (50-67%), including reviewing results and performing documentation, than directly interacting with patient (15-38%) [2-7]. A more recent study of interns found that they spent nearly 40% of their time in front of a computer [8].

Likewise, physicians and medical students have been using health information technology (HIT) for decades. During this time, the role of HIT has changed dramatically from a useful tool for data access and occasional information retrieval to a ubiquitous presence that permeates healthcare and medical practice in myriad ways. Twenty-first century physicians face a much more information-intense world than their predecessors. The field that focuses on how information is acquired, stored, and used is called informatics, and when applied in medicine and other health-related disciplines is called biomedical and health informatics [9].

Why do physicians spend so much time dealing with information? One reason is that the quantity of biomedical knowledge continues to expand, with an attendant increase in the primary scientific literature, i.e., the 75 clinical trials and 11 systematic reviews published each day. [10] Secondary knowledge sources that summarize this information proliferate as well, not only for use by clinicians but also by patients and consumers. Medical knowledge no longer is the exclusive purview of physicians, as 80% of all Internet users have searched for personal health information [11].

Another major change in the use of HIT has been the rapid growth of electronic health record (EHR) adoption. As a result of the “meaningful use” financial incentives of the Health Information for Technology and Clinical Health (HITECH) Act [12, 13], there has been widespread adoption and use of the EHR, growing to 97% in hospitals [14] and 78% in physician offices [15]. Being able to use data and information also means understanding that the EHR is more than “charting,” and that its value goes beyond being able to read it. Clinicians must be facile with all aspects of the EHR, being able to easily move from one vendor system to another. They must also learn to take advantage of clinical decision support that aims to prevent errors in test ordering, prescribing, and other activities that improve diagnosis and treatment of patients [16].

There is also growing adoption of HIT by patients and consumers, who not only want to find information about their health and disease, but also desire to interact with the healthcare system the same way they interact with airlines, banks, and retailers, i.e., through digital means. Growing numbers of patients are participating in their care using technologies such as the personal health record (PHR) [17] and some are even accessing their own progress notes [18] and more [19].

In the meantime, those who purchase and pay for healthcare, along with patients, are demanding more accountability for the quality, safety, and cost of care [20, 21]. This has led to an expectation of measurement and reporting of healthcare quality of care as a routine part of participation in new care delivery mechanisms such as primary care medical homes and accountable care organizations [22]. Likewise, there is a growing application of the data analytics to improving healthcare [23].

Patients may receive care in different places, sometimes by choice but other times by circumstances beyond their control, such as emergencies. Ideally data should “follow the patient” and move readily across organizational boundaries via health information exchange (HIE) [24]. At the same time, telemedicine and telehealth applications extend the reach of healthcare systems and clinicians in both rural and urban settings [25].

The growing quantity of clinical and administrative data in clinical information systems also affords an opportunity for advanced analysis that can enable better deployment of resources and coordination of care, facilitate personalized and precision medicine, and advance clinical and translational research [26]. Together, these advances are moving healthcare toward the global vision of the learning health system, where information systems are used to capture our practice, analyze what we might have done correctly or improperly, and guiding our improvement [27, 28].

Further evidence for importance of these developments comes the recent establishment of the new medical subspecialty of clinical informatics [29]. Practicing physicians are now beginning to become board-certified in this new subspecialty, with the concomitant establishment of fellowship programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). A growing number of physicians hold titles such as Chief Medical Informatics Officer (CMIO).

It should be abundantly clear that information becomes very different when one transitions to becoming a healthcare professional. There are professional and legal expectations that clinicians must acquire, analyze, and evaluate different facets of information to provide the best possible care to individual patients and entire populations. One critical concept is that informatics is not the same as computer literacy. Computer literacy is one of many requirements to use informatics successfully, but knowing how to use a computing device (PC, tablet, or smartphone) is not the same as having skills in informatics, i.e., using that device to improve health, healthcare delivery, public health, or research.

As a physician, information is different in many ways. Critical decisions about patient care are based on information not only in their EHR, but also knowledge retrieved from scientific literature, textbooks, Web sites, and other sources. Information must be accurate, up-to-date, and applied properly. Physicians must also be effective stewards of a patient’s record, both in terms of keeping it accurate and up-to-date, and also doing the utmost to make sure it is kept private and secure.


1.    Shortliffe EH, Biomedical informatics in the education of physicians. Journal of the American Medical Association, 2010. 304: 1227-1228.
2.    Ammenwerth E and Spötl HP, The time needed for clinical documentation versus direct patient care. A work-sampling analysis of physicians' activities. Methods of Information in Medicine, 2009. 48: 84-91.
3.    Tipping MD, Forth VE, Magill DB, Englert K, and Williams MV, Systematic review of time studies evaluating physicians in the hospital setting. Journal of Hospital Medicine, 2010. 5: 353-359.
4.    Kim CS, Lovejoy W, Paulsen M, Chang R, and Flanders SA, Hospitalist time usage and cyclicality: opportunities to improve efficiency. Journal of Hospital Medicine, 2010. 5: 329-334.
5.    Tipping MD, Forth VE, O'Leary KJ, Malkenson DM, Magill DB, Englert K, et al., Where did the day go?--a time-motion study of hospitalists. Journal of Hospital Medicine, 2010. 5: 323-328.
6.    Yousefi V, How Canadian hospitalists spend their time - a work-sampling study within a hospital medicine program in Ontario. Journal of Clinical Outcomes Management, 2011. 18: 159-164.
7.    Victores AJ, Coggins K, and Takashima M, Electronic health records and resident workflow: A time-motion study of otolaryngology residents. Laryngoscope, 2015. 125: 594-598.
8.    Block L, Habicht R, Wu AW, Desai SV, Wang K, Silva KN, et al., In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time? Journal of General Internal Medicine, 2013. 28: 1042-1047.
9.    Hersh W, A stimulus to define informatics and health information technology. BMC Medical Informatics & Decision Making, 2009. 9: 24.
10.    Bastian H, Glasziou P, and Chalmers I, Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Medicine, 2010. 7(9): e1000326.
11.    Fox S and Duggan M, Health Online 2013. 2013, Pew Internet & American Life Project: Washington, DC,
12.    Blumenthal D, Wiring the health system--origins and provisions of a new federal program. New England Journal of Medicine, 2011. 365: 2323-2329.
13.    Blumenthal D, Implementation of the federal health information technology initiative. New England Journal of Medicine, 2011. 365: 2426-2431.
14.    Charles D, Gabriel M, and Searcy T, Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008-2014. 2015, Department of Health and Human Services: Washington, DC,
15.    Hsiao CJ and Hing E, Use and Characteristics of Electronic Health Record Systems Among Office-based Physician Practices: United States, 2001–2013. 2014, National Center for Health Statistics, Centers for Disease Control and Prevention: Hyattsville, MD,
16.    Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al., Grand challenges in clinical decision support. Journal of Biomedical Informatics, 2008. 41: 387-392.
17.    Miller HD, Yasnoff WA, and Burde HA, Personal Health Records: The Essential Missing Element in 21st Century Healthcare. 2009, Chicago, IL: Healthcare Information and Management Systems Society.
18.    Delbanco T, Walker J, Darer JD, Elmore JG, Feldman HJ, Leveille SG, et al., Open Notes: doctors and patients signing on. Annals of Internal Medicine, 2010. 153: 121-125.
19.    Woods SS, Schwartz E, Tuepker A, Press NA, Nazi KM, Turvey CL, et al., Patient experiences with full electronic access to health records and clinical notes through the My HealtheVet Personal Health Record Pilot: qualitative study. Journal of Medical Internet Research, 2013. 15(3): e65.
20.    Berwick DM, Nolan TW, and Whittington J, The triple aim: care, health, and cost. Health Affairs, 2008. 27: 759-769.
21.    McDonald KM, Chang C, and Schultz E, Closing the Quality Gap: Revisiting the State of the Science - Summary Report. 2013, Agency for Healthcare Research and Quality: Rockville, MD,
22.    Longworth DL, Accountable care organizations, the patient-centered medical home, and health care reform: what does it all mean? Cleveland Clinic Journal of Medicine, 2011. 78: 571-589.
23.    Hersh WR, Healthcare Data Analytics, in Health Informatics: Practical Guide for Healthcare and Information Technology Professionals, Sixth Edition, Hoyt RE and Yoshihashi A, Editors. 2014, Pensacola, FL. 62-75.
24.    Kuperman GJ, Health-information exchange: why are we doing it, and what are we doing? Journal of the American Medical Informatics Association, 2011. 18: 678-682.
25.    Kvedar J, Coye MJ, and Everett W, Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Health Affairs, 2014. 33: 194-199.
26.    Safran C, Bloomrosen M, Hammond WE, Labkoff SE, Markel-Fox S, Tang P, et al., Toward a national framework for the secondary use of health data: an American Medical Informatics Association white paper. Journal of the American Medical Informatics Association, 2007. 14: 1-9.
27.    Friedman CP, Wong AK, and Blumenthal D, Achieving a nationwide learning health system. Science Translational Medicine, 2010. 2(57): 57cm29.
28.    Smith M, Saunders R, Stuckhardt L, and McGinnis JM, Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2012, Washington, DC: National Academies Press.
29.    Detmer DE and Shortliffe EH, Clinical informatics: prospects for a new medical subspecialty. Journal of the American Medical Association, 2014. 311: 2067-2068.

Tuesday, July 28, 2015

The ONC Health IT Curriculum Returns to Life

Long-time readers of this blog know that a substantial part of my work life around 2010-2013 involved developing the health information technology curriculum for the Office of the National Coordinator for Health Information Technology (ONC). I posted in this blog about the project when it was funded as well as it came to an end. As the project was winding down, one of my laments was that there was no further funding to maintain the curriculum. This did not mean it was still not a valuable resource, as many educators were continuing to use it and enhance it locally. We were fortunately able to find a home for the materials in the American Medical Informatics Association (AMIA) Knowledge Center.

I was pleased earlier this year when ONC announced a funding opportunity to update the materials and add four new areas of content relevant for improved healthcare delivery: population health, care coordination, new care delivery and payments models, and value-based care. I am even more thrilled to report that OHSU was one of seven institutions awarded nearly a million dollars in funding to carry out this update and enhancement.

The funding is for more than just updating the curriculum and adding the new topic areas. After the curriculum revision is complete, ONC will work with the awardees to establish a program to train incumbent healthcare employees whose roles, duties, or functions involve health IT. The training will be completed in five days or less to accommodate professionals with restricted schedules and will be offered in various settings, such as online, in-person, or train-the-trainer programs. In total, awardees will collectively train about 6000 incumbent healthcare workers (about 1000 per grantee) in team-based care environments, such as long-term care facilities, patient-centered medical homes, accountable care organizations, hospitals, safety net clinics, rural health, and other settings.

I am certain that I will have more to say periodically about the project and its progress. I am also confident that it will help expand capacity of health IT across the country.