Tuesday, May 26, 2015

Is Medicine Precise Enough to Achieve Precision Medicine?

One of the aspects of medicine that struck me as a medical student was its imprecision. I was surprised, sometimes shocked, at decisions that were made based on vague symptoms reported by patients, ambiguous findings detected on physical examination, and even variation in "hard" measurements such as laboratory results. An area of even more imprecision was the data in the patient record, which seemed to matter less when it was scribbled on paper but takes on a whole lot of importance more now that the data is electronic form and advocated for "secondary use."

It is against this backdrop that I view medicine entering the era of "precision medicine" [1]. The vision and potential for precision medicine is compelling and exciting. The notion that we can unravel the myriad of details of a patient's health or disease, and treat the latter more precisely, would be a genuine advance. The charge for precision medicine is being led by the new NIH $215 million Precision Medicine Initiative launched by President Obama.

As a researcher, especially one with an interest in the secondary use of the growing amount of clinical data [2,3], particularly from the electronic health record (EHR), I am naturally exciting about contributing to the advance of precision medicine. But as an educator, I usually try to step back and take a more holistic view. We must evaluate not only the specific components of precision medicine, but also the general paradigm. I believe there are challenges for both.

Focusing first on components of precision medicine, let us look, for example, at an area like genomics. Although I find genomics very intellectually fascinating, its impact on patient outcomes has been modest [4]. While our ability to sequence genomes and measure their expression continues to improve while costs fall at a rate exceeding Moore's Law for computers, only a modest amount of what we can measure has been "clinically actionable." Furthermore, although we tend to think of gene sequencing as very precise, it turns out that it too has imprecision. Last year, a study of two commercial whole genome sequencing platforms found that medians of 9-17% of 56 genes recently identified as having potentially high clinical importance were not covered by sufficient numbers of repeated sequencing reads to achieve clinical grade variant detection [5]. While whole genome sequencing is likely to improve, and it is not the only way to assess genomic variation, these data show that even gene sequencing can be imprecise.

Another specific area of challenge is clinical data, whose imprecision has also been long known. In 2013 I authored a paper on its "caveats" [2], and last year I recounted a situation where getting data back to its native form would be like unscrambling eggs. Just recently I heard an overview about our institutional plans for precision medicine, and walking away from a meeting with a clinical colleague, she was lamenting how the switch-over to ICD-10 for coding diagnoses on radiology ordering had just become a whole lot harder at our hospital because of the vastly increased number of codes. Her residents were overwhelmed by the choices, so often sought out the "not otherwise specified" code, which of course was often not the correct one to choose.

Also a concern about the components of precision medicine is how we will figure out what works. Although a proponent of the evidence-based medicine (EBM) approach, I am well aware of the limits of EBM that homogenize patients into large groups in order to determine an effect of a test or treatment. The nature of "best evidence" studies often glosses over individual differences. This provides a benefit in allowing statistical analysis to discern bias and chance from truth, but at the cost of ignoring personal differences. In precision medicine, when each individual is unique, how will we be able to experimentally compare different diagnostic tests and precision-based treatments?

I also believe that we will need to validate the paradigm of precision medicine. Indeed, this may be a way to overcome some of the EBM-related challenges, in that we may be able to apply experimentation to the precision medicine approach rather than any particular (individualized) therapy. Although hopefully there will be some tests and treatments with widespread enough use to conduct clinical trials.

In any case, the era of precision medicine portends an interesting and likely highly beneficial approach to medicine. The role of informatics will be widespread and important. Many of the issues that plague informatics, especially clinical data, currently (e.g., lack of standards and interoperability, ability to aggregate across healthcare systems, need to integrate with genomics and other bimolecular data) will need to be solved for informatics to make its optimal contribution.


1. Collins, FS and Varmus, H (2015). A new initiative on precision medicine. New England Journal of Medicine. 372: 793-795.
2. Hersh, WR, Weiner, MG, et al. (2013). Caveats for the use of operational electronic health record data in comparative effectiveness research. Medical Care. 51(Suppl 3): S30-S37.
3. Hersh, WR, Cimino, JJ, et al. (2013). Recommendations for the use of operational electronic health record data in comparative effectiveness research. eGEMs (Generating Evidence & Methods to improve patient outcomes). 1: 14. http://repository.academyhealth.org/egems/vol1/iss1/14/.
4. Green, RC, Berg, JS, et al. (2013). ACMG recommendations for reporting of incidental findings in clinical exome and genome sequencing. Genetics in Medicine. 15: 565-574.
5. Dewey, FE, Grove, ME, et al. (2014). Clinical interpretation and implications of whole-genome sequencing. Journal of the American Medical Association. 311: 1035-1044.

Monday, May 11, 2015

Semantic Drift and the Persistence of Informatics

Being concerned with representation of information and knowledge, researchers in informatics sometimes express concern with the concept of "semantic drift," where the meaning of words and concepts changes over time. Semantic drift happens for a variety of reasons, most commonly due to advancing and changing knowledge of health and biomedicine. Another type of semantic drift occurs in many industries, including the information technology (IT) industry, where new terms come along reflecting evolution in technology, although sometimes the new terms are just a different name for a similar or sometimes the same thing. Not infrequently, the new terminology reflects marketing and hype as much as substantive change.

Some terms withstand the test of time, and I am pleased to note that "informatics" fits into that category. The word traces its origins back to the 1960s, and the importance of the discipline has withstood the test of time. As with all fields, the leading edge has changed substantially, but the core function and definition of the field - the use of data, information, and knowledge to improve human health - has not.

Like many fields, informatics has seen the emergence of areas of work that overlap with its work, in essence that provide semantic drift not only from the core definition of informatics but also the description of work that rightfully belongs to it. I am referring to some of the emerging "hot topics" in recent years, such as data science, data analytics, and precision medicine. I suspect that some may argue these are different from informatics, but I would rebut that they really fit under the broad umbrella of informatics.

I also believe these new sub-disciplines need to prove their work, just as informatics has (or in some cases has not). Like most established disciplines, informatics has a long trail of science. Not all of it is strong methodologically, particularly the portion that evaluates systems in the real world. But we can point to techniques and implementations that have been studied enough to demonstrate where they do and do not work [1-4]. Informatics also provides a good deal of experience and perspective in having tried to address some of what these new sub-disciplines are trying to accomplish.

The current hot topic is precision medicine [5-6]. While I share the excitement and recognize its potential, I also know that it is still an unproven science. In other words, there are still few "products" of precision medicine that demonstrated any large-scale success. This does not mean precision medicine will not have such benefit, or that further research should not be pursued. But we also need to look for its results, especially those that lead to improved health and of outcomes from treatment of disease. The same holds true for the previous hot topic before precision medicine, namely data analytics and other aspects of Big Data.

In the meantime, I would encourage those who are pursuing these emerging areas to find a home in the larger science of informatics. Indeed, those from the informatics community are working in them (myself included), and we should show there is a solid trail of science leading into them and eschew that they are somehow completely brand new.


1. Chaudhry, B, Wang, J, et al. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine. 144: 742-752.
2. Goldzweig, CL, Towfigh, A, et al. (2009). Costs and benefits of health information technology: new trends from the literature. Health Affairs. 28: w282-w293.
3. Buntin, MB, Burke, MF, et al. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs. 30: 464-471.
4. Jones, SS, Rudin, RS, et al. (2014). Health information technology: an updated systematic review with a focus on meaningful use. Annals of Internal Medicine. 160: 48-54.
5. Collins, FS and Varmus, H (2015). A new initiative on precision medicine. New England Journal of Medicine. 372: 793-795.
6. Ashley, EA (2015). The Precision Medicine Initiative - a new national effort. Journal of the American Medical Association, Epub ahead of print.

Tuesday, April 28, 2015

Toward Sustainable Funding for Clinical Informatics Fellowships

I have written before that one of the key challenges facing the new clinical informatics subspecialty is the funding of training fellowships. This is just one of the many "square pegs into round holes" problems of clinical informatics training not meshing with the traditional approach to clinical subspecialty fellowships.

The major problem is that most clinical subspecialty fellowship training in medicine is funded in part, even if only indirectly, by the service contributions that fellows make to the clinical unit in which they are training. A cardiology fellow, for example, is providing cardiology service to his or her department, which can either be billed or, if billed by the attending physician, allowing that attending physician to extend his or her capacity. This is in distinction to how most advanced education is funded, where a tuition model (either directly paid by the student or subsidized by someone else, such as a state government or training grant) pays the cost.

While a clinical informatics fellow will hopefully make contributions to the health system in which he or she is training, it is a bit of a stretch to believe they are providing monetary value for their service. At best, the health system is investing in these fellows because they are building informatics capacity in their organizations, which may translate into cost savings as value-based reimbursement models are undertaken (e.g., accountable care organizations).

If we are going to require physicians being trained in clinical informatics to do so via the traditional fellowship model, how are we going to pay the cost of their training? Some health systems might find value and foot the bill, as some already are. But it is not clear how sustainable this model is. If a health system comes on hard times financially and needs to cut costs, the clinical informatics fellowship might be an area that is reduced or eliminated.

Therefore a clinical informatics fellowship model must include a means for fellows to generate revenue for at least part of the their training. One obvious way to do this is allowing these fellows to practice in their primary medical specialty and bill (or otherwise be remunerated) for their work. Accreditation Council for Graduate Medical Education (ACGME) rules actually require clinical informatics fellows to maintain active practice in their primary specialty during their fellowship time (i.e., not moonlighting). Their work in clinical informatics will usually be distinct from their clinical practice, as clinical informatics work is likely to be applied to the entire health system and not just the fellow's primary specialty.

A problem with the fellowship trainee billing, however, is that it comes up against Center for Medical and Medicaid Services (CMS) rules that do not allow clinical trainees to "double dip." That is, most health systems with graduate medical education programs (i.e., residency and fellowship training) receive a subsidy from CMS Medicare funding to pay for physician training. Most health systems interpret this to not allow residents and clinical fellows to bill for their patient care work. But clinical informatics, like a number of other emerging subspecialties that emanate from multiple primary specialties, is truly different from the clinical practice component of the primary specialty.

To this end, I have recently collaborated with several of my colleagues leading clinical informatics fellowship programs to publish an open letter to CMS asking for guidance on clinical fellows being able to bill for their work so that clinical informatics fellowships can achieve financial sustainability [1]. We published this letter in the journal Applied Clinical Informatics and will be engaging with other medical subspecialties to achieve clarification from CMS on this issue. In addition, the leadership of the American Medical Informatics Association (AMIA) is working with other subspecialties in a similar situation to provide a larger picture of the problem, which we have learned is not unique to clinical informatics. The hope of clinical informatics fellowship leaders is that fellows will be allowed to function as attending physicians for their clinical practice in their primary specialty, and that this will allow a more sustainable funding model for clinical informatics fellowships.


1. Lehmann, CU, Longhurst, CA, et al. (2015). Clinical informatics fellowship programs: in search of a viable financial model - an open letter to the Centers for Medicare and Medicaid Services. Applied Clinical Informatics. 6: 267-270.

Thursday, April 9, 2015

Accolades for the Informatics Professor: Leadership Award at HIMSS and Other Notables

Time for one of my periodic postings on accolades for the Informatics Professor. Probably the major one concerns next week's Healthcare Information Management Systems Society (HIMSS) conference, where I will be receiving the HIMSS Physician IT Leadership Award. I am honored to receive this award and to be among so many other special award winners. Others have also picked up on the award, including MedTech Boston and HITECH Answers.

The HIMSS Award is not my only recent accolade. I was quoted in a story in the Portland Business Journal about the role of our program in the IBM-Epic bid to replace the Department of Defense military electronic health record system.

I also gave a Future Talk at New Relic, Inc., a Portland-based software analytics company on March 9, 2015. Entitled, Big Data in Healthcare and Biomedicine: Opportunities and Challenges, the talk, with links to my slides and a video, was written up in their company blog.

Monday, April 6, 2015

HITECH and Meaningful Use at a Crossroads

It is hard to believe that the Health Information Technology for Economic and Clinical Health (HITECH) Act was passed into law five years ago, in 2009, and at the end of the year, the massive legislation was shaped into a program that has profoundly altered the informatics world, not to mention all of healthcare. Like most large undertakings, especially when infused with politics, the results have been mixed. Clearly the goals of EHR adoption have been substantial in hospitals and by physicians, even if the resulting systems have not achieved the ideals we held out for them. Perhaps as much depending on your political views as much as your informatics views, the glass is either partially empty or partially full.

On the positive side, a large proportion of US physicians [1] and nearly all US hospitals [2] now use an electronic health record. While many have argued that there should have been a much greater focus from the start on data interoperability, we are seeing progress with the rapid coalescence behind the FHIR, ReST, and OAuth2 standards in the Argonaut Project of HL7.

On the negative side, the systems we have implemented have been driven by meaningful use criteria. While no one would argue against these criteria generally (e.g., problem lists, electronic prescribing, etc.), many have argued that healthcare organizations have had to devote too much effort to meeting the criteria rather than innovating and leading with the beneficial aspects of technology. By the same token, the focus of vendors has had to be on certification to insure their customers can meet the meaningful use criteria with their products. On top of this is the toxic political environment in the US, with one's views' on HITECH and the Affordable Care Act being a sort of political Rorschach Test, making it even more difficult to have a meaningful conversation.

I tend to be glass-half-full kind of person, although I certainly acknowledge the limitations of the situation we are in now. It is easy to find critics of the current situation, but I tend to prefer to read and converse with those who present a balanced view that recognizes the problems in paper-based healthcare that led us to adopt the (still not achieved) promise of information technology (IT)-enabled healthcare. I give a special call-out to my colleagues Bob Wachter [3] and Jacob Reider [4] for their recent writings, and the former for his book that was just released [5], which I am enjoying but admittedly not done reading yet.

The real question is how we can get from here to where we want to be. This is especially so with the release of the Notice of Proposed Rule Making (NPRM) for Meaningful Use Stage 3 as well as the legislation to solve the Sustainable Growth Rate (SGR) problem (the "doc fix") of Medicare, which contains a proposal to roll the Meaningful Use Program into a more coalesced approach to incentives for quality in the Medicare Program.

My own view is that we should be focusing on data standards and interoperability, aiming to allow innovation to flourish on top of it. We also need to be open and critical of current failings, but also willing to move beyond negativity and linking the current situation to politics and/or greed. Not that both of these are not present, but that we need to come together as a community so those negative attributes are held in check by the greater community working toward more positive goals.


1. Charles, D, Gabriel, M, et al. (2014). Adoption of Electronic Health Record Systems among U.S. Non-federal Acute Care Hospitals: 2008-2013. Washington, DC, Department of Health and Human Services. http://www.healthit.gov/sites/default/files/oncdatabrief16.pdf.
2. Hsiao, CJ and Hing, E (2014). Use and Characteristics of Electronic Health Record Systems Among Office-based Physician Practices: United States, 2001–2013. Hyattsville, MD, National Center for Health Statistics, Centers for Disease Control and Prevention. http://www.cdc.gov/nchs/data/databriefs/db143.htm.
3. Wachter, B (2014). Meaningful Use. Born, 2009, Died, 2014? Wachter's World, November 13, 2014. http://community.the-hospitalist.org/2014/11/13/meaningful-use-born-2009-died-2014/.
4. Reider, J (2015). Spring Deliveries from Washington. The Health Care Blog, March 22, 2015. http://thehealthcareblog.com/blog/2015/03/22/spring-deliveries-from-washington/.
5. Wachter, R (2015). The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age. New York, NY, McGraw-Hill.

Saturday, March 21, 2015

When Bad Governments Happen to Good People

One of the most enjoyable aspects of my job is the opportunity to visit many different countries to collaborate with colleagues and friends in all sorts of informatics settings. I have written before that informatics is a field of global truths, and the benefits and challenges of implementing information systems in healthcare settings are universal across the world. Mixing healthcare and information technology no matter what kind of health system or technology infrastructure a county has.

Some of the countries I visit have governments whose policies and/or actions I consider disagreeable. How do I rectify this? The main way I do is recognize that those who invite me are doing so to share ideas and activities around informatics. As we are part of the larger biomedical and health community, we are driven by the creed that drives all health professions, which is to improve the health of individuals and populations. We do this in informatics by focusing on effort to improve health through better use of data, information, and knowledge.

There are certainly places I visit colleagues where the policies of their particular government are not agreeable to me. On the other hand, I do not always agree with the policies of my own government in the United States, although I do cherish our political system that lets me speak out about it, which is not always the case in places I visit. In any case, my view on visiting countries where I have wonderful colleagues but whose government I find disagreeable is that I do not need to to support a particular government if I am going to interact with my colleagues who live in that country. As my own country's track record in international relations is far from perfect, I can engage in honest discussion when the topic of politics arises. While I do not overtly criticize the governments who policies and actions with which I disagree, I will not hesitate to speak my opinion when asked.

Another tension I sometimes experience concerns religion. My view of religion is that I respect all religions and honor their traditions, even those traditions that are at odds with what I believe is fair and just. (For example, the rights of women and local minorities.)

One additional positive thing I have noted in my travels is that an activity that seems to bridge people around the world is technology, in particular the Internet. The access to facts and ideas that the Internet allows is enabling, and there is a common bond from rich countries to poor ones now that the Internet has become so ubiquitous everywhere on the planet.

While there is very little in the world that one individual can change, I will continue enjoying my travels around the world, especially in advancing the cause of improving health and healthcare through informatics. I will share both my knowledge and my attitude that when the right people are given the right tools, good outcomes can result.

Wednesday, March 18, 2015

Advice for Physicians in Training Seeking a Clinical Informatics Fellowship

Lately I have received requests for advice from physicians in training asking what they can do to make themselves more competitive for clinical informatics fellowship positions. In some sense these are similar to the emails I receive from established physicians asking about eligibility for the clinical informatics subspecialty here and now. To provide answers to the established physicians most efficiently, I prepared a blog post that I send them to as a reply to get them started in thinking about their eligibility. I am now doing likely in having a generic reply to physicians in training, and it is actually similar to another posting of mine from last year that provided advice to any young person seeking a career in informatics.

Let me then focus specifically on the physician in training who is considering pursuing formal training in clinical informatics, since I have increasingly been receiving emails with this question. My advice is really not much different from what anyone might advise a physician seeking training in any specialty or subspecialty. It is, however, important for potential trainees to remember that starting in 2018, fellowships accredited by the Accreditation Council for Graduate Medical Education (ACGME) will be the only pathway to achieve board certification in the new subspecialty.

As for advice, first and foremost, someone seeking formal training in clinical informatics should understand the field and its role in healthcare.  He or she should understand how informatics differs from information technology (IT), computer science, and related areas. The potential informatician should also be aware of the kinds of work that informatics professionals perform and the types of jobs into which they are hired.

A second critical piece of advice is to get involved in some informatics activity in their current medical school or residency program. It need not be a high-profile research project that gets published, but any activity that gives him or her an opportunity to perform and to be able to describe the application of informatics in a healthcare setting that provided value to someone, whether a physician, other healthcare professional, researcher, patient, or even a health system. While it would be ideal for the activity to be a medical school or residency rotation, it could also be volunteer activity. Whatever is done, he or she should be able to describe the work, who benefitted from it, and what principles of informatics it applied.

Another recommendation I can give is to become involved in some sort of professional organization or activity. While participating in a national organization such as the American Medical Informatics Association (AMIA) may be impractical, he or she can try to become involved in a local or regional organization. In Oregon, for example, we have an active local Health Information Management Systems Society (HIMSS) chapter. There are also national as well as local medical societies and other professional organizations that also carry out informatics activity.

A final bit of advice is to choose a fellowship that aligns with one's career goals. The clinical informatics subspecialty fellowship is focused on training for operational informatics work. Those more interested in a research career, especially if training in a medical specialty is not desired, should consider something like the National Library of Medicine (NLM)-sponsored research fellowships, which Oregon Health & Science University (OHSU) offers along with 13 other universities. Our clinical informatics subspecialty fellowship is just one of a family of educational programs in biomedical and health informatics offered by OHSU, and there might be other training options to consider.

In essence, those interested in clinical informatics should understand the field, get involved in it, and connect with professional associations. This advice is really little different from what one might advise someone seeking a career in almost any field. Such activity demonstrates a commitment to the field that will strengthen the application of someone who is seeking a fellowship in clinical informatics.

Sunday, March 15, 2015

Opportunity for Public Input into Evidence Report on Health Information Exchange

In the fall of 2013, I reported on a new project for which OHSU had been awarded a contract to carry out a systematic review of the evidence base for health information exchange (HIE). This project was funded under the Agency for Healthcare Research and Quality (AHRQ) Effective Healthcare Program. Our Department of Medical Informatics & Clinical Epidemiology (DMICE) houses one of the 12 Evidence-Based Practice Centers (EPCs) funded by the AHRQ to create evidence reports on a variety of healthcare topics.

I am pleased to report that the project is just about complete, and those who are interested in it can actually contribute. As is the case with all AHRQ evidence reports, a draft of the report has been posted for public comment. I encourage interested people to download the draft report and provide comments.

While there have unfortunately been two other systematic reviews to appear in the last few months [1,2], our report, based on AHRQ EPC protocols for evidence reports, will hopefully be more comprehensive and have used a more inclusive process. One of those processes is a period of public comment (simultaneous with formal peer review). The public comment period has just started and will be open until April 8, 2015. We look forward to constructive comments that will help improve the final report, which will be available in a few months.


1. Rudin, RS, Motala, A, et al. (2014). Usage and effect of health information exchange: a systematic review. Annals of Internal Medicine. 161: 803-811.
2. Rahurkar, S, Vest, JR, et al. (2015). Despite the spread of health information exchange, there is little evidence of its impact on cost, use, and quality of care. Health Affairs. 34: 477-483.

Thursday, February 26, 2015

Input to the Working Group to Chart the Future Course for the National Library of Medicine

Like many in academic informatics, my career has benefitted greatly from the support of National Library of Medicine (NLM), the institute of the National Institutes of Health (NIH) that has been devoted to, among other things, support for research and training in biomedical and health informatics. I have written over the years in this blog (in 2011 and 2014) of the myriad contributions of the NLM to biomedicine and health, including its unique role in funding basic research in informatics, especially clinical informatics.

The NLM has been guided over the past 30 years by a single leader, Dr. Donald AB Lindberg, whose vision and steady hand have led it to great success. Virtually everyone in healthcare has benefited from the NLM's successful initiatives that have been carried out during Dr. Lindberg's tenure, especially the information resources of Pubmed, MedlinePLUS, ClinicalTrials.gov, and the myriad of genomics resources under the auspices of the National Center for Biotechnology Information (NCBI). Late last year, Dr. Lindberg announced his retirement. This has prompted the NIH to launch a working group and Request for Information (RFI) process to chart the future course of the NLM in advance of appointing a new leader.

Naturally, I view this opportunity as a chance to weigh in on the future of the NLM, which is so critical not only to my own work but also to the informatics field in general and really all of the healthcare enterprise. In the rest of this posting, I list the RFI questions in underline and then provide the answers I entered into the NIH site for collecting them. I look forward to seeing what others write as well as the final report of the working group. Appointing a leader to sustain Dr. Lindberg's contributions is one of the most essential actions for the NLM and the informatics field going forward.

(I do note that one of the challenges with the RFI structure is the lack of a section to make comment about the NLM with regards to all of its missions and constituents. As such, I have placed my comments more disproportionately in the section on the research community, since I believe the issues of basic informatics research are most important to be addressed in the transition to new leadership.)

Current NLM elements that are of the most, or least, value to the research community (including biomedical, clinical, behavioral, health services, public health, and historical researchers) and future capabilities that will be needed to support evolving scientific and technological activities and needs.

The NLM is a unique resource to all communities, especially the research community, in two areas: its basic research and education function in biomedical and health informatics and its library function. The NLM's library function is in excellent shape, and it continues to be an innovator and leader in its world-leading medical library function.

I have more serious concerns about the NLM's research function. Although there are many institutes within NIH (e.g., NCI, NHLBI, and the Fogarty International Center) and other entities outside of NIH (e.g., AHRQ and PCORI) that fund research in informatics-related areas, NLM is the only entity that funds basic research in biomedical and health informatics. Most of the other institutes and entities that fund informatics support projects that are highly applied and/or domain-focused. These projects are important, but basic informatics research is also key to improving both individual health as well as the healthcare system.

The NLM is also nearly unique in funding basic research in clinical informatics. A good deal of informatics research in the other NIH institutes is focused in basic science, e.g., genomics, bioinformatics, and computational biology. AHRQ and PCORI support clinical informatics research, but it is highly applied. Only NLM funds critical basic research in clinical informatics, and this function is vitally important as we strive to use informatics to achieve the triple aim of better health, improved healthcare, and reduced costs. Some of these areas of basic research include standards and interoperability, usability, workflow analysis, natural language understanding, and the intersection of people and organizational issues with information technology.

Informatics research within NIH and other government agencies is also very silo-ed. Why, for example, is the new Data Science (BD2K) program housed in the NIH Director's Office, when it really should be organized as a part of larger informatics science. (Indeed, many BD2K grantees, including myself, receive a good deal of their other research support from NLM.) Initiatives such as data science should really be part of an integrated approach to informatics research and be part of the NLM (or what the NLM should become).

A final critical function of NLM that has provided value and should be maintained is its training programs for those who aspire to careers in informatics research. I count myself among many whose NLM fellowship training led to a successful career as a researcher, educator, and academician generally. NLM training grants have also provided support for my university to educate the next generation of informatics researchers who have gone on to become successful researchers and other leaders in the field.

One part of the problem is that the name itself, "National Library of Medicine," does not connote all of what NLM does. Yes the NLM is a world-renowned biomedical library, and that function is critically important to continue. But NLM also provides cutting-edge research and training in informatics, and an ideal change for NLM would be a name change to something like the "National Biomedical and Health Informatics Institute," of which a robust and innovative National Library of Medicine would be a vital part.

Current NLM elements that are of the most, or least, value to health professionals (e.g., those working in health care, emergency response, toxicology, environmental health, and public health) and future capabilities that will be needed to enable health professionals to integrate data and knowledge from biomedical research into effective practice.

The NLM's library function is critical to all healthcare professionals, and this group probably benefits most from the NLM's excellent use of tax dollars in implementing its freely available resources. However, healthcare professionals have also benefitted from, and will likely continue to benefit from, NLM's basic research, especially in clinical informatics. While the adoption of knowledge systems, electronic health records, health information exchange, and other informatics applications has increased substantially in the last decade, the foundation for these applications emerged in no small part from NLM basic research in informatics. And as these applications are all far from perfect, they will likely need continued research to increase our understanding and optimization of them.

The success of the NLM's work has also led to a new category of health professional, which is the informatics professional, manifested most prominently in the area of clinical informatics. A growing number of healthcare provider organizations have established operational clinical informatics units, which are usually distinct from IT units. These are often directed by a Chief Medical Informatics Officer (CMIO). Another manifestation of this success is the designation of clinical informatics as a physician subspecialty. There are now 787 board-certifiied clinical informaticians, and a number of universities are establishing fellowship programs accredited by ACGME. In the meantime, AMIA has established an Advanced Interprofessional Informatics Certification process that will lead to certification of non-physicians in clinical informatics. These health professionals will play a vital role in applying the results of informatics research to innovating and improving patient care. This also gives impetus for maintaining a strong basic research effort in clinical informatics, and the NLM is uniquely poised to continue that role.

Current NLM elements that are of most, or least, value to patients and the public (including students, teachers, and the media) and future capabilities that will be needed to ensure a trusted source for rapid dissemination of health knowledge into the public domain.

The NLM's library function also ensures rapid dissemination of high-quality knowledge for patients and the general public. Its flagship site, MedlinePLUS, is a gold standard for high-quality, consumer-oriented health information. Of course, there are still a myriad of research issues about optimizing informatics for patients and the public. How do we insure the most appropriate information is delivered to such individuals at an appropriate depth and reading level? What is the role of personal health records in integrating knowledge and guidance? These research questions further demonstrate the importance of basic informatics research.

Current NLM elements that are of most, or least, value to other libraries, publishers, organizations, companies, and individuals who use NLM data, software tools, and systems in developing and providing value-added or complementary services and products and future capabilities that would facilitate the development of products and services that make use of NLM resources.

Likewise, the NLM's library function ensures rapid dissemination of high-quality knowledge for those who produce and disseminate information in the public and private sectors. The standards it sets provide interoperability unparalleled in other areas of the healthcare industry. In this area as well, the basic research of NLM is critical, contributing to more effective ways to produce and disseminate information in a vibrant marketplace.

How NLM could be better positioned to help address the broader and growing challenges associated with:

  • Biomedical informatics, “big data”, and data science;
  • Electronic health records;
  • Digital publications; or
  • Other emerging challenges/elements warranting special consideration.

The NLM, and the research it funds, is well-positioned to address all of these listed challenges.

Individuals who are trained in biomedical and health informatics not only understand Big Data and Data Science, but also bring the perspective of other aspects of informatics, such as standards and interoperability, usability, clinical workflow, and people and organizational issues. Data science transcends algorithms; it requires a thorough understanding of the quality and veracity of data. The understanding of how data, information, and knowledge are generated, organized, critiqued, and maintained is a unique skill of those who are trained in informatics. As noted earlier, the BD2K initiative should really be housed in the NLM, since it is part of larger informatics science and the fact that many who are funded by BD2K also have other funding from NLM.

The same applies to electronic health records (EHRs). While the HITECH Act has led to widespread adoption of EHRs, there are still many challenges associated with their optimal use. Basic research in clinical informatics established the foundation of modern EHRs that enabled companies such as Epic, Cerner, Allscripts, and others to thrive in the market. As such, continued research and training of researchers is necessary to ensure sustained progress, especially with the need to move to standards-based interoperable systems. The presence of academic research will also provide a bulwark against EHR development being driven solely by industry, which has an important role to play, but also requires basic research to push innovation in the market.

There are also emerging technologies, some of which we cannot foresee. When I was an NLM informatics postdoctoral fellow in the late 1980s, I could not have imagined the details of the World Wide Web, the wireless ubiquitous Internet, or modern mobile devices. There are likely new technologies coming down the road that few if any of us can predict that will have major impacts on health and healthcare. It is critical that the NLM and the research it supports enable these technologies to be put to optimal usage.

Saturday, February 7, 2015

2015 Update of Site, What is Biomedical & Health Informatics?

All through my career, I have been asked on a regular basis, What is Medical/Biomedical/Health Informatics? Years ago, to answer this question, I created a Web site that attempted to answer it. Later on, I added some voice-over-Powerpoint lectures, which also provided me the opportunity to demonstrate the technologies we use in our distance learning program at Oregon Health & Science University (OHSU).

Keeping a site like this up to date is no small feat, especially with all my other activities in research, education, and administration. As such, the site has grown out of date from time to time. I am pleased to announce that I have now updated the lecture and references on the site, perhaps being somewhat less ambitious in the breadth of material that I cover. (Though I do hope to add more up-to-date material over time.)

The educational methods I use on this site mirror my on-line teaching. I have always found great value in voice-over-Powerpoint lectures, especially using the Articulate Presenter tool that provides the slides and sound in Flash format and also allows easy navigation among the slides. I also provide PDF files of the slides as well as another PDF that has references to all of the papers, reports, books, and other citations in the lecture. The site also contains a list of key textbooks as well as links to some of my papers and to important organizations and other sites for the field.

I also hope the site will whet peoples' appetites for the AMIA-OHSU 10x10 ("ten by ten") program, the OHSU biomedical informatics graduate program, or other educational programs in the field. I look forward to receiving feedback from people and take full responsibility for any errors in any of the materials I have produced.

Friday, February 6, 2015

The Conundrum of Structured vs. Unstructured Data

As in all complex endeavors, the push for a healthcare system underpinned by structured and interoperable electronic health record (EHR) data has turned out to be more complicated than we might have anticipated when acceleration of EHR adoption was begun about a decade ago. This does not mean that anyone was right or wrong; it just shows the inherent complexities of trying to solve the real problems that motivate data-related problems in healthcare. These healthcare problems have been well-documented over the past couple decades by the Institute of Medicine (IOM) and others, and include incomplete and unavailable records [1], medical errors [2], and suboptimal quality of care [3]. These problems are still every bit as real as they were when the IOM and others first brought them to light, but the solutions have been more challenging to find.

It is almost a holy grail of informatics that the value of EHR data stems from structured and interoperable data, which in turn allows not only better primary use for patient safety, clinical decision support, and other benefits, but also secondary use, such as quality measurement, public health surveillance, and clinical research. Yet it has been known for some time that there is a "tension" between the entry and use of structured vs. unstructured data [4].

A few months ago, I wrote a post on what are realistic goals for EHR interoperability, based on what I saw was positive prioritization by the Office of National Coordinator for Health IT (ONC) on data interoperability within the EHR. There is no question that data flowing seamlessly, and maintaining its meaning, is critical to advance the value of health IT.

That discussion, however, uncovers a challenge of major magnitude within informatics, which is how much data should be structured, and how to best deploy that data. A number of commentators I greatly respect have weighed in on this issue.

My spurring to write on this topic was motivated by Wes Rishel, formerly of Gartner. Mr. Rishel used the challenges of patient summaries to avoid against "de-motivating" interoperability [5]. In particular, he noted the challenge between two views of the interoperable patient summary, one driven by a human-generated narrative that communicates the patient's situation succinctly and other generated by a computer with the goal of transfer of data. He (and others before him, such as Dr. Peter Basch [6]) have noted that clinicians have dissatisfaction and distrust with records generated from structured data.

Other groups have weighed in on this problem as well. Last year, the American Medical Association (AMA) had put forth a succinct piece on improving EHR usability, noting that while data "liquidity" is important, it takes a back seat to the primacy of clinician usage of the EHR for improving care to be its primary motivation [7]. And just recently, the American College of Physicians advocated in a similar manner, releasing a policy paper on clinical documentation also calling for the primary needs to be focused on meeting the needs of clinicians [8].

A major challenge for informatics is how to balance the desire for structured data to add value versus providing readable and succinct documentation to enable the best patient care. Unfortunately, the two can be at odds with each other. If physicians do not like, let alone trust, the kind of structured data that enables other value for EHR data, what is the solution?

When in situations like this, I always remember the words of an elder sage of informatics, Dr. Clement McDonald of the National Library of Medicine, who has noted, Informatics is a journey, not a destination.  We may never achieve the perfect solution, but must continually strive to find the right balance of structure and interoperability. Or, to quote from the decades-old Ten Commandments of Informatics [9], penned by another elder statesman of the field, Dr. Octo Barnett, who stated, Be optimistic about the future, supportive of good work that is being done, passionate in your commitment, but always be guided by a fundamental skepticism.

Or, to be guided by a quote often attributed to Voltaire, which is that we should not let perfect be the enemy of good. It is obvious that the EHR will never, like all of medicine, be perfect. Therefore, we should strive to find the best solution that balances the value of optimally devliered care balanced with the value that structured data can bring.


1. Dick, RS, Steen, EB, et al., Eds. (1997). The Computer-Based Patient Record: An Essential Technology for Health Care, Revised Edition. Washington, DC, National Academies Press.
2. Kohn, LT, Corrigan, JM, et al., Eds. (2000). To Err Is Human: Building a Safer Health System. Washington, DC, National Academies Press.
3. Anonymous (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, National Academies Press.
4. Rosenbloom, ST, Denny, JC, et al. (2011). Data from clinical notes: a perspective on the tension between structure and flexible documentation. Journal of the American Medical Informatics Association. 18: 181-186.
5. Rishel, W (2015). How to Avoid DE-Motivating Interoperability? Retired Healthcare IT Nerd, January 5, 2015. http://rishel.com/blog/2015/01/avoid-de-motivating-interoperability/.
6. Basch, P (2014). ONC’s 10-Year Roadmap Towards Interoperability Requires Changes To The Meaningful Use Program. Health Affairs Blog, November 3, 2014. http://healthaffairs.org/blog/2014/11/03/oncs-10-year-roadmap-towards-interoperability-requires-changes-to-the-meaningful-use-program/.
7. Anonymous (2014). Improving Care: Priorities to Improve Electronic Health Record Usability. Chicago, IL, American Medical Association. https://download.ama-assn.org/resources/doc/ps2/x-pub/ehr-priorities.pdf.
8. Kuhn, T, Basch, P, et al. (2015). Clinical documentation in the 21st century: executive summary of a policy position paper from the American College of Physicians. Annals of Internal Medicine. Epub ahead of print.
9. Barnett, GO (1979). The use of computers in clinical data management: the ten commandments. Society for Computer Medicine Newsletter. 4: 6-8.

Tuesday, January 27, 2015

Accolades For My Informatics Colleagues

Readers of this blog know that from time to time, I post about accolades that I have received. In this posting, I would like to change the focus of the accolades to a number of my colleagues who have been recruited to direct a number of new programs, departments, and institutes that have recently been established in biomedical informatics. These happenings come during challenging times for the biomedical research enterprise generally. However, the informatics field appears to be thriving in these times of overall National Institutes of Health (NIH) support for research declining in terms of real dollars, resulting in the success rates of grant applications falling as more proposals chase fewer dollars.

The tide of new hirings started last fall, when Jason Moore, PhD, previously of Dartmouth College, was tapped to serve as Director of the new Institute for Biomedical Informatics at the University of Pennsylvania. The press release quotes the Dean of Penn's Perelman School of Medicine stating, "Because solving many of the most challenging biomedical problems today depends on our ability to integrate, analyze and interpret complex patterns in 'big data,' establishment of the institute emerged as a top priority in Penn Medicine's strategic plan. Under Jason’s leadership, I expect the institute will become a model of education, innovation and collaborative research at the interface of biomedical informatics, the basic sciences, and the clinical sciences."

Also last fall, Christopher Chute, MD, DrPH announced his departure from his long tenure at Mayo Clinic to join the Johns Hopkins University as the Bloomberg Distinguished Professor of Health Informatics. In addition to also becoming chief health research information officer for the university, he will also take on the creation of a new Institute for Biomedical Informatics across its health-related schools.

These announcements were followed by a spate of others after the new year. First, Atul Butte, MD, PhD of Stanford University was named to head the new Institute for Computational Health Sciences up the road at the University of California San Francisco. Dr. Butte will also serve as Executive Director of Clinical Informatics for University of California Health Sciences and Services.

About the same time, Harvard Medical School announced the transition of its Center for Biomedical Informatics (CBMI) into a full-fledged Department of Biomedical Informatics, with the initial Chair to be Isaac Kohane, MD, PhD. The news release quotes the Dean of Harvard Medical School say, "Based on the increasing impact of the field and the tremendous success of CBMI, we have concluded that biomedical informatics is a field now ready for full academic recognition as a new appointing department at HMS."

Finally, James Cimino, MD, currently Chief of the Laboratory for Informatics Development at the National Institutes of Health's Clinical Center , has been named as the inaugural Director of the new Informatics Institute in the School of Medicine at the University of Alabama at Birmingham. The announcement quotes the Dean of the UAB School of Medicine stating, "Informatics is a relatively new, but incredibly important field, and its reach encompasses all aspects of medicine."

These accolades bode well not only for my colleagues but also the entire informatics field, demonstrating that despite uncertainties in federal and other sources of funding for research, there is a critical role for informatics to play in academic health science centers, where they can contribute to improving individual health, the healthcare system, and population health.

Wednesday, December 31, 2014

Annual Reflections at the End of 2014

It has become a tradition for me in this blog to post some reflections in the last posting of each year. This year is no different, and this posting is the end of 2014 installment.

Each year there has been a theme to my annual reflections. As the start of this blog was very much tied to the Health Information Technology for Clinical and Economic Health (HITECH) Act, the theme of 2009 concerned the deteriorating economy and its impact on the Oregon Health & Science University (OHSU) informatics program, the American Recovery and Reinvestment Act (ARRA), and the HITECH Act within ARRA. In 2010, I focused on the rolling out of the HITECH Act, especially the workforce development grants that were to become a major part of my work life in the following years. In 2011, I described the implementation of our HITECH workforce grants. By 2012, the beginning of the end for the HITECH funding was at hand, while in 2013, I described the transition from HITECH funding and a number of new developments, including the Informatics Discovery Lab (IDL) at OHSU and the rollout of the clinical informatics subspecialty.

What is the theme for 2014? One thing for certain is that work and life have gone on without HITECH. There were many great new accomplishments for the myself and the OHSU informatics program this past year, such as achieving Accreditation Council for Graduate Medical Education (ACGME) accreditation for our new clinical informatics fellowship that will be launched in 2015, new grants from the National Institutes of Health (NIH) Big Data to Knowledge (BD2K) program, and a new focus on competencies for medical (and other health professional) students in clinical informatics.

Despite the grants of HITECH becoming a distant memory, the impact of the HITECH Act on the informatics field cannot be understated. Of course the meaningful use program is still moving along, even if Stage 2 has been daunting and the prospect of penalties for not meeting meaningful use become a possible reality. But the informatics world is truly a different place now than before the HITECH Act. The road has been rocky, but EHR adoption has become near-universal in US hospitals and very substantial in physician offices. The fact that we are now lamenting about the problems of data and its lack of interoperability demonstrates progress in our lamenting less than a decade ago about healthcare being too paper-based. Much has been written about HITECH, often with a tinge (sometimes more) of politics thrown in. My thoughts resonate most with those who view HITECH in the context of its origins and acknowledge its success and limitations, such as Robert Wachter and John Halamka.

What lies ahead for 2015? Certainly the work described above that we have undertaken in 2014 will continue to play an important role. And like in all years, indeed in my whole career, there will be opportunities that emerge out of nowhere and turn out to be major activities.

Friday, December 12, 2014

Education in Informatics: Distinct Yet Integrative

One of the challenges we face in informatics education is how to call out its knowledge, skills, and competencies in the larger context of health and biomedicine. In other words, how do teach its important contributions while recognizing informatics does not exist in a solitary vacuum?

I see this at all levels of education in which I am involved, from that of medical and other health professional students to those training for professional careers in informatics.

One example of this is seen in medical student education. The importance of informatics in the training of physicians is finally being seen as important, yet the challenge is how to integrate appropriate informatics education into an environment where the evolution of the curriculum has been away from discrete courses to integration of all topics, typically organized into blocks and sometimes further divided into cases (i.e., case-based learning). Just as medical education no longer has standalone courses in biochemistry, pathology, physical examination and so forth, we should not aspire to have any sort of standalone informatics course either. Not only is informatics best learned in the context of solving real problems in clinical medicine, it also needs to be seen as integrated with the other subjects being learned.

The same applies to other healthcare professions. We must find ways to make informatics knowledge, skills, and competencies important, yet also integrated with their primary role as deliverers of healthcare.

Even for those training to work in informatics professionally, it is still important to understand its context. Some may be informatics professionals in clinical settings, public health settings, research settings, and even consumer-focused settings. The skilled informatician must know how to add value to those settings by best applying informatics.

This issue also plays out in one of the concerns I have for clinical informatics fellowships. As I have written before, I am troubled the idea of a standalone, one-size-fits-all, two-years-on-the-ground fellowship that is required by ACGME rules. Two additional years of fellowship is a lot to ask of physicians who do not start meaningful earning until into their 30s or later. Several of my clinical faculty colleagues at OHSU have asked why fellows cannot train simultaneously in informatics and another discipline. Not only do I not object to such integrated training, I actually believe it would be a great boon for an oncologist, cardiologist, surgeon, etc. to simultaneously train in informatics along with his or her other discipline, especially if they plan to pursue informatics in the context of that discipline.

But all this integration of informatics aside, I still strongly assert the title of this posting, which is that informatics should be distinct with its knowledge, skills, and competencies. However, its training and practice should be appropriately integrated with other health, clinical, and biomedical aspects of where it is being applied.

Wednesday, December 10, 2014

Accolades for the Informatics Professor - Fall, 2014 Update

As always, I am pleased to share periodically with readers the various accolades and mentions that colleagues, projects, and I at Oregon Health & Science University (OHSU) have received in recent months. This posting covers the mentions in the latter half of 2014.

In the late summer was a mention of my role in the American Medical Association (AMA) Accelerating Change in Medical Education Program of grants to medical schools to advance change in medical education. The OHSU grant has a component of informatics, with a focus on teaching 21st century physicians about data that they will use to facilitate their practices and others will use to assess the quality of care they deliver. One article focused on our development of competencies in clinical informatics for medical students, while the other described how we are implementing them in our AMA grant project.

OHSU also received a mention in a Web page purporting to rank the Top 25 "healthcare informatics" programs by "affordability". I am not sure exactly how they get their cost figures, but the page does accurately describe our program (number 16 on their list).

I received some other mentions concerning the new clinical informatics subspecialty, one in an article just before this year's board exam as well as in an interview with Stanford Program Director, Dr. Chris Longhurst.

Of course, the new subspecialty is one of many changes that informatics education has undergone recently, as noted both in an article I wrote as well as in one where I was interviewed.

I gave a number of talks that were recorded this fall, including my kick-off of our weekly OHSU informatics conference series as well as a talk about our Informatics Discovery Lab at the 2nd Annual Ignite Health event in Portland. The latter has an interesting format of five minutes to talk with slides that automatically advance every 15 seconds (for a total of 20 slides). The talk on the IDL led to my being invited to moderate a panel on business opportunities in health information technology in Portland.

There was also some press around the new National Institutes of Health (NIH) Big Data to Knowledge (BD2K) grants we received. Related to Big Data, another magazine called out my blog posting from last year that data scientists must also understand general research methodology.

Another news item mentioned a project I am likely to write about more in the future that concerns OHSU establishing collaborations in informatics and other areas in Thailand.

Finally, a few accolades came from events of the AMIA Annual Symposium 2014. One was getting my picture in HISTalk in a mention of the Fun Run at this year's symposium. I was also interviewed by a reporter who wanted to follow up on why I selected them items that I did for my top ten events of the year in my Year in Review talk. It was nice to be able to elaborate some and also watch the tweeting that followed.

It is gratifying to receive these accolades and of course I know have to keep doing innovative and important work to maintain them.

Sunday, November 30, 2014

Ten Years of 10x10 ("Ten by Ten")

The completion of the most recent offering of the 10x10 ("the by ten") course at this year's American Medical Informatics Association (AMIA) 2014 Annual Symposium marks ten years of existence of the course. Looking back to its inauspicious start in the fall of 2005, the 10x10 program has been a great success and remains a significant part of my work life. It has not only cemented for my passion and love for teaching, but also gives me great motivation to keep up-to-date broadly across the entire informatics field.

For those who are unfamiliar with the 10x10 course, it is a repackaging of the introductory course in the OHSU Biomedical Informatics Graduate Program. This is the course taken by all students who enter the clinical informatics track of the OHSU program and aims to provide a broad overview of the field and its language. The course has no prerequisites, and does not assume any prior knowledge of healthcare, computing, or other topics. The course has ten units of material, with the graduate course spread over ten weeks and the 10x10 version decompressed to 14 weeks. The 10x10 course also features an in-person session at the end to bring participants together to interact and present project work. (The in-person session is optional for those who might have a hardship in traveling to it.)

The AMIA 10x10 program was launched in 2005 when AMIA wanted to explore online educational offerings. When the cost for development of new materials was found to be prohibitive, I presented a proposal to the AMIA Board of Directors for adapting the introductory online course I had been teaching at Oregon Health & Science University (OHSU) since 1999. Since then-President of AMIA Dr. Charles Safran was calling for one physician and one nurse in each of the 6000 US hospitals to be trained in informatics, I proposed the name 10x10, standing for "10,000 trained by 2010." We all agreed that the course would be mutually non-exclusive, i.e., other universities could offer 10x10 courses while OHSU could continue to employ the course content in other venues.

The OHSU course has, however, been the flagship course of the 10x10 program, and by the end of 2010, a total of 999 had completed it. We did not reach anywhere near that vaunted number of 10,000 by 2010, although probably could have had that many people come forward, since distance learning is very scalable. After 2010 the course continued to be popular and in demand, so we continued to offer "10x10" and have done so to the present time.

This year now marks the tenth year that the course has been offered, and some 1837 people have completed the OHSU offering of 10x10. This includes not only general offerings with AMIA, but those delivered to various partners, including the American College of Emergency Physicians, the Academy of Nutrition and Dietetics, the Mayo Clinic, the Centers for Disease Control and Prevention, the New York State Academy of Family Physicians, and others. The course has also had international appeal as well, with it being translated and then adapted to Latin America by colleagues at Hospital Italiano of Buenos Aires in Argentina as well as being offered in its English version, with some local content and perspective added, in collaboration with Gateway Consulting in Singapore. Additional international offerings have been sponsored by King Saud University of Saudi Arabia and the Israeli Ministry of Health.

All told, the OHSU offering of the 10x10 program has accounted for 76% of the 2406 people who completed various other 10x10 courses. The chart below shows the distribution of the institutions offering English versions of the course.

The 10x10 course has also been good for our informatics educational program at OHSU. As the course is a replication of our introductory course in our graduate program (BMI 510 - Introduction to Biomedical and Health Informatics), those completing the OHSU 10x10 course can optionally take the final exam for BMI 510 and then be eligible for graduate credit at OHSU (if they are eligible for graduate study, i.e., have a bachelor's degree). About half of the people completing the course have taken and passed the final exam, with about half of them (25% of total) enrolling in either our Graduate Certificate or Master of Biomedical Informatics program. Because our graduate program has a "building block" structure, where what is done at lower levels can be applied upward, we have had one individual who even started in the 10x10 course and progressed all the way to obtain a Doctor of Philosophy (PhD) from our program.

As I said at the end of the 2010, the 10x10 program will continue as long as there is interest from individuals who want to take it. Given the continued need for individuals with expertise in informatics, along with rewarding careers for them to pursue in the field, I suspect the course will continue for a long time.

Friday, November 21, 2014

The Year in Review of Biomedical and Health Informatics - 2014

At this year's American Medical Informatics Association (AMIA) 2014 Annual Symposium, I was honored to be asked, along with fellow Oregon Health & Science University (OHSU) faculty member Dr. Joan Ash, to deliver one of the annual Year in Review sessions.

This session was first delivered in 2006 by Dr. Dan Masys, who presented an annual review of the past year's research publications and major events each year. Over time, parts of the annual review were broken off and focused on specific topics. The first of these were the annual reviews in translational bioinformatics (Dr. Russ Altman) and clinical research informatics (Dr. Peter Embi, an OHSU alumnus), presented at the annual AMIA Joint Summits on Translational Science. This year additional topics were peeled off, such as Informatics in the Media Year in Review (Dr. Danny Sands) as well as Public and Global Health Informatics Year in Review (Dr. Brian Dixon, Dr. Jamie Pina, Dr. Janise Richards, Dr. Hadi Kharrazi, and OHSU alumnus Dr. Anne Turner).

This pretty much left clinical informatics as the major topic for Joan and I to cover. However, I had also noted that this separating out of specific aspects of informatics left no one covering the fundamentals of informatics, i.e., topics underlying and germane to all aspects of informatics. We also noted that qualitative and mixed methods research had also been historically underrepresented in these annual reviews. Therefore, Joan and I set the scope of our Year in Review session to clinical informatics and foundations of biomedical and health informatics. For research that was evaluative, Joan would cover qualitative and mixed methods studies, while I would cover studies using predominantly quantitative methods studies.

We also believed that while Dan's methods for gathering publications was sound, different approaches worked better for us. For myself in particular, I decided to plug the annual review process into my existing workflow of uncovering important science and events in the field, which I spend a good amount of time doing in order to keep my introductory (10x10 and OHSU) course up to date. I comprehensively scan the literature as well as the news on a continuous basis to keep my teaching materials (and knowledge!) up to date. I actually created a slide in the presentation to show my normal workflow "methods," which informed my review and is shown below.

Our first annual review was presented at the AMIA 2014 Annual Symposium on November 18, 2014. Continuing Dan's tradition, we created a Web page that has a description of our goals and methods, a link to our slides, and all of the articles cited in our presentation. We also kept the traditional time frame for the "year" in review, which was from October 1, 2013 to September 30, 2014. One additional feature of the session that we added was to offer up the last 15 minutes for attendees to make their own nominations for publications or events to be included.

Joan and I were pleased with how the session went, and we were gratified by the positive response from attendees. We are hopeful to be invited back to present the session again next year!

Saturday, November 15, 2014

Continued Concerns for Building Capacity for the Clinical Informatics Subspecialty - 2014 Update

The first couple years of the clinical informatics subspecialty have been a great success. Last year, about 450 physicians became board-certified after the first certification exam, with many aided by the American Medical Informatics Association (AMIA) Clinical Informatics Board Review Course (CIBRC) that I directed. This year, another cohort took the exam, with many helped by the CIBRC course again. In addition this year, the Accreditation Council for Graduate Medical Education (ACGME) released its initial accreditation guidelines, and four programs (including ours at Oregon Health & Science University [OHSU]) became accredited, with a number of other programs in the process of applying.

Despite these initial positive outcomes, I and others still have many concerns for how we will build appropriate capacity in the new subspecialty. In particular, many of us are concerned that the number of newly certified subspecialists will slow to a trickle after 2018, once the "grandfathering" pathway is no longer available and the only route to certification will be through a two-year, on-site, full-time clinical fellowship. Indeed, the singular bit of advice I give to any physician who is currently "practicing" clinical informatics is to do whatever they can to get certified prior to 2018. It will be much more difficult to become certified after that, since the only pathway will be an ACGME-accredited fellowship.

I previously raised concerns about these challenges in postings last year and the year before, and this one represents an update leading into the annual AMIA Symposium. Colleague Chris Longhurst, whose fellowship program was the first to achieve accreditation, has expressed similar concerns in interviews by CMIO Magazine and HISTalk.

Looking forward, I see four major problems for the subspecialty. I will address each of these and then (since I am a solutions-oriented person) propose what I believe would be a better approach to the subspecialty.

The subspecialty excludes many physicians who do not have a primary specialty

When the AMIA leadership starting development a proposal for professional recognition of physicians in clinical informatics around 2006, they were advised that creating a new primary specialty would be a lot more difficult to sell politically and instead to advised to propose a new subspecialty. This would be unique as a subspecialty of all medical specialties. I am sure that advice was correct, but we have unfortunately excluded those who never obtained a primary clinical specialty or whose specialty certification has lapsed. These individuals can still be highly capable informaticians, and in fact many are. The alternate AMIA Advanced Interprofessional Informatics Certification being developed may serve these physicians, but it would be much better as a profession to have all physicians under a single certification.

The clinical fellowship model will exclude from training the many physicians who gravitate into informatics well after their initial training

The majority of physicians who work in clinical informatics did not start their careers in the field. Many gravitated into the field long after they completed their initial medical training, took a job, and established geographic roots and families. The distance learning graduate programs offered by OHSU and other universities have been a boon to these individuals, as they can train in informatics while keeping their current jobs and not needing to uproot their families. Many of these individuals have great experience, and many passed the initial board exam. They are clearly capable.

After 2018, the "grandfathering" pathway will no longer be an option, and the only way to achieve board certification will be via a full-time two-year fellowship. It is interesting to note the recent advice I heard expressed by Dr. Robert Wah, President of the American Medical Association. He noted that many physicians have moved beyond direct clinical care to have an impact in medicine in other ways. But he advised that every physician should establish their clinical career first and then move on to other pursuits. This too is at odds with the clinical fellowship model that almost by necessity must come during one's primary medical training.

In a similar vein, a number of colleagues who are subspecialists in other fields of medicine express concern that a clinical informatics subspecialty fellowship would add an additional two years of training on to the already lengthy training required of most highly specialized physicians. As much as I am an advocate of formal informatics training, I also recognize, and would even encourage, such training being integrated with other clinical training, especially in subspecialties.

The clinical fellowship model also is not the most appropriate way to train clinical informaticians

Even for those who are able to complete clinical informatics fellowships, the classic clinical fellowship training model is problematic, as those of us applying to ACGME have learned. I likened this process a few months ago to fitting square pegs into round holes.

Clinical medicine is very well suited to episodic learning and hence rotations. A patient comes in, and their current presentation is a nice segue into learning about the diseases they have, the treatments they are being given, and the course of their disease(s). Even patients being followed longitudinally in a continuity clinic have episodes of care with the healthcare system that provide good learning.

But informatics is a different kind of topic. Informatics is not an activity that takes place in episodes. You can't really learn from episodic exposure to it. Good informatics projects, such as a clinical decision support implementation or a quality improvement initiative, take place over time. In fact, learning is compromised when you jump in and/or leave in the middle. Informatics projects are also carried out by teams of people with diverse skills with whom the informatician must work. I would assert that better learning takes place when the informatics trainee encounters specific informatics issues (standards, security, change management, etc.) in the context of long-term projects.

There are other concerns that have arisen about various aspects of the ACGME accreditation progress. One program was declined accreditation because a program director was not in the same primary specialty as the Residency Review Committee (RRC), despite the fact that clinical informatics is supposed to span all specialties. ACGME also requires any fellowship program, no matter how small, to have 2.0 FTE of combined director and faculty time. This may make sense in a clinical setting where faculty are simultaneously engaged in care of the same patient, but does not fit well when a trainee is working on single aspects of a larger project. Another ACGME requirement is for all faculty who teach to be named, and for those who are named to be board-certified clinical informaticians. This again does not make sense in the context of informatics being an activity with participants from many disciplines outside of medicine, some even outside of healthcare. Finally, ACGME requires fellows to be paid. This is easier to do when fellows are actively involved in the clinical operations of the hospital. Even if these trainees cannot bill, they do make it easier for attending physicians and hospitals to bill.

The funding model for fellowships creates challenges for their sustainability

A final challenge for clinical informatics fellowships is their funding and sustainability. Most subspecialty training in the US is funded by academic hospitals, and part of the "grand bargain" of such training is that clinical trainees provide inexpensive labor, which "extends" the ability of their teachers to provide care. The various clinical units have incentive to do this because it increases the ability of the units to provide and bill for services. Clinical informatics is different in that fellows will be unlikely to provide direct capacity benefit to academic clinical informatics departments. Our department at OHSU, for example, does not have operational clinical IT responsibilities.

Furthermore, these fellows will be doing their clinical practice in their primary specialties, and not their clinical informatics subspecialty. The primary specialties will include the full gamut of medical specialties such as internal medicine, radiology, pathology, and others. Even if fellows will be able to bill, it will be challenging within organizations for units to divide up the revenues.


In last year's post I proposed a solution addressing last year's description of these problems, and what follows is an updated version. There are approaches that could be rigorous enough to ensure an equally if not more robust educational and training experience than the proposed fellowship model. It would no doubt test the boundaries of a tradition-bound organization like ACGME but could also show innovation reflective (and indeed required) of modern medical training generally.

A first solution is to provide a pathway for any physician to become certified in clinical informatics, whether having a primary board certification or not. Informatics as a subspecialty of any medical specialty is a contortion. I do not buy that one cannot be a successful clinical informatician without having a primary board certification. I and likely everyone else in the field know of too many counter-examples to that.

Moving on to specifics of training, last year I noted that there should be three basic activities of clinical informatics subspecialty trainees:
  • Clinical informatics education to master the core knowledge of the field
  • Clinical informatics project work to gain skills and practical experience
  • Clinical practice to maintain their skills in their primary medical specialty
How can we insure that these three general areas are met without requiring a trainee to be in a fixed location or to train over a fixed period of time? The first activity area is easy. There is no question any more that essentially any learning experience can be delivered using online learning. There are many educational programs in informatics, including ours at OHSU, that have a strong track record in successfully imparting knowledge to students remotely.

Next, how would trainees get their practical hands-on project work? Again, many informatics programs, certainly ours, have developed mechanisms by which students can do internships or practicums in remote location through a combination of affiliation agreements, local mentoring, and remote supervision. While our program currently has students performing 3-6 months at a time of these, I see no reason why the practical experience could not be expanded to a year or longer. Strict guidelines for experience and both local and remote mentoring could be put in place to insure quality.

Lastly, what about clinical practice? As noted above, I disagree that this should even be a requirement. But if it were, requiring a trainee to perform a certain volume of clinical practice, while adhering to all appropriate requirements for licensure and maintenance of certification, should be more than adequate to insure practice in their primary specialty. Many informatics distance learning students are already maintaining their clinical practices to maintain their livelihood. Making clinical practice explicit, instead of as something requiring supervision, will also allow training to be more financially viable for the fellow. Any costs of tuition and practical work could easily be offset by clinical practice revenue.

There would need to be some sort of national infrastructure to set standards and monitor progress of clinical informatics trainees. There are any number of organizations that could perform this task, such as AMIA, and it could perhaps be a requirement of accreditation. Indeed, ACGME and the larger medical education community may learn from alternative approaches like this for training in other specialties. One major national concern these days is that number of residency positions for medical school graduates is not keeping up with the increases of medical school enrollment or, for that matter, the national need for physicians. It is possible that alternative approaches like this could expand the capacity of all medical specialties and subspecialties, and not just clinical informatics.

Wednesday, November 12, 2014

Ebola is a Reason for Implementing ICD-10, or is it? What is the Role of Coding in the Modern EHR Era?

The recent Ebola outbreak has been used to justify or advocate many things. Among them is further advocacy for the transition to the ICD-10 coding system. However, when discussing Ebola and coding, it also gives us a chance to pause and address some larger issues around coding in the modern era of the electronic health record (EHR).

Coding of medical records is a requirement for billing, i.e., diagnosis codes must be included on a claim to obtain reimbursement for services from an insurer, whether a private insurance company or the government. The coding system currently used in the United States is ICD-9-CM, but its replacement with ICD-10-CM has been mandated, although the deadline has been postponed three times over the last four years. This coding also potentially creates a vast source of data for research, surveillance, and other purposes. Indeed, there is a whole body of research based on such "claims data," with one of the arguments for its use being that what this data lacks in depth or completeness is made up for by its volume [1].

ICD-9-CM has many limitations as a coding system. Probably its biggest limitation is that many codes cover a whole swath of diseases. In addition, its "not otherwise specified" may change over time when one of the components of not being otherwise specified becomes specified.

What does this have to do with Ebola? In ICD-9-CM, Ebola is one of many diagnoses covered by the code, 078.89 - Other specified diseases due to viruses. There are about 35 viruses that map to this code, including some common ones such as coronavirus and rotavirus. ICD-10-CM, on the other hand, has a specific code, A98.4 - Ebola virus disease.

Does this provide justification for the move to ICD-10-CM? ICD-10-CM is clearly more detailed and granular, and in fact may be excessively granular. Another concern about implementing ICD-10-CM is the cost to physicians and hospitals, which are mostly unknown although estimates vary widely [2, 3]. There is no question that ICD-9-CM falls short and that ICD-10-CM does have a specific code in the case of Ebola. But is this itself a reason justifying the move to ICD-10-CM, or are there other ways to determine from a medical record whether a diagnosis has been made?

When controversial questions arise, I always find it useful to step back and ask some questions, such as what we are trying to accomplish and whether it is the best way for doing so? There is actually a body of scientific literature that has assessed the consistency and value of coding medical records. One systematic review of United Kingdom coding studies found that coding accuracy in the UK varied widely, with a mean accuracy of 80.3% for diagnoses and 84.2% for procedures [4]. The range of accuracy, however, was from 50.5-97.8%. Another systematic review looked at heart failure diagnoses in Canadian hospitals, finding both ICD-9 and ICD-10 coding to vary widely in sensitivity of actual diagnosis (29-89%) and kappa scores of inter-assigner agreement (0.39-0.84) [5]. A US-based systematic review of identifying heart failure with diagnostic coding data found positive predictive value to be reasonably high (87-100%) but sensitivity to be lower [6]. Studies of diagnosis codes for hypertension [7] and obesity [8] found low sensitivity but higher specificity.

While these studies show that coding data is imperfect, there was a time when the predominance of paper medical records meant there was no alternative to data that could be analyzed. However, we are now in an era of widespread EHR adoption, which means that there are other sources of data to document diagnoses, testing, and treatments. In the case of Ebola, we have many other possible sources of data, as described by the Centers for Disease Control.

While there is certainly no evidence that our entire medical record coding enterprise should be immediately abandoned, there is definitely a case to reassess its necessity and value in the modern EHR era. This is especially the case when we are using EHR data for so many other purposes [9]. As with many questions, dispassionate science and analysis is the best approach to providing us with answers.


1. Ferver, K, Burton, B, et al. (2009). The use of claims data in healthcare research. The Open Public Health Journal. 2: 11-24.
2. Hartley, C and Nachimson, S (2014). The Cost of Implementing ICD‐10 for Physician Practices – Updating the 2008 Nachimson Advisors Study. Baltimore, MD, Nachimson Advisors, LLC. http://www.ama-assn.org/resources/doc/washington/icd-10-costs-for-physician-practices-study.pdf.
3. Kravis, TC, Belley, S, et al. (2014). Cost of converting small physician offices to ICD-10 much lower than previously reported. Journal of AHIMA, http://journal.ahima.org/wp-content/uploads/Week-3_PDFpost.FINAL-Estimating-the-Cost-of-Conversion-to-ICD-10_-Nov-12.pdf.
4. Burns, EM, Rigby, E, et al. (2012). Systematic review of discharge coding accuracy. Journal of Public Health. 34: 138-148.
5. Quach, S, Blais, C, et al. (2010). Administrative data have high variation in validity for recording heart failure. Canadian Journal of Cardiology. 26: e306-e312.
6. Saczynski, JS, Andrade, SE, et al. (2012). A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiology and Drug Safety. 21: 129-140.
7. Tessier-Sherman, B, Galusha, D, et al. (2013). Further validation that claims data are a useful tool for epidemiologic research on hypertension. BMC Public Health. 13: 51. http://www.biomedcentral.com/1471-2458/13/51.
8. Lloyd, JT, Blackwell, SA, et al. (2014). Validity of a claims-based diagnosis of obesity among medicare beneficiaries. Evaluation and the Health Professions. Epub ahead of print.
9. Hersh, WR (2014). Healthcare Data Analytics. In Health Informatics: Practical Guide for Healthcare and Information Technology Professionals, Sixth Edition. R. Hoyt and A. Yoshihashi. Pensacola, FL, Lulu.com: 62-75.