CATCH-IT Final Report: The Relationship between Electronic Health Record Use and Quality of Care over Time

Abstract and initial discussion

CATCH-IT Draft Report

Presentation on Slideshare

Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, Simon, SR. Relationship between EHR use and Quality of Care over Time. J Am Med Inform Assoc. Jul-Aug 2009;16:457– 464.

Introduction

The paper chosen for this particular CATCH-IT report is entitled, The Relationship between Electronic Health Record Use and Quality of Care over Time. It was written by the research team of Zhou, Soran, Jenter, Volk, Orav, Bates and Simon. This team consisted of the leading researchers in electronic health records (EHR) from Columbia University and Harvard University. This paper can be considered a high quality paper because this group has conducted what little research there has been in this field. The results of this paper may have a potential impact on the decision of hospital administrators and clinicians to use EHRs in their respective practice settings. It was also picked because electronic health records are currently a very timely issue on the political agenda of many Western countries.

This paper would be interesting for a number of stakeholders. It would interest researchers because it suggests that further research is required. This would enable future work for researchers in the field of quality and EHRs. There are also the quality measurement criteria that could be improved. It may also appeal to researchers because there is a potential for other quality of care data sets to be used to compare against individual clinician EHR usage. It would be of interest to politicians and other policy makers, such as lobbyists, researchers, ministries of health, etc. It may be especially useful to the critics of eHealth because the conclusions drawn state that the use of EHR is not associated with improved healthcare outcomes. This can be used as an argument for government excess or spending. Other policy makers may be able to conclude that there is no inherent value in EHRs. This would affect the policies for the particular jurisdiction these policy makers reside. The media may also use the results of this study to show that there may not be any inherent value in having an EHR at all. However, since this paper only discusses the quality aspect of care, this may not be as much of a disincentive to stop the investment in EHRs. There are other aspects to an EHR which may be of more importance such as reduction of paper, improved process flows and greater physician communication, to name a few.

Discussion

Some of the background information drawn for this paper is the methodology for the clinician EHR usage survey. To obtain this (and the survey) one would have to check the reference list and contact the author of that particular study which only measured physician EHR usage. It was noted upon reading the reference paper by Simon et al. that there were many physicians who had moved, were deceased or were no longer practicing medicine who received the survey.

This article studied the use of EHR and the quality of care given by clinicians in various practices settings. The study employed the use of two data sets. It is of interest to note that both data sets used are secondary data and were not created for the purpose intended by the study authors. One survey measured physicians’ adoption and use of EHR in the state of Massachusetts. It used an eight page questionnaire with a stratified random sample, i.e., hospital-based vs. large vs. rural practices, by specialty, etc. Sample weights were used to adjust for overall representation. For example, if there was a total of 70 physicians, and only 50 responded, the authors would perform an adjustment calculation, i.e., 50/70 multiplied by each answer to represent the entire population. The second data set was statewide data on physicians’ quality of care as indicated by their performance on widely used quality measures. The secondary data set was drawn from a HEDIS database. A HEDIS database is populated using insurance claims data. There were certain exclusion criteria set for the HEDIS data which made respondents younger, more recently graduated, in smaller practices with more patient visits and more likely to be female. These two data sets were integrated and linked for each physician. The outcome measures used were core EHR functions and associated features (i.e., health information and data, result management, order entry and management, decision support, and electronic communication and connectivity) as well as the HEDIS quality measuses which were aggregated into six clinical categories (i.e., asthma care, behavioural and mental health, cancer screening, diabetes care, well child and adolescent visit, and women’s health). Also, the use of a feature such as decision support in this type of measure is misleading. The extent of decision support varies as well as the functionality as this feature has progressed from 2001 through 2005.

As mentioned, this study examined the cross-sectional relationship between having an EHR and concurrent indicators of quality of care. The authors carried out a longitudinal analysis to study the trend of EHR adoption and usage as well as to examine the association between the duration of EHR usage and quality of care. They compared the HEDIS quality measures of respondents to the length of time using an EHR. The results show that EHR adoption increased over time. For the purposes of this study, adoption was equated to the availability of an EHR in a particular practice. This may not be the case. There can be physician who have an EHR available yet choose not to use it. The results also showed that the availability and use of EHR core functions increased. The usage of specific features was measured and presented in the paper as used most of the time, some of the time or none of the time. There was no statistical difference between EHR users and non-users. Finally, and most importantly, usage of an EHR was not related to increased quality of care.

There were only two references cited for relationship of EHR use and quality of care. This would seem to indicate a need for more research to be conducted in this field. However, the difficulties with conducting this type of research have already been explained. The implications for policy makers would be that there may be a need to pay more for higher quality care. This may ensure that physicians are using EHRs to their full capacity. This can be achieved through education and workflow transformation. There may need to be stronger incentives or more extensive programs to support physician office transformation. One strong argument the authors suggest is that The 2005 EHR adoption rate will need to triple in less than a decade (for MA) for 100% physician EHR usage. This represents a challenge for health care policy.

Questions to the authors:

Would it be possible to use a different data set in order to measure quality of care? Do you believe that lack of effect on HEDIS data represents a lack of impact on quality?

Would/did you consider creating your own survey instrument for this study rather than secondary data?

Would the use of an insurance claim dataset automatically exclude the care of those without health insurance? If so, how would you account for this?

How were the adjusted calculations performed for the quality of care measures?

Is there any possible way the HEDIS data set could have been used for longitudinal analysis of EHR adoption/use and quality of care?

Why was the availability of a system equated to adoption whereas, usage was separately measured?

Conclusion

The results are of this paper are valid. This paper addressed the research question it proposed from the outset. The data used shows that there is no clear connection between EHR usage and quality of care. However, it should be noted that the results are shown with adjusted values. It should be recognized that it is difficult to conduct these types of studies. There are measurement issues, one cannot prove correlation can be equated to causation and there are adjustments for variables. Upon initial implementation, health care quality cannot be positively associated with the systems which are supposed to improve it. It is much easier to document negative outcomes. It is very difficult to measure “quality of care” and as such, the attempt by these authors was a good one. In order to operationalize this variable, the researchers made a valiant attempt to create a measure which would be both useful and meaningful for readers. It is a difficult concept in which to prove an increase, or even to quantify, and this paper shows just how difficult it can be.