An “Epidemic” of Medical Malpractice? A Commentary on the Harvard Medical Practice Study

Richard E. Anderson, MD, FACP, is chairman and CEO of The Doctors Company.

Medical malpractice litigation is one of the largest components of the tort system. It is widely held responsible in part for the high cost of healthcare both because its expense is passed along directly in insurance rates and because it compels physicians to practice costly defensive medicine. Its costs appear to be on the rise: settlements and judgments against New York City alone for alleged malpractice in its municipal hospitals have increased from $20.6 million in 1984 to nearly $80 million in 1994. The Tillinghast-Towers Perrin surveys have estimated the cost of malpractice insurance to have mounted from $8.5 billion in 1990 to $12.7 billion, faster than the tort system as a whole.

Is there an “epidemic” of malpractice that explains these high outlays? Some claim there is, and for evidence they point to what is known as the Harvard Medical Practice Study. Both advocates and critics of malpractice reform have cited the study since its initial release in 1991. It has figured prominently in congressional hearings and is the basis for two large grants from the Robert Wood Johnson Foundation for new studies in Colorado and Utah.

In this month’s Civil Justice Memo we offer one critique of the Harvard study. Richard E. Anderson, MD, FACP, is a specialist in medical oncology, a clinical professor of medicine at the University of California San Diego, and chairman of the board of a national medical malpractice insurer, The Doctors Company.

The Harvard Medical Practice Study was first published in 1991 and was based on 1984 case records. The study attempts to measure the extent of medical malpractice in hospitals in the state of New York, and compare the resulting patterns with the negligence claims actually filed. The researchers have subsequently written a number of articles and a book, and popular discussion of “the Harvard study” has come to refer to these collective works.^{1, 2, 3, 4, 5, 6}

Both supporters and opponents of malpractice legal reform can agree that the Harvard study provides us with a wealth of valuable data on questions of medical negligence. More than with most studies, however, the Harvard group’s results are subject to sharply different interpretations and depend heavily on definitions and assumptions with which many will differ. Unfortunately, the study tended to gloss over these differences, and by the time the “Harvard findings” made their way into the hands of organized advocates of the tort system, they were badly distorted and sometimes patently false.

The principal conclusions of the Harvard study are: 1) malpractice was frequent in the hospitals studied, 2) a high toll of avoidable medical injury nationwide can be inferred from this frequency rate; and 3) relatively few of the injured patients actually sued. From this it is deduced that the “real problem” is not that there is too much malpractice litigation but that there is too little, and that the admittedly high rate of meritless suits against physicians is preferable to reform of the tort system. Consumers Union, which has worked closely with anti-tort-reform efforts in Washington, has claimed that the Harvard study proves doctors “kill” 80,000 patients a year.⁸

In fact, each of these purported findings is open to serious question:

• Counting incidents of malpractice in a hospital file is not a simple matter. The Harvard researchers were willing to label episodes as negligence or as adverse events in cases where their own reviewers disagreed on the question, and disagreements occurred frequently. Ultimately they employed a relaxed “more likely than not” standard, apparently borrowed from the legal context, which does not adequately account for the uncertainty inherent in measuring the impact of differences in medical care on the prognosis of complex illness.
• The study lumped together adverse events both grave and minor, whether caused by doctors or simply occurring anywhere in a hospital; a slip and fall in a corridor was grouped with surgical error. The result was to obtain a high number of adverse events, which were then categorized as potentially negligent by merely averaging the assessments of each reviewer. The majority of these events either resulted in transient injuries (full recovery within a month) or occurred during the final stages of life in which therapeutic misadventures may subtract only hours or days from the life of a dying patient. For very good reasons, families seldom seek legal compensation in either of these situations, and it would be difficult to imagine a workable system in which they were encouraged to do so.
• The study is based exclusively on a sample of hospital inpatients. By definition, they are the most seriously ill and the interaction of therapy and disease will inevitably produce a significant fraction of adverse events.
• The researchers have de-emphasized various considerations that might raise questions about their portrait of widespread iatrogenic (medically caused) injuries. Among them are the likelihood that some failure to provide patients with aggressive treatment reflected the patients’ own wishes, and the paradoxical finding that university teaching hospitals, widely considered our best, had higher rates of adverse events.
• Perhaps most remarkable of all, the researchers admit they could not find negligence in most cases where legal claims were actually filed—casting serious doubt on the rationality of the current tort system, and strongly supporting physicians’ belief that many doctors are wrongly sued. In reality, the study’s findings make an excellent case for tort reform. However, the Harvard researchers claim that the study “proves” that lawmakers should not enact reforms intended to shield innocent doctors from these unfounded suits or take steps to cool the lottery mentality that encourages filing of such an extraordinary percentage of meritless claims.

Summary Description of the Study

From the 2,671,863 nonpsychiatric patients discharged from nonfederal acute care hospitals in New York in 1984, a weighted sample of 31,429 records was selected for review. Trained nurses and medical records analysts screened the records for evidence of at least one of eighteen criteria that would suggest a possible adverse event. The definition of an “adverse event” was “an injury that was caused by medical management (rather than the underlying disease) and that prolongs the hospitalization, produced a disability at the time of discharge, or both.”¹ “Negligence” was defined as “care that fell below the standard expected of physicians in their community.”¹ Records that met any one of the screening criteria were then referred to two physicians who separately evaluated the cause of injury. The physicians were specially trained for this project but were not necessarily specialists in the area of care they were asked to review. To be classified as a negligent adverse event, it was merely necessary for the average of the two physicians’ evaluations to represent a judgment of at least “more likely than not.”²

The statewide incidence of adverse events was estimated at 3.7 percent and the rate of adverse events due to negligence was estimated at 1.0 percent. ¹Thus, the percentage of all adverse events that was attributed to negligence was 27 percent.¹

These numbers were derived as follows: 31,429 charts were identified in the original sample. Of the 7,817 that met screening criteria, 7,743 were reviewed by physicians. They found 1,278 adverse events and attributed 306 of those to negligence, 280 of which formed the basis for this study.¹

Over half (56 percent) of these adverse events resulted in “minor impairment with complete recovery in one month.” ¹ Nearly 14 percent led to disabilities that lasted between one and six months, 2.6 percent produced permanent disability, and 13.6 percent caused death.¹

The incidence of adverse events increased with patients’ age. Nearly 48 percent of the adverse events were surgical complications, but only 17 percent of these were judged to be secondary to negligence.² Rates of adverse events and negligence varied widely among specialties. Obstetrics had the lowest adverse event rate of 1.5 percent, but 38.3 percent of these were judged negligent. Vascular surgery had the highest rate of adverse events (16.1 percent) and 18.0 percent of these were judged negligent.

At least one of the two physician reviewers identified management errors in 58 percent of all the adverse events, but only 28 percent of the events ultimately met study criteria for negligence.³ “Errors of omission” were the highest percentage of medical error. Included in this category were failure to prevent accidental injury or to use appropriate tests, avoidable delays in treatment, and failed diagnoses.

There was marked variation among individual hospitals with adverse event rates ranging from 0.2 percent to 7.9 percent.⁴ The percentage of these adverse events judged due to negligence ranged from 1 percent to 60 percent.⁴ University teaching hospitals had an adverse event rate of 4.1 percent, nearly double that of nonteaching hospitals—2.3 percent. Only 10.7 percent of the adverse events at university hospitals were ascribed to negligence, however, versus 26.9 percent in the nonteaching hospitals.⁴

Using a weighting procedure, these numbers were extrapolated to the 2,671,863 patients discharged from acute care hospitals in New York in 1984 to hypothesize 98,609 adverse events, and 27,179 adverse events due to negligence.¹

Forty-seven actual malpractice claims were found to have arisen from the records reviewed.² The physician reviewers found no evidence of any medical injury, negligent or not, in 26 of the 47 claims. In only 8 of the 47 did they find evidence that medical malpractice had caused such an injury. However, 40 percent of the cases where they found no evidence of negligence nonetheless resulted in indemnity payment.

Study Findings

In their examination of the role of the legal system, the researchers noted two striking findings: most persons with potentially legitimate claims appeared not to file them, but most claims that were filed had no evident basis. They make much of the first finding, while downplaying the second. The researchers proclaim that “the chances that a claim would be filed by a patient with an identifiable negligent injury is only one in fifty.”⁴

They view this failure as a serious “gap,” a “tort gap,” which public policy should remedy. Remarkably, given their own finding that fewer than 20 percent of malpractice claims actually filed had an identifiable basis in medical negligence, they assert that “the underlying assumption that too many groundless malpractice suits are initiated is unfounded.”⁴ And, “...not too many, but rather too few suits were brought for the negligent injuries inflicted on patients.”⁴

They excuse the fact that most filed claims were unfounded by saying that the legal system “appears to do a surprisingly accurate job of sifting out the valid from the invalid claims (in the latter cases paying the claimants nothing or just a small amount of damages).” Still, they concede that “many innocent doctors are subjected to unwarranted tort suits.” Although a general increase in claim-filing might be expected to bring more unfounded as well as more well-founded suits, the researchers are sure this would be a price worth paying: “the focus of legislative concern should be that the malpractice system is too inaccessible, rather than too accessible, to the victims of negligent medical treatment.”⁴

The researchers’ central proposal is for something that does not currently exist in any state: a government-mandated scheme that would provide patients with no-fault compensation for medical “adverse events.” (The Harvard study has become the basis for proposed demonstration projects of this idea in Colorado and Utah.) The aim would be to compensate for a considerably broader range of harms than the current legal system, but at more modest financial levels. In particular, the scheme would provide payment for financial losses (not pain and suffering) occurring after a six-month deductible period, and only for losses not otherwise compensated. Only causation and not negligence would have to be shown, and patients would agree to waive their rights under traditional tort liability theories. The institution with which the individual physician is affiliated legally and financially would be held responsible for all injuries.

Notably, their plan would incorporate two major tort reforms, eliminating damages for pain and suffering and disallowing double recovery for expenses already covered by other forms of insurance and benefits (“collateral sources”). Despite these retrenchments, a no-fault scheme could be expected to result in larger outlays than today’s tort system. Once negligence did not have to be proved, the number of claims would likely rise rapidly, especially since the prospect of compensation would encourage speculative filing of cases where causation itself was uncertain.

The researchers offer scant discussion of these likely effects. They do, however, conclude that in the meantime “legal measures” should not be adopted “that could swell the already extensive ranks of deserving patients with serious bodily injuries who receive no financial redress from the legal system.”⁴ If this is a reference to tort reform, it is a highly disputable one, since few if any tort reform proposals would in fact deny financial redress to such patients. In fact two of the most common reform ideas—placing limits on currently unlimited pain-and-suffering damages and collateral-source reform—turn up as key elements in the study’s own proposal.

Discussion

The problems with the study begin with its very broad definition of its most fundamental concept, the negligent adverse event. To count such an event, the study required only that the average of two reviewers’ assessments rank it as more likely than not that an injury was caused by substandard care. The frequency of disagreement between reviewers may be judged by the study’s striking admission that in an astounding 58 percent of all adverse events at least one of the physician reviewers purported to identify medical negligence, but in the study’s final conclusion only 27 percent of adverse events were called negligent.

One contributing factor in this high rate of disagreement may have been the reviewers’ unfamiliarity with practice specialties. The study made no apparent attempt to match reviewers’ specialties to those of the medical problems at issue. Although a panel of specialists was said to be available to answer questions, the actual determinations were made by the general physician reviewers. This system of review would be considered unacceptable under contemporary standards of either medical or legal analysis of treatment decisions.

Even given all their own assumptions, the authors were unable to reproduce their own data. In attempting to assess the consistency of their observations, a second team of reviewers evaluated the same medical records. “In the duplicate review of a subsample of 318 medical records . . . a second team of physicians did not identify the same group of adverse events as did the first team, but they did find about the same incidence of adverse events and adverse events due to negligence” (emphasis added).² By their own admission, their methodology fails to consistently identify the same incidents as adverse events. The identification of medical negligence would then necessarily be even less reliable. The authors are inexplicably reassured by the fact that the second review team produced numbers for adverse events and medical negligence that were similar to those of the first team. In fact, this notion is remarkably similar to physicians’ view of medical liability proceedings as resembling nothing more than a lottery. If plaintiffs file enough suits, they will ultimately win some percentage of them.

A number of additional issues suggest the limits of the Harvard data. The review is limited to hospital patients. By definition, they require the complexities of inpatient treatment. Few diseases have simple cures, and modern medicine does not offer remedy without risk. Even the finest surgeons operating in America’s best hospitals will have a finite incidence of wound infection. The pharmaceuticals used in both prevention and treatment programs all, without exception, carry with them the risk of adverse side effects. These risks will be greatest among a population of acutely ill patients whose care requires a hospital’s complex structured environment.

The researchers’ own examples underscore this point. One patient had lung cancer so advanced he was on life support equipment when a pneumothorax occurred. (Pneumothorax is the presence of air in the pleural space external to the lung, which can cause collapse of the lung itself.) The complication proved fatal in this terminally compromised patient. The study counts the pneumothorax as the cause of death, not the incurable cancer. Thus the researchers’ methodology eliminates the context; the adverse event occurred as a complication of treatment for an incurable condition.¹

Many of the patients studied were elderly and suffering from catastrophic, multisystem diseases. The very effort to offer treatment to patients in the final stages of advanced illness carries with it high risk. The most effective way to prevent complications of therapy is to offer none. Alternatively, treatment may be given only to those overwhelmingly likely to benefit from it, and leave those whose medical outcomes are uncertain to suffer the fate of the disease process. Only in an obscure paragraph is this significant caveat partially acknowledged, as the researchers allow that “. . . some patients may have requested and received limited care without that fact being documented in the medical record.”¹

The researchers also concede that “. . . modern medical care is an inherently risky enterprise.”⁴ Moreover, “[a] prime reason that modern medical care is so hazardous is that it so ambitiously attempts to cure the even greater hazards of nature. Many of today’s procedures are extremely risky simply by virtue of their complexity.”⁴ They further concede, in the case of the fatalities they blame on medical negligence, that “a substantial proportion of patients were gravely ill, and many would have died from their underlying illnesses in months, days, perhaps hours, even absent the mishap in treatment . . . Unfortunately, we cannot say what proportion of deaths from medically adverse events involved patients with relatively short life expectancies.”⁴

They also prudently “caution” against “too quick a comparison” of the fatalities in their data with any presumed national toll taken by malpractice. This did not, however, keep them from offering just such an extrapolation which was immediately picked up by partisans on the issue: “If New York’s adverse event-related death total can be extrapolated to the U.S. population as a whole, we would estimate over 150,000 iatrogenic fatalities annually, more than half of which are due to negligence.”⁴ Consumers Union soon claimed that the Harvard study showed “negligent doctors kill more than 80,000 people each year, more than the number who die in firearms or automobile accidents.”⁸

Additional points bear scrutiny. The study presents a picture of extraordinary variation among hospitals and specialists. Medical injury rates among the hospitals analyzed varied nearly forty-fold (from 0.2 percent to 7.9 percent),⁴ with rates of negligence varying sixty-fold (1 percent to 60 percent).⁴ There is a thirty-two-fold variation in the purported rate of iatrogenic injury from 0.5 percent in the treatment of newborns to 16.1 percent in patients undergoing vascular surgery. The percentage of adverse events due to negligence ranged from 18 percent to 38 percent among specialties, again suggesting how much of what was detected in the Harvard study is related to disease process rather than avoidable medical error.⁴ Variations of this magnitude underline the inappropriateness of extrapolating the results to larger populations. It bears repeating that nowhere in the study was any effort made to limit the inclusion of adverse events to those that were actually under the control of physicians.

One might also note the large percentage of adverse events attributed to medication reaction: 19.4 percent of the total.⁴ This figure includes even those events most obviously unavoidable with current medical technology, such as the side effects of the potent medications used to treat cancer. Though side effects of therapeutic agents were not regarded as negligent adverse events, the inclusion of such outcomes in the study’s primary database of adverse events contributes to the impression, generated by the study itself, of abundant avoidable error. This point cannot be overemphasized because the researchers’ proposal of a no-fault system would compensate all eligible adverse events.

Conclusion

The study broadly defines negligent adverse event and then, premised on this inappropriate definition, concludes that too few lawsuits are filed. A physician need not even have been involved, let alone responsible. In fact, a separate team of Harvard reviewers was not even able to identify the same set of adverse outcomes, let alone medical negligence. It is troubling that such a study is so widely cited to prove that medical malpractice is common.

The Harvard study proposes a new entitlement program that does not distinguish unavoidable from negligent injury. The researchers would impose a no-fault administrative system for those claiming injury from medical care, in large part because they view the current tort system as too restrictive. Regardless, they would encourage more litigation in one forum or the other by families who currently do not sue.

It is ironic that the study amply documents some of the things tort reformers have been saying for many years. Limiting compensation for noneconomic injury and duplicate recovery of expenses already covered are fundamental tort reforms already in place in a number of states, most notably California (MICRA—Medical Injury Compensation Reform Act of 1975).

The Harvard researchers seem unconcerned by the change in the climate of medical practice that would inevitably occur in a system offering the prospect of continuous, burdensome investigation of treatment outcomes, whether or not categorized as no-fault. For example, the majority of patients choosing intensive hospital-based cancer treatment and its unavoidable toxicities would be candidates for an administratively based compensation hearing. This is a prospect that gives pause to the medical profession of America—and it should be of even greater concern to all of us who would inevitably receive care under such a system.

For additional information on tort reform contact:

The Manhattan Institute
52 Vanderbilt Avenue, New York, NY 10017
Telephone (212) 599-7000, Fax (212) 599-3494
or visit http://www.manhattan-institute.org

---