Changes in Mortality After Massachusetts Health Care Reform: A Quasi-experimental Study
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Background:
The Massachusetts 2006 health care reform has been called a model for the Affordable Care Act. The law attained near-universal insurance coverage and increased access to care. Its effect on population health is less clear.
Objective:
To determine whether the Massachusetts reform was associated with changes in all-cause mortality and mortality from causes amenable to health care.
Design:
Comparison of mortality rates before and after reform in Massachusetts versus a control group with similar demographics and economic conditions.
Setting:
Changes in mortality rates for adults in Massachusetts counties from 2001 to 2005 (prereform) and 2007 to 2010 (postreform) were compared with changes in a propensity score–defined control group of counties in other states.
Participants:
Adults aged 20 to 64 years in Massachusetts and control group counties.
Measurements:
Annual county-level all-cause mortality in age-, sex-, and race-specific cells (n = 146 825) from the Centers for Disease Control and Prevention's Compressed Mortality File. Secondary outcomes were deaths from causes amenable to health care, insurance coverage, access to care, and self-reported health.
Results:
Reform in Massachusetts was associated with a significant decrease in all-cause mortality compared with the control group (−2.9%; P = 0.003, or an absolute decrease of 8.2 deaths per 100 000 adults). Deaths from causes amenable to health care also significantly decreased (−4.5%; P < 0.001). Changes were larger in counties with lower household incomes and higher prereform uninsured rates. Secondary analyses showed significant gains in coverage, access to care, and self-reported health. The number needed to treat was approximately 830 adults gaining health insurance to prevent 1 death per year.
Limitations:
Nonrandomized design subject to unmeasured confounders. Massachusetts results may not generalize to other states.
Conclusion:
Health reform in Massachusetts was associated with significant reductions in all-cause mortality and deaths from causes amenable to health care.
Primary Funding Source:
None.
Get full access to this article
View all available purchase options and get full access to this article.
Supplemental Material
References
1.
Long SK, Stockley K, Dahlen H. Massachusetts health reforms: uninsurance remains low, self-reported health status improves as state prepares to tackle costs. Health Aff (Millwood). 2012;31:444-51. [PMID: 22282572]
2.
Kolstad JT, Kowalski AE. The impact of health care reform on hospital and preventive care: evidence from Massachusetts. J Public Econ. 2012;96:909-929. [PMID: 23180894]
3.
Zhu J, Brawarsky P, Lipsitz S, Huskamp H, Haas JS. Massachusetts health reform and disparities in coverage, access and health status. J Gen Intern Med. 2010;25:1356-62. [PMID: 20730503]
4.
Clark CR, Soukup J, Govindarajulu U, Riden HE, Tovar DA, Johnson PA. Lack of access due to costs remains a problem for some in Massachusetts despite the state's health reforms. Health Aff (Millwood). 2011;30:247-55. [PMID: 21289346]
5.
Key indicators: quarterly enrollment update. Boston: Massachusetts Division of Health Care Finance and Policy; 2011. Accessed at www.mass.gov/chia/docs/r/pubs/12/2011-june-key-indicators.pdf on 5 March 2014.
6.
Long SK. On the road to universal coverage: impacts of reform in Massachusetts at one year. Health Aff (Millwood). 2008;27:w270-84. [PMID: 18522948]
7.
Long SK, Masi PB. Access and affordability: an update on health reform in Massachusetts, fall 2008. Health Aff (Millwood). 2009;28:w578-87. [PMID: 19477874]
8.
Pande AH, Ross-Degnan D, Zaslavsky AM, Salomon JA. Effects of healthcare reforms on coverage, access, and disparities: quasi-experimental analysis of evidence from Massachusetts. Am J Prev Med. 2011;41:1-8. [PMID: 21665057]
9.
Van Der Wees PJ, Zaslavsky AM, Ayanian JZ. Improvements in health status after Massachusetts health care reform. Milbank Q. 2013;91:663-89. [PMID: 24320165]
10.
Miller S. The effect of the Massachusetts reform on health care utilization. Inquiry. 2012;49:317-26. [PMID: 23469675]
11.
Courtemanche CJ, Zapata D. Does universal coverage improve health? The Massachusetts experience. J Policy Anal Manage. 2014;33:36-69. [PMID: 24358528]
12.
Wilper AP, Woolhandler S, Lasser KE, McCormick D, Bor DH, Himmelstein DU. Health insurance and mortality in US adults. Am J Public Health. 2009;99:2289-95. [PMID: 19762659]
13.
Care without coverage: too little, too late. Washington, DC: Institute of Medicine; 2002.
14.
Sommers BD, Baicker K, Epstein AM. Mortality and access to care among adults after state Medicaid expansions. N Engl J Med. 2012;367:1025-34. [PMID: 22830435]
15.
Kronick R. Health insurance coverage and mortality revisited. Health Serv Res. 2009;44:1211-31. [PMID: 19453392]
16.
Finkelstein A, Taubman S, Wright B, Bernstein M, Gruber J, Newhouse JP, et al; Oregon Health Study Group. The Oregon health insurance experiment: evidence from the first year. Q J Econ. 2012;127:1057-1106. [PMID: 23293397]
17.
Weathers RR 2nd, Stegman M. The effect of expanding access to health insurance on the health and mortality of Social Security Disability Insurance beneficiaries. J Health Econ. 2012;31:863-75. [PMID: 23000873]
18.
Bertrand M, Duflo E, Mullainathan S. How much should we trust differences-in-differences estimates? Quarterly Journal of Economics. 2004;119:249-75.
19.
Dehejia RH, Wahba S. Propensity-score matching methods for nonexperimental causal studies. Rev Econ Stat. 2002;84:151-61.
20.
Abadie A, Diamond A, Hainmueller J. Comparative politics and the synthetic control method. Cambridge, MA: Harvard Kennedy School; 2012. Accessed at www.hks.harvard.edu/fs/aabadie/comparative.pdf on 5 March 2014.
21.
McDonough JE, Rosman B, Phelps F, Shannon M. The third wave of Massachusetts health care access reform. Health Aff (Millwood). 2006;25:w420-31. [PMID: 16973653]
22.
Compressed Mortality File 1999–2010. CD ROM series 20, no. 2P, ASCII version. Hyattsville, MD: National Center for Health Statistics; 2012. Accessed at www.cdc.gov/nchs/data/mortab/1999-2010CMFDocumentation.pdf on 5 March 2014.
23.
Area Health Resource File (AHRF), 2011–2012. Rockville, MD: U.S. Department of Health and Human Services; 2012.
24.
U.S. Census Bureau. Model-based small area health insurance estimates (SAHIE) for counties and states, 2005. Accessed at www.census.gov/did/www/sahie/data/files/SAHIE2005.TXT on 3 October 2012.
25.
Keating NL, Kouri EM, He Y, West DW, Winer EP. Effect of Massachusetts health insurance reform on mammography use and breast cancer stage at diagnosis. Cancer. 2013;119:250-8. [PMID: 22833148]
26.
Charlton JR, Velez R. Some international comparisons of mortality amenable to medical intervention. Br Med J (Clin Res Ed). 1986;292:295-301. [PMID: 3080144]
27.
Mackenbach JP, Bouvier-Colle MH, Jougla E. “Avoidable” mortality and health services: a review of aggregate data studies. J Epidemiol Community Health. 1990;44:106-11. [PMID: 2196328]
28.
Nolte E, McKee M. Measuring the health of nations: analysis of mortality amenable to health care. BMJ. 2003;327:1129. [PMID: 14615335]
29.
Nolte E, McKee CM. Measuring the health of nations: updating an earlier analysis. Health Aff (Millwood). 2008;27:58-71. [PMID: 18180480]
30.
Nolte E, McKee M. Variations in amenable mortality—trends in 16 high-income nations. Health Policy. 2011;103:47-52. [PMID: 21917350]
31.
Singer JD, Willett JB. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. New York: Oxford Univ Pr; 2003.
32.
Afendulis CC, He Y, Zaslavsky AM, Chernew ME. The impact of Medicare Part D on hospitalization rates. Health Serv Res. 2011;46:1022-38. [PMID: 21306]
33.
Bendavid E, Bhattacharya J. The President's Emergency Plan for AIDS Relief in Africa: an evaluation of outcomes. Ann Intern Med. 2009;150:688-95. [PMID: 19349625]
34.
Cooper AL, Trivedi AN. Fitness memberships and favorable selection in Medicare Advantage plans. N Engl J Med. 2012;366:150-7. [PMID: 22236225]
35.
Cotet AM, Benjamin DK. Medical regulation and health outcomes: the effect of the physician examination requirement. Health Econ. 2013;22:393-409. [PMID: 22450959]
36.
Lien DS, Evans WN. Estimating the impact of large cigarette tax hikes: the case of maternal smoking and infant birth weight. J Human Resources. 2005;40:373-92.
37.
Besley T, Burgess R. Can labour regulation hinder economic performance: Evidence from India. Quarterly Journal of Economics. 2004;119:91-134.
38.
Karaca-Mandic P, Norton EC, Dowd B. Interaction terms in nonlinear models. Health Serv Res. 2012;47:255-74. [PMID: 22091735]
39.
Gray BH, Scheinmann R, Rosenfeld P, Finkelstein R. Aging without Medicare? Evidence from New York City. Inquiry. 2006;43:211-21. [PMID: 17176965]
40.
Baicker K, Taubman SL, Allen HL, Bernstein M, Gruber JH, Newhouse JP, et al; Oregon Health Study Group. The Oregon experiment—effects of Medicaid on clinical outcomes. N Engl J Med. 2013;368:1713-22. [PMID: 23635051]
41.
Miilunpalo S, Vuori I, Oja P, Pasanen M, Urponen H. Self-rated health status as a health measure: the predictive value of self-reported health status on the use of physician services and on mortality in the working-age population. J Clin Epidemiol. 1997;50:517-28. [PMID: 9180644]
42.
DeSalvo KB, Bloser N, Reynolds K, He J, Muntner P. Mortality prediction with a single general self-rated health question. A meta-analysis. J Gen Intern Med. 2006;21:267-75. [PMID: 16336622]
43.
Eisenberg JM, Power EJ. Transforming insurance coverage into quality health care: voltage drops from potential to delivered quality. JAMA. 2000;284:2100-7. [PMID: 11042759]
44.
DeLeire T, Dague L, Leininger L, Voskuil K, Friedsam D. Wisconsin experience indicates that expanding public insurance to low-income childless adults has health care impacts. Health Aff (Millwood). 2013;32:1037-45. [PMID: 23733977]
45.
Sommers BD, Kenney GM, Epstein AM. New evidence on the Affordable Care Act: coverage impacts of early Medicaid expansions. Health Aff (Millwood). 2014;33:78-87. [PMID: 24395938]
46.
U.S. Census Bureau. State rankings—Statistical Abstract of the United States: doctors per 100,000 resident population, 2007. Accessed at www.census.gov/compendia/statab/2012/ranks/rank18.html on 12 November 2012.
Comments
Sign In to Submit A CommentInformation & Authors
Information
Published In
Annals of Internal Medicine
Volume 160 • Number 9 • 6 May 2014
Pages: 585 - 593
History
Published online: 6 May 2014
Published in issue: 6 May 2014
Keywords
Copyright
© 2014 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Changes in Mortality After Massachusetts Health Care Reform: A Quasi-experimental Study. Ann Intern Med.2014;160:585-593. [Epub 6 May 2014]. doi:10.7326/M13-2275
Changes in Mortality After Massachusetts Health Care Reform: A Quasi-experimental Study. Ann Intern Med.2014;160:585-593. [Epub 6 May 2014]. doi:10.7326/M13-2275
Cited by
View More
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Comment
REFERENCE
1. Sommers BD, Long SK, Baicker K. Changes in Mortality After Massachusetts Health Care Reform: A Quasi-experimental Study. Annals of Internal Medicine 2014; 160(9): 585-93.
Comment
One potential problem with the analysis was that inferences were based on an approach that was unlikely to be valid. The natural experiment that was the basis of the analysis was characterized by only one treatment state among 47 states and sample sizes in the states that were quite uneven in terms of the number of observations per state. In these settings, common approaches to inference have been shown to significantly over reject the null hypothesis. (3-5)
I assessed the extent of the potential inference problem by calculating p-values for estimates in the article using randomization inference methods. (6) The p-values derived from this method were very large and ranged from 0.277 to 0.783. These p-values indicate that none of the estimates reported in the article are statistically significant at commonly accepted levels of significance. Moreover, given the relatively large magnitudes of the estimates that were reported, the large p-values associated with these estimates suggest that the analysis lacked adequate statistical power to detect plausible effect sizes. In sum, my analysis indicates that the question of whether Massachusetts health reform affected mortality remains unanswered.
Robert Kaestner, Ph.D.
Institute of Government and Public Affairs
References
1. Sommers, Benjamin D., Sharon K. Long, and Katherine Baicker. 2014. “Changes in Mortality After Massachusetts Health Care Reform: A Quasi-Experimental Study.” Annals of Internal Medicine 160 (9): 585–93. doi:10.7326/M13-2275.
2. Tavernise, Sabrina. “Mortality Drop Seen to Follow ’06 Health Law,” The New York Times, May 5 2014. http://www.nytimes.com/2014/05/06/health/death-rate-fell-in-massachusetts-after-health-care-overhaul.html?_r=0, last accessed January 30, 2015. (A version of this article appears in print on May 6, 2014, on page A16 of the New York edition with the headline: Mortality Drop Follows Massachusetts Health Law.)
3. Cameron, A. Colin, Jonah B. Gelbach, and Douglas L. Miller. 2008. “Bootstrap-Based Improvements for Inference with Clustered Errors.” The Review of Economics and Statistics 90 (3): 414–27.
4. Conley, Timothy G., and Christopher R. Taber. 2011. “Inferences with “Difference in Differences” with a Small Number of Policy Changes.” The Review of Economics and Statistics 93 (1): 113-125.
5. MacKinnon, James G., and Matthew D. Webb. 2014. Wild Bootstrap Inference for Wildly Different Cluster Sizes. Working Paper 1314. Queen’s University, Department of Economics. https://ideas.repec.org/p/qed/wpaper/1314.html, last accessed January 30, 2015.
6. Rosenbaum, Paul R. 2002. “Covariance Adjustment in Randomized Experiments and Observational Studies.” Statistical Science 17 (3): 286–327. doi:10.1214/ss/1042727942.
Authors' Response to Dr. Kaestner's Comment
Lacking sufficient detail on the analysis conducted by Dr. Kaestner, we cannot evaluate the soundness of his approach or the validity of his results. However, we believe his concern about uneven cluster size driving spurious results is addressed by the state-level analysis presented in Appendix Table 4 of our paper, described in detail in the on-line Supplemental Appendix.(1) We analyzed the study’s 47 states using annual state race-age-sex mortality rates. The analysis used a sample with an identical number of observations per state cluster – 288 race-sex-age yearly mortality rates – in 43 of the 47 states, while the other four states had some empty race-sex-age cells, leading to state samples ranging from 241-285. In that analysis, which avoids Dr. Kaestner’s primary methodological concern, we found large and statistically significant reductions in unadjusted all-cause mortality, as well as adjusted and unadjusted heath-care amenable mortality. Thus, the key results of the paper do not appear to be driven by uneven cluster sizes.
More generally, our paper builds on numerous studies of Massachusetts health reform that have taken similar approaches with state-level clustering, using well-established methods for these sorts of analyses.(2,3) Our results remained statistically significant (Appendix Table 4) when using county-level clustering, which has been used by other researchers in county-based analyses of Massachusetts health reform such as ours.(4,5)
Of course, all statistical estimation techniques have strengths and weaknesses, and the standard errors in quasi-experimental approaches are often sensitive to assumptions. This concern motivated the multiple approaches taken in the Appendix. We were reassured by the robustness of the results across approaches. We agree that great care must be taken with statistical inference in designs such as ours, but our analysis indicates that our use of comprehensive mortality statistics capturing every death in the study population of nearly 50 million people over nine years generated more than sufficient power to detect clinically meaningful mortality changes.
REFERENCES
1. Sommers BD, Long SK, Baicker K. Changes in Mortality After Massachusetts Health Care Reform: A Quasi-experimental Study. Ann Intern Med. May 6 2014;160(9):585-593.
2. Courtemanche CJ, Zapata D. Does universal coverage improve health? The Massachusetts experience. J Policy Anal Manage. Winter 2014;33(1):36-69.
3. Van Der Wees PJ, Zaslavsky AM, Ayanian JZ. Improvements in health status after Massachusetts health care reform. Milbank Q. Dec 2013;91(4):663-689.
4. Miller S. The effect of insurance on emergency room visits: An analysis of the 2006 Massachusetts health reform. J Pub Econ. 2012;96(11-12):893-908.
5. Hanchate AD, Lasser KE, Kapoor A, et al. Massachusetts reform and disparities in inpatient care utilization. Med Care. Jul 2012;50(7):569-577.
Disclosures: Dr. Sommers is an advisor in the Office of the Assistant Secretary for Planning and Evaluation at the U.S. Department of Health and Human Services. However, this paper was written in Dr. Sommers’ capacity as a Harvard employee and does not represent the views of the
U.S. Department of Health and Human Services. We received no funding for this work.