Abstract
- Issue: No two countries are alike when it comes to organizing and delivering health care for their people, creating an opportunity to learn about alternative approaches.
- Goal: To compare the performance of health care systems of 11 high-income countries.
- Methods: Analysis of 71 performance measures across five domains — access to care, care process, administrative efficiency, equity, and health care outcomes — drawn from Commonwealth Fund international surveys conducted in each country and administrative data from the Organisation for Economic Co-operation and Development and the World Health Organization.
- Key Findings: The top-performing countries overall are Norway, the Netherlands, and Australia. The United States ranks last overall, despite spending far more of its gross domestic product on health care. The U.S. ranks last on access to care, administrative efficiency, equity, and health care outcomes, but second on measures of care process.
- Conclusion: Four features distinguish top performing countries from the United States: 1) they provide for universal coverage and remove cost barriers; 2) they invest in primary care systems to ensure that high-value services are equitably available in all communities to all people; 3) they reduce administrative burdens that divert time, efforts, and spending from health improvement efforts; and 4) they invest in social services, especially for children and working-age adults.
Introduction
No two nations are alike when it comes to health care. Over time, each country has settled on a unique mix of policies, service delivery systems, and financing models that work within its resource constraints. Even among high-income nations that have the option to spend more on health care, approaches often vary substantially. These choices affect health system performance in terms of access to care, patients’ experiences with health care, and people’s health outcomes. In this report, we compare the health systems of 11 high-income countries as a means to generate insights about the policies and practices that are associated with superior performance.
With the COVID-19 pandemic imposing an unprecedented stress test on the health care and public health systems of all nations, such a comparison is especially germane. Success in controlling and preventing infection and disease has varied greatly. The same is true of countries’ ability to address the challenges that the pandemic has presented to the workforce, operations, and financial stability of the organizations delivering care. And while the comparisons we draw are based on data collected prior to the pandemic or during the earliest months of the crisis, the prepandemic strengths and weaknesses of each country’s preexisting arrangements for health care and public health have undoubtedly been shaping its experience throughout the crisis.
For our assessment of health care system performance in Australia, Canada, France, Germany, the Netherlands, New Zealand, Norway, Sweden, Switzerland, the United Kingdom, and the United States, we used indicators available across five domains:
- Access to care
- Care process
- Administrative efficiency
- Equity
- Health care outcomes.
For more information on these performance domains and their component measures, see How We Measured Performance. Most of the data were drawn from surveys examining how members of the public and primary care physicians experience health care in their respective countries. These Commonwealth Fund surveys were conducted by SSRS in collaboration with partner organizations in the 10 other countries. Additional data were drawn from the Organisation for Economic Co-operation and Development (OECD) and the World Health Organization (WHO).
How the 11 Countries Rank on Performance
The top-performing countries overall are Norway, the Netherlands, and Australia (Exhibit 1).
The next three countries in the ranking — the U.K., Germany, and New Zealand — perform very similarly to one another (Exhibit 2). The U.S. ranks #11 — last. Exhibit 2 shows the extent to which the U.S. is an outlier: its performance falls well below the average of the other countries and far below the two countries ranked directly above it, Switzerland and Canada. In fact, the U.S. is such an outlier that we have calculated the average performance based on the other 10 countries, excluding the U.S. (see How We Measured Performance). The U.S. is last on all domains of performance except care process, on which it ranks #2.
Exhibit 3 shows that while spending as a share of gross domestic product (GDP) has increased in all countries, spending growth in the U.S. — by far the worst performer overall — has greatly exceeded growth in the other 10 nations. In 1980, high-income countries spent between 5 percent and 8 percent of GDP on health care. But as U.S. spending accelerated over the decades, the U.S. was spending a substantially larger share of its GDP on health care by 2019 than every other high-income country.
Exhibit 4 starkly shows just how much the U.S. is an outlier from the other nations when its performance as a health care system is compared to its spending as a share of GDP.
Change in Rankings Since the 2017 Edition of Mirror, Mirror
Readers familiar with the previous edition of this report (2017) will notice that some of the country ranks have changed. These changes should be interpreted with caution. While most of the 71 measures included in the new edition are identical to those used in 2017, 10 measures were modified because survey items, response categories, or available data changed. We replaced 17 of the 2017 measures with 16 new measures to reflect newly available data as well as to better represent previously defined performance domains and subdomains. An expert advisory panel reviewed the proposed changes. See Appendix 2 for more detail on the changes by domain.
Readers should interpret changes in ranks in the context of the statistical variation in countries’ performance scores (as visualized in Exhibit 2, for example). We calculated performance differences as the standard deviation from “average performance” — a measure of the degree of difference between countries given the range of variation in this set of countries.
Depending on the domain, some countries have quantitatively similar performance scores, meaning that very small differences can produce changes in rankings. The U.K.’s drop in rank from #1 to #4 is associated with that country’s lower performance on several domains (such as access to care and equity) compared to 2017.
For more on the differences between the 2017 and 2021 editions of this report, please see How We Conducted This Study.
Access to Care: Universal, Affordable Coverage Is Paramount
Access to care includes measures of health care’s affordability and timeliness. The Netherlands performs best on this performance domain among the 11 countries, ranking at or near the top in both subdomains. Norway and Germany also performed well on access to care (Exhibit 1), but all three are outranked on affordability by the U.K. (Exhibit 5).
Overall, the U.S. is #11 — last — on access to care (Exhibit 1). The U.S. has the poorest performance on the affordability subdomain, scoring much lower than even the next-lowest country, Switzerland (Exhibit 5). Compared to residents of the U.S., residents of the Netherlands, the U.K., Norway, and Germany are much less likely to report that their insurance denied payment of a claim or paid less than expected. Residents of these countries are also less likely to report difficulty in paying medical bills (Appendix 4).
People in the countries performing the best on the timeliness subdomain are more likely to be able to get same-day care and after-hours care. The U.S. ranked #9 on timeliness.
Care Process: The U.S. Compares Favorably on Preventive Care, Safe Care, and Engagement and Patient Preferences
Care process includes measures of preventive care, safe care, coordinated care, and engagement and patient preferences. The U.S. ranks #2 on this performance domain (Exhibit 1). Along with the U.K. and Sweden, the U.S. achieves higher performance on the preventive care subdomain, which includes rates of mammography screening and influenza vaccination as well as the percentage of adults who talked with their provider about nutrition, smoking, and alcohol use. New Zealand and the U.S. perform best on the safe care subdomain, with higher reported use of computerized alerts and routine review of medications. Still, in all countries, more than 10 percent of adults report experiencing medical or medication mistakes in their care.
New Zealand, Switzerland, and the Netherlands perform best among countries on the coordinated care subdomain. Switzerland, New Zealand, Australia, Norway, and France perform well on measures related to communication between primary care doctors and specialists. No country stood out at achieving good communication between the primary care and hospital, emergency department, and home-based care provider or coordination with local social services providers.
The U.S. and Germany achieve the highest performance on the engagement and patient preferences subdomain, although U.S. adults have the lowest rates of continuity with the same doctor. Among people with chronic illness, U.S. adults are among the most likely to discuss goals, priorities, and treatment options with their provider, though less likely to receive as much support from health professionals as they felt was needed.
Use of web-based portals for communicating medical concerns and refilling medications is highest among adults in Norway and the U.S. In the year prior to the COVID-19 pandemic, primary care clinicians in Sweden and Australia were the most likely to report using video consultations.
Administrative Efficiency: Many Countries Simplify Insurance Coverage, Billing, and Payment
Administrative efficiency refers to how well health systems reduce documentation (paperwork) and other bureaucratic tasks that patients and clinicians frequently face during care. The top performers on the administrative efficiency domain are Norway, Australia, New Zealand, and the U.K. (Exhibit 1). The U.S. ranks last.
U.S. doctors are the most likely to have trouble getting their patients medication or treatment because of restrictions on insurance coverage. Compared to most of the other countries, larger percentages of adults in the U.S. say they spend a lot of time on paperwork related to medical bills. For nonemergency care, U.S. and Canadian adults are also more likely to visit the emergency department — a less efficient option than seeing a regular doctor.
Equity: Income-Related Disparities Are Largest in the U.S., Canada, New Zealand, and Norway
Our analysis of equity focuses on income-related disparities, based on standardized data across the 11 countries, in the access to care, care process, and administrative efficiency performance domains. Similar standardized data are not available for measuring equity in performance with respect to different racial and ethnic groups (see How We Measured Performance for more detail).
Australia, Germany, and Switzerland rank highest on the equity domain, meaning these countries had the smallest income-related disparities in performance based on the included measures (Exhibit 6).
Within these countries, experiences reported by people in lower- and higher-income groups on 11 indicators in the affordability, timeliness, preventive care, safe care, and engagement and patient preferences subdomains are less divergent than they are within other countries (Appendix 7).
In contrast, the U.S. consistently demonstrated the largest disparities between income groups, except for those measures related to preventive services and safety of care. U.S. disparities are especially large when looking at financial barriers to accessing medical and dental care, medical bill burdens, difficulty obtaining after-hours care, and use of web portals to facilitate patient engagement. Compared to the other countries, the United States and Canada had larger income-related inequities in patient reported experiences.
Exhibit 7 illustrates the importance of comparing country performance on equity: relatively good performance on a health care measure overall may mask pronounced gaps in the experiences of lower-income versus higher-income groups. It also illustrates the challenge that arises in assessing equity without also considering performance overall: income-related differences on a measure may be small, but a nation’s performance may be comparatively poor for both higher- and lower-income groups.
In Exhibit 7, income-related performance disparities in Switzerland and Australia are as small as those in Germany and the U.K. But the cost-related access problems for higher-income residents of Switzerland and Australia resemble the levels seen among lower-income residents of the Netherlands and Canada. Adults with higher incomes in the U.S., Switzerland, and Australia are as likely as, or more likely than, adults with lower incomes in five countries to report cost-related access problems.
Health Care Outcomes: Many Countries Achieve Better Outcomes Despite Lower Spending
Health care outcomes reported here refer to those health outcomes that are most likely to be responsive to health care. On this domain, Australia, Norway, and Switzerland rank at the top of our 11-nation group (Exhibit 1). Norway has the lowest infant mortality rate (two deaths per 1,000 live births), while Australia has the highest life expectancy after age 60 (25.6 years of additional life expectancy for those who survive to age 60).
The U.S. ranks last overall on the health care outcomes domain (Exhibit 1). On nine of the 10 component measures, U.S. performance is lowest among the countries (Appendix 8), including having the highest infant mortality rate (5.7 deaths per 1,000 live births) and lowest life expectancy at age 60 (23.1 years). The U.S. ranks last on the mortality measures included in this report, with the exception of 30-day in-hospital mortality following stroke. The U.S. rate of preventable mortality (177 deaths per 100,000 population) is more than double the best-performing country, Switzerland (83 deaths per 100,000).
The U.S. has exceptionally poor performance on two other health care outcome measures. Maternal mortality is one: the U.S. rate of 17.4 deaths per 100,000 live births is twice that of France, the country with the next-highest rate (7.6 deaths per 100,000 live births).
The second is the 10-year trend in avoidable mortality. As depicted in Exhibit 8, all countries reduced their rate of avoidable mortality over 10 years, but the U.S., with the highest level in 2007, reduced it by the least amount — 5 percent reduction in deaths per 100,000 population by 2017 — compared to 25 percent in Switzerland (by 2017) and 24 percent in Norway (by 2016).
Discussion
Some high-income nations get more for their health dollars than the U.S. does. As nations strive for better health care and better health for their residents, several basic lessons emerge from our findings.
Achieving better health outcomes will require policy changes within and beyond health care.
The striking contrast in performance between the U.S. and other high-income countries on avoidable mortality measures points to several intervention or policy targets. How have top-performing countries reduced avoidable mortality? A comparison of the features of top-performing countries and poorer-performing countries suggests that top-performing countries rely on four features to attain better and more equitable health outcomes:
- They provide for universal coverage and remove cost barriers so people can get care when they need it and in a manner that works for them.
- They invest in primary care systems to ensure that high-value services are equitably available locally in all communities to all people, reducing the risk of discrimination and unequal treatment.
- They reduce the administrative burdens on patients and clinicians that cost them time and effort and can discourage access to care, especially for marginalized groups.
- They invest in social services that increase equitable access to nutrition, education, child care, community safety, housing, transportation, and worker benefits that lead to a healthier population and fewer avoidable demands on health care.
Prioritizing maternal health is critical for reducing maternal mortality. Top-performing countries have had success in preventing maternal deaths through the removal of cost sharing for maternal care. They invest in primary care models that ensure continuity of care from conception through the postpartum period, including midwife-led models. They offer social support benefits, including parental leave.
Several additional causes of avoidable mortality are linked to mental health. Higher rates of suicide in the U.S. — rates that have increased every year since 2000 — could be addressed by expanding the capacity of primary care to diagnose comorbid mental health conditions and provide early intervention and treatment as well as promote social connectedness and suicide prevention. Compared to other countries, the U.S. has a comparatively smaller workforce dedicated to meeting mental health needs. Countries like the Netherlands, Sweden, and Australia more frequently include mental health providers on primary care teams.
Health Care Outcomes vs. Health Outcomes
Health outcomes are influenced by a wide variety of social and economic factors, many of them outside the control of health care systems. Policies and public investments in education, employment, nutrition, housing, transportation, and environmental safety shape the health of the population. Our report focuses on health care outcome metrics — those outcomes that can be improved by the delivery of health care services.
Compared to other OECD countries, the U.S. spends relatively less on social programs such as early childhood education, parental leave, and income supports for single parents. The U.S. also spends less on supports for workers, such as unemployment protections and labor market incentives. Labor market policies in particular have been linked to so-called deaths of despair, including suicides and overdose deaths.
U.S. health outcomes could therefore be improved through actions targeting factors beyond health care. Accountable Communities for Health offer one promising approach to improving health outcomes as well as equity.
Improving access to care requires expanding and strengthening insurance coverage.
The U.S. remains the only high-income country lacking universal health insurance coverage. With nearly 30 million people still uninsured and some 40 million with health plans that leave them potentially underinsured, out-of-pocket health care costs continue to mar U.S. health care performance.
Top-performing countries achieve near-universal coverage and much higher levels of protection against medical costs in the form of annual out-of-pocket caps on covered benefits and full coverage for highly beneficial preventive services, primary care, and effective treatments for chronic conditions. Germany abolished copayments for physician visits in 2013, while several countries have fixed annual out-of-pocket maximums for health expenditures (ranging from about USD 300 per year in Norway to USD 2,645 in Switzerland).
Australia addresses income-related equity through a mix of annual spending caps that are lower for low-income individuals as well as incentives for people to seek primary care. In 2019, 86 percent of Australians faced no out-of-pocket costs for primary care visits.
Improving access to care requires strengthening primary care and extending it to every local community.
Access to care, however, requires more than insurance coverage. Convenient and timely primary care is also vital. Top-ranking countries like the Netherlands and Norway ensure timely availability to care by phone on nights and weekends (with in-person follow-up at home as needed). In the Netherlands, cooperative “GP posts” are staffed by general practitioners (primary care physicians), who are obligated to provide at least 50 hours of after-hours care (between 5:00 pm and 8:00 am) annually in order to maintain their professional licensure. In Norway, the Patients’ Rights Act specifies a right to receive care within specific timeframes and with maximum wait times applying to covered services, including general practitioner visits, hospital care, mental health care, and substance use treatment.
In top-performing countries, workforce policy is geared to ensuring access within communities, especially those that have been historically marginalized. Norway, with the highest number of doctors per person among the 11 countries in our study, has a much larger supply of physicians relative to its population than the U.S. has. Outside the U.S., a larger proportion of clinicians are devoted to primary care and are geographically distributed to match population needs. For example, Norwegian local municipalities, which are responsible for the supply of GPs, may apply to the national government for extra funding to ensure they have an adequate number of physicians.
Reducing administrative burden can free up resources to devote to improving health.
Administrative requirements cost both time and money for patients, clinicians, and managers while also diverting resources away from efforts to improve care. Our results are consistent with other studies showing that administrative costs are more substantial in the U.S. than in other high-income countries. Many countries have simplified their health insurance and payment systems, usually through legislation, regulation, and standardization. For example, top-ranked Norway determines patient copayments for physician fees on a regional basis, applying the standardized copayments to all physicians practicing in the public sector within a specialty within a geographic area.
In countries where private insurance companies compete for customers, such as the Netherlands, standards including a mandatory minimum basic benefit package, community rating to keep premiums lower for sicker individuals, and cost-sharing caps to simplify choice for beneficiaries. These features create an incentive for insurers to compete on service and quality rather than on avoidance of people with higher health risks, similar to the marketplace insurance plans introduced by the Affordable Care Act. Germany and Canada negotiate provider payments administratively, as the U.S. Medicare and Medicaid programs do. As other countries have demonstrated, collective negotiation and standardized payment for services, at either the national or regional level, can greatly simplify transactions, reducing errors and appeals, and making time and attention available to improve care.
Smarter spending — not more spending — is required to achieve better health system performance.
The U.S. continues to outspend other nations on health care, devoting nearly twice as much of its GDP as the average OECD country. U.S. health spending reached nearly 17 percent of GDP in 2019, far above the 10 other countries compared in this report. Moreover, high U.S. out-of-pocket health spending per person, the second-highest in the OECD, makes it difficult for many Americans to access needed care.
The U.S. has managed to keep pace with or exceed other countries on several measures of care process included in the report, such as influenza vaccination rates for older adults, lower rates of postoperative sepsis after abdominal surgery, and more use of patient-facing health information technology for provider communications and prescription filling. But the U.S. still lags other nations on measures of health care outcomes, access to care, equity, and administrative efficiency. What explains the apparent disconnect?
First, many process measures focus on the care available to people who actually have access to care. For example, a measure of care quality for hospitalized patients focuses on those who had access to hospital care in the first place and ignores those who died before reaching a hospital. It is possible to deliver high-quality care to the population that has access to care and the means to pay for it, while delivering poor-quality care to the smaller share of the population that lacks those means. The result may be an average level of performance overall, but a health system that nevertheless inadequately serves the sickest and most vulnerable.
Second, administrative barriers may disproportionately deter poorer and marginalized individuals from receiving health services. Low-income people who work long hours or those with limited health literacy or support from family, friends, or neighbors may have difficulty navigating complex insurance eligibility rules, a maze of application procedures, or getting online access. In fact, this is why the U.S. is the only country among those compared here that employs health navigators to help direct patients through both insurance and the wider health care system.
Third, the relationship between health care outcomes and care process is inevitably complex, especially if the population is less healthy because of economic and social policies that produce inequities or fail to mitigate their consequences. The U.S. population is sicker on average than the populations of other high-income countries, with a high prevalence of chronic conditions like obesity, diabetes, heart disease, and respiratory ailments. This disease burden, coupled with insufficient access to care, partially explains the shorter and declining life expectancy in the U.S. compared to other countries. Even excellent care process, health information technology, and patient engagement may be no match for insufficient access, administrative deterrents, and inadequate chronic disease management. The high U.S. death toll during the COVID-19 pandemic illustrates the difficulty of achieving good health care outcomes if the population is sicker and access to preventive and primary care is limited, particularly because of affordability barriers.
It appears, then, that the U.S. health system delivers too little of the care that’s most needed — and often delivers it too late — especially for people with complex chronic illness, mental health problems, or substance use disorders, many of whom have faced a lifetime of inequitable access to care.
Conclusion
International comparisons allow the public, policymakers, and health care leaders to see alternative approaches to delivering health care, ones that might be borrowed to build better health systems that yield better health outcomes. Lessons from the three top performers we highlight in this report — Norway, the Netherlands, and Australia — can inform the United States and other countries seeking to improve.
As the COVID-19 pandemic has amply shown, no nation has the perfect health system. Health care is a work in progress; the science continues to advance, creating new opportunities and challenges. But by learning from what’s worked and what hasn’t elsewhere in the world, all countries have the opportunity to try out new policies and practices that may move them closer to the ideal of a health system that achieves optimal health for all its people at a price the nation can afford.
HOW WE MEASURED PERFORMANCE
Access to Care. The access to care domain encompasses two subdomains: affordability and timeliness. The five measures of affordability include patient reports of avoiding medical care or dental care because of cost, having high out-of-pocket expenses, facing insurance shortfalls, or having problems paying medical bills. One 2017 measure was dropped (not available from a recent survey).
The timeliness subdomain includes six measures (one reported by primary care clinicians) summarizing how quickly patients can obtain information, make appointments, and obtain urgent care after hours. The 2021 report includes a new measure of the percentage of respondents who received counseling or treatment for mental health issues if they wanted or needed it. The wording of two survey-based measures was modified since 2017. Five 2017 measures were not included. Two were not available from a recent survey. Three other measures of wait times were excluded because they were asked early in the 2020 COVID-19 pandemic and results were thought to be unreliable.
Care Process. The care process domain encompasses four subdomains relevant to health care for the general population: preventive care, safe care, coordinated care, and engagement and patient preferences.
The preventive care subdomain includes three survey items related to counseling by health professionals on healthy behaviors, three OECD measures of mammography screening and influenza and measles vaccination (new for the 2021 rankings), and three OECD measures of rates (age- and sex-standardized) of avoidable hospital admissions for three prevalent chronic conditions: diabetes, asthma, and congestive heart failure. The wording or timeframe differed slightly for three measures. One 2017 measure was not available from a recent survey.
The safe care subdomain includes three survey items: two indicators of safe care based on patient reports of experiencing medical, medication, or laboratory mistakes, and failure to receive effective prescription medication management, as well as one measure indicating whether primary care doctors receive an electronic alert or prompt to provide patients with test results. One measure’s wording was modified since 2017. Two OECD measures related to adverse events occurring after hospital procedures are new in the 2021 report.
The coordinated care subdomain uses seven measures to summarize timely sharing of information among primary care clinicians, specialists, emergency departments, and hospitals. It includes five physician-reported measures of effective communication among primary care clinicians and home care, social service providers, and emergency departments. Wording of four measures was modified slightly since 2017.
The engagement and patient preferences subdomain consists of 13 measures that evaluate the delivery of patient-centered care, which includes effective and respectful clinician–patient communication and care planning that reflects the patient’s goals and preferences. New measures in the 2021 report include the percentage of chronically ill patients who felt they got the support they needed from health professionals to manage their health problems, and three measures related to how patients and health care professionals use health information technology (IT) or video consultations. One 2017 measure was excluded because it was not available from a recent survey.
Administrative Efficiency. The administrative efficiency domain includes five measures. Four assess patients’ and primary care clinicians’ reports of time and effort spent dealing with paperwork or administrative issues, as well as disputes related to documentation requirements of insurance plans and government agencies. One patient-reported measure evaluates barriers to care because of limited availability of the regular doctor. Two 2017 measures were excluded because they were not included in all of the countries surveyed.
Equity. The equity domain compares performance for higher- and lower-income individuals within each country, using 11 selected survey measures from the care process and access to care domains. The analysis stratifies the surveyed populations based on reported income (above-average vs. below-average, relative to the country’s median income) and calculates a percentage-point difference in performance between the two groups. A larger percentage-point difference represents lower equity between income groups in that country. A negative percentage-point difference indicates better performance among those with below-average income. Two new 2021 measures are related to patient use of health IT and one measure of patient-reported levels of medical or medication mistakes. Two 2017 measures related to wait times were dropped and one measure was unavailable from a recent survey (see access to care, above).
Health Care Outcomes. The health care outcomes domain includes 10 measures of the health of populations selected to focus on outcomes that can be modified by health care (in contrast to public health measures such as life expectancy at birth, which may be affected more by social and economic conditions). The measures fall into three categories:
- Population health outcomes reflect the chronic disease and mortality burden of selected populations. We include two measures comparing countries on mortality defined by age (infant mortality, life expectancy at age 60) and one measure on the proportion of nonelderly adults who report having multiple common chronic conditions (arthritis, asthma or chronic lung disease, diabetes, heart disease, high blood pressure).
- Mortality amenable to health care reflects deaths under age 75 from specific causes that are considered preventable in the presence of timely and effective health care. In the 2021 edition of Mirror, Mirror we dropped two previous measures replacing them with new standardized and publicly available OECD measures of mortality that consist of deaths considered preventable through effective primary prevention and other public health measures (“preventable mortality”) and of deaths that were considered treatable through more effective and timely health care interventions (“treatable mortality”).1 OECD combines these two measures to report “avoidable mortality” — for which we report the 10-year trend as an additional new measure.
- Condition-specific health outcomes measures include measures on 30-day in-hospital mortality following myocardial infarction and stroke, as well as two new measures in this section: maternal mortality and deaths from suicide. We dropped two OECD measures related to five-year cancer survival rates (breast and colon), because recent data were not available.
HOW WE CONDUCTED THIS STUDY
The 2021 edition of Mirror, Mirror was constructed using the same methodological framework developed for the 2017 report in consultation with an expert advisory panel.2 Another expert advisory panel was convened to review the data, measures, and methods used in the 2021 edition.3
Using data available from Commonwealth Fund international surveys of the public and physicians and other sources of standardized data on quality and health care outcomes, and with the guidance of the independent expert advisory panel, we carefully selected 71 measures relevant to health care system performance, organizing them into five performance domains: access to care, care process, administrative efficiency, equity, and health care outcomes. The criteria for selecting measures and grouping within domains included: importance of the measure, standardization of the measure and data across the countries, salience to policymakers, and relevance to performance-improvement efforts. We examined correlations among indicators within each domain, removing a few highly correlated measures. Mirror, Mirror is unique in its inclusion of survey measures designed to reflect the perspectives of patients and professionals — the people who experience health care in each country during the course of a year. Nearly three-quarters of the measures come from surveys designed to elicit the public’s experience of its health system.
Changes Since 2017
The majority of measures included in this report are the same as in the 2017 edition of Mirror, Mirror (Appendix 2). Seventeen measures were dropped if a survey question was no longer included in the Commonwealth Fund International Health Policy Survey or if we had reason to believe the response to the measure might be less valid because of effects of the COVID-19 pandemic, such as questions in the timeliness subdomain related to wait times, which were being fielded during the spring of 2020. Ten measures were considered “modified” in the 2021 report because the wording of a survey item was altered since the 2017 version.
We worked to include new measures to fill previously identified gaps in performance measurement across the 11 countries and considered a wide array of potential new measures related to topics such as quality of behavioral and mental health care, hospital care, pediatric care, and safety. We considered the data availability of new measures, how recently they had been updated, and how they correlated with other measures in each domain. In the end we included 16 new measures across the five domains (see How We Measured Performance for details).
Data
Data for this report were derived from several sources. Survey data are drawn from Commonwealth Fund International Health Policy Surveys fielded during 2017, 2019, and 2020. Since 1998, in collaboration with international partners, the Commonwealth Fund has supported these surveys of the public’s and primary care physicians’ experiences of their health care systems. Each year, in collaboration with researchers in the 11 countries, a common questionnaire is developed, translated, adapted, and pretested. The 2020 survey was of the general population; the 2017 survey surveyed adults age 65 and older. The 2020 and 2017 surveys examined patients’ views of the health care system, quality of care, care coordination, medical errors, patient–physician communication, wait times, and access problems. The 2019 survey was administered to primary care physicians and examined their experiences providing care to patients, use of information technology, and use of teams to provide care.
The Commonwealth Fund International Health Policy Surveys (2017, 2019, and 2020) include nationally representative samples drawn at random from the populations surveyed. The 2017 and 2020 surveys’ sampling frames were generated using probability-based overlapping landline and mobile phone sampling designs and in some countries, listed or nationwide population registries; the 2019 survey was drawn from government or private company lists of practicing primary care doctors in each country, except in France, where they were selected from a nationally representative panel of primary care physicians. Appendix 9 presents the number of respondents and response rates for each survey, and further details of the survey methods are described elsewhere.4,5,6
In addition to the survey items, standardized data were drawn from recent reports of the Organisation for Economic Co-operation and Development (OECD) and the World Health Organization (WHO). Our study included data from the OECD on screening, immunization, preventable hospital admissions, population health, and disease-specific outcomes. WHO data were used to measure health care outcomes.
Analysis
The method for calculating performance scores and rankings is similar to that used in the 2017 report, except that we modified the calculation of relative performance because the U.S. was a distinct and substantial outlier (see below).
Measure performance scores: For each measure, we converted each country’s result (e.g., the percentage of survey respondents giving a certain response or a mortality rate) to a measure-specific, “normalized” performance score. This score was calculated as the difference between the country result and the 10-country mean, divided by the standard deviation of the results for each measure (see Appendix 3). Normalizing the results based on the standard deviation accounts for differences between measures in the range of variation among country-specific results. A positive performance score indicates the country performs above the group average; a negative score indicates the country performs below the group average. Performance scores in the equity domain were based on the difference between higher-income and lower-income groups, with a wider difference interpreted as a measure of lower equity between the two income strata in each country.
The normalized scoring approach assumes that results are normally distributed. In 2021, we noted that the U.S. was such a substantial outlier that it was negatively skewing the mean performance, violating the assumption. In 2017, we had included all 11 countries to calculate the mean and standard deviation of each measure. After conducting an outlier analysis (see below), we chose to adjust the calculation of average performance by excluding the U.S., using the other 10 countries as the sample group for calculating the mean performance score and standard deviation. This modification changes a country’s performance scores relative to the mean but does not affect the ranking of countries relative to one another.
Domain performance scores and ranking: For each country, we calculated the mean of the measure performance scores in that domain. Then we ranked each country from 1 to 11 based on the mean domain performance score, with 1 representing the highest performance score and 11 representing the lowest performance score.
Overall performance scores and ranking: For each country, we calculated the mean of the five domain-specific performance scores. Then, we ranked each country from 1 to 11 based on this summary mean score, again with 1 representing the highest overall performance score and 11 representing the lowest overall performance score.
Outlier analysis: We applied Tukey’s boxplot method of detecting statistical outliers and identified several domains or subdomains (affordability, preventive care, equity, and health care outcomes) in which the U.S. was a statistical outlier. The test identified isolated instances of other countries as statistical outliers on specific measures, but the pattern for other countries was inconsistent and the outlier differences were smaller than in the U.S.
Sensitivity Analysis. We checked the sensitivity of the results to different methods of excluding the U.S. as an outlier (see above). We removed the U.S. from the performance score calculation of each domain in which it was a statistical outlier on at least one indicator (otherwise keeping the U.S. in calculation of other domains where it was not an outlier (see Appendix 3). In another sensitivity analysis, we excluded the U.S. and other countries from the domains in which they were outliers, but the results were essentially similar.
We tested the stability of the ranking method by running two tests based on Monte Carlo simulation to observe how changes in the measure set or changes in the results on some measures would affect the overall rankings. For the first test, we removed three measure results from the analysis at random and then calculated the overall rankings on the remaining 68 measure results, repeating this procedure for 1,000 combinations selected at random. For the second test, we reassigned at random the survey measure results derived from the Commonwealth Fund International Health Policy surveys across a range of plus or minus 3 percentage points — approximately the 95 percent confidence interval for most measures — recalculating the overall rankings based on the adjusted data and repeating this procedure 1,000 times.
The sensitivity tests showed that the overall performance scores for each country varied but that the ranks clustered within several groups similar to that shown in Exhibit 2. Among the simulations, Norway, the Netherlands, and Australia were nearly always ranked among the three top countries; the U.S. was always ranked at the bottom, while Canada, France, and Switzerland were nearly always ranked between eighth and tenth. The other four countries varied in order between the fourth and seventh ranks. These results suggest that the selected ranking method was only slightly sensitive to the choice of indicators.
Four OECD indicators from the health care outcomes domain (30-day in-hospital mortality rate following acute myocardial infarction, 30-day in-hospital mortality rate following ischemic stroke, maternal mortality, and deaths from suicides) are included in the OECD measures of treatable and preventable mortality. To evaluate the potential impact of double-counting these four measures, we examined the correlations between each of the four measures and the two composite measures and recalculated the performance scores after removing these four measures. The correlations were modest or low. We found little difference in the overall performance scores for the 11 countries after removing the four potentially duplicative OECD indicators.
Limitations
This report has limitations. Some are particular to our analysis, while some are inherent in any effort to assess overall health system performance. No international comparative report can encapsulate every aspect of a complex health care system. As described above, our sensitivity analyses suggests that country rankings in the middle of the distribution (but not the extremes) are somewhat sensitive to small changes in the data or indicators included in the analysis.
Second, despite improvements in recent years, standardized cross-national data on health system performance are limited. The Commonwealth Fund surveys offer unique and detailed data on the experiences of patients and primary care physicians but do not capture important dimensions that might be obtained from medical records or administrative data. Furthermore, patients’ and physicians’ assessments might be affected by their expectations, which could differ by country and culture. Augmenting the survey data with standardized data from other international sources adds to our ability to evaluate population health and disease-specific outcomes. Some topics, such as hospital care and mental health care, are not well covered by currently available international data.
Third, we base our assessment of overall health system performance on five domains — access to care, care process, administrative efficiency, equity, and health care outcomes — which we weight equally in calculating each countries’ overall performance score. Other elements of system performance, such as innovative potential or public health preparedness, are important. We continue to seek feasible standardized indicators to measure other domains.
Fourth, in defining the five domains, we recognize that some measures could plausibly fit within several domains. To inform action, country performance should be examined at the level of individual measures in addition to the domains we have constructed.
Acknowledgments
The authors would like to thank the members of the 2021 advisory panel (Marc Elliott, Niek Klazinga, Jennifer Nuzzo, and Irene Papanicolas); our Commonwealth Fund colleagues including David Blumenthal, Melinda Abrams, Chris Hollander, Jen Wilson, Paul Frame, David Radley, Jesse Baumgartner, and Gaby Aboulafia; and Rie Fujisawa from OECD for their insights and assistance in producing this report.
NOTES
- Organisation for Economic Co-operation and Development (OECD), Avoidable Mortality: OECD/Eurostat Lists of Preventable and Treatable Causes of Death (November 2019 version).
- Eric C. Schneider et al., Mirror, Mirror 2017: International Comparison Reflects Flaws and Opportunities for Better U.S. Health Care (Commonwealth Fund, July 2017).
- Members of the 2021 advisory panel include: Marc Elliott, M.A., Ph.D., Distinguished Chair in Statistics and Senior Principal Researcher, RAND Corporation; Niek Klazinga, M.D., Ph.D., Head of the Health Care Quality Indicators (HCQI) Project, Organisation for Economic Co-operation and Development Health Division; Jennifer Nuzzo, Dr.P.H., Senior Scholar, Johns Hopkins Center for Health Security; Irene Papanicolas, Ph.D., Associate Professor of Health Economics, Department of Health Policy, London School of Economics and Political Science.
- Michelle M. Doty et al., “Income-Related Inequalities in Affordability and Access to Primary Care in Eleven High-Income Countries: 2020 Commonwealth Fund International Health Policy Survey,” Health Affairs 40, no. 1 (Jan. 1, 2021): 113–20.
- Michelle M. Doty et al., “Primary Care Physicians’ Role in Coordinating Medical and Health-Related Social Needs in Eleven Countries: Results from a 2019 Survey of Primary Care Physicians in Eleven High-Income Countries About Their Ability to Coordinate Patients’ Medical Care and with Social Service Providers,” Health Affairs 39, no. 1 (Jan. 1, 2020): 115–23.
- Robin Osborn et al., “Older Americans Were Sicker and Faced More Financial Barriers to Health Care Than Counterparts in Other Countries,” Health Affairs 36, no. 12 (Dec. 1, 2017): 2123–32.