When you come up with a policy that you think, on average, will help society, there can be winners and losers. In fact, the ideal policy should be Pareto efficient; which makes individuals better off without making other individuals worse off. Population-level models only focus on the average person; therefore, they can miss an intervention’s nuanced effect on a few individuals (outliers) despite benefiting the health of most people. As a solution, researchers should know when and how to estimate individualized treatment effects using local instrumental variables (IV). Understanding how to use LIV will allow us to appreciate the importance and usefulness of patient-centered treatment (PeT) effects, especially within the context of patient’s choice to seek (or not seek) treatment.

We believe that the use of individualized treatment effects are critical to informing decision makers about policy, especially in regards to health.  In this article, I summarize the history, equations, and terminology of PeT to stimulate conversations between epidemiologists, biostatisticians, mathematicians, health services researchers, and economists. The goal of the article is to introduce the ideas of PeT so it is easy to synthesize without the complicated mathematics. Ultimately, we want to develop an intuition that flags scenarios where PeT effects can be applied. 

The methods to estimate PeT effects can generate vital information to help with many types of decisions, such as:

• Kaiser covering a new medicine in a health plan
• Medicaid eligibility extending to a broader population
• New York City health department offering financial incentives for viral suppression to HIV patients

Notes: 1) If you are feeling lost in the econometrics literature on this topic, check out the glossary and abbreviations provided at the end of this post; 2) I use the words policy, intervention, treatment, and medicine interchangeably in this discussion: intending to cast a broad net.

THE HISTORY

Heckman’s Legacy

James Hickman developed the ideas and proof for using PeT in policy analysis. Watch this video of Professor James Heckman at the University of Chicago explaining why we should discourage policies based merely on ideology, and instead evaluate and promote empirical-based policies that have credibility. Healthcare is an excellent opportunity for the application of these principles.

Through a series of papers in 1999, 2001, and 2005, Heckman taught us how to use a special type of instrumental variable (called “local”) to adjust for unmeasured confounders in policy analysis and the unique, individualized impacts of these confounders on different types of people (we call this “heterogeneity of effect”). The importance of his ideas earned him a Nobel prize in 2000. Although developed by an economist, Heckman’s methods can be applied to epidemiology, policy analysis, and health services research. However, it is still common for health-related studies to list “unmeasured confounding” as a limitation. Use of this method has the potential to address this limitation, especially in the context of health.

Basu’s Breakthroughs

In the early 2000’s at the University of Chicago, Heckman was an essential influence on economist Anirban Basu, who took Heckman’s algorithms to the next level by applying it in health.

“PeT effects are more individualized than conditional treatment effects from a randomized setting with the same observed characteristics. PeT effects can be easily aggregated to construct any of the mean treatment effect parameters and, more importantly, are well suited to comprehend individual-level treatment effect heterogeneity.”                         – Basu 2013

Recently, Basu and colleagues published this tutorial in the Journal of Health Economics using data his students (my classmates) wrestle with in a dauntingly advanced econometrics class at the University of Washington. Basu and colleagues report that the impact of schooling type (e.g., selective early-tracking system versus non-selective comprehensive schooling) on health using publicly available data from Britain. Below is a step-by-step summary of the paper’s approach to applying PeT to generate causal interpretation for the individual-level effects.

The Step-by-Step Guide

To estimate the impact of [INSERT YOUR FAVORITE INTERVENTION HERE] and the individual heterogeneity in the effects, you can use local instrumental variable methods (LIV) to compute person-centered treatment (PeT) effects by following these steps*:

1. Run a regression that has treatment (yes/no) on the left side of the equation and the IV and measured patient characteristics on the right side (our short primer could help)
2. Use that fit to predict propensity scores for everyone (see our reminders about propensity scores)
3. Check to make sure the scores are spread up and down between 0 and 1 for people in both treatment groups (and trim it up if you don’t)
4. Run a probit regression where the left-hand side is the health outcome and on the right-hand side you have the propensity scores and other things you know about the patient
5. Draw a bunch of random numbers (let’s pick 1,000!) for each person in your data and pretend they are fake propensity scores
6. Use that probit model to re-calculate what you think each person’s health outcome with each fake propensity score
7. For each person, you now have enough information to guess what would have happened to them if they had received the treatment and what would have happened if they had not received the treatment (dependent on their measured characteristics and lots of things we don’t know)
8. For each person in your data, subtract the average health outcome for scores corresponding to not being treated from the average scores for if they had been treated. This is the marginal treatment effect, unique to each person!
9. Taking the mean of this across of the people in your data gets you back to the average treatment effect (ATE).

*My oversimplification of the algorithm provided by Basu et al. (2018)

Conceptually, a PeT effect is a weighted version of the marginal treatment effect. It gives you tons of flexibility to do incredible things, like:

• Look at the inequality of effectiveness between different people
• Figure out who might be hurt instead of helped
• Figure out who would benefit the most
• Understand the effect on multi-dimensional outcomes
• Make a prediction algorithm

Methods We Need to Develop Next

Current instrumental variable PeT methods are designed to evaluate a binary treatment variable. However, we do not yet have methods to answer questions like the impact of clinic quality on adherence to preventative medications and whether the effect is different for different people, such as those with low income. Clinic quality is continuous, and it would be hard to conduct a valid randomized study of this. Basu suggests we start by converting continuous or multi-value treatments into ordered categories. As I write this article, no proof has been developed to satisfy this problem. Please notify me if you solve it.

Conclusion

Associations are easy; valid causal inference is hard. If we want to be honest in evaluating the winners and losers in healthcare policies, then we argue more people and disciplines should learn and use these methods if the necessary conditions are satisfied. Failure to consider these methods could result in unintended consequences and exacerbate existing inequalities in health between patients who are “average” and “outliers.”

 

RESOURCES

To overcome the high barrier that your discipline-specific vocabulary has created to prohibit clear communication with colleagues outside your department, I recommend reading a ridiculously helpful article about synonyms in health research methodology by Maciejewski, Weaver, and Hebert (the last author, coincidentally, also attended the journal club that inspired writing this post).

 

GLOSSARY (in Blythe speak)

heterogeneity –  really different from each other. Example: I have a very heterogeneous bag of marbles with lots of different colors and sizes).

instrumental variable – something that I can find in my non-randomized data this is related to the likelihood that someone received a treatment but is not related to the outcome you are trying to change. You have to run a lot of tests you to make sure you have a good one. Refresh your knowledge with our IV primer.

local instrumental variable – a continuous or multi-value instrumental variable. You need this to vary between people and still be tied to how likely they are to receive a treatment.

person-centered treatment (PeT) effects – how much and in what way the treatment could impact a unique individual

propensity score – the probability that a person given a treatment, adjusting for all the characteristics we know about them; considering these reminders.

ABBREVIATIONS

• ATE: average treatment effect
• CATE: conditional average treatment effects
• C-LATE: conditional local average treatment effects
• LATE: local average treatment effect
• MTE: marginal treatment effect
• PeT: person-centered treatment effect
• TT: treatment effect on the treated
• TUT: treatment effect on the untreated

 

Author’s Note

The views expressed are my own and do not reflect the views of the University of Washington or my research grant funders. I don’t think I have any conflicts of interest here and received no compensation for this article. 

Acknowledgements

This article was inspired by a CHOICE Institute journal club meeting in January 2018 and conversations with Professor Heckman at the Lindau Nobel Laureate Meeting on Economics Sciences in August 2017. Mark Bounthavong, Kangho Suh, and Lauren Strand contributed to the content.

Keywords: patient-level, methods, causal inference, propensity scores, instrumental variables.

SOURCES

1. Basu A, Jones A, Rosa Dias P. Heterogeneity in the impact of type of schooling on adult health and lifestyle. Journal of Health Economics. 57 (2018) 1-14.
2. Basu, A., 2014. Person-Centered Treatment (PeT) effects using instrumental variables: an application to evaluating prostate cancer treatments. J. Appl. Econometrics 29, 671–691.
3. Bounthavong M, Adamson B, Basu A. Instrumental Variables: A Tool to Reduce Bias in Non-Randomized Studies. Value and Outcomes Spotlight. 2016 Feb;2(1) 24-25.
4. Heckman, J.J., Vytlacil, E.J., 1999. Local instrumental variables and latent variable models for identifying and bounding treatment effects. Proc. Natl. Acad. Sci. 96 (8), 4730–4734.
5. Heckman JJ. 2001. Accounting for heterogeneity, diversity and general equilibrium in evaluating social programmes. Economic Development Journal 111: F654–F699.
6. Heckman JJ, Vytlacil E. 2005. Structural equations, treatment effects and econometric policy evaluation. Econometrica 73(3): 669–738.
7. Maciejewski ML, Weaver EM, Hebert PL.  Synonyms in health services research methodology. Med Care Res Rev. 2011 Apr;68(2):156-76.
8. Strand L, Adamson B, Delaney J, Basu A. The Good, the Bad, and the Ugly: Reminders About Propensity Scores. Value and Outcomes Spotlight. 2017 Nov;3(5):Epub ahead of print.
9. Image credit: Photo by Dennis Wise, University of Washington Visual Asset Collection, 2018.