Optimizing Outcomes Through Patient-Therapist Matching with Dr. Michael Constantino

Dan welcomes Dr. Michael Constantino, a professor of clinical psychology and director of the Psychotherapy Research Lab at the University of Massachusetts Amherst.

Dr. Constantino discusses the evolution of psychotherapy research, and the importance of understanding therapist effects and patient-therapist matching. Dr. Constantino emphasizes the limitations of one-size-fits-all treatments and explores how matching patients to therapists based on their strengths can lead to better outcomes, particularly for those with severe issues. Dan and Dr. Constantino dive into findings of the Match Trial, which demonstrated significant benefits of matching patients to therapists with proven effectiveness in treating their specific problems.

Special Guest: Dr. Michael Constantino

Psychotherapy Research Lab

https://www.pcori.org/research-results/2015/matching-patients-therapists-improve-mental-health-care

https://www.pcori.org/research-results/2022/implementing-matching-patients-mental-healthcare-therapists-strengths

Constantino, M. J., Coyne, A. E., Boswell, J. F., Goldfried, M. R., & Castonguay, L. G. (2023). Training on context-responsive psychotherapy integration: An evidence-informed framework. In L. G. Castonguay & C. E. Hill (Eds.), Becoming better psychotherapists: Advancing training and supervision (pp. 85–105). American Psychological Association. https://doi.org/10.1037/0000364-005

🎞️ Video version of the show@PsychotherapyAppliedPsychology on YouTube
🛜 Check out the website: Listen to every episode on your podcast player of choice

Connect with Dan
☏ Leave a voice message on Speakpipe
🔗 LinkedIn
📬 TheAppliedPsychologyPodcast@gmail.com

🦋@danielwcox.bsky.social

Chapters

• 0:00 Introduction
• 5:27 Process vs. Outcome Research & Limitations of One-Size Treatments
• 10:48 Understanding Therapist Effects: Between vs. Within Therapists
• 16:19 Matching Therapists to Client Needs & Therapist Self-Awareness
• 24:00 Designing and Running the Therapist-Client Match Trial
• 33:38 Results, Expectancy Effects, and Implementation in Clinics
• 36:57 Who Benefits Most from Matching
• 39:41 Implications for Mild vs. Severe Clients
• 46:25 Severity Profiles & Research Directions

Transcript

0:00

[Music] Psychotherapy is fundamentally about relationships, relationships between therapists and clients. But what makes some relationships work better than others? In this episode, I'm joined by a leading psychotherapy researcher, whose work on therapist effects and patient therapist matching has reshaped the way that we think about effective treatment. So first, if you're new here, I'm your host, Dr. Dan Cox, the professor of counseling psychology at the University of British Columbia. Welcome to psychotherapy and applied psychology, where I dive deep with leading researchers to uncover practical insights, pull back the curtain, and hopefully have a little bit of fun along the way. And if you enjoy the show, consider sharing it with someone who might find it meaningful, so one of the best ways to keep the conversation going. This episode begins with my guest talking about how we got into studying psychotherapy. So without further ado, it is my absolute pleasure to welcome my very special guest, Dr. Michael Constantino.[Music] I tend to think my pathway has been pretty linear, and I have felt fortunate about that. So I think, like as early as high school, I was interested in psychology. Of course, I don't know that I knew what all of that entailed. But then, you know, became a psych major, and I think I was drawn a lot to trying to understand and even create some of my own very crude theories about, like self-concept. And how self-concept, or one self-concept, or the makeup of one self, really helped people develop interpersonal effectiveness or interfered with that. So I think I had early on this idea of self and other, as sort of a foundational idea of interest. And I probably could have very easily been a social psychologist if I had had certain formative experiences, you know, that took me that way. And in fact, a lot of my psychotherapy research, you know, kind of brings in social psych constructs. We won't necessarily talk about those today, but I've just had a longstanding interest in, you know, self and relationships. And so, you know, when I started looking toward grad schools, you know, this idea of interpersonal effectiveness, I was like, oh, so if you're not interpersonal effective, what might you do? Well, maybe go to therapy. And then of course, light bulbs go off that probably should have always been on, which is like, oh, therapy's a relationship. And in therapy, you're in a relationship, you're probably working on relationships or your sense of self. And Louis Castingay was doing research on the therapeutic alliance at Penn State. And I thought, well, that could be a perfect blend of all of these things. And so fortunately, I applied to work with Louis and got into Penn State. And that was the beginning of a career in psychotherapy research that really centered on, you know, patient therapist and relational dynamics in the context of the therapeutic endeavor. That's very like, and you describing that that feels very sophisticated for, you know, an undergrad. I think part of what, you know, looking, I'm probably telling it much better looking back than I would have saying, where am I going back then? But, you know, part of it was I never wavered on my interest in psychology. So I, you know, I immediately became a psych major, I stayed a psych major, I got to work in both clinical and social labs. And it really just seemed like, you know, overly simplistic, but that the idea of, you know, relationship dynamics and how, how individual self concepts like bear on those. And I just thought it was really interesting to think about a treatment that was not a prescription, but rather a relationship that there must be people studying that process. Right. And I will say in a very unsophisticated way, it took me a long time to know how to define process research or process outcome research, even though I was doing it in the Wies lab, I would go to a conference and, you know, one of my former undergrad pals, I might have seen somewhere and they said, oh, what do you do? And I was like, every process recently, like, what's that? And I was like, hmm, how do I define this? So how do you define it? Yeah. So it wasn't luckily, Louis and I got asked to define it for it in psychopedia chapter. And so we had these two really nice concise entries, you know, about outcomes research really focuses on often comparing full treatment packages and the effect that they have on patients mental health. And this process research really tries to uncover questions of like, how do they work? What are various things that predict outcomes, you know, either within or across different therapies and really opening the black box of the psychotherapy encounter to know, you know, what does it look like? And what are the things that are most facilitated? And what are things that are most hindered? So, yeah, so I guess it, you know, there became a more sophisticated narrative eventually. But I guess I've always been interested in the participants of psychotherapy and what they bring into it. And then the dynamics that unfold. So this goes into a question I like to ask people so, you know, for the last three decades or so, a lot of psychotherapy research and what I don't know if is at least was has been the gold standard is sort of this treatment for this problem. Yeah. What do you think about that? And what do you think about the potential limitations of that? Yeah. Yeah, it's really interesting. I think a lot of my research, even before I could have said it in a well articulated way, especially early on focusing on things like the therapeutic alliance or patient expectations or hope, you know, a lot of that was born out of an inherent. And then I think that frustration with the idea that, you know, one size treatment should fit all. And really in my mind, reading a lot of things that were about how this treatment should target these problems, which should then result in, you know, post treatment big outcomes. And it great like if you had the perfect candidate for therapy and they believed in the rationale for the treatment, it fit their understanding of human suffering and human change. And when that works fantastic, right, and maybe when it works and all of those things aligned, maybe therapy can be more prescriptive so to speak, you know, do this because they're a good aggregated data that say this is a good path. But, you know, how many times do all of those things align, right? And, you know, the idea of the idea of treatment almost being handed out like a pill or, you know, like delivered as though you can control and mask the person of the therapist. And it just has to be something that you learn the skills and then you deliver in the same way to everyone. That was just always really limiting and frustrating in some ways to me. And so I think my early, the way I channeled that frustration early on was to say, okay, what are the things that cut across treatments that seem to be helpful because then we have these common factors where we don't have to be so biased toward or clinging to, you know, a label, right, with that label often being an acronym, you know, some kind of, you know, treatment that has three letters, you know, ending in tea. And so I originally got really interested in this idea of common factors, but it was still very much within, you know, the framework of evidence based treatment is delivering, it's delivering manualized treatments or at least, you know, theory informed treatments, but my interest at the time was, can we identify any principles, you know, that are just, you know, relevant to all of them, you know, or at least not unique to any of them. And so that was my first sort of take on addressing my sort of own frustrations or ideas of limitations about that model of evidence based therapies or treatment packages. I was far more interested in evidence based principles, you know, or just calling it evidence based practice, which sometimes might mean adhering to certain manualized principles, but other times might mean changing course, because your patient tells you this is working. Or your patient tells you, I don't really believe in that idea of like how people, what you're telling me, I don't really think that's going to be the thing that allows at least me to improve or I've tried that before, you know, something like that. Yeah, and then I would say a later take on that was really trying to understand more empirically how to personalize. So not just the idea of can we locate common factors, but can we also find treatments or ultimately, as we'll talk more about today, providers who are particularly well suited to certain patients, or can we find moments in therapy that are going to be much more likely to be addressed successfully. If you do this instead of that and doing this instead of that might mean temporarily putting some kind of treatment package on the shelf and changing course departing. We've called them departure modules because we've tried to test whether there are specific ways that therapists can behave in the face of certain markers like patient resistance. It's not uncommon that a patient at some point will tell you this isn't working or I don't like how it's going. I didn't like that homework assignment, whatever. And you know, some of our work about a decade ago was to show that being more Carl Rogers like in those moments is better than trying to double down on your agenda or your rationale that you might be using, for example, from a cognitive or some other model. And I'm just a listener. So just to let you know, you know, when you're listening to Mike sort of push back on some of those. Just you know, this treatment for this problem, something to know and we'll get into this is that Mike has done. You know, he's very much in the experimental world where he's doing randomized control trials of these very nuanced processes. Which is one of the great reasons to talk to you is that, you know, you get some people who are sort of push back against the RCT approach in general. And a lot of your work, at least as some of your work as I read it is, you're still doing RCTs, but you're doing RCTs of these other processes in these really sophisticated and creative ways, which will get to in a few minutes. So, but before we get there, so we're going to talk mostly about therapist effects today. So when you say therapist effects, what does that mean? Yeah, it's a really important question, you know, because actually I've contributed to projects that have been labeled something like a therapist effect. And really what people are talking about are therapists variables, right? Anything that relates to a therapist or that you measure from a therapist perspective is a therapist factor or variable, right? But the therapist effect has a really particular meaning both empirically and clinically, right? And the classic effect. Well, let me first say to cover the various kinds of therapist effects, it's simply the idea that who the therapist is has some influence on their patients outcomes beyond the treatment they're using, beyond other characteristics. The patients have that may may possess beyond different contextual factors. It's simply the idea that some variability in patients outcome is explained by who they see. All right, now the classic therapist effect is referred to as the between therapist effect. And this simply means that if you compared the average patients outcomes for therapist a, they may be significantly higher than the average patient outcome for therapist b, meaning the average patient on each of their caseloads. So it could be that therapist a and therapist b both saw 100 patients. And if you average their outcome score on some measure, let's say the outcome questioner, so symptom and functional impairment. It could be that therapist a average patient is significantly better in their outcome than therapist b's, right? And that has been replicated well established meta analytically. And if you want to put it in, you know, that there are some therapists that are better than others. Some that the therapist significantly influences patient outcomes in the most simplest manifestation of that or the most straightforward manifestation of that is that some therapists are better than others in general on their patients global outcome. And so, you know, that what that basically, you know, told us if you want to put it in terms of like percent variance explained, I think that's a, that's a pretty easy one for people to understand that who the therapist is explains about 5% of differences people have or patients have in their outcomes. And if we want to compare that to something like the therapeutic alliance, the alliance explains about 7 and a half percent, right? So that doesn't sound like a lot, but of course in psychotherapy, there are so many things that contribute to why some patients do better than others. That explaining 5% is significant. And even the variable like the alliance that we tend to think is the most predictive of outcome doesn't explain much more than that. So it's a meaningful percentage. Now that that kind of classic effect largely stems from having some kind of routine outcome measure that gives you some kind of global score. And then you compare therapists, you know, again, their average patients outcomes to each other. Well, we started to become interested and was a second type of therapist effect and, you know, the first way to refer to it would be a within therapist effect, but the, I think, an easier way to think of it is that therapists themselves have strengths and weaknesses within their own practice in terms of who or what they treat. And so we got very interested in this notion over the past, you know, 10 to 15 years, we have been using a multi-dimensional outcome measure that looks at different types of problem areas like depression, anxiety, substance use, you know, relational conflict, et cetera. And we have 12 scales in this measure that is called the treatment outcome package or top. And we have done a lot of, or we did a lot of preliminary work just to show that therapists indeed have empirical strengths and weaknesses when treating their average patient whose primary problem is depression versus anxiety versus substance misuse, et cetera. And so, in the news department, most therapists show some area of effectiveness or even exceptional effectiveness. And it's often in at least maybe three or so of those 12 domains. That's the good news. And the good news is that there are no therapists in our thousands of therapists we've studied in this way, who are exceptional at all 12, perhaps we should have expected that. So maybe that's not terrible news. But there is about, there are about 4% of therapists who are either ineffective or harmful in all 12 domains. And so, that's the idea of a harmful effect or a harmful therapist effect. But again, we want to hone in on the sort of positive ways in which we might leverage this for a few minutes. And so, that's the idea of a harmful effect or a harmful therapist effect. And so, that's the idea of a harmful effect or a harmful therapist effect. But again, we want to hone in on the sort of positive ways in which we might leverage this for clinical benefit. And that is, can we learn, and the answer is, yes, we can learn it. We can talk more in a minute about what we can do with it. But can we learn that certain therapists are just going to be better suited to using their talents or their skills or their effectiveness when they're treating certain types of patient presenting problems relative to others. And there are two immediate implications one could think about that, right? As if we learn that, then that therapist might choose to specialize to what they're excellent at. Or if they're really wed to being a generalist and treating just about anyone who comes through their door, then it might suggest they areas where they need more training. Or some kind of way in which they can address those areas where they're weaker. And again, weaker can sometimes just mean empirically their average patient who has a certain type of problem is not changing to an expected degree. And all of this, by the way, is when you adjust for risk. So we're not just saying that if a therapist only saw the hardest to treat patients, we would call them ineffective. We would only call them ineffective if those very difficult patients weren't changing to a degree a statistical model would predict they should, even if that degree was just they should change a little bit. So, okay, so there's so let so you've done a lot of work on this patient therapist matching idea. So could we could you give us a just general overview overview and you did one particularly notable RCT where you did this could you give us an overview of that project and please feel free to talk about related whatever you think would fit for helping the listener understand. And, oh, okay, this is what they did. This is how this looked. Right. Yeah. So, so we have this we have the, you know, several studies where again, we established that therapists have strengths and weaknesses empirically based on what type of problem they treat. So we had done that also before we get to the trial, the match trial, we had also just done some qualitative research with patients and therapists and other administrators in community mental health clinics just to say, what would you make of this and would you be interested in helping in using therapist performance data so to speak to inform clinical decisions. In terms of who sees who who you might want to see who you got referred to, would you want to know that a therapist, for example, had a history of being really successful at treating depression, but they've been less successful at treating anxiety, would you want to know that. And of course, most patients said, sure, if you could give me that information and you could help me understand it and those therapists were available, that would be great. And that's, you know, just that's what I think about, you know, we have in our lives, people approach you and say, hey, I'm struggling with this or my sister struggling with this. Do you have any recommendations? And in your head, I assume you do the exact same thing that I do, which is I think, okay, that's their problem. I know who their sister is or something about them, who would be most likely to help that sister with that particular problem. Right. That's sort of the thing we do in that role of Dix in our head. Absolutely do. And then the question is, what are we basing that on? Is it because we like that therapist, our colleague, is it because we know they tend to treat a lot of depressed patients, for example, is it because their website says I'm an expert on this. And all of that makes sense and there that isn't a knock, it just it's sort of the way it has been, right. Well, so in one other preliminary study to the trial that I'll mention, we also not only measured therapist strengths and weaknesses across these 12 domains. But separately, without therapists knowing their strengths or weaknesses, we gave them a questionnaire that just said in these 12 areas, do you think you are exceptionally effective, relatively neutral, meaning your patients change right about to the degree we would expect, or do you think you're ineffective or harmful. And then it turns out that therapists were no better than chance at predicting their actual measurement based strengths. So this was the study that you know sort of built on prior surveys that simply asked therapists how effective do you think you are in almost every therapist said I'm near the top. So that's the way to show, look, if we measure your actual patient outcomes and we ask you to predict where you think you're really effective or not, therapists are not very good at that. So that tells us when therapists are advertising on their websites, when we are making referrals to our friends and colleagues because we think it's a good fit. And then we have a chance that therapists don't know their own areas of expertise through no fault of their own because of course how we train we think that you build expertise by getting certain content and getting certain experience. And that is where our field has been. And it's just intuitive as well, right? At some point you start seeing, when you start seeing patients and well maybe not when you start, but over time you start to say like, oh man, I really, I really, I think it seems like these people with moderate depression, I'm really knocking out of the part or the people who are struggling with this transition or the substance use folks or whatever it happens to be. Like it's just being human. So to an extent when I read that in, you know, in preparation for this, I was sort of like, oh yeah, that makes sense. We aren't good at just generally with, you know, just insight, right? Humans aren't that great with this sort of predicting what. But then the more I thought about the more I'm like, huh, that actually, no, I don't know if you were to put a gun to my head. Would I say coin flip? No, I don't think I would. I would say that that therapist had a better than average. I'm not saying they're going to be 100% that, but I would think it would be better than 50/50 chance. And your data don't suggest that they suggest we don't know. We don't know. We don't know more people should do this kind of work. You know, this is a study. So like it's I won't say it, you know, definitively. And I will say some other things about that study. So, you know, one thing is that it seemed to be for the conditions that are most prevalent that that therapist treat and outpatient settings like depression and anxiety. That was where we saw the overconfidence bias. Therapists were better at saying if I see a patient with psychotic symptoms, I'm probably average or maybe even below average. Therapists were far more willing or at least accurate to classify themselves in a way that supported their own data when it was for harder to treat less prevalent problems that they might see in an outpatient community based therapy setting. So the overconfidence bias tends to be for the things that therapist treat, you know, most frequently. But we seem to understand is therapists that some patients are hard and I'm not going to judge myself as though I'm awesome at treating them. I get it that I'm probably average, you know, when I when I treat these folks. So I thought that was an interesting little nuance. But anyway, the for us, we thought we had accumulated enough data, both the data that established the within therapist strengths and weaknesses affect the fact that we had patients, therapists, administrators telling us, yeah, I'd be interested in that idea. And the fact that therapists aren't particularly good at judging their own effectiveness in these 12 domains. So we thought, let's apply for funding and see if we can do a trial where the main manipulation happens before therapy. We never changed or manipulated anything about treatment. We went into a community mental health clinic or network. And we basically asked the question, if patients are assigned prospectively and intentionally to a therapist who has a history of being exceptionally effective in treating their primary problem, will that outperform or will those patients do better by the end of naturalistic, unmanipulated treatment, you know, therapists just doing what they do. Will those folks do better than ones who were assigned as usual and as usual fits the example you gave before they heard from a friend that this person's great or as usual for a person walking into an intake center might be this therapist is available tomorrow. So just so just so just you took half the patients and you assigned them to the based on the patient's primary problem challenge you assigned them to the therapist who are supposed to kick ass with that particular problem and the other half you didn't touch. So some of them just by coincidence did end up with the excellent right so whatever right whatever normally happens in a center or in that particular center that particular network just let that happen. So we're not saying we're putting people with the bad ones we're just letting natural stuff happen exactly and indeed you know their patients in the control group. Some of them were assigned to a really good fitting therapist empirically yep and so it's a great way of summarizing it to other quick points one is that the patient and the therapist were masked so therapist didn't know their historical strengths even though we measured them and they didn't know that the intake center was making assignments according to those historical strengths and the patients didn't know they were being matched or not the patients only knew that we were doing a study that was looking at different ways. To optimize the patient therapist referral connection that's all they knew so any effect that I'm going to talk about is it's important to keep in mind that neither the patient or the therapist knew this matching took place. And that they're presenting their highest presenting concern was based on the first thing they did when they arrived at the intake was take the treatment outcome package measure that established their most elevated concern. So the when you're talking about the therapist who sort of are like really good at a particular problem and you were talking about sort of the what you would so you're talking about client outcome in terms of how much you would expect the client to improve so the way that expect to improve is based on thousands of patients over the years that like the patient with this particular profile if you will they have a typical trajectory. Of improvement that's a beautiful way of saying what was a really sophisticated statistical machine learning model in a reference sample of thousands of patients and hundreds of therapists right yeah we learned that when patients had certain demographic and clinical and historical characteristics. That we could we could use a model to say this is how much we would predict them to get better on on the global outcome score of the top meaning their total symptomatology their total level of functioning. This is how much this patient would be expected to change right so right so if you have certain characteristics some patients are going to pretty much all patients are predicted to get better pretty much. So that's how you would have certain challenges certain situations they're going to get better more quickly and more versus more slowly so just okay great okay so they're predictive somebody might come in and be worried well so to speak and their their prediction is that they're going to you know I mean that's not a great example because they might have less room to change but like let's say somebody comes in and they're high distress. So that's what they're going to do is they're going to get you know whatever their characteristics are in their history and their their personality their attachment etc those things might predict that they're going to change substantially by the end of their and a typical routine outcome monitoring tool. So that's the typical routine outcome monitoring and feedback literature that's been out there since my Lambert's work several decades ago and we were just basically saying since we have those data we can also analyze them at the therapist level. So what we did was at the baseline of this trial before we started recruiting trial patients for the 48 therapists who participated we spent several months tracking how their average patient their sort of average patients pre post outcome in each of these 12 domains and we made sure that they had at least 15 or so patients it turns out we got these performance profiles on a mean of about 28 pre trial cases. Yep and so since these are not diagnoses they're 12 dimensions of symptoms are functioning every time a patient completes the top that adds a patient to that therapist caseload that we are analyzing to say to classify them in one of three ways for each of the 12 domains effective simply means their average patient in a given domain changed more than the algorithm suggested that they should. Right their average patient relative to themselves and their own predicted change rate got changed more than we would expect that is what we called effective or maybe even better yet exceptionally effective right so the so the exceptionally effective so those that grouping of patients with that particular profile they're basically yeah beating what's expected from an out. Yep right the like if they meet the algorithm or third sort of that that still in most cases those patients are still improving yeah yeah we think we think of a good outcome like you're you're getting your patients right about to where you would expect them to to change not exceptional we called it neutral I actually don't think that's a great label looking back. You know but but you know maybe think a bit more average you have average outcomes but your patients are changing right about to where the model would expect them to right so you sort of expected better than expected worse than expected exactly yeah I think now we might be in some of our writing it's more of like exceptionally effective effective or average average effectiveness and then ineffective and recall that ineffective can be one of two things ineffective could be your average patient changes a little bit. It changes a little bit less than expected or your average patient gets worse than where they started and we do know that some patients are harmed by therapy and that some therapists are more likely to have patients deteriorate than others again that the percentage is low which is the good news part but it's meaningful which is the bad news part. So yeah so once we had so at the baseline of this trial 48 therapists had a classification across an average of 28 historical patients and then we started randomly assigning patients and again you you were matched or or you got assigned as usual which meant if you were to be matched it was purely by chance. And we then looked you know to just to oversimplify our analyses you know did one group do better than the other and as it turns out our hypothesis was supported with a large effect size large how many patients were in this study large between group of effect size so over 200 I'm trying to remember the exact number per per group but you know we're talking about yeah over 200 yeah over there you're some patients. And the you know to the to the level that we needed to be statistically powered going in we we saw an effective match that with a with a large between group effect size I think the D was 0.75 so close to the whole yeah that's huge and I mean the other thing to note about that is that in the in the treatment is usual condition in the control condition. You know some of those folks are still getting matched to the optimal therapist so that really because I mean going into that you had to be a little bit nervous I would have been very like yeah because because I mean right in the in the in the perfect science well I don't know depends on what but like one can say we sort of want to do match to the optimal match to the worst right like that would be the cleanest way if our goal is to just find an effect but to have it be clinically murky yeah we could have we could have made it so much better that we could have made it better to be more accurate and more accurate and more accurate and more accurate and more accurate. So you know we could have made the control group be like you're going to get somebody who's not an expert for sure right but to so like you were there is right in any study there still is a chance that your random is a or your in your case your treatment is usual that you would have lousy luck yeah and that a bunch of your patients would have been assigned to a bunch of the best therapist and then you would have been so but and so some of that still did happen and for that effect size to be that the way it's supposed to be that huge is really saying something.- Yeah, and a couple of thoughts about that, I mean, it's a, it's a perfect way of saying it. You know, so we were nervous. And we knew that this was a, a chance that, you know, because it was as usual, who knew if in this particular community as usual, resulted in a lot of the patients they see seeing a therapist who had an expertise in that area. Maybe it was a unique therapist sample where there was a lot of expertise in, let's say, depression and that happened to be a clinic where a lot of depressed patients came, and by chance they got well matched. So we were nervous about that. We were also nervous that we knew we weren't going to be able to capitalize on expectancy effects, right? Because our patients didn't know they were being matched. And in fact, in a future trial, we actually think we'll see a bigger effect of matching if the patient on the therapist know it's happening, right? Because then they would have this expectancy effect of like,"Ooh, I'm seeing a therapist who I know is really good at this." And the therapist is saying, "Yep, I have this expertise and it's supported by data." And, you know, whatever the mechanisms of expectancies or placebo are, that can be an additive effect to the match itself, right? Right. And the expectancy effect, and I think this is where you're getting at, at least in some ways, is that would be more actually what would happen in the real world. So if you have worked at a center where the typical way of operating was folks got matched to the therapist that are best, but that's just what we do here. Then that, and the patients got explained that on the front end. The therapist obviously knew that. Everybody knew that. That, I mean, maybe there could be some confounds I'm missing here. But like that, that would be built into the system, which you actually systematically removed from your first RCT looking. Exactly. And I'll skip ahead a little bit, but then I'll back up to where we are, which is our funder, Patient-centered outcomes research institute of PCORI, has also funded us post trial to do an implementation study. And the implementation work is looking just like you said. The clinics that we're working with in this implementation project are basically just saying, "We are just going to adopt this. Our therapists are going to know, our patients are going to know, our intake workers are going to know, and it's just how we're going to do business." So now we have at least a naturalistic way of seeing, you know, without setting up a trial or an experiment, we will have some data to tell us, do we see a bump just if we benchmark people's improvement levels by virtue of being matched to the trial? Do we maybe even see bigger amounts of change because patients and therapists know it's happening? It just becomes standard operating procedure in those clinics. Yeah. But back to the trial, I mean, yeah, we were really nervous and then we were really excited that we saw an effect. And we, you know, like any trial, we looked at whether our manipulation worked and indeed more patients were matched in 100% of the patients were matched in the match group. And I can't remember quite the percentage, but some in the as usual, but like you said, not enough to dilute the effect. Like less than 25%. Yeah, less than that. Yeah, exactly. So that also told us what was happening by chance in this clinic. Yeah, right. Yeah, about a less than a quarter of the people were getting matched by chance to a therapist who was empirically good-fitting. And so we found this effect. We also found an expected and then an unexpected moderator of the effect. So the effect became stronger, perhaps unsurprisingly, for patients who presented with the most severe pathology. So for patients who's overall score on a measure of symptoms and functioning combined, when that was more severe at baseline, being matched was even more effective for that group. And another way to put that is for the patients, and we might come back to this later when you, I think you might end up wanting to ask about a pushback on some of this stuff. We'll come back to this idea, but another way to say that is for patients who are less severe, so for the patients with milder pathology entering treatment, matching really didn't matter. That those patients could see just about any therapist-- At all, it didn't matter at all. It didn't matter. At least at an aggregate between group level, matching did not have an effect at the lower end of presenting severity. So we can think about that later in terms of resource allocation. Like what if a clinic didn't have a certain expertise? And it's like, well, maybe those therapists who are just average at everything. If you knew that, those therapists are still very valuable resources because they could see the more milder or mild to moderate patients. And we know empirically that those patients would still get better, at least to the degree that we would expect. Another way of putting that is matching to a higher performing therapy really wouldn't bump them up at all. So it was for the most severe patients that matching was-- it almost like doubly effective for them. And the same thing was true for people who identified as a person of color. And that one we didn't expect to find, but in looking back on it, if you think about the marginalization and the harms that these folks have traditionally faced in the psychotherapy world, somehow-- again, they were masked-- but somehow they experienced the therapist's expertise in a way that helped translate into better outcomes when they were in that match condition than when they weren't. We are really interested in following this up to learn more about what that was all about because it wasn't like we got to tell them, hey, we know you might have had challenging experiences in therapy in the past. But what we're going to do now is we're going to match you to someone who we know is really effective at working with the problem you have. We didn't tell them that. So it's hard to know the mechanism of that moderated effect. But we hope to understand that better. At just an outcome level, maybe this type of matching is at least one pathway to improving mental health care equity. So I think there's a lot there. So let's jump back real fast just to put a bow on this. So for the folks who were the most severe-- for the folks that had the most severe symptoms, the life disruption, whatever-- on your outcome measure, when they started, for them, matching to therapists who are particularly strong in whatever domain was their most significant, most important, most severe domain-- that really mattered a lot in terms of them improving when you looked at folks who were moderate severity, were just using that language, generically not right, that matching doesn't seem to matter that much. Matching on effectiveness for their primary presenting problem didn't matter that much. OK. It didn't seem to help or hurt. Right. The as usual way of assigning for those milder or milder moderate cases would be empirically supported. OK. So let's just roominate on this for a moment. So I think-- I'm trying to remember-- but I remember some research from a while ago sort of making the argument that-- what that-- so not the matching thing, but just generally speaking, that when you look at really good therapists versus weaker therapists or average therapists or whatever, that the difference does have to do with working with really difficult clients. So if the work you're referring to now is reverting back to the idea of the between therapists, that we're just generally speaking, when therapists have differences in overall effectiveness, not domain-specific, it is true that one of the things that amplifies therapist differences is working with more severe patients. And if you think about what that means, it means that when you see a patient who's presenting with severe pathology, some therapists get even better, some therapists stay the same, and some therapists get worse. That is what we mean by the therapist effect getting bigger. The amount of variability explained in the severe patient's outcomes that is explained by the therapist gets bigger. And again, what that means is some therapists will do really well with those more challenging cases for lack of a better phrase. Others will do about the same, and other therapists will get worse. They'll get really disrupted in their effectiveness by seeing that particular group. So you're right that there has been a longstanding link between severity and the between therapist effect. In this case, it was the idea that severity here, think of it as a patient level moderator of the match condition. And in the match condition, we essentially adjusted for severity because everybody's own expected change rate took that into account in getting matched to a therapist who had that particular level of effectiveness. So it became this moderator variable for the match condition relative to the control condition. So it's a nuanced difference. But you're right to say that there's a long history of connecting therapist effectiveness differences, mattering more-- or in this case, how you leverage those differences matters more when you're talking about high severity cases. Well, it seems to me like one of the contributions of your work is you took this existing finding of between therapist effects of some therapist get better-- or basically, some therapist-- there's not that much difference between therapist when the patients at a lower moderate or whatever level of dysfunction, but at higher levels of dysfunction, there is a big between therapist effect. But you took that between therapist effect and opened it up and said, let's get some more nuance of what's going on here. And you extended it by saying, yeah, that might be true that on average, therapists are-- that it doesn't matter so much what therapist when you have mild, moderate, level severity. But what we're seeing is so the reason-- or one of the reasons why at higher levels of severity, some therapist or better or worse-- has to do with this matching phenomena, at least in part. So like, this is where-- so that really expands that finding to be more nuanced to say-- Exactly. It's the matching piece that matters. And so a couple-- so a question-- I guess this might not be anything you reported in any of your papers about this, but just generally, were the folks who were more-- you might not know. But I'm sort of curious, were the folks that were more severe, were they more differentiated in terms of their profile, like were they had, what, like, their presenting problem was spiked through the roof and everything else was moderate? You know what I mean? Or was it all-- Yeah, I mean, so because the top gives you these 12 domains, you do see variability in how many standard deviations away from a community sample of non-distressed and on treatment-seeking individuals, are they in a given area? But then you also get this total score across the 12 domains. And that's the score we're talking about here as the moderator of the match effect. And so it was really just one score. Of course, there was variability. But I think your question is a good one. And that for those who were the most standard deviations away from a non-distressed community norm, did their profiles look like one or two areas were through the roof and everything else was-- we haven't really done a profile analysis like that. I think that's really interesting, though, because that will allow us to better know what severity looks like, at least in this sample. And we might be able to think of other ways to take that into account in case assignment or treatment personalization. Yeah. Yeah, I'm just sort of thinking about like, what is the-- Yeah, you know, just like you're doing-- trying to explain what's actually happening here, you know? What's really interesting about your question is that you're thinking of it like, what does an individual patient's profile look like? We were thinking of it as what is a therapist's strengths and weaknesses profile look like across these 12 domains. And I will tell you, I don't want to get too into this because I think right now we have a nice, clean understanding of the trial. But we did have levels of matching. So I have been talking about it as matching was simply when a patient, when you matched a therapist who had effectiveness in the patient's single, highest presenting problem area. But we also had a higher level of match than that. The highest level of match was when the therapist was effective at treating the patient's three most elevated problems. And that therapist was never ineffective on any of the other domains. So it's like the highest level of matches you have a therapist who has strength in your three highest areas of concern. And that therapist is never harmful. That was the highest level. The lowest level of matching, there were five levels. I know I won't go into all of them, but the level-- Yeah, but the middle level is what we've been talking about. You match on your primary concern, one primary concern. The lowest level of matching was simply, you got a therapist who was average across the board. They were never harmful, but they also didn't have an exceptional strength in your area. And it turns out that level of matching expectedly did moderate the effect. The match effect was stronger at higher levels of matching. But it was still present at the lowest level of matching, meaning simply getting assigned to a therapist who would not harm you or would not be ineffective outperformed, getting assigned to a therapist by chance. So that, I thought, was really cool. Like the lowest level of our matching still outperformed case assignment, as usual. And of course, the more effective your therapist was for more of your problems, and the more that therapist never had any ineffectiveness in any other domains, then the match effect got bigger, and it got stronger, of course. But to us, it's like even if you are willing to assign patients away from harm, that can be better than, as usual, chance-based matching. Well, I think what you're saying is really interesting, because I could totally see-- you said, of course, but to me, I think it's fascinating, because one could imagine a threshold effect whereby if the therapist is particularly good with their one presenting problem, or that there's a certain level of competence, I don't know if that-- effectiveness will probably be a better term here. As long as they meet that threshold, then it's not really going to make that much of a difference. But what you're saying is it's more of a continuum, where you get more variables. If you can match on more of these variables that actually continues to be improved, improved, improved. It can be used to matter, yeah, exactly. So if those therapists exist in a given network, then an intake worker should absolutely try to make that assignment. It's like, oh, right, because this whole notion is based on a system that will produce a short list of therapists for any given patient who are considered empirically well-matched at one of our five levels. And so the idea is that hopefully many therapists will show up on that short list. And that some of those therapists will also take the patient's insurance, will also have an opening. But if any of those therapists, when you're taking these other factors into account, happen to be strong on all three of the patients, most significant presenting problems, then try to get the patient to that therapist. But you still know that even if you can't, anybody else on that short list would outperform making a random assignment or just basing an untherapist, self-described expertise. So what ballpark, I don't know, you only didn't know these numbers had these numbers memorized. What percentage of the 200 and something patients were categorized as severe? Yeah, yeah, I'd have to go back to the-- I mean, is it like 25%, 50%, and-- It was a continuous variable, right? So there was no particular cutoff. It was a continuous interaction of being in the mass condition and having more versus less severity. So there is no clinical cut point that we're saying, once you get here, you're now moderately severe or very severe. It was a continuous variable. And so it's hard to say precisely when it starts to matter that somebody's severe enough. We just know that as you got more severe matching matter even more, and we've now talked about three distinct moderators. So matching worked better for high severity cases, matching worked better for people of color, and matching worked better when there was a higher versus lower level of match. But even the lowest level of matching worked better significantly than doing nothing. Yeah, I think that you have-- having these moderator effects-- it's a large study, obviously. And it's not, right? So the fact that you found this for that-- for that the amount of matching is almost just from a disherum sort of a statistical-- it's pretty impressive. You know, I mean, it really speaks to your large effect size. It absolutely does because you're right. We had 48 therapists in 218, I think, patients, which is big for a trial, but it's not enormous. This is not big data we're working with. We are working with big data to establish patients predicted change profiles or slopes. We are working with big data to establish therapists' strengths and weaknesses. But when we talk about manipulating the case assignment for one particular trial, it was a good size trial, but it wasn't huge. Right. And so you're absolutely right. That created anxiety going in. Will we find this? But also coming out, it tells us how powerful the main effect of matching was that you then also saw some of these differential results or amplifying of the match condition for certain levels of a patient characteristic or a match characteristic. So were there like, did you-- how can I say this? So you had the therapists, therapists who were particularly good, particularly effective with certain presenting problems or primary problems. Did it look like-- and I think it was about three was the average-- I think it was two points, something if I remember correctly-- was the average number of presenting problems that a person was really good at. Or probably-- Go ahead and look at the main effect. Yeah, yeah. So did you see in those data, like, there are these 48 therapists. There are some percentage that are expert therapists who have a whole bunch of domains that they're really good at. And then most therapists are good at one. And then there are some therapists who aren't particularly good at any. Or is it really like-- no, when we look at it, a lot of people are really two, three, four-- does that make sense? Does that make sense? That's exactly-- yeah. If you combine the trial data with our much bigger data sets with David Kraus and James Boswell and I and Alice Coin that we-- other studies we published, it's the latter. It's most-- the most therapists are going to be good at three or four domains. And I think the highest we see for a handful of therapists is they're good at like eight of them. Again, we never saw therapists who were good at all 12. So this notion of the super shrink that has been out there, where they're just good at everything, our data wouldn't say that they're necessarily good at everything. At least when you think about being good in being tethered to presenting problems, right? Again, let's be very clear that what we are talking about right now is problem-based matching. We're not talking about anything else. And importantly, I can tell you a little bit about some of the new work we're doing, where we're looking at strengths and weaknesses in other ways, not just tethered to patients presenting problems. But at least when we think of it that way, we are more likely to see that most therapists who are out there practicing in the community do indeed have some strengths in about three areas. I think that for the listener, I mean, I think that this is a really good news story, at least. To me, it is. I mean, to me, it says that if you're in training or you're in practice, the typical person is the typical therapist is going to be good at-- and again, we're talking about very specific domains. But they're really good with some percentage of patients who are really struggling with particular domains of challenge or struggle. And that's a much more-- compared to some of the conversations I've had-- that's a much more optimistic-- I mean, that's the problem of getting client to therapist. Right? Client to right therapist is a-- as you've demonstrated-- a very solvable problem. Right? It's a software problem. It's, you know, I mean, hell, you've already done the programming, right? Somebody-- you're willing to measure patients outcomes, at least at pre- and post-treatment. And you have a therapist ID in your electronic health record. Then you can get therapist effectiveness profiles in this way. If your measure is multi-dimensional-- if you just have a measure that's one global score, then you can get the between therapist effect, but you can't get the strengths and weaknesses. So that you're right, that the technology is there, part of what we're talking about. And maybe the optimism you're expressing here is what I hope will be contagious, right? Is that the main thing is that I don't think clinics are having a hard time anymore with the idea of patients completing a measure-- routine outcomes monitoring-- which helps at the case level, measurement-based care. If you are willing, as an administrator, a system, and as a clinician, to find out what you are particularly exceptionally good at-- and areas where you're OK, and areas where you're poor-- if you're open to that, if you're open to knowing yourself, then I think it's a solvable problem that we can do matching. And we can do nuance matching, right? We can even say, even if what you're finding out is that you're kind of average in this one area, if a patient comes in and overall they're not that distressed, no big deal. Still see them. That doesn't have to affect your job. It doesn't mean your livelihood is in threat. But there might also be a situation where a therapist says, gosh, it turns out I'm really, really exceptionally effective at treating-- the data tell me at treating depression. And I don't know, social conflict, right? I think I'm going to start to specialize. I'm going to start to put those data on my website, really become a specialist. And maybe I'm less likely to burn out because I know my patients are getting better. My patients are better off because they're having this experience. So there are various ways you can go with it. And part of the pushback, if you will, has really not been the-- surprisingly for us, when we talk to clinicians, or clinicians are in an audience of a talk, they're not telling us, nope, I'm resistant to anybody measuring my outcomes. Nobody understands how to measure outcomes in the way that I do work and whatever. It's not that. It's that they don't want to lose their job or they don't want to get pigeonholed into only treating a certain type of patient. They don't want to believe that they can't improve in areas where they're not doing as well. And so part of what we want to do as a follow-up is to really find out if we do more pointed continuing education to areas in which therapists are weaker, do they get better, as opposed to just letting therapists sign up for hours whenever they can. So there are a lot of ways to go with this that don't have to just end up being match, no match. Right, there are ways to think about in terms of professional development for clinicians. And part of it will be their autonomous choice. Do you want to be a specialist based on what you learn? Do you want to improve in certain areas? Are you happy that your average across the board and just make sure you don't see patients who are really, really severe? For some people, that might be perfectly fine for their practice. Right? And I think what I'm thinking about too is that based on your data, so if you and I are both working at a center, that for a bunch of the patients who walk in the door, unless they you and I have different strengths, it doesn't matter if they see you or me because they're going to be at a moderate or whatever level of severity. So then where our caseloads are going to be varied, right? But you have three areas of strength, I have three areas of strength. They don't overlap. So it's only going to be for the quote-unquote severe patients where your caseload will have more of those people who are struggling in those three areas. Mine will have more of those patients who are struggling in my three areas where I have strength. But that's only going to make up-- I'm making up this number-- 50%, some percent, 50%, 30%, 40%, 60%, I don't know, of our caseload, where the rest of it, since they're not as severe, those just get thrown out to everybody. So that's one way of thinking about mitigating people's concern. Right, because you've written about this, because you've written about that's a concern that people have. And it's totally reasonable, which is, if all I have are people with this particular problem in the severe group, Jesus, that's exhausting, right? And like-- Or does a network run out of clinicians who have the necessary strength? And then it's like, we do matching, but your short list came up to zero. Sorry. So we're going to have to go back to just randomly picking somebody for you, right? And so yeah, I do think there needs to be sort of an interaction or a nuanced way of thinking about populating one short list of good-fitting therapists. But I think the way you explained it could be a manifestation of how therapists still have a variety on their caseloads, how administrators can make sure that the right patients get the right therapist, because not every network is just going to have 10 experts in each of the 12 domains that always have an opening, always take the right insurance, always work at the convenient office. There's still going to be other pragmatics that matter. And of course, there's also patient preferences. So we're also working on an app that allows patients to sort of drive their own matching right on their phone, where it sort of populates in a given area. That's a wrap on the first part of our conversation. As noted at the top of the show, be much appreciated if you spread the word to anyone else who you think might enjoy it. Until next time.[Music][Music]