Building my Scout Mindset: #2
edit: Upon further reflection, I don’t think this entry should be considered a genuine attempt at building a scout mindset. I am leaving it up as an example of what an unsuccessful attempt looks like and to show that building this habit can be difficult, but I aim to revise this entry in the future.
By the way, if you have any feedback on the format of this series, I’d love to hear it!
Upon First Glance
This section excerpts my stream-of-consciousness upon first reading the article.
Headline: ‘No, social workers don’t do better than cops at mental-health response’
Totally not surprised by this, given the praise this program has received. This response was going to come … Wonder what it’ll say?
Context: I’d been seeing coverage of New York’s B-HEARD program for the past week as part of my internship.
Annotations
This subsection annotates the article with my immediate thoughts in italic.
Last month, New York City joined dozens of other cities experimenting with “alternatives” to policing by launching the Behavioral Health Emergency Assistance Response Division, or B-HEARD, in a pilot study in parts of Harlem. Gotta love the quote marks around ‘alternatives.’ … Supporters argue that this approach reorients the response to these calls, from public safety toward public health. Well, I think supporters intend to frame the intervention as advancing both public health and public safety by targeting a root cause (mental health). But this is nitpicky. Or maybe not, because I think the little things typically matter in communication! Now the mayor’s office has put out its first report on the program to a glowing, but misleading, reception in the media. Nice, not burying the lede.
Feeling: Curious
Nothing out of the blue here—all expected and not hyperbolic: viz. “misleading”
While I’m biased against this opinion, as my prior is that mental health is a big crime factor that needs addressing, I also believe that there is little data on police reform and am very interested in any analysis on these new initiatives.
Outlets such as Business Insider and NPR highlighted the finding that those approached by B-HEARD responders were more likely to accept help than those approached by police. No, I’m not going to read through those articles. But I bet you I’ve probably seen the headlines of those anyway, as part of my internship. NBC New York identified a causal relationship, claiming that B-HEARD is “reducing unnecessary hospitalizations, while increasing the percentage of people who accept help when offered.” Okay, this is really silly to me. I think it’s clear that most people casually imply causality when they really mean to identify a correlation. But, I mean, fine because technically true? At first glance, the results look impressive. Just 5 percent of those approached by B-HEARD in its first month refused assistance, compared with 18 percent who received a “traditional mental-health response” in the same period. What the heck is a “traditional” MH response? Conducted by the police? (Yep, according to the Business Insider article: “When there is a traditional response from police and EMS workers, about 82% of people in crisis accept help.”) And B-HEARD cases were much less likely to be hospitalized. Good! Can you imagine—being hospitalized from a police call? Good god. Unfortunately, a closer look at the data reported by the mayor’s office suggests a deck stacked for B-HEARD’s success. Responders appear to have received calls that were — perhaps unwittingly — selected to produce better outcomes. Ooh, I am both intrigued and not surprised. It makes perfect sense and I think might have been predicted? After all, I took an intro Public Policy course in 2019 that mentioned the idea of mental health response teams, so it’s a more ‘conventional’ police reform (compared to defunding the police) and has been criticized for being inadequate per se.
Feeling: Pleasantly surprised
Wild, being pleasantly surprised to ‘expect’ this criticism of an initiative I support? But indeed—I think understanding a critique nudges me towards calmly acknowledging and engaging with it, while being familiar with the point allows me to recall counterpoints.
Notice that I instinctively slip into soldier mindset!
I think there’s a very fine line between rationally engaging with an argument and only looking for counterpoints … I don’t think Scout Mindset argues that you cannot bring up counterarguments, but I’m not sure how to avoid instinctively focusing on counterarguments.
I think I’ll just have to keep reminding myself that the question is, “Can I believe this?” and not, “Must I?”
As the mayor’s report notes, the month following B-HEARD’s launch saw roughly 500 mental-health-related calls in the covered area. No link to the report? Sigh. It’s here. Of those, roughly a quarter were referred to B-HEARD, which then kicked back a further 1 in 5. In total, the civilian responders considered themselves able to handle just 107 calls, or about 21 percent of the total. Did some quick Math to check and, indeed, 107⁄500 = .214 The calls referred to B-HEARD weren’t a random sample. Yeah, fair, this is not an RCT by any means. I’m amused that they are bringing this up, though, as I think this is somewhat strawmanny because I think supporters see MH responders as substituting only this select slice of calls. Responders weren’t routed calls “that involve a weapon, an imminent risk of violence or where NYPD or EMS call-takers know that an individual has an immediate need for transportation to a medical facility.” B-HEARD responders were also on call for only 16 hours a day; if they took the night off, they would have been spared dealing with mental-health crises among individuals who had declined shelter for the evening. Partly my fault but I’m not sure why this is relevant. I’m inferring that individuals refusing shelter maybe are more likely to have more severe mental health issues? But that doesn’t fit my priors for reasons for unhoused individuals refusing shelters, and I don’t like the possible implication that unhoused individuals are more prone to violence.
Feeling: Bit confused, skeptical.
Hence an apples-to-apples comparison between B-HEARD and “traditional” response suffers from selection bias. Okay. Yes. I think this is a fair assessments and suggests advocates may need to be more careful about the framing. Now, I haven’t actually read coverage of B-HEARD so I’m not comfortable concluding that advocates aren’t using this frame. The difference in outcomes between the two groups owes at least partially, even mostly, to the difference between the pools of calls that each is responsible for handling. Now this is just a sneaky claim. “Partially, even mostly”—where’s the justification for that jump? We don’t have a counterfactual to indicate that that B-HEARD performs worse than the NYPD for a random pool of calls. The most we could say is that B-HEARD seems to do better with nonviolent crises, but that is more than saying that B-HEARD is not actually better than the NYPD at handling anything. If the NYPD is called in on more serious and violent calls, the people with whom the police interact will be, on average, less willing to accept help — regardless of whatever techniques cops use to de-escalate the situation. I think this as a fair assumption. This pattern is common across civilian-led mental-health-response initiatives. This seems framed as evidence for the author’s argument but I think this supports my theory equally well: these mental health responders are intended to substitute for traditional police in some, but not all, cases. CAHOOTS, the original alternative program from Eugene, Ore., is often touted as a model for other cities looking to reduce police activity. But as my recent Manhattan Institute report found, CAHOOTS responders cover fewer than 20 percent of 911 calls, with 3 in 4 of those involving routine welfare checks or transporting homeless people. LOL, Manhattan Institute, checks out. Anyway—isn’t a substitution of ~20% still a reduction in traditional policing?
Feeling: Somehow, simultaneously tired/bored/annoyed and vindicated.
I think I’ve just slipped into soldier mindset at this point but I have to say, I feel like I’m being very open and rational! The disparity between the claims and the evidence seems so obvious!
Let’s take a step back, though, and remember: “Can I believe this?” I definitely believe that mental health responders cannot fully replace police. This article, however, also implies that they do not reduce police activity—which just does not match the stats they’re citing.
I do think, though, that I could believe that MH responders do not reduce police activity at all—in the sense that maybe that in reality, they are so ineffective that they have to call in the police 100% of the time. I don’t know what kind of training the B-HEARD team went through and I am confident that not just any MH professional could handle such de-escalation.
But I also don’t think that’s what the evidence (in this article) shows!
I feel like I’ve sufficiently given this question a shot.
Civilian alternatives, in other words, take the easy calls and mostly do a good job. “Easy calls”—I mean, maybe they do better than the NYPD at these calls? In which case, maybe they’re not so easy for everyone? But I have no idea and this is a pointless quibble. If cities want that, it’s not a bad use of resources — if anything, it frees up cops’ time to focus on fighting crime. Ughhhh. This is somehow even worse than the poor logic. This article keeps pushing the narrative that cops reduce crime! Honestly that’s probably more impactful than the actual argumentation. But we should not expect B-HEARD’s high rates of success to persist if it starts picking up more challenging calls. Sure. Okay. This reinforces the possibility that some advocates framed this as a complete replacement for traditional policing, in which case I am on board with the author. The reality is that police do a pretty good job handling mental-health situations, and unarmed civilian alternatives will always be reliant on the police as a backup; B-HEARD sought support in seven of the 107 calls it managed. ??? Okay I don’t know what evidence they are referencing when claiming that “police do a pretty good job.” And I guess I spoke too soon when I labelled my earlier contention a “pointless quibble” because this just went to a weird place. Yes, they will be reliant on the police as a backup, which isn’t an issue for advocates who see MH responders as a partial (not full) replacement. Also, 7⁄107 = 6.5% which doesn’t seem high, although I have no idea what rate we should ‘expect.’ That could also speak to the lack of calibration re: appropriate calls to refer to the B-HEARD team, rather than their ineptitude. Advocates of defunding police will point to B-HEARD’s success as evidence that we can dramatically curtail policing, but let’s be clear about what the data show: We can trim around the edges, but civilians can’t replace cops. This is not an objectionable conclusion … But is anyone claiming otherwise? Agh. It’s just, so common for political analyses to push a narrative through strawmanning (this goes for both sides), and I’m tired of it.
Feeling: Exasperated
I kind of feel bad for being so annoyed but at least in the communities I run, this article feels like it is strawmanning hard.
Not bothered to run a Google search to figure out whether this is a strawman or a significant portion of reformer messaging, but I did finally read the three linked news articles.
Business Insider does not claim that B-HEARD is or should be a full replacement: “better serve individuals in crisis by sending specialists who are trained in treating mental health conditions. These programs also aim to reduce violent encounters between police and people in crisis, while easing strain on police resources.” (emphasis mine)
In fact this is basically … What the author concedes.
NPR really just reports stats, but it does quote Jamaal Bowman: “This is great news. A smarter approach to public health and public safety. A smarter use of resources. And the evidence — from Denver to New York — shows that responding with care works,” U.S. Rep. Jamaal Bowman, D-N.Y., tweeted.”
I don’t think this implies that B-HEARD is a full replacement for traditional policing but I guess one could (not generously) assume that.
Also, “Mental health-related calls accounted for 22% of cases in which on-duty police used lethal force and killed someone, according to data from 2009 to 2012 from 17 states where data was available.” Funny how the author doesn’t mention this when claiming that police respond well to MH calls!
NBC also limits B-HEARD response to mental health: “The B-HEARD teams are currently responding to about a quarter of daily mental health calls in the pilot region; the city expects to gradually expand that to half.” (I am assuming that it’s “half of daily mental health calls”)
Again, huh! Funny that the author doesn’t mention that “One of the major concerns going into the program was the safety of the first responders, but so far the program has only called for NYPD backup seven times. On the other hand, the city said, the NYPD has called in B-HEARD teams 14 times after finding police services weren’t needed.” (emphasis mine)
It’s almost like the NYPD appreciates having this policing alternative! But that wouldn’t fit the narrative now, would it?
Charles Fain Lehman is a fellow at the Manhattan Institute and a contributing editor of City Journal, from which this column was adapted.
Feeling: Unsurprised.
City Journal, duh. I actually don’t hate all their stuff, especially their housing stuff, but eh.
Also, the New York Post filed this under “homeless” which … Hmm. I am ungenerously assuming that is because they think many of these MH calls are for unhoused individuals (which actually is probably true, so I should not feel any way about this).
Ideological Turing Test
I think a fair summary of the author’s position is,
Advocates claim that the early success of New York City’s B-HEARD program demonstrates that mental health response teams can fully replace traditional police. However, that does not account for the selection bias inherent to B-HEARD (and similar programs), and so that is a faulty conclusion. Civilians cannot fully replace traditional police.
Inspired by Julia Galef’s suggestion:
Pick a belief you hold strongly and attempt an ideological Turing test of the other side. (Bonus points if you can actually find someone from the other side to judge your attempt.)
Do let me know if you happen to be (or know) “someone from the other side” and would be interested in judging my attempt!
Reflections
I am sorely disappointed by this! I am eager to engage with analysis of new criminal justice reforms, so we can really figure out the most evidence-backed interventions, and this just fell flat. It’s not well-argued, I have no idea who the target audience is (besides anti-reformers), and it insidiously reinforces regressive narratives.
I am, however, concerned about how I can better identify soldier mindset going forward. I really feel like I tried to be open to the author’s argument but I’m not sure if I was… It’s very easy to pretend like you are being generous and rational.
I think in the future, instead of an ideological Turing test, I might try to steelman the author’s position.
I’m not sure your Ideological Turing Test really captures the essence of the article. The key conclusion isn’t just that civilians can’t fully replace traditional police, it’s that (this piece of) the evidence doesn’t support the activists’ claims that there is any aspect of policing which should be de-policed. Presumably the target audience is anyone who might have otherwise believed the suggestions in the other referenced articles that this de-policing policy was beneficial.
Is this not true? My very quick googling (1) (2) suggests homeless people are disproportionately violent, and I would guess the problem is worse among those not using shelters, as being violent or abusing drugs might have been the reason they were kicked out. Given the demographics of the homeless it would be pretty surprising if they were not more prone to violence than the average person.
Thanks for reading + responding! Some responses:
I think you’re right that my Turing Test doesn’t capture the full extent of the author’s implications, but I’m not sure I agree on your reframing.
Given the headline and the phrase, “unarmed civilian alternatives will always be reliant on the police as a backup,” I can see how the article is implying that mental health responders might not replace police at all (if police are equally effective). However, the author also suggests that “it [B-HEARD] frees up cops’ time to focus on fighting crime” which reads to me as reducing police presence in specific cases (i.e. the MH calls that B-HEARD would be taking).
Agree on the target audience—I was too blinded by my bubble to consider that a significant audience!
It also makes sense to me that unhoused individuals are statistically more likely to have a violent history. I didn’t consider that some individuals might have been barred from shelters—I was very much pulling from priors of individuals voluntarily refusing to enter NYC shelters—and so that is a fair point!
My comment was not that the implication is not true but rather, that it’s not a very insightful or helpful point. To me, it’s akin to citing statistics showing that Black people commit crimes at higher rates, which seems disingenuous given that the clearer statistical relationship is between socioeconomic status and crime.
I don’t understand this comparison. If there’s a clear statistical relationship between socioeconomic status and crime, isn’t the implication that unhoused status is correlated with criminal behavior making use of exactly that relationship? It’s hard to think of a more effective proxy for socioeconomic status than not having a place to live.
Epistemic status: I read pretty quickly, so some of my points or numbers could be way off.
Weakly upvoted; I love these posts, but I thought this one was less successful than the first. I’m not sure what was happening internally, but I think some of your statistical claims just look shakier than I think they would if you were reading + responding to a story you felt neutrally about.
One useful rule for these posts (which I really like!) would be that if you feel yourself getting annoyed by someone’s citing a source, the proper reaction seems like either “I’m going to assume this source is saying something true” or “I’m going to look at the source and figure out whether something is false”. In this case, it felt like your mind did a sneaky “aha, this organization probably said something untrue” without checking the data.
Of course, some organizations really do say a lot of untrue things, and this may be more prevalent the further out you go on the political spectrum (in both directions). But these data checks are still good practice — and if an org lies a lot, the checks may not take very long.
Some of the biggest shifts to my own worldview have been when I did “epistemic spot-checking” on claims I was sure were exaggerated or fake, only to find that they really did seem fair. You never know when one of those might hit you, and once you’re in “LOL, checks out” mode, you’re probably no longer open to them.
(Massive credit for typing “LOL, checks out” and acknowledging that was going through your head — that’s a big part of the battle! Er, the scouting mission.)
In order to claim this statistic as evidence that police handle these calls badly, it feels like you need (a) some sense of the absolute numbers (22% of what?), and (b) a sense of how “mental health-related calls” are defined. What fraction of these calls involve an in-progress assault, or someone brandishing a weapon, vs. an unarmed person who isn’t seriously threatening anyone? The NPR article hints that few such calls involve this kind of danger, but I could imagine a world where:
There are 500,000 such calls per year in the US
90% of them are nonthreatening, 10% are threatening (50,000)
There are 1,000 police killings per year, 22% of which happen in mental health incidents (220)
In this set of imaginary calculations, the chance of a person being killed when the police deal with a threatening mental health event is ~1/200. This doesn’t seem unreasonable if these incidents almost always involve weapons/assault/some other form of threatening behavior (though again, I don’t know what the actual numbers are).
That said, I’m also leaving out cases of serious injury, etc., which could make police numbers look much worse.
I say this as someone who’s really excited about actually trying new things like B-HEARD, and who likes the idea of “experts dealing with the things they’re good at” all over society. But I’d want to know more before I concluded that overall, the police are really bad at this particular thing.
Maybe I’m not reading correctly, but it seems like B-HEARD called the police for 7⁄107 instances, while the NYPD called in B-HEARD in 14/X instances. What is X? If it’s 393 (500 mental health calls − 107 taken by B-HEARD), then the police call in B-HEARD less often, per case, than B-HEARD calls in the cops.
This doesn’t mean B-HEARD is a bad idea or anything — as you point out, people might be poorly calibrated on what they should be handling — but it doesn’t seem to warrant an “on the other hand” that implies B-HEARD helps the police more often than they receive help.