Thank you very much for taking the time to write this detailed reply, Michael! I havenât read the To WELLBY or not to WELLBY? post, but definitely want to check that out to understand this all better.
I also want to apologize for my language sounding overly critical/âharsh in my previous comment. E.g. Making my first sentence âThis post didnât address my concerns related to using WELLBYs...â when I knew full well that wasnât what the post was intending to address was very unfair of me.
I know youâve put a lot of work into researching the WELLBY approach and are no doubt advancing our frontier of knowledge of how to do good effectively in the process, so I want to acknowledge that I appreciate what you do regardless of any level of skepticism I may still have related to heavily relying on WELLBY measurements as the best way to evaluate impact.
As a final note, I want to clarify that while my previous comment may have made it sound like I was confident that the WELLBY approach was no good, in fact my tone was more reflective of my (low-information) intuitive independent impression, not my all-things-considered view. I think thereâs a significant chance that when I read into your research on neutrality, linearity, and comparability, etc, that Iâll update toward thinking that the WELLBY approach makes considerably more sense than I initially assumed.
Thanks for saying that. Yeah, I couldnât really understand where you were coming from (and honestly ended up spending 2+ hours drafting a reply).
On reflection, we should probably have done more WELLBY-related referencing in the post, but we were trying to keep the academic side light. In fact, we probably need to recombine our various scratching on the WELLBY and put them onto a single page on our websiteâitâs been a lower priority than doing the object-level charity analysis work.
If youâre doing the independent impression thing again, then, as a recipient, it would have been really helpful to know that. Then I would have read it more as a friendly âIâm new to this and sceptical and X and Yâwhatâs going on with those?â and less as a âIâm sceptical, you clearly have no idea what youâre talking aboutâ (which was more-or-less how I initially interpreted it⌠:) )
Then I would have read it more as a friendly âIâm new to this and sceptical and X and Yâwhatâs going on with those?â and less as a âIâm sceptical, you clearly have no idea what youâre talking aboutâ
Ah, Iâm really sorry I didnât clarify this!
For the record, youâre clearly an expert on WELLBYs and Iâm quite new to thinking about them.
My initial exposure to HLIâs WELLBY approach to evaluating interventions was the post Measuring Good Better and this post is only my second time reading about WELLBYs. I also know very little about subjective wellbeing surveys. Iâve been asked to report my subjective wellbeing on surveys before, but Iâve basically never read about them before besides that chapter of WWOTF.
The rest of this comment is me offering an explanation on what I think happened here:
Scott Alexander has a post called Socratic Grilling that I think offers useful insight into our exchange. In particular, while I absolutely could and should have written my initial comment to be a lot friendlier, I think my comment was essentially an all-at-once example of Socratic grilling (me being the student and you being the teacher). As Scott points out, thereâs a known issue with this:
Second, to a hostile observer, it would sound like the student was challenging the teacher. Every time the teacher tried to explain germ theory, the student âpouncedâ on a supposed inconsistency. When the teacher tried to explain the inconsistency, the student challenged her explanations. At times he almost seems to be mocking the teacher. Without contextual clues â and without an appreciation for how confused young kids can be sometimes â it could sound like this kid is an arrogant know-it-all who thinks heâs checkmated biologists and proven that germ theory canât possibly be true. Or that he thinks that he, a mere schoolchild, can come up with a novel way to end all sickness forever that nobody else ever thought of.
Later:
Tolerating this is harder than it sounds. Most people can stay helpful for one or two iterations. But most people are bad at explaining things, so one or two iterations isnât always enough Iâve had times when I need five or ten question-answer rounds with a teacher in order to understand what theyâre telling me. The process sounds a lot like âThe thing you just said is obviously wrongââŚâno, that explanation you gave doesnât make sense, youâre still obviously wrongââŚâyou keep saying the same thing over and over again, and it keeps being obviously wrongââŚâno, thatâs irrelevant to the point thatâs bothering meââŚâno, thatâs also irrelevant, you keep saying an obviously wrong thingââŚâOh! That word means something totally different from what I thought it meant, now your statement makes total sense.â
But itâs harder even than that. Sometimes there is a vast inferential distance between you and the place where your teacherâs model makes sense, and you need to go through a process as laborious as converting a religious person to a materialist worldview (or vice versa) before the gap gets closed.
When I first read about HLIâs approach in the Measuring Good Better article my reaction was âHuh, this seems like a poor way to evaluate impact given [all the aspects of subjective wellbeing surveys that intuitively seemed problematic to me].â
If I was talking with you in person about it I probably would have done a back-and-forth Socratic grilling with you about it. But I didnât comment. I then got to this post some weeks later and was hoping it would provide some answer to my concerns, was disappointed that that was not the postâs purpose, and proceeded to write a long post explaining all my concerns with the WELLBY approach so that you or someone could address them. In short, I dumped a lot of work on you and completely failed to think about how (Scottâs words:) â it would sound like the student was challenging the teacher,â and how I could come across as an âarrogant know-it-all who thinks heâs checkmatedâ you, and how âTolerating this is harder than it soundsâ.
So Iâm really sorry about that and will make it a point to make sure I actually think about how my comments will be received next time Iâm tempted to âSocratically grillâ someone, that way I can make sure my comment comes across as friendly.
Thank you very much for taking the time to write this detailed reply, Michael! I havenât read the To WELLBY or not to WELLBY? post, but definitely want to check that out to understand this all better.
I also want to apologize for my language sounding overly critical/âharsh in my previous comment. E.g. Making my first sentence âThis post didnât address my concerns related to using WELLBYs...â when I knew full well that wasnât what the post was intending to address was very unfair of me.
I know youâve put a lot of work into researching the WELLBY approach and are no doubt advancing our frontier of knowledge of how to do good effectively in the process, so I want to acknowledge that I appreciate what you do regardless of any level of skepticism I may still have related to heavily relying on WELLBY measurements as the best way to evaluate impact.
As a final note, I want to clarify that while my previous comment may have made it sound like I was confident that the WELLBY approach was no good, in fact my tone was more reflective of my (low-information) intuitive independent impression, not my all-things-considered view. I think thereâs a significant chance that when I read into your research on neutrality, linearity, and comparability, etc, that Iâll update toward thinking that the WELLBY approach makes considerably more sense than I initially assumed.
Hello William,
Thanks for saying that. Yeah, I couldnât really understand where you were coming from (and honestly ended up spending 2+ hours drafting a reply).
On reflection, we should probably have done more WELLBY-related referencing in the post, but we were trying to keep the academic side light. In fact, we probably need to recombine our various scratching on the WELLBY and put them onto a single page on our websiteâitâs been a lower priority than doing the object-level charity analysis work.
If youâre doing the independent impression thing again, then, as a recipient, it would have been really helpful to know that. Then I would have read it more as a friendly âIâm new to this and sceptical and X and Yâwhatâs going on with those?â and less as a âIâm sceptical, you clearly have no idea what youâre talking aboutâ (which was more-or-less how I initially interpreted it⌠:) )
Ah, Iâm really sorry I didnât clarify this!
For the record, youâre clearly an expert on WELLBYs and Iâm quite new to thinking about them.
My initial exposure to HLIâs WELLBY approach to evaluating interventions was the post Measuring Good Better and this post is only my second time reading about WELLBYs. I also know very little about subjective wellbeing surveys. Iâve been asked to report my subjective wellbeing on surveys before, but Iâve basically never read about them before besides that chapter of WWOTF.
The rest of this comment is me offering an explanation on what I think happened here:
Scott Alexander has a post called Socratic Grilling that I think offers useful insight into our exchange. In particular, while I absolutely could and should have written my initial comment to be a lot friendlier, I think my comment was essentially an all-at-once example of Socratic grilling (me being the student and you being the teacher). As Scott points out, thereâs a known issue with this:
Later:
When I first read about HLIâs approach in the Measuring Good Better article my reaction was âHuh, this seems like a poor way to evaluate impact given [all the aspects of subjective wellbeing surveys that intuitively seemed problematic to me].â
If I was talking with you in person about it I probably would have done a back-and-forth Socratic grilling with you about it. But I didnât comment. I then got to this post some weeks later and was hoping it would provide some answer to my concerns, was disappointed that that was not the postâs purpose, and proceeded to write a long post explaining all my concerns with the WELLBY approach so that you or someone could address them. In short, I dumped a lot of work on you and completely failed to think about how (Scottâs words:) â it would sound like the student was challenging the teacher,â and how I could come across as an âarrogant know-it-all who thinks heâs checkmatedâ you, and how âTolerating this is harder than it soundsâ.
So Iâm really sorry about that and will make it a point to make sure I actually think about how my comments will be received next time Iâm tempted to âSocratically grillâ someone, that way I can make sure my comment comes across as friendly.