I’m a huge fan of having high standards. Posts that are like “we reproduced this published output and think they made these concrete errors” are often great. But I notice much more “these people did a bad job or spent too much money” takes often from people who afaict haven’t done a bunch of stuff themselves so aren’t very calibrated, and don’t seem very scope sensitive. If people saw their projects being critiqued and were then motivated to go and do more things more quickly I’d think that was great (or were encouraged to do more things more quickly from “fear” of critiques) I think we’d be in a better equilibrium.
For example people often point out that LW and the forum are somewhat expensive per user as evidence they are being mismanaged and imo this is a bad take which is rarely made by people who have built or maintained popular software projects/forums or used the internet enough to know that discussion of the kind in these venues is really quite rare and special.
To be clear, I think the “but have they actually done stuff” critique should also be levelled at grantmakers. I’m sympathetic to grantmakers who are like “the world is burning and I just need to do a leveraged thing right now” but my guess is that if more grantmakers had run projects in the reference class of things they want to fund (or founded any complicated or unusual and ambitious projects) we’d be in a better position. I think this general take is very common in YC/VC spaces, which perform a similar function to grantmaking for their ecosystem.
Many examples of criticism in replies, are high quality posts that I think improve standards. I may spend an hour going through the criticism tag and sorting them into posts I think are useful/anti-useful to check.
I’m not quite as convinced of the much greater cost of “bad criticism” over “good criticism”. I’m optimistic that discussions on the forum tend to come to a reflective equilibrium that agrees with valid criticism and disregards invalid criticism. I’ll give some examples (but pre-committing to not rehashing these too much):
I think HLI is a good example of long-discussion-that-ends-up-agreeing-with-valid-criticism, and as discussed by other people in this thread this probably led to capital + mind share being allocated more efficiently.
I think the recent back and forth between VettedCauses and Sinergia is a good example of the other side. Setting aside the remaining points of contention, I think commenters on the original post did a good job of clocking the fact that there was room for the reported flaws to have a harmless explanation. And then Carolina from Sinergia did a good job of providing a concrete explanation of most of the supposed issues[1].
It’s possible that HLI and Sinergia came away equally discouraged, but if so I think that would be a misapprehension on Sinergia’s part. Personally I went from having no preconceptions about them to having mildly positive sentiment towards them.
Perhaps we could do some work to promote the meme that “reasonably-successfully defending yourself against criticism is generally good for your reputation not bad”.
(Stopped writing here to post something rather than nothing, I may respond to some other points later)
You could also argue that not everyone has time to read through the details of these discussions, and so people go away with a negative impression. I don’t think that’s right because on a quick skim you can sort of pick up the sentiment of the comment section, and most things like this don’t escape the confines of the forum.
I’m a huge fan of having high standards. Posts that are like “we reproduced this published output and think they made these concrete errors” are often great. But I notice much more “these people did a bad job or spent too much money” takes often from people who afaict haven’t done a bunch of stuff themselves so aren’t very calibrated, and don’t seem very scope sensitive. If people saw their projects being critiqued and were then motivated to go and do more things more quickly I’d think that was great (or were encouraged to do more things more quickly from “fear” of critiques) I think we’d be in a better equilibrium.
For example people often point out that LW and the forum are somewhat expensive per user as evidence they are being mismanaged and imo this is a bad take which is rarely made by people who have built or maintained popular software projects/forums or used the internet enough to know that discussion of the kind in these venues is really quite rare and special.
To be clear, I think the “but have they actually done stuff” critique should also be levelled at grantmakers. I’m sympathetic to grantmakers who are like “the world is burning and I just need to do a leveraged thing right now” but my guess is that if more grantmakers had run projects in the reference class of things they want to fund (or founded any complicated or unusual and ambitious projects) we’d be in a better position. I think this general take is very common in YC/VC spaces, which perform a similar function to grantmaking for their ecosystem.
Many examples of criticism in replies, are high quality posts that I think improve standards. I may spend an hour going through the criticism tag and sorting them into posts I think are useful/anti-useful to check.
I’m not quite as convinced of the much greater cost of “bad criticism” over “good criticism”. I’m optimistic that discussions on the forum tend to come to a reflective equilibrium that agrees with valid criticism and disregards invalid criticism. I’ll give some examples (but pre-committing to not rehashing these too much):
I think HLI is a good example of long-discussion-that-ends-up-agreeing-with-valid-criticism, and as discussed by other people in this thread this probably led to capital + mind share being allocated more efficiently.
I think the recent back and forth between VettedCauses and Sinergia is a good example of the other side. Setting aside the remaining points of contention, I think commenters on the original post did a good job of clocking the fact that there was room for the reported flaws to have a harmless explanation. And then Carolina from Sinergia did a good job of providing a concrete explanation of most of the supposed issues[1].
It’s possible that HLI and Sinergia came away equally discouraged, but if so I think that would be a misapprehension on Sinergia’s part. Personally I went from having no preconceptions about them to having mildly positive sentiment towards them.
Perhaps we could do some work to promote the meme that “reasonably-successfully defending yourself against criticism is generally good for your reputation not bad”.
(Stopped writing here to post something rather than nothing, I may respond to some other points later)
You could also argue that not everyone has time to read through the details of these discussions, and so people go away with a negative impression. I don’t think that’s right because on a quick skim you can sort of pick up the sentiment of the comment section, and most things like this don’t escape the confines of the forum.