I don’t think most of the costs that I described that come from legibility differ that much between research and educational institutions? The american public education system, as well as many other public education systems actually strike me as core examples of systems that have suffered greatly due to very strong forces on legibility in all of their actions (like standardized curricula combined with standardized testing). I think standardized testing is pretty good in a lot of situations, but that in this case it resulted in a massive reduction in variance in a system where most of the value comes from the right tail.
I agree that there are also other separate costs to legibility in cutting-edge domains, but the costs on educational institutions still seem quite significant to me. And most of the costs are relatively domain-general.
Thanks, that helps me understand where you’re coming from, though it doesn’t change my views on CFAR. My guess is we disagree about various more general claims around the costs and benefits of legibility, but unfortunately I don’t have time right now to articulate my view on this.
Very roughly, I think I (i) agree with you that excessive optimization for easily measurable metrics has harmed the public education system, and in particular has reduced benefits from the right tail, (ii) disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole, (iii) don’t think this tells us much about CFAR as I both think that CFAR’s environment makes increased legibility less risky (due to things like high goal-alignment with important stakeholders such as funders, a more narrow target audience, …) and also that there are plenty of ways to become more legible that don’t incur risks similar to standardized testing or narrow optimization for quantitative metrics (examples: qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these; maintain a list of lessons learned).
(I upvoted your reply, not sure why it was downvoted by someone else.)
(Reply written after the paragraph was added above)
Thanks for the elaboration! Some quick thoughts:
qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these
I think CFAR has done at least everything on this list of examples. Which you might already be aware of, but wanted to make sure is common knowledge. There are a significantnumberof posts trying to explain CFAR at a high-level, and the example workshop schedule summarizes all the classes at a high-level. CFAR has also published the number of workshops they’ve run and their total attendance in their impact reports and on their homepage (currently listing 1045 alumni). Obviously I don’t think that alone is sufficient, but it seemed plausible that a reader might walk away thinking that CFAR hadn’t done any of the things you list.
disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole
I think there is some truth to this interpretation, but I think it’s overall still wrong enough that I would want to correct it. I think the education system has many goals, and I don’t think I would summarize it’s primary output as “quality-weighted sum of generated research”. I don’t think going into my models of the education system here is going to be super valuable, though happy to do that at some other point if anyone is interested in them. My primary point was that optimizing for legibility clearly has had large effects on educational institutions, in ways that would at least be harmful to CFAR if affected in the same way (another good example here might be top universities and the competition for getting into all the top 10 ranking, though I am less confident of the dynamics of that effects).
(Edit the below was written before Max edited the second paragraph into his comment)
Seems good! I actually think considerations around legibility are quite important and where I expect a good amount of intellectual progress to be made by talking to each other, so I would like to see your perspective written up and engage with it.
I also want to make sure that it’s clear that I do think CFAR should be more legible and transparent (as I said in the writeup above). I have some concerns with organizations trying to be overly legible, but I think we both agree that at the current margin it would be better for CFAR to optimize more for legibility.
(I’ve sadly had every single comment of mine on this thread strong-downvoted by at least one person, and often multiple people. My sense is that CFAR is a pretty polarizing topic, which I think makes it particularly important to have this conversation, but seems to also cause some unfortunate voting patterns that feel somewhat stressful to deal with.)
I’m sorry to see the strong downvotes, especially when you’ve put in more effort on explaining your thinking and genuinely engaging with critiques than perhaps than all other EA Fund granters put together. I want you to know that I found your explanations very helpful and thought provoking, and really like how you’ve engaged with criticisms both in this thread and the last one.
(I’m wondering whether this phenomenon could also be due to people using downvotes for different purposes. For example, I use votes roughly to convey my answer to the question “Would I want to see more posts like this on the Forum?”, and so I frequently upvote comments I disagree with. By contrast, someone might use votes to convey “Do I think the claims made in this comment are true?”.)
Data point: I often feel a pull towards up-voting comments that I feel have stimulated or advanced my thinking or exemplify a valuable norm of transparency and clarity, but then I hold back because I think I might disagree with the claims made or I think I simply don’t know enough to judge those claims. This is based on a sense that I should avoid contributing to information cascade-type situations (even if, in these cases, any contribution would only be very slight).
This has happened multiple times in this particular thread; there’ve been comments of Oliver’s that I’ve very much appreciated the transparency of, but with which I felt like I still might slightly disagree overall, so I avoided voting either way.
(I’m not saying this is the ideal policy, just that it’s the one I’ve taken so far.)
I don’t think most of the costs that I described that come from legibility differ that much between research and educational institutions? The american public education system, as well as many other public education systems actually strike me as core examples of systems that have suffered greatly due to very strong forces on legibility in all of their actions (like standardized curricula combined with standardized testing). I think standardized testing is pretty good in a lot of situations, but that in this case it resulted in a massive reduction in variance in a system where most of the value comes from the right tail.
I agree that there are also other separate costs to legibility in cutting-edge domains, but the costs on educational institutions still seem quite significant to me. And most of the costs are relatively domain-general.
Thanks, that helps me understand where you’re coming from, though it doesn’t change my views on CFAR. My guess is we disagree about various more general claims around the costs and benefits of legibility, but unfortunately I don’t have time right now to articulate my view on this.
Very roughly, I think I (i) agree with you that excessive optimization for easily measurable metrics has harmed the public education system, and in particular has reduced benefits from the right tail, (ii) disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole, (iii) don’t think this tells us much about CFAR as I both think that CFAR’s environment makes increased legibility less risky (due to things like high goal-alignment with important stakeholders such as funders, a more narrow target audience, …) and also that there are plenty of ways to become more legible that don’t incur risks similar to standardized testing or narrow optimization for quantitative metrics (examples: qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these; maintain a list of lessons learned).
(I upvoted your reply, not sure why it was downvoted by someone else.)
(Reply written after the paragraph was added above)
Thanks for the elaboration! Some quick thoughts:
I think CFAR has done at least everything on this list of examples. Which you might already be aware of, but wanted to make sure is common knowledge. There are a significant number of posts trying to explain CFAR at a high-level, and the example workshop schedule summarizes all the classes at a high-level. CFAR has also published the number of workshops they’ve run and their total attendance in their impact reports and on their homepage (currently listing 1045 alumni). Obviously I don’t think that alone is sufficient, but it seemed plausible that a reader might walk away thinking that CFAR hadn’t done any of the things you list.
I think there is some truth to this interpretation, but I think it’s overall still wrong enough that I would want to correct it. I think the education system has many goals, and I don’t think I would summarize it’s primary output as “quality-weighted sum of generated research”. I don’t think going into my models of the education system here is going to be super valuable, though happy to do that at some other point if anyone is interested in them. My primary point was that optimizing for legibility clearly has had large effects on educational institutions, in ways that would at least be harmful to CFAR if affected in the same way (another good example here might be top universities and the competition for getting into all the top 10 ranking, though I am less confident of the dynamics of that effects).
(Edit the below was written before Max edited the second paragraph into his comment)
Seems good! I actually think considerations around legibility are quite important and where I expect a good amount of intellectual progress to be made by talking to each other, so I would like to see your perspective written up and engage with it.
I also want to make sure that it’s clear that I do think CFAR should be more legible and transparent (as I said in the writeup above). I have some concerns with organizations trying to be overly legible, but I think we both agree that at the current margin it would be better for CFAR to optimize more for legibility.
(I’ve sadly had every single comment of mine on this thread strong-downvoted by at least one person, and often multiple people. My sense is that CFAR is a pretty polarizing topic, which I think makes it particularly important to have this conversation, but seems to also cause some unfortunate voting patterns that feel somewhat stressful to deal with.)
I’m sorry to see the strong downvotes, especially when you’ve put in more effort on explaining your thinking and genuinely engaging with critiques than perhaps than all other EA Fund granters put together. I want you to know that I found your explanations very helpful and thought provoking, and really like how you’ve engaged with criticisms both in this thread and the last one.
Seconded.
(I’m wondering whether this phenomenon could also be due to people using downvotes for different purposes. For example, I use votes roughly to convey my answer to the question “Would I want to see more posts like this on the Forum?”, and so I frequently upvote comments I disagree with. By contrast, someone might use votes to convey “Do I think the claims made in this comment are true?”.)
Data point: I often feel a pull towards up-voting comments that I feel have stimulated or advanced my thinking or exemplify a valuable norm of transparency and clarity, but then I hold back because I think I might disagree with the claims made or I think I simply don’t know enough to judge those claims. This is based on a sense that I should avoid contributing to information cascade-type situations (even if, in these cases, any contribution would only be very slight).
This has happened multiple times in this particular thread; there’ve been comments of Oliver’s that I’ve very much appreciated the transparency of, but with which I felt like I still might slightly disagree overall, so I avoided voting either way.
(I’m not saying this is the ideal policy, just that it’s the one I’ve taken so far.)
Thank you! :)