Iāve been saying this in various comments for a long time, and I was glad to see the point laid out here in more detail.
My comments often look like: āWhen you say that āEA should do Xā, which people and organizations in EA are you referring to? What should we do more/āless of in order to do less/āmore of X? What are cases where X would clearly be useful?ā
Iām a big fan of the two-paper rule, and will try to remember to apply it when I respond to methodology-driven posts in the future.
Regarding this claim:
EA has made exactly one major methodological step forward since its beginnings, which was identifying the optimizerās curse about eight years ago, something which had the benefit of a mathematical proof.
I appreciate that you went on to qualify this statement, but Iād still have appreciated some more justification. Namely, what are some popular ideas that many people thought were a step forward, but that you believe were not?
If methodological ideas generally havenāt been popular, EA wouldnāt be emphasizing methodology; if they were popular, Iād be curious to see any other writing youāve done on reasons you donāt think they helped. (I realize that would be a lot of work, and it may not be a good use of your time to satisfy my curiosity.)
When I look at the top ~50 Forum posts of all time (sorted by karma), I only see one that is about methodology, and itās not as much prescriptive as it is descriptive (āEA is biased towards some methodologies, other methodologies exist, but Iām not actively recommending any particular alternativesā). Almost all the posts are about object-level research or community work, at least as far as I understand the term āobject-levelā.
I can only think of a few cases when established EA orgs/āresearchers explicitly recommended semi-novel approaches to methodology, and Iām not sure whether my examples (cluster thinking, epistemic modesty) even count. People who recommend, say, using anthropological methods in EA generally havenāt gotten much attention (as far as I can recall).
I am also thinking of how there has been more back-and-forth about the optimizerās curse, people saying it needs to be taken more seriously etc.
I donāt think that the prescriptive vs descriptive nature really changes things, descriptive philosophizing about methodology is arguably not as good as just telling EAs what to do differently and why.
Methodology comment:
Iāve been saying this in various comments for a long time, and I was glad to see the point laid out here in more detail.
My comments often look like: āWhen you say that āEA should do Xā, which people and organizations in EA are you referring to? What should we do more/āless of in order to do less/āmore of X? What are cases where X would clearly be useful?ā
Iām a big fan of the two-paper rule, and will try to remember to apply it when I respond to methodology-driven posts in the future.
Regarding this claim:
I appreciate that you went on to qualify this statement, but Iād still have appreciated some more justification. Namely, what are some popular ideas that many people thought were a step forward, but that you believe were not?
If methodological ideas generally havenāt been popular, EA wouldnāt be emphasizing methodology; if they were popular, Iād be curious to see any other writing youāve done on reasons you donāt think they helped. (I realize that would be a lot of work, and it may not be a good use of your time to satisfy my curiosity.)
When I look at the top ~50 Forum posts of all time (sorted by karma), I only see one that is about methodology, and itās not as much prescriptive as it is descriptive (āEA is biased towards some methodologies, other methodologies exist, but Iām not actively recommending any particular alternativesā). Almost all the posts are about object-level research or community work, at least as far as I understand the term āobject-levelā.
I can only think of a few cases when established EA orgs/āresearchers explicitly recommended semi-novel approaches to methodology, and Iām not sure whether my examples (cluster thinking, epistemic modesty) even count. People who recommend, say, using anthropological methods in EA generally havenāt gotten much attention (as far as I can recall).
I am also thinking of how there has been more back-and-forth about the optimizerās curse, people saying it needs to be taken more seriously etc.
I donāt think that the prescriptive vs descriptive nature really changes things, descriptive philosophizing about methodology is arguably not as good as just telling EAs what to do differently and why.
I grant that #3 on this list is the rarest out of the 4. The established EA groups are generally doing fine here AFAIK. There is a CSER writeup on methodology here which is perfectly good: https://āāwww.cser.ac.uk/āāresources/āāprobabilities-methodologies-and-evidence-base-existential-risk-assessments-cccr2018/āā itās about a specific domain that they know, rather than EA stuff in general.