There’s a difficult trade-off between the high-fidelity communication of our long explainer posts and the concision that is necessary to get people to actually read a post when it comes to participating in a contest. Our explainer posts get very little engagement. To participate in the contest it’s not necessary to understand exactly how our mechanisms work, so we hope to reach more people by explaining things in simpler terms without words like “ex ante” and comparisons to constructed counterfactual world histories.
After this contest, it will still be the case that most people will be more likely to read and use instructions that are short and simple. It may be very hard to later “fix” the influence that posts like the OP has on potential future retro funders. Simpler instructions are more prone to become a meme. Therefore, retro funders may predict that some (most?) future retro funders will use the simple “buy likable impact” rule rather than the “adhere to the safety solutions in the Toward Impact Markets post” rule, and thus be incentivized to follow the simple rule themselves. Posts like the OP risk pushing everyone towards the Schelling point of “retro funders buy likable impact”. (All this becomes more worrisome if you or someone else in EA ends up launching a decentralized impact market).
Regarding the claim that “articles on the EA Forum” are “very unlikely to be extremely harmful”: EA Forum posts can disseminate info hazards that can be extremely harmful. (And this does not seem very unlikely, considering that the ideas that are discussed on the EA Forum are often related to anthropogenic x-risks.)
Hmm, I love writing high-fidelity content. Just thinking, “how can I express what I mean as clearly as I can” rather than “how can I simplify what I mean to maximize the fidelity/complexity ratio” is a lot easier for me. But a lot of smart people disagree, and point to how shallow heuristics and layered didactic approaches are essential bridge inferential gaps under time constraints.
So I would like to pose the question to anyone else reading this: If you read “Toward Impact Markets” and you read the above post, do you think we should’ve gone for the same level of fidelity above? Or not? Or something in between?
EA Forum posts can disseminate info hazards that can be extremely harmful. (And this does not seem very unlikely, considering that the ideas that are discussed on the EA Forum are often related to anthropogenic x-risks.)
Excluding whole categories of usually valuable content from contests, though, seems like a very uncommon level of caution. I’m not saying that I *know* that it’s exaggerated caution, but there have been many prize contests for content on the EA Forum, and none of them were so concerned about info hazards. Some of them have had bigger prize pools too. And in addition the EA Forum is moderated, and the moderators probably have a protocol for how to respond to info hazards.
I’ve long pushed for something like the “EA Criticism and Red Teaming” contest (though I usually had more specific spins on the idea in mind), I’m delighted it exists, and I think it’ll be good. But it is a lot more risky than ours. It has a greater prize pool, the most important red-teaming should focus on topics that are important to EA at the moment, so “longtermism” (i.e. “how do we survive the next 20 years”) topics like biosecurity and AI safety, and the whole notion of red-teaming is conceptually close to info hazards too. (E.g., some people claim that some others invoke “info hazard” as a way to silence epistemic threats to their power. I mostly disagree, but my point is about how close the concepts are to each other.)
The original EA Forum Prize referred readers to the About page at the time (note that they, too, opted to put the details on a separate linked page), which explicitly discourages info hazards, rudeness, illegal activities, etc., but spends about a dozen words on fleshing this out precisely as opposed to our 10k+ words. Of course if you can communicate the same thing in a dozen and in 10k+ words, then a dozen is better, but if you think that “non-risky” is not clear about whether it refers to actions that are risky while they’re being performed or only to actions whose results remain risky indefinitely, then “What we discourage (and may delete) … Information hazards that concern us” is also unclear like that. Maybe someone is aware of an info hazard so dangerous that the moment they post it they can see from their own state of existence or nonexistence whether they got lucky or not. I think that both framings clearly discourage such sharing, but regardless, the contests are parallel in this regard. (Or, if anything, ours is safer because we are very, very explicit about the ex ante ceiling in our detailed explainer, with definitions, examples, diagrams, etc.)
But I don’t want to just throw this out there as an argument from authority, “If the EA Forum gods do it, it got to be okay.” It’s just that there is a precedent (over the course of four years or so) for lower levels of caution than ours and nothing terrible happening. That is valuable information for us when we try to make our own trade-off between risks and opportunity costs. (But of course all the badness can be contained in one Black Swan event that is yet to come, so there’s no certainty.)
The original EA Forum Prize does not seem to have had the distribution mismatch problem; the posts were presumably evaluated based on their ex-ante EV (or something like that?).
I don’t know if they were, so either way it was probably also not obvious to some post authors that they’d be judged by ex ante EV, and it’s enough for one of them to only think that they’ll be judged by ex post value to run into the distribution mismatch.
At least to the same extent – whatever it may be – as our contest. Expectational consequentialism seems to me like the norm, though that may be just my bubble, so I would judge both contests to be benign and net positive because I would expect most people to not want to gamble with everyone’s lives, to not think that a contest tries to encourage them to gamble with everyone’s lives, and to not want to just disguise their gamble from the prize committee.
In the original EA Forum Prize, the ex-post EV at the time of evaluation is usually similar to the ex-ante EV assuming that the evaluation happens closely after the post was written. (In a naive impact market, the price of a certificate can be high due to the chance that 3 years from now its ex-post EV will be extremely high.)
So you’re saying it’s fine for them not to make the distinction because they’re so quick that it hardly matters, but that it’s important for us? That makes sense. I suppose that circles back to my earlier comment that I think that our wording is pretty clear about the ex ante nature of the riskiness, but that we can make it even more clear by inserting a few more sentences into the post that make the ex ante part very explicit.
After this contest, it will still be the case that most people will be more likely to read and use instructions that are short and simple. It may be very hard to later “fix” the influence that posts like the OP has on potential future retro funders. Simpler instructions are more prone to become a meme. Therefore, retro funders may predict that some (most?) future retro funders will use the simple “buy likable impact” rule rather than the “adhere to the safety solutions in the Toward Impact Markets post” rule, and thus be incentivized to follow the simple rule themselves. Posts like the OP risk pushing everyone towards the Schelling point of “retro funders buy likable impact”. (All this becomes more worrisome if you or someone else in EA ends up launching a decentralized impact market).
Regarding the claim that “articles on the EA Forum” are “very unlikely to be extremely harmful”: EA Forum posts can disseminate info hazards that can be extremely harmful. (And this does not seem very unlikely, considering that the ideas that are discussed on the EA Forum are often related to anthropogenic x-risks.)
Hmm, I love writing high-fidelity content. Just thinking, “how can I express what I mean as clearly as I can” rather than “how can I simplify what I mean to maximize the fidelity/complexity ratio” is a lot easier for me. But a lot of smart people disagree, and point to how shallow heuristics and layered didactic approaches are essential bridge inferential gaps under time constraints.
So I would like to pose the question to anyone else reading this: If you read “Toward Impact Markets” and you read the above post, do you think we should’ve gone for the same level of fidelity above? Or not? Or something in between?
Excluding whole categories of usually valuable content from contests, though, seems like a very uncommon level of caution. I’m not saying that I *know* that it’s exaggerated caution, but there have been many prize contests for content on the EA Forum, and none of them were so concerned about info hazards. Some of them have had bigger prize pools too. And in addition the EA Forum is moderated, and the moderators probably have a protocol for how to respond to info hazards.
I’ve long pushed for something like the “EA Criticism and Red Teaming” contest (though I usually had more specific spins on the idea in mind), I’m delighted it exists, and I think it’ll be good. But it is a lot more risky than ours. It has a greater prize pool, the most important red-teaming should focus on topics that are important to EA at the moment, so “longtermism” (i.e. “how do we survive the next 20 years”) topics like biosecurity and AI safety, and the whole notion of red-teaming is conceptually close to info hazards too. (E.g., some people claim that some others invoke “info hazard” as a way to silence epistemic threats to their power. I mostly disagree, but my point is about how close the concepts are to each other.)
The original EA Forum Prize referred readers to the About page at the time (note that they, too, opted to put the details on a separate linked page), which explicitly discourages info hazards, rudeness, illegal activities, etc., but spends about a dozen words on fleshing this out precisely as opposed to our 10k+ words. Of course if you can communicate the same thing in a dozen and in 10k+ words, then a dozen is better, but if you think that “non-risky” is not clear about whether it refers to actions that are risky while they’re being performed or only to actions whose results remain risky indefinitely, then “What we discourage (and may delete) … Information hazards that concern us” is also unclear like that. Maybe someone is aware of an info hazard so dangerous that the moment they post it they can see from their own state of existence or nonexistence whether they got lucky or not. I think that both framings clearly discourage such sharing, but regardless, the contests are parallel in this regard. (Or, if anything, ours is safer because we are very, very explicit about the ex ante ceiling in our detailed explainer, with definitions, examples, diagrams, etc.)
But I don’t want to just throw this out there as an argument from authority, “If the EA Forum gods do it, it got to be okay.” It’s just that there is a precedent (over the course of four years or so) for lower levels of caution than ours and nothing terrible happening. That is valuable information for us when we try to make our own trade-off between risks and opportunity costs. (But of course all the badness can be contained in one Black Swan event that is yet to come, so there’s no certainty.)
The original EA Forum Prize does not seem to have had the distribution mismatch problem; the posts were presumably evaluated based on their ex-ante EV (or something like that?).
I don’t know if they were, so either way it was probably also not obvious to some post authors that they’d be judged by ex ante EV, and it’s enough for one of them to only think that they’ll be judged by ex post value to run into the distribution mismatch.
At least to the same extent – whatever it may be – as our contest. Expectational consequentialism seems to me like the norm, though that may be just my bubble, so I would judge both contests to be benign and net positive because I would expect most people to not want to gamble with everyone’s lives, to not think that a contest tries to encourage them to gamble with everyone’s lives, and to not want to just disguise their gamble from the prize committee.
In the original EA Forum Prize, the ex-post EV at the time of evaluation is usually similar to the ex-ante EV assuming that the evaluation happens closely after the post was written. (In a naive impact market, the price of a certificate can be high due to the chance that 3 years from now its ex-post EV will be extremely high.)
So you’re saying it’s fine for them not to make the distinction because they’re so quick that it hardly matters, but that it’s important for us? That makes sense. I suppose that circles back to my earlier comment that I think that our wording is pretty clear about the ex ante nature of the riskiness, but that we can make it even more clear by inserting a few more sentences into the post that make the ex ante part very explicit.