A clear problem with this model is that AFAICT, it assumes that (i) the size of the research community working on safety when AI is developed is independent of (ii) the the degree to which adding a researcher now will change the total number of researchers.
Both (i) and (ii) can vary by orders of magnitude, at least on my model, but are very correlated, because they depend on timelines. This means I get an oddly high chance of averting existential risk. If the questions where combined together into “what fraction of the AI community will the community by enlarged by adding an extra person” then I think my chance of averting existential risk would come out much lower.
Yes, I think this is a significant concern with this version of the model (somewhat less so with the original cruder version using something like medians, but that version also fails to pick up on legitimate effects of “what if these variables are all in the tails”). Combining the variables as you suggest is the easiest way to patch it. More complex would be to add in explicit time-dependency.
A clear problem with this model is that AFAICT, it assumes that (i) the size of the research community working on safety when AI is developed is independent of (ii) the the degree to which adding a researcher now will change the total number of researchers.
Both (i) and (ii) can vary by orders of magnitude, at least on my model, but are very correlated, because they depend on timelines. This means I get an oddly high chance of averting existential risk. If the questions where combined together into “what fraction of the AI community will the community by enlarged by adding an extra person” then I think my chance of averting existential risk would come out much lower.
Yes, I think this is a significant concern with this version of the model (somewhat less so with the original cruder version using something like medians, but that version also fails to pick up on legitimate effects of “what if these variables are all in the tails”). Combining the variables as you suggest is the easiest way to patch it. More complex would be to add in explicit time-dependency.