I see, I double checked the model. But obviously there are an infinite number of distributions that can be set for that assumption, and I don’t have any particular reason to believe the chance of MIRI solving its research agenda (in such a time as it could actually be used) is as high as to be modeled with a Beta(10, 45) distribution. In fact the problems I see identified in MIRI’s research agenda are extremely difficult, probably far more difficult than almost any Millennium Prize problem.
Considering solving the Poincare Conjecture problem took more than a century of effort with intense attention from the academic mathematical community (and was finally solved by a mathematics professor, not a “lone wolf” type), what reason do we have to believe that a small team of researchers could solve a set of probably far more difficult problems working more or less in a silo, outside of the purview of traditional academia? Has MIRI even produced any peer-reviewed scientific papers? I’d expect solving some of these problems to be about on par with the difficulty of solving P=NP, something I’d expect a small team to have something on the order of a 1 in 1 million chance of solving before anyone else given a refusal to work within the existing dominant form of mass intellectual collaboration (the system of academia). I could fit any distribution to that I wanted with at least as much plausibility as the estimates that went into this post, and get a far reduced overall value of MIRI with a point estimate somewhere around 1 in 1 million.
In fact, P=NP very plausibly might even be a prerequisite for a provably safe or friendly AI to even exist, or an explosion of computational power allowing every solution to actually be checked by brute force. Within certain domains there could be relatively simple software checks ton the output that ensure it is “safe” but whether that’s even possible and how it manifests depends on the particular application.
Also, using a skewed (i.e. alpha =/= beta) beta distribution is suspect to me. Intuitively I’d have little reason not to initially expect a symmetric distribution like a uniform distribution around the estimates.
I see, I double checked the model. But obviously there are an infinite number of distributions that can be set for that assumption, and I don’t have any particular reason to believe the chance of MIRI solving its research agenda (in such a time as it could actually be used) is as high as to be modeled with a Beta(10, 45) distribution. In fact the problems I see identified in MIRI’s research agenda are extremely difficult, probably far more difficult than almost any Millennium Prize problem.
Considering solving the Poincare Conjecture problem took more than a century of effort with intense attention from the academic mathematical community (and was finally solved by a mathematics professor, not a “lone wolf” type), what reason do we have to believe that a small team of researchers could solve a set of probably far more difficult problems working more or less in a silo, outside of the purview of traditional academia? Has MIRI even produced any peer-reviewed scientific papers? I’d expect solving some of these problems to be about on par with the difficulty of solving P=NP, something I’d expect a small team to have something on the order of a 1 in 1 million chance of solving before anyone else given a refusal to work within the existing dominant form of mass intellectual collaboration (the system of academia). I could fit any distribution to that I wanted with at least as much plausibility as the estimates that went into this post, and get a far reduced overall value of MIRI with a point estimate somewhere around 1 in 1 million.
In fact, P=NP very plausibly might even be a prerequisite for a provably safe or friendly AI to even exist, or an explosion of computational power allowing every solution to actually be checked by brute force. Within certain domains there could be relatively simple software checks ton the output that ensure it is “safe” but whether that’s even possible and how it manifests depends on the particular application.
Also, using a skewed (i.e. alpha =/= beta) beta distribution is suspect to me. Intuitively I’d have little reason not to initially expect a symmetric distribution like a uniform distribution around the estimates.