So jobs don’t go away, they are just created in other areas.
This isn’t really true. Yes, probably there is some job replacement so that the jobs don’t literally disappear 1-for-1. But there will probably be fewer jobs, and I don’t think it’s easy to say (without doing some research) whether it’s 0.1 or 0.5 or 0.9 fewer jobs for each malaria net maker that goes away.
I like this idea. One danger (in both directions) with comparing to VC is that my impression is venture capital is way more focused on prestige and connections than funding charities is. In particular, if you can successfully become a prestigious, well-connected VC firm, then all of the Stanford/MIT students (for instance) will want you to fund their start-up, and picking with only minimal due diligence from among that group is likely to already be fairly profitable. [Disclaimer: I’m only tangentially connected to the VC world so this could be completely wrong, feel free to correct me.]
If this is true, what should we expect to see? We should expect that (1) VCs put in less research than OpenPhil (or similar organizations) when making investments, (2) hits-based is very successful for VC firms conditioned on having a strong established reputation. I would guess that both of these are true, though I’m unsure of the implications.
Also, I realized it might not be clear why I thought the quotes above are relevant to whether the reviews addressed the “theory-building” aspect. The point is it seems to me that the quoted parts of the reviews are directly engaging with whether the definitions make sense / the results are meaningful, which is a question about the adequacy of the theory for addressing the claimed questions, and not of its technical impressiveness. (I could imagine you don’t feel this addresses what you meant by theory-building, but in that case you’ll have to be more specific for me to understand what you have in mind.)
I feel like I care a lot about theory-building, and at least some of the other internal and external reviewers care a lot about it as well. As an example, consider External Review #1 of Paper #3 (particularly the section starting “How significant do you feel these results are for that?”). Here are some snippets (link to document here):
The first paragraph suggests that this problem is motivated by the concern of assigning probabilities to computations. This can be viewed as an instance of the more general problems of (a) modeling a resource-bounded decision maker computing probabilities and (b) finding techniques to help a resource-bounded decision maker compute probabilities. I find both of these problems very interesting. But I think that the model here is not that useful for either of these problems. Here are some reasons why:
It’s not clear why the properties of uniform coherence are the “right” ones to focus on. Uniform coherence does imply that, for any fixed formula, the probability converges to some number, which is certainly a requirement that we would want. This is implied by the second property of uniform coherence. But that property considers not just constant sequences of formulas, but sequence where the nth formula implies the (n+1)st. Why do we care about such sequences? [...]
The issue of computational complexity is not discussed in the paper, but it is clearly highly relevant. [...]
Several more points are raised, followed by (emphasis mine):
I see no obvious modification of uniformly coherent schemes that would address these concerns. Even worse, despite the initial motivation, the authors do not seem to be thinking about these motivational issues.
For another example, see External Review #1 of Paper #4 (I’m avoiding commenting on internal reviews because I want to be sensitive to breaking anonymity).
On the website, it is promised that this paper makes a step towards figuring out how to come up with “logically non-omniscient reasoners”. [...]
This surely sounds impressive, but there is the question whether this is a correct interpretation of Theorem 5. In particular, one could imagine two cases: a) we are predicting a single type of computation, and b) we are predicting several types of computations. In case (a), why would the delays matter in asymptotic convergence in the first place? [...] In case (b), the setting that is studied is not a good abstraction: in this case there should be some “contextual information” available to the learner, otherwise the only way to distinguish between two types of computations will be based on temporal relation, which is a very limiting assumption here.
To end with some thoughts of my own: in general, when theory-building I think it is very important to consider both the relevance of the theoretical definitions to the original problem of interest, and the richness of what can actually be said. I don’t think that definitions can be assessed independently of the theory that can be built from them. At the danger of self-promotion, I think that my own work here, which makes both definitional and theoretical contributions relevant to ML + security, does a good job of putting forth definitions and justifying them (by showing that we can get unexpectedly strong results in the setting considered, via a nice and fairly general algorithm, and that these results have unexpected and important implications for initially unrelated-seeming problems). I also claim that this work is relevant to AI safety but perhaps others will disagree.
(including the bizarre-ness of OpenPhil’s analysis of number-of-papers-written, which is not how one measures progress of fundamentals research.)
What in the grant write-up makes you think the focus was on number-of-papers-written? I was one of the reviewers and that was definitely not our process.
(Disclaimer: I’m a scientific advisor for OpenPhil, all opinions here are my own.)
Thanks. I think my reasons are basically the same as those in this post: http://effective-altruism.com/ea/14d/donor_lotteries_demonstration_and_faq/.