Three things:
more or less any level of intelligence could in principle be combined with more or less any final goal.
This seems to me kind of a weird statement of the thesis, “could in principle” being too weak.
If I understand, you’re not actually denying that just about any combination of intelligence and values could in principle occur. As you said, we can take a fact like the truth of evolution and imagine an extremely smart being that’s wrong about that specific thing. There’s no obvious impossibility there. It seems like the same would go for basically any fact or set of facts, normative or not.
I take it the real issue is one of probability, not possibility. Is an extremely smart being likely to accept what seem like glaringly obvious moral truths (like “you shouldn’t turn everyone into paperclips”) in virtue of being so smart?
(2) I was surprised to see you say your case depended completely on moral realism. Of course, if you’re a realist, it makes some sense to approach things that way. Use your background knowledge, right?
But I think even an anti-realist may still be able to answer yes to the question above, depending on how the being in question is constructed. For example, I think something in this anti-orthogonality vein is true of humans. They tend to be constructed so that understanding of certain non-normative facts puts pressure on certain values or normative views: If you improve a human’s ability to imaginatively simulate the experience of living in slavery (a non-moral intellectual achievement), they will be less likely to support slavery, and so on.
This is one direction I kind of expected you to go at some point after I saw the Aaronson quote mention “the practical version” of the thesis. That phrase has a flavor of, “Even if the thesis is mostly true because there are no moral facts to discover, it might still be false enough to save humanity.”
(3) But perhaps the more obvious the truths, the less intelligence matters. The claim about slavery is clearer to me than the claim that learning more about turning everyone into paperclips would make a person less likely to do so. It seems hard to imagine a person so ignorant as to not already appreciate all the morally relevant facts about turning people into paperclips. It’s as if, when the moral questions get so basic, intelligence isn’t going to make a difference. You’ve either got the values or you don’t. (But I’m a committed anti-realist, and I’m not sure how much that’s coloring these last comments.)
Here are some attempts:
“Apt is a tool that fits all hands.” (Emphasizes the wisdom of avoiding overly targeted interventions.) Or perhaps, “Apt is the ladder that anyone can stand on.”
“A seed well planted sprouts a forest.” (Emphasizes that money is an investment, not a short-term benefit.)
“Within the owl’s tree is what only the owl can see.” (Emphasizes that recipients have special knowledge about what’s good for them.)
A rough formula for generating these might be this:
Figure out the specific message/insight you want to convey. (The contest instructions leave this pretty wide open. I think saying a little more about what the organization would like to emphasize could help steer people in the right direction.)
Come up with a metaphor to express that message.
Tinker with it until it has a nice, poetic sound.
Another thing it would be helpful to know is what context you have in mind for this proverb. Are you guys looking for something you can use as a rebuttal when someone says “give a man a fish...”, or are you looking for something you can use as a slogan in its own right? (Or both? Or something else?)