Undergraduate in Cognitive Science
Currently writing my thesis on genetic engineering attribution with deep learning under the supervision of Dr Oliver Crook at Oxford University
aaron_mai
Karma: 235
(Hastily written, sry)
I would love to see more of the theories of change that researchers in EA have for their own career! I’m particularly interested to see them in Global Priorities Research as its done at GPI (because I find that both extremely interesting and I’m very uncertain how useful it is apart from field-building).Two main reasons:
It’s not easy at all (in my experience) to figure out which claims are actually decision relevant in major ways. Seeing these theories of change might make it much easier for junior researchers to develop a “taste” for which research directions are tractable and important.
Publishing their theories of change would allow researchers to get more feedback on their project and perhaps realise earlier why it’s not that important. (This point of course applies not only to researchers). As a result of the second point, it seems less likely that EA researchers get off-track following intellectual curiosities rather than what’s most important (which I suspect researchers, in general, are prone to)
How do EA grantmakers take expert or peer opinions on decision-relevant claims into account? More precisely, if there’s some claim X that’s crucial to an EA grantmakers’ decision and probabilistic judgements from others are available on X (e.g. from experts) -- how do EA grantmakers tend to update on those judgements?
Motivation: I suspect that in these situations it’s common to just take some weighted average of the various credences and use that as one’s new probability estimate. I have some strong reasons to think that this is incompatible with bayesian updating (post coming soon).
I wonder if it would be good to create another survey to get some data not only on who people update on but also on how they update on others (regarding AGI timelines or something else). I was thinking of running a survey where I ask EAs about their prior on different claims (perhaps related to AGI development), present them with someone’s probability judgements and then ask them about their posterior. That someone could be a domain expert, non-domain expert (e.g., professor in a different field) or layperson (inside or outside EA).
At least if they have not received any evidence regarding the claim before, then there is a relatively simple and I think convincing model of how they should update: they should set their posterior odds in the claim to the product of their prior odds and someone else’s odds (this is the result of this paper, see e.g. p.18). It would then be possible to compare the way people update to this rational ideal. Running such a survey doesn’t seem very hard or expensive (although I don’t trust my intuition here at all) and we might learn a few interesting biases in how people defer to others in the context of (say) AI forecasts.I have a few more thoughts on exactly how to do this, but I’d be curious if you have any initial thoughts on this idea!
Cool idea to run this survey and I agree with many of your points on the dangers of faulty deference.
A few thoughts:
(Edit: I think my characterisation of what deference means in formal epistemology is wrong. After a few minutes of checking this, I think what I described is a somewhat common way of modelling how we ought to respond to experts)
-
The use of the concept of deference within the EA community is unclear to me. When I encountered the concept in formal epistemology I remember “deference to someone on claim X” literally meaning (a) that you adopt that persons probability judgement on X. Within EA and your post (?) the concept often doesn’t seem to be used in this way. Instead, I guess people think of deference as something like (b) “updating in the direction of a persons probability judgement on X” or (c)”taking that person’s probability estimate as significant evidence for (against) X if that person leans towards X (not-X)”?
-
I think (a) - (c) are importantly different. For instance, adopting someones credence doesn’t always mean that you are taking their opinion as evidence for the claim in question even if they lean towards it being true: you might adopt someones high credence in X and thereby lowering your credence (because yours was even higher before). In that case, you update as though their high credence was evidence against X. You might also update in the direction of someones credence without taking on their credence. Lastly, you might lower your credence in X by updating in someones direction even if they lean towards X.
Bottom line: these three concepts don’t refer to the same “epistemic process” so I think its good to make clear what we mean by deference.
Here is how I would draw the conceptual distinctions:
(I) deference to someones credence in X = you adopt their probability in X (II) positively updating on someone’s view = increasing your confidence in X upon hearing their probability on X (III) negatively updating on someones view = decreasing your confidence in X upon hearing their probability in X
I hope this comment was legible, please ask for clarification if anything was unclearly expressed :)
-
[Question] How valueable are external reviews?
Heuristics for making theoretical progress (in philosophy) from Alan Hajek (ANU)
However, even if we’d show that the repugnance of the repugnant conclusion is influenced in these ways or even rendered unreliable, I doubt the same would be true for the “very repugnant conclusion”:
for any world A with billions of happy people living wonderful lives, there is a world Z+ containing both a vast amount of mildly-satisfied lizards and billions of suffering people, such that Z+ is better than A.
(Credit to joe carlsmith who mentioned this on some podcast)
Thanks for the post!
I’m particularly interested in the third objection you present—that the value of “lives barely worth living” may be underrated.
I wonder to what extent the intuition that world Z is bad compared to A is influenced by framing effects. For instance, if I think of “lives net positive but not by much”, or something similar, this seems much more valueable than “lives barely worth living”, allthough it means the same in population ethics (as I understand it).
I’m also sympathetic to the claim that ones response to world Z may be affected by ones perception of the goodness of the ordinary (human) life. Perhaps, buddhists, who are convinced that ordinary life is pervaded with suffering, view any live that is net-positive as remarkably good.
Do you know if there exists any psychological literature on any of these two hypotheses? I’d be interested to research both.
I agree that it seems like a good idea to get somewhat familiar with that literature if we want to translate “longtermism” well.
I think I wouldn’t use “Langzeitethik” as this suggests, as you say, that longtermism is a field of research. In my mind, “longtermism” typically refers to a set of ethical views or a group of people/institutions. Probably people sometimes use the term to refer to a research field, but my impression is that this rather rare. Is that correct? :)
Also, I think that a new term—like “Befürworter der Langzeitverantwortung”—which is significantly longer than the established term, is unlikely to stick around both in conversation or in writing. “Longtermists” is faster to say and, at least in the beginning, easier to understand among EAs, so I think that people will prefer that. This might matter for the translation. It could be kind of confusing if the term in the new German EA literature is quite different from the one that is actually used by people in the German community
Thanks :)
Out of curiosity: how do you adjust for karma inflation?
This seems a bit inaccurate to me in a few ways, but I’m unsure how accurate we want to be here.
First, when the entry talks about “consequentialism” it seems to identify it with a decision procedure: “Consequentialists are supposed to estimate all of the effects of their actions, and then add them up appropriately”. In the literature, there is usually a distinction made between consequentialism as a criterion of rightness and a decision procedure, and it seems to me like many endorse the latter and not the former.Secondly, it seems to identify consequentialism with act-consequentialism, because it only refers to consequences of individual actions as the criterion for evaluation.
Red team: is it actually rational to have imprecise credences in possible longrun/indirect effects of our actions rather than precise ones?
Why: my understanding from Greaves (2016) and Mogensen (2020) is that this has been necessary to argue for the cluelessness worry.
Thanks! :) And great to hear that you are working on a documentary film for EA, excited to see that!
Re: EA-aligned Movies and Documentaries
I happen to know a well-established documentary filmmaker, whos areas of interest overlap with EA topics. I want to pitch him to work on a movie about x-risks. Do you have any further info about the kinds of documentaries you’d like to fund? Anything that’s not obvious from the website.
Hey! I wonder how flexible the starting date is. My semester ends mid-July, so I couldn’t start before. This is probably the case for most students from Germany. Is that too late?
Thanks for the post!
Does this apply at all to undergrads or graduate students who haven’t published any research yet?
Why is there so much more talk about the existential risk from AI as opposed to the amount by which individuals (e.g. researchers) should expect to reduce these risks through their work?
The second number seems much more decision-guiding for individuals than the first. Is the main reason that its much harder to estimate? If so, why?