bgarfinkel

Karma: 2,775

bgarfinkel Jun 30, 2021, 6:57 PM
7 points
0 ∶ 0
in reply to: Paul_Christiano’s comment on: Taboo “Outside View”

I suspect you are more broadly underestimating the extent to which people used “insect-level intelligence” as a generic stand-in for “pretty dumb,” though I haven’t looked at the discussion in Mind Children and Moravec may be making a stronger claim.

I think that’s good push-back and a fair suggestion: I’m not sure how seriously the statement in Nick’s paper was meant to be taken. I hadn’t considered that it might be almost entirely a quip. (I may ask him about this.)

Moravec’s discussion in Mind Children is similarly brief: He presents a graph of the computing power of different animal’s brains and states that “lab computers are roughly equal in power to the nervous systems of insects.”He also characterizes current AI behaviors as “insectlike” and writes: “I believe that robots with human intelligence will be common within fifty years. By comparison, the best of today’s machines have minds more like those of insects than humans. Yet this performance itself represents a giant leap forward in just a few decades.” I don’t think he’s just being quippy, but there’s also no suggestion that he means anything very rigorous/specific by his suggestion.

Rodney Brooks, I think, did mean for his comparisons to insect intelligence to be taken very seriously. The idea of his “nouvelle AI program” was to create AI systems that match insect intelligence, then use that as a jumping-off point for trying to produce human-like intelligence. I think walking and obstacle navigation, with several legs, was used as the main dimension of comparison. The Brooks case is a little different, though, since (IIRC) he only claimed that his robots exhibited important aspects of insect intelligence or fell just short insect intelligence, rather than directly claiming that they actually matched insect intelligence. On the other hand, he apparently felt he had gotten close enough to transition to the stage of the project that was meant to go from insect-level stuff to human-level stuff.

A plausible reaction to these cases, then, might be:

OK, Rodney Brooks did make a similar comparison, and was a major figure at the time, but his stuff was pretty transparently flawed. Moravec’s and Bostrom’s comments were at best fairly off-hand, suggesting casual impressions more than they suggest outcomes of rigorous analysis. The more recent “insect-level intelligence” claim is pretty different, since it’s built on top of much more detailed analysis than anything Moravec/Bostrom did, and it’s less obviously flawed than Brooks’ analysis. The likelihood that it reflects an erroneous impression is, therefore, a lot lower. The previous cases shouldn’t actually do much to raise our suspicion levels.

I think there’s something to this reaction, particularly if there’s now more rigorous work being done to operationalize and test the “insect-level intelligence” claim. I hadn’t yet seen the recent post you linked to, which, at first glance, seems like a good and clear piece of work. The more rigorous work is done to flesh out the argument, the less I’m inclined to treat the Bostrom/Moravec/Brooks cases as part of an epistemically relevant reference class.

My impression a few years ago was that the claim wasn’t yet backed by any really clear/careful analysis. At least, the version that filtered down to me seemed to be substantially based on fuzzy analogies between RL agent behavior and insect behavior, without anyone yet knowing much about insect behavior. (Although maybe this was a misimpression.) So I probably do stand by the reference class being relevant back then.

Overall, to sum up, my position here is something like: “The Bostrom/Moravec/Brooks cases do suggest that it might be easy to see roughly insect-level intelligence, if that’s what you expect to see and you’re relying on fuzzy impressions, paying special attention to stuff AI systems can already do, or not really operationalizing your claims. This should make us more suspicious of modern claims that we’ve recently achieved ‘insect-level intelligence,’ unless they’re accompanied by transparent and pretty obviously robust reasoning. Insofar as this work is being done, though, the Bostrom/Moravec/Brooks cases become weaker grounds for suspicion.”

bgarfinkel Jun 29, 2021, 1:46 PM
15 points
0 ∶ 0
in reply to: bgarfinkel’s comment on: Taboo “Outside View”
As a last thought here (no need to respond), I thought it might useful to give one example of a concrete case where: (a) Tetlock’s work seems relevant, and I find the terms “inside view” and “outside view” natural to use, even though the case is relatively different from the ones Tetlock has studied; and (b) I think many people in the community have tended to underweight an “outside view.”

A few years ago, I pretty frequently encountered the claim that recently developed AI systems exhibited roughly “insect-level intelligence.” This claim was typically used to support an argument for short timelines, since the claim was also made that we now had roughly insect-level compute. If insect-level intelligence has arrived around the same time as insect-level compute, then, it seems to follow, we shouldn’t be at all surprised if we get ‘human-level intelligence’ at roughly the point where we get human-level compute. And human-level compute might be achieved pretty soon.

For a couple of reasons, I think some people updated their timelines too strongly in response to this argument. First, it seemed like there are probably a lot of opportunities to make mistakes when constructing the argument: it’s not clear how “insect-level intelligence” or “human-level intelligence” should be conceptualised, it’s not clear how best to map AI behaviour onto insect behaviour, etc. The argument also hadn’t yet been vetted closely or expressed very precisely, which seemed to increase the possibility of not-yet-appreciated issues.

Second, we know that there are previous of examples of smart people looking at AI behaviour and forming the impression that it suggests “insect-level intelligence.” For example, in Nick Bostrom’s paper “How Long Before Superintelligence?” (1998) he suggested that “approximately insect-level intelligence” was achieved sometime in the 70s, as a result of insect-level computing power being achieved in the 70s. In Moravec’s book Mind Children (1990), he also suggested that both insect-level intelligence and insect-level compute had both recently been achieved. Rodney Brooks also had this whole research program, in the 90s, that was based around going from “insect-level intelligence” to “human-level intelligence.”

I think many people didn’t give enough weight to the reference class “instances of smart people looking at AI systems and forming the impression that they exhibit insect-level intelligence” and gave too much weight to the more deductive/model-y argument that had been constructed.

This case is obviously pretty different than the sorts of cases that Tetlock’s studies focused on, but I do still feel like the studies have some relevance. I think Tetlock’s work should, in a pretty broad way, make people more suspicious of their own ability to perform to linear/model-heavy reasoning about complex phenomena, without getting tripped up or fooling themselves. It should also make people somewhat more inclined to take reference classes seriously, even when the reference classes are fairly different from the sorts of reference classes good forecasters used in Tetlock’s studies. I do also think that the terms “inside view” and “outside view” apply relatively neatly, in this case, and are nice bits of shorthand — although, admittedly, it’s far from necessary to use them.

This is the sort of case I have in the back of my mind.

(There are also, of course, cases that point in the opposite direction, where many people seemingly gave too much weight to something they classified as an “outside view.” Early under-reaction to COVID is arguably one example.)

bgarfinkel Jun 29, 2021, 1:05 PM
4 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
Thank you (and sorry for my delayed response)!

I shudder at the prospect of having a discussion about “Outside view vs inside view: which is better? Which is overrated and which is underrated?” (and I’ve worried that this thread may be tending in that direction) but I would really look forward to having a discussion about “let’s look at Daniel’s list of techniques and talk about which ones are overrated and underrated and in what circumstances each is appropriate.”

I also shudder a bit at that prospect.

I am sometimes happy making pretty broad and sloppy statements. For example: “People making political predictions typically don’t make enough use of ‘outside view’ perspectives” feels fine to me, as a claim, despite some ambiguity around the edges. (Which perspectives should they use? How exactly should they use them? Etc.)

But if you want to dig in deep, for example when evaluating the rationality of a particular prediction, you should definitely shift toward making more specific and precise statements. For example, if someone has based their own AI timelines on Katja’s expert survey, and they wanted to defend their view by simply evoking the principle “outside views are better than inside views,” I think this would probably a horrible conversation. A good conversation would focus specifically on the conditions under which it makes sense to defer heavily to experts, whether those conditions apply in this particular case, etc. Some general Tetlock stuff might come into the conversation, like: “Tetlock’s work suggests it’s easy to trip yourself up if you try to use your own detailed/causal model of the world to make predictions, so you shouldn’t be so confident that your own ‘inside view’ prediction will be very good either.” But mostly you should be more specific.
Now I’ll try to say what I think your position is:
If people were using “outside view” without explaining more specifically what they mean, that would be bad and it should be tabood, but you don’t see that in your experience
If the things in the first Big List were indeed super diverse and disconnected from the evidence in Tetlock’s studies etc., then there would indeed be no good reason to bundle them together under one term. But in fact this isn’t the case; most of the things on the list are special cases of reference-class / statistical reasoning, which is what Tetlock’s studies are about. So rather than taboo “outside view” we should continue to use the term but mildly prune the list.
There may be a general bias in this community towards using the things on the first Big List, but (a) in your opinion the opposite seems more true, and (b) at any rate even if this is true the right response is to argue for that directly rather than advocating the tabooing of the term.
How does that sound?
I’d say that sounds basically right!

The only thing is that I don’t necessarily agree with 3a.

I think some parts of the community lean too much on things in the bag (the example you give at the top of the post is an extreme example). I also think that some parts of the community lean too little on things in the bag, in part because (in my view) they’re overconfident in their own abilities to reason causally/deductively in certain domains. I’m not sure which is overall more problematic, at the moment, in part because I’m not sure how people actually should be integrating different considerations in domains like AI forecasting.

There also seem to be biases that cut in both directions. I think the ‘baseline bias’ is pretty strongly toward causal/deductive reasoning, since it’s more impressive-seeming, can suggest that you have something uniquely valuable to bring to the table (if you can draw on lots of specific knowledge or ideas that it’s rare to possess), is probably typically more interesting and emotionally satisfying, and doesn’t as strongly force you to confront or admit the limits of your predictive powers. The EA community has definitely introduced an (unusual?) bias in the opposite direction, by giving a lot of social credit to people who show certain signs of ‘epistemic virtue.’ I guess the pro-causal/deductive bias often feels more salient to me, but I don’t really want to make any confident claim here that it actually is more powerful.

bgarfinkel Jun 28, 2021, 8:12 PM
3 points
0 ∶ 0
in reply to: Rohin Shah’s comment on: Ben Garfinkel’s Shortform

I’m not sure if you think this is an interesting point to notice that’s useful for building a world-model, and/or a reason to be skeptical of technical alignment work. I’d agree with the former but disagree with the latter.

Mostly the former!

I think the point may have implications for how much we should prioritize alignment research, relative to other kinds of work, but this depends on what the previous version of someone’s world model was.

For example, if someone has assumed that solving the ‘alignment problem’ is close to sufficient to ensure that humanity has “control” of its future, then absorbing this point (if it’s correct) might cause them to update downward on the expected impact of technical alignment research. Research focused on coordination-related issues (e.g. cooperative AI stuff) might increase in value, at least in relative terms.

bgarfinkel Jun 23, 2021, 12:03 PM
12 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
It’s definitely entirely plausible that I’ve misunderstood your views.

My interpretation of the post was something like this:

There is a bag of things that people in the EA community tend to describe as “outside views.” Many of the things in this bag are over-rated or mis-used by members of the EA community, leading to bad beliefs.

One reason for this over-use or mis-use is that the the term “outside view” has developed an extremely positive connotation within the community. People are applauded for saying that they’re relying on “outside views” — “outside view” has become “an applause light” — and so will rely on items in the bag to an extent that is epistemically unjustified.

The things in the bag are also pretty different from each other — and not everyone who uses the term “outside view” agrees about exactly what belongs in the bag. This conflation/ambiguity can lead to miscommunication.

More importantly, when it comes to the usefulness of the different items in the bag, some have more evidential support than others. Using the term “outside view” to refer to everything in the bag might therefore lead people to overrated certain items that actually have weak evidential support.

To sum up, tabooing the term “outside view” might solve two problems. First, it might reduce miscommunication. Second, more importantly, it might cause people to stop overrating some of the reasoning processes that they currently characterize as involving “outside views.” The mechanisms by which tabooing the term can help to solve the second problem are: (a) it takes away an “applause light,” whose existence incentivizes excessive use of these reasoning processes, and (b) it allows people to more easily recognize that some of these reasoning processes don’t actually have much empirical support.

I’m curious if this feels roughly right, or feels pretty off.

Part of the reason I interpreted your post this way: The quote you kicked the post off suggested to me that your primary preoccupation was over-use or mis-use of the tools people called “outside views,” including more conventional reference-class forecasting. It seemed like the quote is giving an example of someone who’s refusing to engage in causal reasoning, evaluate object-level arguments, etc., based on the idea that outside views are just strictly dominant in the context of AI forecasting. It seemed like this would have been an issue even if the person was doing totally orthodox reference-class forecasting and there was no ambiguity about what they were doing.^[1]

I don’t think that you’re generally opposed to the items in the “outside view” bag or anything like that. I also don’t assume that you disagree with most of the points I listed in my last comment, for why I think intellectuals probably on average underrated the items in the bag. I just listed all of them because you asked for an explanation for my view, I suppose with some implication that you might disagree with it.

You’ve also given two rough definitions of the term, which seem quite different to me, and also quite fuzzy. (e.g. if by “reference class forecasting” you mean the stuff Tetlock’s studies are about, then it really shouldn’t include the anti-weirdness heuristic, but it seems like you are saying it does?)

I think it’s probably not worth digging deeper on the definitions I gave, since I definitely don’t think they’re close to perfect. But just a clarification here, on the anti-weirdness heuristic: I’m thinking of the reference class as “weird-sounding claims.”

Suppose someone approaches you not the street and hands you a flyer claiming: “The US government has figured out a way to use entangled particles to help treat cancer, but political elites are hoarding the particles.” You quickly form a belief that the flyer’s claim is almost certainly false, by thinking to yourself: “This is a really weird-sounding claim, and I figure that virtually all really weird-sounding claims that appear in random flyers are wrong.”

In this case, you’re not doing any deductive reasoning about the claim itself or relying on any causal models that directly bear on the claim. (Although you could.) For example, you’re not thinking to yourself: “Well, I know about quantum mechanics, and I know entangled particles couldn’t be useful for treating cancer for reason X.” Or: “I understand economic incentives, or understand social dynamics around secret-keeping, so I know it’s unlikely this information would be kept secret.” You’re just picking a reference class — weird-sounding claims made on random flyers — and justifying your belief that way.

I think it’s possible that Tetlock’s studies don’t bear very strongly on the usefulness of this reference class, since I imagine participants in his studies almost never used it. (“The claim ‘there will be a coup in Venezuela in the next five years’ sounds really weird to me, and most claims that sound weird to me aren’t true, so it’s probably not true!”) But I think the anti-weirdness heuristic does fit with the definitions I gave, as well as the definition you give that characterizes the term’s “original meaning.” I also do think that Tetlock’s studies remain at least somewhat relevant when judging the potential usefulness of the heuristic.
1. ↩︎
  I initially engaged on the miscommunication, point, though, since this is the concern that would mostly strongly make me want to taboo the term. I’d rather address the applause light problem, if it is a problem, but trying get people in the EA community stop applauding, and the evidence problem, if it is a problem, by trying to just directly make people in the EA community more aware of the limits of evidence.

bgarfinkel Jun 21, 2021, 11:26 AM
19 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
On the contrary; tabooing the term is more helpful, I think. I’ve tried to explain why in the post. I’m not against the things “outside view” has come to mean; I’m just against them being conflated with / associated with each other, which is what the term does. If my point was simply that the first Big List was overrated and the second Big List was underrated, I would have written a very different post!

My initial comment was focused on your point about conflation, because I think this point bears on the linguistic question more strongly than the other points do. I haven’t personally found conflation to be a large issue. (Recognizing, again, that our experiences may differ.) If I agreed with the point about conflation, though, then I would think it might be worth tabooing the term “outside view.”

By what definition of “outside view?”

By “taking an outside view on X” I basically mean “engaging in statistical or reference-class-based reasoning.” I think it might also be best defined negatively: “reasoning that doesn’t substantially involve logical deduction or causal models of the phenomenon in question. “^[1]

I think most of the examples in your list fit these definitions.

Epistemic deference is a kind of statistical/reference-class-based reasoning, for example, which doesn’t involve applying any sort of causal model of the phenomenon in question. The logic is “Ah, I should update downward on this claim, since experts in domain X disagree with it and I think that experts in domain X will typically be right.”

Same for anti-weirdness: The idea is that weird claims are typically wrong.

I’d say that trend extrapolation also fits: You’re not doing logical reasoning or relying on a causal model of the relevant phenomenon. You’re just extrapolating a trend forward, largely based on the assumption that long-running trends don’t typically end abruptly.

“Foxy aggregation,” admittedly, does seem like a different thing to me: It arguably fits the negative definition, depending on how you generate your weights, but doesn’t seem to fit statistical/reference-class one. It also feels like more of a meta-level thing. So I wouldn’t personally use the term “outside view” to talk about foxy aggregation. (I also don’t think I’ve personally heard people use the term “outside view” to talk about foxy aggegration, although I obviously believe you have.)

There is some evidence that in some circumstances people don’t take reference class forecasting seriously enough; that’s what the original term “outside view” meant. What evidence is there that the things on the Big List O’ Things People Describe as Outside View are systematically underrated by the average intellectual?

A condensed answer is: (a) I think most public intellectuals barely use any of the items on this list (with the exception of the anti-weirdness heuristic); (b) I think some of the things on this list are often useful; (c) I think that intellectuals and people more generally are very often bad at reasoning causally/logically about complex social phenomena; (d) I expect intellectuals to often have a bias against outside-view-style reasoning, since it often feels somewhat unsatisfying/unnatural and doesn’t allow them to display impressive-seeming domain-knowledge, interesting models of the world, or logical reasoning skills; and (e) I do still think Tetlock’s evidence is at least somewhat relevant to most things on the list, in part because I think they actually are somewhat related to each other, although questions of external validity obviously grow more serious the further you move from the precise sorts of questions asked in his tournaments and the precise styles of outside-view reasoning displayed by participants. ^[2]

There’s also, of course, a bit of symmetry here. One could also ask: “What evidence is there that the things on the Big List O’ Things People Describe as Outside View are systematically overrated by the average intellectual?” :)
1. ↩︎
  These definitions of course aren’t perfect, and other people sometimes use the term more broadly than I do, but, again, some amount of fuzziness seems OK to me. Most concepts have fuzzy boundaries and are hard to define precisely.
2. ↩︎
  On the Tetlock evidence: I think one thing his studies suggest, which I expect to generalize pretty well to many different contexts, is that people who are trying to make predictions about complex phenemona (especially complex social phenemona) often do very poorly when they don’t incorporate outside views into their reasoning processes. (You can correct me if this seems wrong, since you’ve thought about Tetlock’s work far more than I have.) So, on my understanding, Tetlock’s work suggests that outside-view-heavy reasoning processes would often substitute for reasoning processes that lead to poor predictions anyways. At least for most people, then, outside-view-heavy reasoning processes don’t actually need to be very reliable to constitute improvements—and they need to be pretty bad to, on average, lead to worse predictions.
  
  Another small comment here: I think Tetlock’s work also counts, in a somewhat broad way, against the “reference class tennis” objection to reference-class-based forecasting. On its face, the objection also applies to the use of reference classes in standard forecasting tournaments. There are always a ton of different reference classes someone could use to forecast any given political event. Forecasters need to rely on some sort of intuition, or some sort of fuzzy reasoning, to decide on which reference classes to take seriously; it’s a priori plausible that people would be just consistently very bad at this, given the number of degrees of freedom here and the absence of clear principles for making one’s selections. But this issue doesn’t actually seem to be that huge in the context of the sorts of questions Tetlock asked his participants. (You can again correct me if I’m wrong.) The degrees-of-freedom problem might be far larger in other contexts, but the fact that the issue is manageable in Tetlockian contexts presumably counts as at least a little bit of positive evidence.

bgarfinkel Jun 19, 2021, 11:54 AM
19 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
I agree that people sometimes put too much weight on particular outside views—or do a poor job of integrating outside views with more inside-view-style reasoning. For example, in the quote/paraphrase you present at the top of your post, something has clearly gone wrong.^[1]

But I think the best intervention, in this case, is probably just to push the ideas “outside views are often given too much weight” or “heavily reliance on outside views shouldn’t be seen as praiseworthy” or “the correct way to integrate outside views with more inside-view reasoning is X.” Tabooing the term itself somehow feels a little roundabout to me, like a linguistic solution to a methodological disagreement.

I think you’re right that “outside view” now has a very positive connotation. If enough community members become convinced that this positive connotation is unearned, though, I think the connotation will probably naturally become less positive over time. For example, the number of upvotes on this post is a signal that people shouldn’t currently expect that much applause for using the term “outside view.”
1. ↩︎
  As a caveat, although I’m not sure how much this actually matters for the present discussion, I probably am significantly less concerned about the problem than you are. I’m pretty confident that the average intellectual doesn’t pay enough attention to “outside views”—and I think that, absent positive reinforcement from people in your community, it actually does take some degree of discipline to take outside views sufficiently seriously. I’m open to the idea that the average EA community member has over-corrected, here, but I’m not yet convinced of it. I think it’s also possible that, in a lot of cases, the natural substitute for bad outside-view-heavy reasoning is worse inside-view-heavy reasoning.

bgarfinkel Jun 18, 2021, 10:20 PM
24 points
0 ∶ 0
on: Taboo “Outside View”
When people use “outside view” or “inside view” without clarifying which of the things on the above lists they mean, I am left ignorant of what exactly they are doing and how well-justified it is. People say “On the outside view, X seems unlikely to me.” I then ask them what they mean, and sometimes it turns out they are using some reference class, complete with a dataset. (Example: Tom Davidson’s four reference classes for TAI). Other times it turns out they are just using the anti-weirdness heuristic. Good thing I asked for elaboration!

FWIW, as a contrary datapoint, I don’t think I’ve really encountered this problem much in conversation. In my own experience (which may be quite different from yours): when someone makes some reference to an “outside view,” they say something that indicates roughly what kind of “outside view” they’re using. For example, if someone is just extrapolating a trend forward, they’ll reference the trend. Or if someone is deferring to expert opinion, they’ll reference expert opinion. I also don’t think I’d find it too bothersome, in any case, to occasionally have to ask the person which outside view they have in mind.

So this concern about opacity wouldn’t be enough to make me, personally, want people to stop using the term “outside view.”

If there’s a really serious linguistic issue, here, I think it’s probably that people sometimes talk about “the outside view” as though there’s only a single relevant outside view. I think Michael Aird made a good comment on my recent democracy post, where he suggests that people should taboo the phrase “the outside view” and instead use the phrase “an outside view.” (I was guilty of using the phrase “the outside view” in that post — and, arguably, of leaning too hard on one particular way of defining a reference class.) I’d be pretty happy if people just dropped the “the,” but kept talking about “outside views.”^[1]
1. ↩︎
  It’s of course a little ambiguous what counts as an “outside view,” but in practice I don’t think this is too huge of an issue. In my experience, which again may be different from yours, “taking an outside view” still does typically refer to using some sort of reference-class-based reasoning. It’s just the case that there are lots of different reference classes that people use. (“I’m extrapolating this 20-year trend forward, for another five years, because if a trend has been stable for 20 years it’s typically stable for another five.” “I’m deferring to the experts in this survey, because experts typically have more accurate views than amateurs.” Etc.) I do feel like this style of reasoning is useful and meaningfully distinct from, for example, reasoning based on causal models, so I’m happy to have a term for it, even if the boundaries of the concept are somewhat fuzzy.

bgarfinkel May 18, 2021, 11:03 PM
10 points
0 ∶ 0
in reply to: Max_Daniel’s comment on: What are things everyone here should (maybe) read?
Fortunately, if I remember correctly, something like the distinction between the true criterion of rightness and the best practical decision procedure actually is a major theme in the Kagan book. (Although I think the distinction probably often is underemphasized.)

It is therefore kind of misleading to think of consequentialism vs. deontology vs. virtue ethics as alternative theories, which however is the way normative ethics is typically presented in the analytic tradition.

I agree there is something to this concern. But I still wouldn’t go so far as to say that it’s misleading to think of them as alternative theories. I do think they count as conceptually distinct (even if the boundaries are sometimes a bit muddy), and I think they do sometimes have different implications for how you should in fact make moral decisions.

Beyond the deontology/consequentialism debate, I think there are also relevant questions around demandingness (how strong are our moral obligations, if any?), on the nature of well-being (e.g. hedonistic vs. preference-based vs. objective list theories), on the set of things that count as morally relevant consequences (e.g. do things beyond well-being matter? should we care more about totals or averages?), and so on.

bgarfinkel May 18, 2021, 9:52 PM
22 points
0 ∶ 0
on: What are things everyone here should (maybe) read?
A slightly boring answer: I think most people should at least partly read something that overviews common theories and frameworks in normative ethics (and the arguments for and against them) and something that overviews core concepts and principles in economics (e.g. the idea of expected utility, the idea of an externality, supply/demand, the basics of economic growth, the basics of public choice).

In my view, normative ethics and economics together make up a really large portion of the intellectual foundation that EA is built on.

One good book that overviews normative ethics is Shelly Kagan’s Normative Ethics, although I haven’t read it since college (and I think it has only a tiny amount of coverage of population ethics and animal ethics). One thing I like about it is it focuses on laying out the space of possible ethical views in a sensible way, rather than tracing the history of the field. If I remember correctly, names like Aristotle, Kant, etc. never show up. It’s also written in a very conversational style.

One good introductory economics textbook is Tyler Cowen’s and Alex Tabarrok’s Modern Principles of Economics. I don’t know how it stacks up to other intro textbooks, since it’s the only one that I’ve read more than a little of, but it’s very readable, has very little math, and emphasizes key concepts and principles. Reading just the foundational chapters in an intro textbook, then the chapters whose topics sound important, can probably get most people a decent portion of the value of reading a full textbook.

bgarfinkel May 4, 2021, 12:18 PM
1 point
0 ∶ 0
in reply to: Linch’s comment on: Ben Garfinkel’s Shortform
That’s a good example.

I do agree that quasi-random variation in culture can be really important. And I agree that this variation is sometimes pretty sticky (e.g. Europe being predominantly Christian and the Middle East being predominantly Muslim for more than a thousand years). I wouldn’t say that this kind of variation is a “rounding error.”

Over sufficiently long timespans, though, I think that technological/economic change has been more significant.

As an attempt to operationalize this claim: The average human society in 1000AD was obviously very different than the average human society in 10,000BC. I think that the difference would have been less than half as large (at least in intuitive terms) if there hadn’t been technological/economic change.

I think that the pool of available technology creates biases in the sorts of societies that emerge and stick around. For large enough amounts of technological change, and long enough timespans (long enough for selection pressures to really matter), I think that shifts in these technological biases will explain a large portion of the shifts we see in the traits of the average society.^[1]
1. ↩︎
  If selection pressures become a lot weaker in the future, though, then random drift might become more important in relative terms.

bgarfinkel May 3, 2021, 2:01 PM
2 points
0 ∶ 0
in reply to: abergal’s comment on: Ben Garfinkel’s Shortform
FWIW, I wouldn’t say I agree with the main thesis of that post.

However, while I expect machines that outcompete humans for jobs, I don’t see how that greatly increases the problem of value drift. Human cultural plasticity already ensures that humans are capable of expressing a very wide range of values. I see no obviously limits there. Genetic engineering will allow more changes to humans. Ems inherit human plasticity, and may add even more via direct brain modifications.

In principle, non-em-based artificial intelligence is capable of expressing the entire space of possible values. But in practice, in the shorter run, such AIs will take on social roles near humans, and roles that humans once occupied....

I don’t see why people concerned with value drift should be especially focused on AI. Yes, AI may accompany faster change, and faster change can make value drift worse for people with intermediate discount rates. (Though it seems to me that altruistic discount rates should scale with actual rates of change, not with arbitrary external clocks.)

I definitely think that human biology creates at least very strong biases toward certain values (if not hard constraints) and that AI system would not need to have these same biases. If you’re worried about future agents having super different and bad values, then AI is a natural focal point for your worry.

A couple other possible clarifications about my views here:
- I think that the outcome of the AI Revolution could be much worse, relative to our current values, than the Neolithic Revolution was relative to the values of our hunter-gatherer ancestors. But I think the question “Will the outcome be worse?” is distinct from the question “Will we have less freedom to choose the outcome?”
- I’m personally not so focused on value drift as a driver of long-run social change. For example, the changes associated with the Neolithic Revolution weren’t really driven by people becoming less egalitarian, more pro-slavery, more inclined to hold certain religious beliefs, more ideologically attached to sedentism/farming, more happy to accept risks from disease, etc. There were value changes, but, to some significant degree, they seem to have been downstream of technological/economic change.

bgarfinkel May 3, 2021, 11:44 AM
2 points
0 ∶ 0
in reply to: abergal’s comment on: Ben Garfinkel’s Shortform

Do you have the intuition that absent further technological development, human values would drift arbitrarily far?

Certainly not arbitrarily far. I also think that technological development (esp. the emergence of agriculture and modern industry) has played a much larger role in changing the world over time than random value drift has.

[E]ven non-extinction AI is enabling a new set of possibilities that modern-day humans would endorse much less than the decisions of future humans otherwise.

I definitely think that’s true. But I also think that was true of agriculture, relative to the values of hunter-gatherer societies.

To be clear, I’m not downplaying the likelihood or potential importance of any of the three crisper concerns I listed. For example, I think that AI progress could conceivably lead to a future that is super alienating and bad.

I’m just (a) somewhat pedantically arguing that we shouldn’t frame the concerns as being about a “loss of control over the future” and (b) suggesting that you can rationally have all these same concerns even if you come to believe that technical alignment issues aren’t actually a big deal.

bgarfinkel May 2, 2021, 3:30 PM
35 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
A thought on how we describe existential risks from misaligned AI:

Sometimes discussions focus on a fairly specific version of AI risk, which involves humanity being quickly wiped out. Increasingly, though, the emphasis seems to be on the more abstract idea of “humanity losing control of its future.” I think it might be worthwhile to unpack this latter idea a bit more.

There’s already a fairly strong sense in which humanity has never controlled its own future. For example, looking back ten thousand years, no one decided that the sedentary agriculture would increasingly supplant hunting and gathering, that increasingly complex states would arise, that slavery would become common, that disease would take off, that social hierarchies and gender divisions would become stricter, etc. The transition to the modern world, and everything that came with this transition, also doesn’t seem to have been meaningfully chosen (or even really understood by anyone). The most serious effort to describe a possible future in detail — Hanson’s Age of Em — also describes a future with loads of features that most present-day people would not endorse.

As long as there are still strong competitive pressures or substantial random drift, it seems to me, no generation ever really gets to choose the future.^[1] It’s actually sort of ambiguous, then, what it means to worry about “losing control of our future.”

Here are a few alternative versions of the concern that feel a bit crisper to me:
1. If we ‘mess up on AI,’ then even the most powerful individual humans will have unusually little influence over their own lives or the world around them.^[2]
1. If we ‘mess up on AI,’ then future people may be unusually dissatisfied about the world they live in. In other words, people’s preferences will be unfilled to an unusually large degree.
2. Humanity may have a rare opportunity to take control of its own future, by achieving strong coordination and then locking various things in. But if we ‘mess up on AI,’ then we’ll miss out on this opportunity.^[3]
Something that’s a bit interesting about these alternative versions of the concern, though, is that they’re not inherently linked to AI alignment issues. Even if AI systems behave roughly as their users intend, I believe each of these outcomes is still conceivable. For example, if there’s a missed opportunity to achieve strong coordination around AI, the story might look like the failure of the Baruch Plan for international control of nuclear weapons: that failure had much more to do with politics than it had to do with the way engineers designed the technology in question.

In general, if we move beyond discussing very sharp alignment-related catastrophes (e.g. humanity being quickly wiped out), then I think concerns about misaligned AI start to bleed into broader AI governance concerns. It starts to become more ambiguous whether technical alignment issues are actually central or necessary to the disaster stories people tell.
1. ↩︎
  Although, admittedly, notable individuals or groups (e.g. early Christians) do sometimes have a fairly lasting and important influence.
2. ↩︎
  As an analogy, in the world of The Matrix, people may not actually have much less control over the long-run future than hunter-gatherers did twenty thousand years ago. But they certainly have much less control over their own lives.
3. ↩︎
  Notably, this is only a bad thing if we expect the relevant generation of humans to choose a better future than would be arrived at by default.

bgarfinkel Apr 30, 2021, 9:25 PM
2 points
0 ∶ 0
in reply to: JP Addison🔸’s comment on: Ben Garfinkel’s Shortform
Good point!

That consideration—and the more basic consideration that more junior people often just know less—definitely pushes in the opposite direction. If you wanted to try some version of seniority-weighted epistemic deference, my guess is that the most reliable cohort would have studied a given topic for at least a few years but less than a couple decades.

bgarfinkel Apr 30, 2021, 12:18 PM
29 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
A thought on epistemic deference:

The longer you hold a view, and the more publicly you hold a view, the more calcified it typically becomes. Changing your mind becomes more aversive and potentially costly, you have more tools at your disposal to mount a lawyerly defense, and you find it harder to adopt frameworks/perspectives other than your favored one (the grooves become firmly imprinted into your brain). At least, this is the way it seems and personally feels to me.^[1]

For this reason, the observation “someone I respect publicly argued for X many years ago and still believes X” typically only provides a bit more evidence than the observation “someone I respect argued for X many years ago.” For example, even though I greatly respect Daron Acemoglu, I think the observation “Daron Acemoglu still believes that political institutions are the central determinant of economic growth rates” only gives me a bit more evidence than the observation “15 years ago Daron Acemoglu publicly argued that institutions are the central determinant of economic growth rates.”

A corollary: If there’s an academic field that contains a long-standing debate, and you’d like to defer to experts in this field, you may want to give disproportionate weight to the opinions of junior academics. They’re less likely to have responded to recent evidence and arguments in an epistemically inflexible way.
1. ↩︎
  Of course, there are exceptions. The final chapter of Scout Mindset includes a moving example of a professor publicly abandoning a view he had championed for fifteen years, after a visiting academic presented persuasive new evidence. The reason these kinds of stories are moving, though, is that they describe truly exceptional behavior.

bgarfinkel Apr 19, 2021, 10:08 AM
8 points
0 ∶ 0
in reply to: Stefan_Schubert’s comment on: Ben Garfinkel’s Shortform
I’d actually say this is a variety of qualitative research. At least in the main academic areas I follow, though, it seems a lot more common to read and write up small numbers of detailed case studies (often selected for being especially interesting) than to read and write up large numbers of shallow case studies (selected close to randomly).

This seems to be true in international relations, for example. In a class on interstate war, it’s plausible people would be assigned a long analysis of the outbreak WW1, but very unlikely they’d be assigned short descriptions of the outbreaks of twenty random wars. (Quite possible there’s a lot of variation between fields, though.)

bgarfinkel Apr 18, 2021, 11:19 PM
36 points
0 ∶ 0
in reply to: bgarfinkel’s comment on: Ben Garfinkel’s Shortform
In general, I think “read short descriptions of randomly sampled cases” might be an underrated way to learn about the world and notice issues with your assumptions/models.

A couple other examples:

I’ve been trying to develop a better understanding of various aspects of interstate conflict. The Correlates of War militarized interstate disputes (MIDs) dataset is, I think, somewhat useful for this. The project files include short descriptions of (supposedly) every case between 1993 and 2014 in which one state “threatened, displayed, or used force against another.” Here, for example, is the set of descriptions for 2011-2014. I’m not sure I’ve had any huge/concrete take-aways, but I think reading the cases: (a) made me aware of some international tensions I was oblivious to; (b) gave me a slightly better understanding of dynamics around ‘micro-aggressions’ (e.g. flying over someone’s airspace); and (c) helped me more strongly internalize the low base rate for crises boiling over into war (since I disproportionately read about historical disputes that turned into something larger).

Last year, I also spent a bit of time trying to improve my understanding of police killings in the US. I found this book unusually useful. It includes short descriptions of every single incident in which an unarmed person was killed by a police officer in 2015. I feel like reading a portion of it helped me to quickly notice and internalize different aspects of the problem (e.g. the fact that something like a third of the deaths are caused by tasers; the large role of untreated mental illness as a risk factor; the fact that nearly all fatal interactions are triggered by 911 calls, rather than stops; the fact that officers are trained to interact importantly differently with people they believe are on PCP; etc.). l assume I could have learned all the same things by just reading papers — but I think the case sampling approach was probably faster and better for retention.

I think it’s possible there might be value in creating “random case descriptions” collections for a broader range of phenomena. Academia really doesn’t emphasize these kinds of collections as tools for either research or teaching.

EDIT: Another good example of this approach to learning is Rob Besinger’s recent post “thirty-three randomly selected bioethics papers.”

bgarfinkel Apr 9, 2021, 7:39 PM
18 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
The O*NET database includes a list of about 20,000 different tasks that American workers currently need to perform as part of their jobs. I’ve found it pretty interesting to scroll through the list, sorted in random order, to get a sense of the different bits of work that add up to the US economy. I think anyone who thinks a lot about AI-driven automation might find it useful to spend five minutes scrolling around: it’s a way of jumping yourself down to a lower level of abstraction. I think the list is also a little bit mesmerizing, in its own right.

One update I’ve made is that I’m now more confident that more than half of present-day occupational tasks could be automated using fairly narrow, non-agential, and boring-looking AI systems. (Most of them don’t scream “this task requires AI systems with long-run objectives and high levels of generality.”) I think it’s also pretty interesting, as kind of a game, to try to imagine as concretely as possible what the training processes might look like for systems that can perform (or eliminate the need for) different tasks on the list.

As a sample, here are ten random tasks. (Some of these could easily be broken up into a lot of different sub-tasks or task variants, which might be automated independently.)
- Cancel letter or parcel post stamps by hand.
- Inquire into the cause, manner, and circumstances of human deaths and establish the identities of deceased persons.
- Teach patients to use home health care equipment.
- Write reports or articles for Web sites or newsletters related to environmental engineering issues.
- Supervise and participate in kitchen and dining area cleaning activities.
- Intervene as an advocate for clients or patients to resolve emergency problems in crisis situations.
- Mark or tag material with proper job number, piece marks, and other identifying marks as required.
- Calculate amount of debt and funds available to plan methods of payoff and to estimate time for debt liquidation.
- Weld metal parts together, using portable gas welding equipment.
- Provide assistance to patrons by performing duties such as opening doors and carrying bags.

bgarfinkel Mar 30, 2021, 1:59 PM
4 points
0 ∶ 0
in reply to: Eevee🔹’s comment on: Is Democracy a Fad?
Thanks for the comment!

I think endnotes 12 and 13, within my cave of endnotes, may partly address this concern.

I don’t think the prediction that the labor share will fall in the future depends on (a) the assumption that the amount of work to be done in the economy is constant, (b) the assumption that automation is currently reducing the demand for labor, or (c) the assumption that individual AI systems will tend to have highly general capabilities. I do agree that the first two assumptions are wrong. I also think the third assumption is very plausibly wrong, in line with some of the analysis in Reframing Superintelligence.

I think the prediction only depends on the assumption that, in the future, it will become unnecessary (and comparatively more expensive) to hire humans workers to produce goods and services. I find this assumption really plausible. The human brain is ultimately just a physical thing, so there’s no fundamental physical reason why (at least in aggregate) human-made machines couldn’t perform all of the same tasks that the brain is capable of.^[1] I also think it’s likely that engineers will eventually be able to make these kinds of machines; seemingly, the vast majority of AI researchers expect this to happen eventually. There are also, I think, very strong economic incentives to make and use these machines. If a business or state can produce goods and services more cheaply or effectively, by escaping the need to hire human workers, then it will typically want to do this. Any group that continues to pay for a lot of unnecessary human workers will be at a disadvantage.

This prediction is consistent with the observation that, historically, automation has tended to increase overall demand for labor. When one domain becomes highly automated, this tends to increase the demand for labor in complementary domains (inc. domains that did not previously exist) which are not highly automated. My understanding is that this dynamic explains why automation has mainly been driving wages up for the past couple hundred years. But the dynamic seems to break down once there are no longer any complementary automation-resistant domains.

For example: Suppose we live in a cheese-and-bread economy. People like eating cheese sandwiches, but don’t like eating cheese on its own. It then seems like completely automating cheese production (using machines that are more efficient than humans) will tend to increase demand for workers to staff bread factories. Automating both cheese and bread production, though, seems like it would pretty much eliminate the demand for labor. If either factory has an extra ten thousand dollars to spare, then (seemingly) they have no incentive to use it to pay a human worker a living wage, rather than spending it on capital that will increase output by a larger amount.^[2]

My thought process here is largely based on my memory of this paper and this paper. I’m not an economist, though, so I’m curious whether you or anyone else reading this comment thinks there’s a significant gap/mistake in this analysis.
1. ↩︎
  As a caveat, in some cases, people might intrinsically prefer for certain goods or services to be provided by humans. For example, people might naturally prefer to watch human athletes, talk to human therapists, listen to sermons by human religious leaders, etc. Human labor could also become a kind of status good in its own right; paying people do things could be sort of the future equivalent of buying rare paintings or NFTs. As a more direct and ominous analogy, my impression is that slaves used to be a really common status/luxury good for elites in lots of different parts of the world; maybe free human workers could play a similar social role in the future.
  
  This would prevent the labor share from going to zero, even if AI systems can (at the physical level) do everything that human workers can do. But I’d find it kind of surprising if this kind of work was enough to maintain very high labor force participation. It also seems like, if all remaining work was in this category, then we should still be worried about democracy. If military operations, law enforcement, the production of nearly all physical stuff, etc., were all highly and effectively automated, then that would still seem to undercut a lot of the hypothesized economic basis for democracy.
2. ↩︎
  I don’t think comparative advantage arguments ultimately help here. At the same time, though, I also don’t feel like I have a great grasp of how to apply them to capital-labor substitution.