bgarfinkel

Karma: 2,776

bgarfinkel Jun 28, 2021, 8:12 PM
3 points
0 ∶ 0
in reply to: Rohin Shah’s comment on: Ben Garfinkel’s Shortform

I’m not sure if you think this is an interesting point to notice that’s useful for building a world-model, and/or a reason to be skeptical of technical alignment work. I’d agree with the former but disagree with the latter.

Mostly the former!

I think the point may have implications for how much we should prioritize alignment research, relative to other kinds of work, but this depends on what the previous version of someone’s world model was.

For example, if someone has assumed that solving the ‘alignment problem’ is close to sufficient to ensure that humanity has “control” of its future, then absorbing this point (if it’s correct) might cause them to update downward on the expected impact of technical alignment research. Research focused on coordination-related issues (e.g. cooperative AI stuff) might increase in value, at least in relative terms.

bgarfinkel Jun 23, 2021, 12:03 PM
12 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
It’s definitely entirely plausible that I’ve misunderstood your views.

My interpretation of the post was something like this:

There is a bag of things that people in the EA community tend to describe as “outside views.” Many of the things in this bag are over-rated or mis-used by members of the EA community, leading to bad beliefs.

One reason for this over-use or mis-use is that the the term “outside view” has developed an extremely positive connotation within the community. People are applauded for saying that they’re relying on “outside views” — “outside view” has become “an applause light” — and so will rely on items in the bag to an extent that is epistemically unjustified.

The things in the bag are also pretty different from each other — and not everyone who uses the term “outside view” agrees about exactly what belongs in the bag. This conflation/ambiguity can lead to miscommunication.

More importantly, when it comes to the usefulness of the different items in the bag, some have more evidential support than others. Using the term “outside view” to refer to everything in the bag might therefore lead people to overrated certain items that actually have weak evidential support.

To sum up, tabooing the term “outside view” might solve two problems. First, it might reduce miscommunication. Second, more importantly, it might cause people to stop overrating some of the reasoning processes that they currently characterize as involving “outside views.” The mechanisms by which tabooing the term can help to solve the second problem are: (a) it takes away an “applause light,” whose existence incentivizes excessive use of these reasoning processes, and (b) it allows people to more easily recognize that some of these reasoning processes don’t actually have much empirical support.

I’m curious if this feels roughly right, or feels pretty off.

Part of the reason I interpreted your post this way: The quote you kicked the post off suggested to me that your primary preoccupation was over-use or mis-use of the tools people called “outside views,” including more conventional reference-class forecasting. It seemed like the quote is giving an example of someone who’s refusing to engage in causal reasoning, evaluate object-level arguments, etc., based on the idea that outside views are just strictly dominant in the context of AI forecasting. It seemed like this would have been an issue even if the person was doing totally orthodox reference-class forecasting and there was no ambiguity about what they were doing.^[1]

I don’t think that you’re generally opposed to the items in the “outside view” bag or anything like that. I also don’t assume that you disagree with most of the points I listed in my last comment, for why I think intellectuals probably on average underrated the items in the bag. I just listed all of them because you asked for an explanation for my view, I suppose with some implication that you might disagree with it.

You’ve also given two rough definitions of the term, which seem quite different to me, and also quite fuzzy. (e.g. if by “reference class forecasting” you mean the stuff Tetlock’s studies are about, then it really shouldn’t include the anti-weirdness heuristic, but it seems like you are saying it does?)

I think it’s probably not worth digging deeper on the definitions I gave, since I definitely don’t think they’re close to perfect. But just a clarification here, on the anti-weirdness heuristic: I’m thinking of the reference class as “weird-sounding claims.”

Suppose someone approaches you not the street and hands you a flyer claiming: “The US government has figured out a way to use entangled particles to help treat cancer, but political elites are hoarding the particles.” You quickly form a belief that the flyer’s claim is almost certainly false, by thinking to yourself: “This is a really weird-sounding claim, and I figure that virtually all really weird-sounding claims that appear in random flyers are wrong.”

In this case, you’re not doing any deductive reasoning about the claim itself or relying on any causal models that directly bear on the claim. (Although you could.) For example, you’re not thinking to yourself: “Well, I know about quantum mechanics, and I know entangled particles couldn’t be useful for treating cancer for reason X.” Or: “I understand economic incentives, or understand social dynamics around secret-keeping, so I know it’s unlikely this information would be kept secret.” You’re just picking a reference class — weird-sounding claims made on random flyers — and justifying your belief that way.

I think it’s possible that Tetlock’s studies don’t bear very strongly on the usefulness of this reference class, since I imagine participants in his studies almost never used it. (“The claim ‘there will be a coup in Venezuela in the next five years’ sounds really weird to me, and most claims that sound weird to me aren’t true, so it’s probably not true!”) But I think the anti-weirdness heuristic does fit with the definitions I gave, as well as the definition you give that characterizes the term’s “original meaning.” I also do think that Tetlock’s studies remain at least somewhat relevant when judging the potential usefulness of the heuristic.
1. ↩︎
  I initially engaged on the miscommunication, point, though, since this is the concern that would mostly strongly make me want to taboo the term. I’d rather address the applause light problem, if it is a problem, but trying get people in the EA community stop applauding, and the evidence problem, if it is a problem, by trying to just directly make people in the EA community more aware of the limits of evidence.

bgarfinkel Jun 21, 2021, 11:26 AM
19 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
On the contrary; tabooing the term is more helpful, I think. I’ve tried to explain why in the post. I’m not against the things “outside view” has come to mean; I’m just against them being conflated with / associated with each other, which is what the term does. If my point was simply that the first Big List was overrated and the second Big List was underrated, I would have written a very different post!

My initial comment was focused on your point about conflation, because I think this point bears on the linguistic question more strongly than the other points do. I haven’t personally found conflation to be a large issue. (Recognizing, again, that our experiences may differ.) If I agreed with the point about conflation, though, then I would think it might be worth tabooing the term “outside view.”

By what definition of “outside view?”

By “taking an outside view on X” I basically mean “engaging in statistical or reference-class-based reasoning.” I think it might also be best defined negatively: “reasoning that doesn’t substantially involve logical deduction or causal models of the phenomenon in question. “^[1]

I think most of the examples in your list fit these definitions.

Epistemic deference is a kind of statistical/reference-class-based reasoning, for example, which doesn’t involve applying any sort of causal model of the phenomenon in question. The logic is “Ah, I should update downward on this claim, since experts in domain X disagree with it and I think that experts in domain X will typically be right.”

Same for anti-weirdness: The idea is that weird claims are typically wrong.

I’d say that trend extrapolation also fits: You’re not doing logical reasoning or relying on a causal model of the relevant phenomenon. You’re just extrapolating a trend forward, largely based on the assumption that long-running trends don’t typically end abruptly.

“Foxy aggregation,” admittedly, does seem like a different thing to me: It arguably fits the negative definition, depending on how you generate your weights, but doesn’t seem to fit statistical/reference-class one. It also feels like more of a meta-level thing. So I wouldn’t personally use the term “outside view” to talk about foxy aggregation. (I also don’t think I’ve personally heard people use the term “outside view” to talk about foxy aggegration, although I obviously believe you have.)

There is some evidence that in some circumstances people don’t take reference class forecasting seriously enough; that’s what the original term “outside view” meant. What evidence is there that the things on the Big List O’ Things People Describe as Outside View are systematically underrated by the average intellectual?

A condensed answer is: (a) I think most public intellectuals barely use any of the items on this list (with the exception of the anti-weirdness heuristic); (b) I think some of the things on this list are often useful; (c) I think that intellectuals and people more generally are very often bad at reasoning causally/logically about complex social phenomena; (d) I expect intellectuals to often have a bias against outside-view-style reasoning, since it often feels somewhat unsatisfying/unnatural and doesn’t allow them to display impressive-seeming domain-knowledge, interesting models of the world, or logical reasoning skills; and (e) I do still think Tetlock’s evidence is at least somewhat relevant to most things on the list, in part because I think they actually are somewhat related to each other, although questions of external validity obviously grow more serious the further you move from the precise sorts of questions asked in his tournaments and the precise styles of outside-view reasoning displayed by participants. ^[2]

There’s also, of course, a bit of symmetry here. One could also ask: “What evidence is there that the things on the Big List O’ Things People Describe as Outside View are systematically overrated by the average intellectual?” :)
1. ↩︎
  These definitions of course aren’t perfect, and other people sometimes use the term more broadly than I do, but, again, some amount of fuzziness seems OK to me. Most concepts have fuzzy boundaries and are hard to define precisely.
2. ↩︎
  On the Tetlock evidence: I think one thing his studies suggest, which I expect to generalize pretty well to many different contexts, is that people who are trying to make predictions about complex phenemona (especially complex social phenemona) often do very poorly when they don’t incorporate outside views into their reasoning processes. (You can correct me if this seems wrong, since you’ve thought about Tetlock’s work far more than I have.) So, on my understanding, Tetlock’s work suggests that outside-view-heavy reasoning processes would often substitute for reasoning processes that lead to poor predictions anyways. At least for most people, then, outside-view-heavy reasoning processes don’t actually need to be very reliable to constitute improvements—and they need to be pretty bad to, on average, lead to worse predictions.
  
  Another small comment here: I think Tetlock’s work also counts, in a somewhat broad way, against the “reference class tennis” objection to reference-class-based forecasting. On its face, the objection also applies to the use of reference classes in standard forecasting tournaments. There are always a ton of different reference classes someone could use to forecast any given political event. Forecasters need to rely on some sort of intuition, or some sort of fuzzy reasoning, to decide on which reference classes to take seriously; it’s a priori plausible that people would be just consistently very bad at this, given the number of degrees of freedom here and the absence of clear principles for making one’s selections. But this issue doesn’t actually seem to be that huge in the context of the sorts of questions Tetlock asked his participants. (You can again correct me if I’m wrong.) The degrees-of-freedom problem might be far larger in other contexts, but the fact that the issue is manageable in Tetlockian contexts presumably counts as at least a little bit of positive evidence.

bgarfinkel Jun 19, 2021, 11:54 AM
19 points
0 ∶ 0
in reply to: kokotajlod’s comment on: Taboo “Outside View”
I agree that people sometimes put too much weight on particular outside views—or do a poor job of integrating outside views with more inside-view-style reasoning. For example, in the quote/paraphrase you present at the top of your post, something has clearly gone wrong.^[1]

But I think the best intervention, in this case, is probably just to push the ideas “outside views are often given too much weight” or “heavily reliance on outside views shouldn’t be seen as praiseworthy” or “the correct way to integrate outside views with more inside-view reasoning is X.” Tabooing the term itself somehow feels a little roundabout to me, like a linguistic solution to a methodological disagreement.

I think you’re right that “outside view” now has a very positive connotation. If enough community members become convinced that this positive connotation is unearned, though, I think the connotation will probably naturally become less positive over time. For example, the number of upvotes on this post is a signal that people shouldn’t currently expect that much applause for using the term “outside view.”
1. ↩︎
  As a caveat, although I’m not sure how much this actually matters for the present discussion, I probably am significantly less concerned about the problem than you are. I’m pretty confident that the average intellectual doesn’t pay enough attention to “outside views”—and I think that, absent positive reinforcement from people in your community, it actually does take some degree of discipline to take outside views sufficiently seriously. I’m open to the idea that the average EA community member has over-corrected, here, but I’m not yet convinced of it. I think it’s also possible that, in a lot of cases, the natural substitute for bad outside-view-heavy reasoning is worse inside-view-heavy reasoning.

bgarfinkel Jun 18, 2021, 10:20 PM
24 points
0 ∶ 0
on: Taboo “Outside View”
When people use “outside view” or “inside view” without clarifying which of the things on the above lists they mean, I am left ignorant of what exactly they are doing and how well-justified it is. People say “On the outside view, X seems unlikely to me.” I then ask them what they mean, and sometimes it turns out they are using some reference class, complete with a dataset. (Example: Tom Davidson’s four reference classes for TAI). Other times it turns out they are just using the anti-weirdness heuristic. Good thing I asked for elaboration!

FWIW, as a contrary datapoint, I don’t think I’ve really encountered this problem much in conversation. In my own experience (which may be quite different from yours): when someone makes some reference to an “outside view,” they say something that indicates roughly what kind of “outside view” they’re using. For example, if someone is just extrapolating a trend forward, they’ll reference the trend. Or if someone is deferring to expert opinion, they’ll reference expert opinion. I also don’t think I’d find it too bothersome, in any case, to occasionally have to ask the person which outside view they have in mind.

So this concern about opacity wouldn’t be enough to make me, personally, want people to stop using the term “outside view.”

If there’s a really serious linguistic issue, here, I think it’s probably that people sometimes talk about “the outside view” as though there’s only a single relevant outside view. I think Michael Aird made a good comment on my recent democracy post, where he suggests that people should taboo the phrase “the outside view” and instead use the phrase “an outside view.” (I was guilty of using the phrase “the outside view” in that post — and, arguably, of leaning too hard on one particular way of defining a reference class.) I’d be pretty happy if people just dropped the “the,” but kept talking about “outside views.”^[1]
1. ↩︎
  It’s of course a little ambiguous what counts as an “outside view,” but in practice I don’t think this is too huge of an issue. In my experience, which again may be different from yours, “taking an outside view” still does typically refer to using some sort of reference-class-based reasoning. It’s just the case that there are lots of different reference classes that people use. (“I’m extrapolating this 20-year trend forward, for another five years, because if a trend has been stable for 20 years it’s typically stable for another five.” “I’m deferring to the experts in this survey, because experts typically have more accurate views than amateurs.” Etc.) I do feel like this style of reasoning is useful and meaningfully distinct from, for example, reasoning based on causal models, so I’m happy to have a term for it, even if the boundaries of the concept are somewhat fuzzy.

bgarfinkel May 18, 2021, 11:03 PM
10 points
0 ∶ 0
in reply to: Max_Daniel’s comment on: What are things everyone here should (maybe) read?
Fortunately, if I remember correctly, something like the distinction between the true criterion of rightness and the best practical decision procedure actually is a major theme in the Kagan book. (Although I think the distinction probably often is underemphasized.)

It is therefore kind of misleading to think of consequentialism vs. deontology vs. virtue ethics as alternative theories, which however is the way normative ethics is typically presented in the analytic tradition.

I agree there is something to this concern. But I still wouldn’t go so far as to say that it’s misleading to think of them as alternative theories. I do think they count as conceptually distinct (even if the boundaries are sometimes a bit muddy), and I think they do sometimes have different implications for how you should in fact make moral decisions.

Beyond the deontology/consequentialism debate, I think there are also relevant questions around demandingness (how strong are our moral obligations, if any?), on the nature of well-being (e.g. hedonistic vs. preference-based vs. objective list theories), on the set of things that count as morally relevant consequences (e.g. do things beyond well-being matter? should we care more about totals or averages?), and so on.

bgarfinkel May 18, 2021, 9:52 PM
22 points
0 ∶ 0
on: What are things everyone here should (maybe) read?
A slightly boring answer: I think most people should at least partly read something that overviews common theories and frameworks in normative ethics (and the arguments for and against them) and something that overviews core concepts and principles in economics (e.g. the idea of expected utility, the idea of an externality, supply/demand, the basics of economic growth, the basics of public choice).

In my view, normative ethics and economics together make up a really large portion of the intellectual foundation that EA is built on.

One good book that overviews normative ethics is Shelly Kagan’s Normative Ethics, although I haven’t read it since college (and I think it has only a tiny amount of coverage of population ethics and animal ethics). One thing I like about it is it focuses on laying out the space of possible ethical views in a sensible way, rather than tracing the history of the field. If I remember correctly, names like Aristotle, Kant, etc. never show up. It’s also written in a very conversational style.

One good introductory economics textbook is Tyler Cowen’s and Alex Tabarrok’s Modern Principles of Economics. I don’t know how it stacks up to other intro textbooks, since it’s the only one that I’ve read more than a little of, but it’s very readable, has very little math, and emphasizes key concepts and principles. Reading just the foundational chapters in an intro textbook, then the chapters whose topics sound important, can probably get most people a decent portion of the value of reading a full textbook.

bgarfinkel May 4, 2021, 12:18 PM
1 point
0 ∶ 0
in reply to: Linch’s comment on: Ben Garfinkel’s Shortform
That’s a good example.

I do agree that quasi-random variation in culture can be really important. And I agree that this variation is sometimes pretty sticky (e.g. Europe being predominantly Christian and the Middle East being predominantly Muslim for more than a thousand years). I wouldn’t say that this kind of variation is a “rounding error.”

Over sufficiently long timespans, though, I think that technological/economic change has been more significant.

As an attempt to operationalize this claim: The average human society in 1000AD was obviously very different than the average human society in 10,000BC. I think that the difference would have been less than half as large (at least in intuitive terms) if there hadn’t been technological/economic change.

I think that the pool of available technology creates biases in the sorts of societies that emerge and stick around. For large enough amounts of technological change, and long enough timespans (long enough for selection pressures to really matter), I think that shifts in these technological biases will explain a large portion of the shifts we see in the traits of the average society.^[1]
1. ↩︎
  If selection pressures become a lot weaker in the future, though, then random drift might become more important in relative terms.

bgarfinkel May 3, 2021, 2:01 PM
2 points
0 ∶ 0
in reply to: abergal’s comment on: Ben Garfinkel’s Shortform
FWIW, I wouldn’t say I agree with the main thesis of that post.

However, while I expect machines that outcompete humans for jobs, I don’t see how that greatly increases the problem of value drift. Human cultural plasticity already ensures that humans are capable of expressing a very wide range of values. I see no obviously limits there. Genetic engineering will allow more changes to humans. Ems inherit human plasticity, and may add even more via direct brain modifications.

In principle, non-em-based artificial intelligence is capable of expressing the entire space of possible values. But in practice, in the shorter run, such AIs will take on social roles near humans, and roles that humans once occupied....

I don’t see why people concerned with value drift should be especially focused on AI. Yes, AI may accompany faster change, and faster change can make value drift worse for people with intermediate discount rates. (Though it seems to me that altruistic discount rates should scale with actual rates of change, not with arbitrary external clocks.)

I definitely think that human biology creates at least very strong biases toward certain values (if not hard constraints) and that AI system would not need to have these same biases. If you’re worried about future agents having super different and bad values, then AI is a natural focal point for your worry.

A couple other possible clarifications about my views here:
- I think that the outcome of the AI Revolution could be much worse, relative to our current values, than the Neolithic Revolution was relative to the values of our hunter-gatherer ancestors. But I think the question “Will the outcome be worse?” is distinct from the question “Will we have less freedom to choose the outcome?”
- I’m personally not so focused on value drift as a driver of long-run social change. For example, the changes associated with the Neolithic Revolution weren’t really driven by people becoming less egalitarian, more pro-slavery, more inclined to hold certain religious beliefs, more ideologically attached to sedentism/farming, more happy to accept risks from disease, etc. There were value changes, but, to some significant degree, they seem to have been downstream of technological/economic change.

bgarfinkel May 3, 2021, 11:44 AM
2 points
0 ∶ 0
in reply to: abergal’s comment on: Ben Garfinkel’s Shortform

Do you have the intuition that absent further technological development, human values would drift arbitrarily far?

Certainly not arbitrarily far. I also think that technological development (esp. the emergence of agriculture and modern industry) has played a much larger role in changing the world over time than random value drift has.

[E]ven non-extinction AI is enabling a new set of possibilities that modern-day humans would endorse much less than the decisions of future humans otherwise.

I definitely think that’s true. But I also think that was true of agriculture, relative to the values of hunter-gatherer societies.

To be clear, I’m not downplaying the likelihood or potential importance of any of the three crisper concerns I listed. For example, I think that AI progress could conceivably lead to a future that is super alienating and bad.

I’m just (a) somewhat pedantically arguing that we shouldn’t frame the concerns as being about a “loss of control over the future” and (b) suggesting that you can rationally have all these same concerns even if you come to believe that technical alignment issues aren’t actually a big deal.

bgarfinkel May 2, 2021, 3:30 PM
35 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
A thought on how we describe existential risks from misaligned AI:

Sometimes discussions focus on a fairly specific version of AI risk, which involves humanity being quickly wiped out. Increasingly, though, the emphasis seems to be on the more abstract idea of “humanity losing control of its future.” I think it might be worthwhile to unpack this latter idea a bit more.

There’s already a fairly strong sense in which humanity has never controlled its own future. For example, looking back ten thousand years, no one decided that the sedentary agriculture would increasingly supplant hunting and gathering, that increasingly complex states would arise, that slavery would become common, that disease would take off, that social hierarchies and gender divisions would become stricter, etc. The transition to the modern world, and everything that came with this transition, also doesn’t seem to have been meaningfully chosen (or even really understood by anyone). The most serious effort to describe a possible future in detail — Hanson’s Age of Em — also describes a future with loads of features that most present-day people would not endorse.

As long as there are still strong competitive pressures or substantial random drift, it seems to me, no generation ever really gets to choose the future.^[1] It’s actually sort of ambiguous, then, what it means to worry about “losing control of our future.”

Here are a few alternative versions of the concern that feel a bit crisper to me:
1. If we ‘mess up on AI,’ then even the most powerful individual humans will have unusually little influence over their own lives or the world around them.^[2]
1. If we ‘mess up on AI,’ then future people may be unusually dissatisfied about the world they live in. In other words, people’s preferences will be unfilled to an unusually large degree.
2. Humanity may have a rare opportunity to take control of its own future, by achieving strong coordination and then locking various things in. But if we ‘mess up on AI,’ then we’ll miss out on this opportunity.^[3]
Something that’s a bit interesting about these alternative versions of the concern, though, is that they’re not inherently linked to AI alignment issues. Even if AI systems behave roughly as their users intend, I believe each of these outcomes is still conceivable. For example, if there’s a missed opportunity to achieve strong coordination around AI, the story might look like the failure of the Baruch Plan for international control of nuclear weapons: that failure had much more to do with politics than it had to do with the way engineers designed the technology in question.

In general, if we move beyond discussing very sharp alignment-related catastrophes (e.g. humanity being quickly wiped out), then I think concerns about misaligned AI start to bleed into broader AI governance concerns. It starts to become more ambiguous whether technical alignment issues are actually central or necessary to the disaster stories people tell.
1. ↩︎
  Although, admittedly, notable individuals or groups (e.g. early Christians) do sometimes have a fairly lasting and important influence.
2. ↩︎
  As an analogy, in the world of The Matrix, people may not actually have much less control over the long-run future than hunter-gatherers did twenty thousand years ago. But they certainly have much less control over their own lives.
3. ↩︎
  Notably, this is only a bad thing if we expect the relevant generation of humans to choose a better future than would be arrived at by default.

bgarfinkel Apr 30, 2021, 9:25 PM
2 points
0 ∶ 0
in reply to: JP Addison🔸’s comment on: Ben Garfinkel’s Shortform
Good point!

That consideration—and the more basic consideration that more junior people often just know less—definitely pushes in the opposite direction. If you wanted to try some version of seniority-weighted epistemic deference, my guess is that the most reliable cohort would have studied a given topic for at least a few years but less than a couple decades.

bgarfinkel Apr 30, 2021, 12:18 PM
29 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
A thought on epistemic deference:

The longer you hold a view, and the more publicly you hold a view, the more calcified it typically becomes. Changing your mind becomes more aversive and potentially costly, you have more tools at your disposal to mount a lawyerly defense, and you find it harder to adopt frameworks/perspectives other than your favored one (the grooves become firmly imprinted into your brain). At least, this is the way it seems and personally feels to me.^[1]

For this reason, the observation “someone I respect publicly argued for X many years ago and still believes X” typically only provides a bit more evidence than the observation “someone I respect argued for X many years ago.” For example, even though I greatly respect Daron Acemoglu, I think the observation “Daron Acemoglu still believes that political institutions are the central determinant of economic growth rates” only gives me a bit more evidence than the observation “15 years ago Daron Acemoglu publicly argued that institutions are the central determinant of economic growth rates.”

A corollary: If there’s an academic field that contains a long-standing debate, and you’d like to defer to experts in this field, you may want to give disproportionate weight to the opinions of junior academics. They’re less likely to have responded to recent evidence and arguments in an epistemically inflexible way.
1. ↩︎
  Of course, there are exceptions. The final chapter of Scout Mindset includes a moving example of a professor publicly abandoning a view he had championed for fifteen years, after a visiting academic presented persuasive new evidence. The reason these kinds of stories are moving, though, is that they describe truly exceptional behavior.

bgarfinkel Apr 19, 2021, 10:08 AM
8 points
0 ∶ 0
in reply to: Stefan_Schubert’s comment on: Ben Garfinkel’s Shortform
I’d actually say this is a variety of qualitative research. At least in the main academic areas I follow, though, it seems a lot more common to read and write up small numbers of detailed case studies (often selected for being especially interesting) than to read and write up large numbers of shallow case studies (selected close to randomly).

This seems to be true in international relations, for example. In a class on interstate war, it’s plausible people would be assigned a long analysis of the outbreak WW1, but very unlikely they’d be assigned short descriptions of the outbreaks of twenty random wars. (Quite possible there’s a lot of variation between fields, though.)

bgarfinkel Apr 18, 2021, 11:19 PM
36 points
0 ∶ 0
in reply to: bgarfinkel’s comment on: Ben Garfinkel’s Shortform
In general, I think “read short descriptions of randomly sampled cases” might be an underrated way to learn about the world and notice issues with your assumptions/models.

A couple other examples:

I’ve been trying to develop a better understanding of various aspects of interstate conflict. The Correlates of War militarized interstate disputes (MIDs) dataset is, I think, somewhat useful for this. The project files include short descriptions of (supposedly) every case between 1993 and 2014 in which one state “threatened, displayed, or used force against another.” Here, for example, is the set of descriptions for 2011-2014. I’m not sure I’ve had any huge/concrete take-aways, but I think reading the cases: (a) made me aware of some international tensions I was oblivious to; (b) gave me a slightly better understanding of dynamics around ‘micro-aggressions’ (e.g. flying over someone’s airspace); and (c) helped me more strongly internalize the low base rate for crises boiling over into war (since I disproportionately read about historical disputes that turned into something larger).

Last year, I also spent a bit of time trying to improve my understanding of police killings in the US. I found this book unusually useful. It includes short descriptions of every single incident in which an unarmed person was killed by a police officer in 2015. I feel like reading a portion of it helped me to quickly notice and internalize different aspects of the problem (e.g. the fact that something like a third of the deaths are caused by tasers; the large role of untreated mental illness as a risk factor; the fact that nearly all fatal interactions are triggered by 911 calls, rather than stops; the fact that officers are trained to interact importantly differently with people they believe are on PCP; etc.). l assume I could have learned all the same things by just reading papers — but I think the case sampling approach was probably faster and better for retention.

I think it’s possible there might be value in creating “random case descriptions” collections for a broader range of phenomena. Academia really doesn’t emphasize these kinds of collections as tools for either research or teaching.

EDIT: Another good example of this approach to learning is Rob Besinger’s recent post “thirty-three randomly selected bioethics papers.”

bgarfinkel Apr 9, 2021, 7:39 PM
18 points
0 ∶ 0
on: Ben Garfinkel’s Shortform
The O*NET database includes a list of about 20,000 different tasks that American workers currently need to perform as part of their jobs. I’ve found it pretty interesting to scroll through the list, sorted in random order, to get a sense of the different bits of work that add up to the US economy. I think anyone who thinks a lot about AI-driven automation might find it useful to spend five minutes scrolling around: it’s a way of jumping yourself down to a lower level of abstraction. I think the list is also a little bit mesmerizing, in its own right.

One update I’ve made is that I’m now more confident that more than half of present-day occupational tasks could be automated using fairly narrow, non-agential, and boring-looking AI systems. (Most of them don’t scream “this task requires AI systems with long-run objectives and high levels of generality.”) I think it’s also pretty interesting, as kind of a game, to try to imagine as concretely as possible what the training processes might look like for systems that can perform (or eliminate the need for) different tasks on the list.

As a sample, here are ten random tasks. (Some of these could easily be broken up into a lot of different sub-tasks or task variants, which might be automated independently.)
- Cancel letter or parcel post stamps by hand.
- Inquire into the cause, manner, and circumstances of human deaths and establish the identities of deceased persons.
- Teach patients to use home health care equipment.
- Write reports or articles for Web sites or newsletters related to environmental engineering issues.
- Supervise and participate in kitchen and dining area cleaning activities.
- Intervene as an advocate for clients or patients to resolve emergency problems in crisis situations.
- Mark or tag material with proper job number, piece marks, and other identifying marks as required.
- Calculate amount of debt and funds available to plan methods of payoff and to estimate time for debt liquidation.
- Weld metal parts together, using portable gas welding equipment.
- Provide assistance to patrons by performing duties such as opening doors and carrying bags.

bgarfinkel Mar 30, 2021, 1:59 PM
4 points
0 ∶ 0
in reply to: Eevee🔹’s comment on: Is Democracy a Fad?
Thanks for the comment!

I think endnotes 12 and 13, within my cave of endnotes, may partly address this concern.

I don’t think the prediction that the labor share will fall in the future depends on (a) the assumption that the amount of work to be done in the economy is constant, (b) the assumption that automation is currently reducing the demand for labor, or (c) the assumption that individual AI systems will tend to have highly general capabilities. I do agree that the first two assumptions are wrong. I also think the third assumption is very plausibly wrong, in line with some of the analysis in Reframing Superintelligence.

I think the prediction only depends on the assumption that, in the future, it will become unnecessary (and comparatively more expensive) to hire humans workers to produce goods and services. I find this assumption really plausible. The human brain is ultimately just a physical thing, so there’s no fundamental physical reason why (at least in aggregate) human-made machines couldn’t perform all of the same tasks that the brain is capable of.^[1] I also think it’s likely that engineers will eventually be able to make these kinds of machines; seemingly, the vast majority of AI researchers expect this to happen eventually. There are also, I think, very strong economic incentives to make and use these machines. If a business or state can produce goods and services more cheaply or effectively, by escaping the need to hire human workers, then it will typically want to do this. Any group that continues to pay for a lot of unnecessary human workers will be at a disadvantage.

This prediction is consistent with the observation that, historically, automation has tended to increase overall demand for labor. When one domain becomes highly automated, this tends to increase the demand for labor in complementary domains (inc. domains that did not previously exist) which are not highly automated. My understanding is that this dynamic explains why automation has mainly been driving wages up for the past couple hundred years. But the dynamic seems to break down once there are no longer any complementary automation-resistant domains.

For example: Suppose we live in a cheese-and-bread economy. People like eating cheese sandwiches, but don’t like eating cheese on its own. It then seems like completely automating cheese production (using machines that are more efficient than humans) will tend to increase demand for workers to staff bread factories. Automating both cheese and bread production, though, seems like it would pretty much eliminate the demand for labor. If either factory has an extra ten thousand dollars to spare, then (seemingly) they have no incentive to use it to pay a human worker a living wage, rather than spending it on capital that will increase output by a larger amount.^[2]

My thought process here is largely based on my memory of this paper and this paper. I’m not an economist, though, so I’m curious whether you or anyone else reading this comment thinks there’s a significant gap/mistake in this analysis.
1. ↩︎
  As a caveat, in some cases, people might intrinsically prefer for certain goods or services to be provided by humans. For example, people might naturally prefer to watch human athletes, talk to human therapists, listen to sermons by human religious leaders, etc. Human labor could also become a kind of status good in its own right; paying people do things could be sort of the future equivalent of buying rare paintings or NFTs. As a more direct and ominous analogy, my impression is that slaves used to be a really common status/luxury good for elites in lots of different parts of the world; maybe free human workers could play a similar social role in the future.
  
  This would prevent the labor share from going to zero, even if AI systems can (at the physical level) do everything that human workers can do. But I’d find it kind of surprising if this kind of work was enough to maintain very high labor force participation. It also seems like, if all remaining work was in this category, then we should still be worried about democracy. If military operations, law enforcement, the production of nearly all physical stuff, etc., were all highly and effectively automated, then that would still seem to undercut a lot of the hypothesized economic basis for democracy.
2. ↩︎
  I don’t think comparative advantage arguments ultimately help here. At the same time, though, I also don’t feel like I have a great grasp of how to apply them to capital-labor substitution.

bgarfinkel Mar 30, 2021, 12:21 PM
2 points
0 ∶ 0
in reply to: Nathan Calvin’s comment on: Is Democracy a Fad?
Thanks for sharing this, Nathan! Very interesting graph (and a metric I haven’t ever thought to consider.)

I’m curious if you have any views on what we should take away from trends in “the portion of output produced by democracies” vs. “the portion of people living under democracy” vs. “the portion of states that are democratic.”

Am I right to think that “portion of output produced by democracies” is most useful as a measure of the global power/influence of democracies? If so, that does seem like an interesting trend to track. I could also imagine it being interesting to look at secondary metrics of national power, if you haven’t already. For example, I think some IR scholars argue for the use of GDP multiplied by GDP-per-capita, based on the intuition that poor-but-highly-populated countries (e.g. Indonesia) seem to have less global power than their GDPs would suggest. You’re also probably already familiar with this sort of unprincipled metric of “national material capabilities” that international relations people sometimes use. Although my guess is that the trends would probably look pretty similar.

It seems like “portion of output produced by democracies” also functions as a combined metric of the prevalence of democracy, the strength of the development/democracy correlation, and the weakness of the (I think slightly negative?) population/democracy correlation. I suppose it’s a bad sign for democracy if any of these components decrease.

[[Edit: One more thought. If you haven’t already done this, it might also be interesting to look at trends in Polity-score-weighted GDP as a more continuous measure of the financial power of democracy. I think the trend would probably look about the same, since China’s polity score has been pretty stable over time, but there’s some chance it’d be interestingly different. I might also just do myself, out of curiosity.]]

bgarfinkel Mar 19, 2021, 10:05 AM
7 points
0 ∶ 0
in reply to: MichaelA🔸’s comment on: Is Democracy a Fad?

So that makes it sound like we might want to aim for good post-human/transhuman scenarios (if aiming for the good versions specifically is relatively tractable), or for good scenarios in which something non-human is very much in control (like developing a friendly agential AI).

I’m not sure if that follows. I mainly think that the meaning of the question “Will the future be democratic?” becomes much less clear when applied to fully/radically post-human futures. But I’m not sure if I see a natural reason to think that the futures would be ‘politically better’ than futures that are more recognizably human. So, at least at the moment, I’m not inclined to treat this as a major reason to push for a more or less post-human future.

That sounds to me like a 4-in-5 chance of something that might probably itself be an existential catastrophe (global authoritarianism that lasts indefinitely long), or might substantially increase the chances of some other existential catastrophe (e.g., because it’s harder to have a long reflection and so bad values get locked in).… But maybe you don’t see [this possibility] as necessarily that concerning? E.g., maybe you think that something like mild or genuinely enlightened and benevolent authoritarianism accounts for a substantial part of the likelihood of authoritarianism?

On the implications of my prediction for future people:

I definitely think of my prediction as, at least, bad news for future people. I’m a little unsure exactly how bad the news is, though.

Democratic governments are currently, on average, much better for the people who live under them. It’s not always possible to be totally sure of causation, but massacres, famines, serious suppressions of liberties, etc., have clearly been much more common under dictatorial governments than democratic governments. There are also pretty basic reasons to expect democracies to typically better for the people under them: there’s a stronger link between government decisions and people’s preferences. I expect this logic to hold, even if a lot of the specific ways in which dictatorships are on average worse than democracies (like higher famine risk) become less relevant in the future.

At the same time, I’m not sure we should be imagining a dystopia. Most people alive today live under dictatorial governments, and, for most of these people, daily life doesn’t feel like a boot on the face. The average person in Hanoi, for example, doesn’t think of themselves as living in the midst of catastrophe. Growing prosperity and some forms of technological progress are also reasons to expect quality of life to go up over time, even if the political situation deteriorates.

So I just want to clarify that, even though I’m predicting a counterfactually worse outcome, I’m not necessarily predicting a dystopia for most people, or a scenario in which most people’s lives are net negative. A dystopian future is conceivable, but doesn’t necessarily follow from a lack of democracy.

On the implications of my prediction for “value lock-in,” more broadly:

I think the main benefit of democracy, in this case, is that we should probably expect a wider range of values to be taken into account when important decisions with long-lasting consequences are made. Inclusiveness and pluralism of course doesn’t always imply morally better outcomes. But moral uncertainty considerations probably push in the direction of greater inclusivity/pluralism being good, in expectation. From some perspectives, it’s also inherently morally valuable for important decisions to be made in inclusive/pluralistic ways. Finally, I expect the average dictator to have worse values than the average non-dictator.

I actually haven’t thought very hard about the implications of dictatorship and democracy for value lock-in, though. I think I also probably have a bit of a reflexive bias toward democracy here.

bgarfinkel Mar 17, 2021, 3:18 PM
3 points
0 ∶ 0
in reply to: MichaelA🔸’s comment on: Is Democracy a Fad?

Are you seeing this prediction as including scenarios in which TAI has been developed by then, but things are basically going well, at least one million beings roughly like humans still exist, and the TAI is either agential and well-aligned with humanity and deferring to our wishes[1] or CAIS-like / tool-like?

Yep! I’m including these scenarios in the prediction.

I suppose I’m conditioning on either:

(a) AI has already been truly transformative, but people are still around and still meaningfully responsible for some important political decisions.*

(b) AI hasn’t yet been truly transformative , but people haven’t gone extinct

I actually haven’t thought enough about the relative probability of these two cases or my actual conditional probabilities for each of them. So my “4-in-5” prediction shouldn’t be taken as very rigorously thought through. I think the outside view is relevant to both cases, but the automation argument is only very relevant to the first case.

*I agree with your analogy here: People might be “meaningfully responsible” in the same way that US citizens are “meaningfully responsible” for US government actions, even though they only provide very occasional and simple inputs.

A related uncertainty I have is what you mean by “individual people still at least sort of exist” in that quote. E.g., would you include whole brain emulations with a fairly similar mind design to current humans?

I’m a little torn here. I’ve gone back and forth on this point, but haven’t really settled on how much including emulations should or should not influence the prediction. (Another sign that my “4-in-5” shouldn’t be taken too seriously.)

If whole brains emulations have largely replaced regular biological people, and mostly aren’t doing work (because other AI systems can do better jobs for most relevant cognitive tasks), then the automation argument still applies. But we should also assume, if we’re talking about emulations, that there have been an incredible number of other changes, some of which might be much more relevant than the destruction of the value of labor. For example, surely the ability to make copies of an emulation has implications for the nature of voting.

So, although I still feel that automation pushes in the direction of dictatorship, in the emulation case, I do feel a bit silly making mechanistic or “inside view” arguments given how foreign this possible future is to us. I also think the outside view continues to be relevant. At the same time, though, there might be a somewhat stronger case for just throwing up our hands and beginning from a non-informative ⁵⁰⁄₅₀ prior instead of trying to think too hard about base rates.