Matthew_Barnett

Karma: 3,300

Matthew_Barnett 14 Dec 2017 4:50 UTC
5 points
0 ∶ 0
on: We Could Move $80 Million to Effective Charities, Pineapples Included
Just a thought: if you think that earning-to-give is a good strategy, then this is one of the best things you can do as an effective altruist. Just to put things in perspective here, if you donated $50,000 to an effective charity for 20 years, then you would be doing just about as much good as merely leaving a good comment in that thread. I hope that helps to internalize just what’s at stake here.

Just make sure that the Pineapple fund doesn’t generate some animosity towards EA. If it takes 100 good reasons to change someone’s mind, it only takes 1 really bad one to turn them away. The person doing the giveaway said that they are interested in the SENS foundation. This is pretty good evidence that they care about the long-term future. We might be able to do the most good if we focus our efforts on that cause area specifically.

Matthew_Barnett 1 Jan 2018 20:49 UTC
2 points
0 ∶ 0
on: Cosmic EA: How Cost Effective Is Informing ET?
There seems to be somewhat a consensus among effective altruists that the Rare Earth explanation is the most likely resolution to the Fermi Paradox. I tend to agree, but like you, I think that effective altruists generally underestimate the risk from aliens.

However, I would caution against a few assumptions that you made in the article. The first is the assumption that aliens would be anything like they show in the movies—rouge civilizations restricted to quadrants in the galaxy. As many have pointed out in the past, a civilization with artificial superintelligence would likely be able to colonize the entire galaxy within just a few million years, which means that if aliens with advanced artificial intelligence existed, we probably would have seen evidence of them existing already. Of course, maybe they’re hiding, but now you’re running up against Occam’s razor.

The second assumption is that we can affect the state of affairs of civilizations at our stage of development. Now, even given the generous assumption that we have the ability to share useful knowledge with aliens at our stage of development, it would be unlikely that we ever find aliens that are exactly at our development stage. A civilization just decades younger would be unavailable to contact without radio, and a civilization just centuries more advanced would probably have artificial intelligence already.

Matthew_Barnett 21 Feb 2018 7:36 UTC
11 points
1 ∶ 0
in reply to: Larks’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment

But it seems that it would be very bad if everyone took this advice literally.

Fortunately, not everyone does take this advice literally :).

This is very similar to the tragedy of the commons. If everyone acts out of their own self motivated interests, then everyone will be worse off. However, the situation as you described does not fully reflect reality because none of the groups you mentioned are actually trying to influence AI researchers at the moment. Therefore, MCE has a decisive advantage. Of course, this is always subject to change.

In contrast, preventing the extinction of humanity seems to occupy a privileged position

I find that it is often the case that people will dismiss any specific moral recommendation for AI except this one. Personally I don’t see a reason to think that there are certain universal principles of minimal alignment. You may argue that human extinction is something that almost everyone agrees is bad—but now the principle of minimal alignment has shifted to “have the AI prevent things that almost everyone agrees is bad” which is another privileged moral judgement that I see no intrinsic reason to hold.

In truth, I see no neutral assumptions to ground AI alignment theory in. I think this is made even more difficult because even relatively small differences in moral theory from the point of view of information theoretic descriptions of moral values can lead to drastically different outcomes. However, I do find hope in moral compromise.

Matthew_Barnett 21 Feb 2018 7:47 UTC
4 points
0 ∶ 0
in reply to: adamaero’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
Just because an event is theoretical doesn’t mean that it won’t occur. An asteroid hitting the Earth is theoretical, but something I think you might realize is quite real when it impacts.

Some say that superintelligence doesn’t have precedence, but I think that’s overlooking a key fact. The rise of homo sapiens has radically altered the world—and all signs point toward intelligence as the cause. We think at the moment that intelligence is just a matter of information processing, and therefore, there should be a way that it could be done by our own computers some day, if only we figured out the right algorithms to implement.

If we learn that superintelligence is impossible, that means our current most descriptive scientific theories are wrong, and we will have learned something new. That’s because that would indicate that humans are somehow cosmically special, or at least have hit the ceiling for general intelligence. On the flipside, if we create superintelligence, none of our current theories of how the world operates must be wrong.

That’s why it’s important to take seriously. Because the best evidence we have available tells us that it’s possible, not that it’s impossible.

Matthew_Barnett 21 Feb 2018 8:01 UTC
9 points
0 ∶ 0
on: Why I prioritize moral circle expansion over artificial intelligence alignment
A very interesting and engaging article indeed.

I agree that people often underestimate the value of strategic value spreading. Oftentimes, proposed moral models that AI agents will follow have some lingering narrowness to them, even when they attempt to apply the broadest of moral principles. For instance, in Chapter 14 of Superintelligence, Bostrom highlights his common good principle:

Superintelligence should be developed only for the benefit of all of humanity and in the service of widely shared ethical ideals.

Clearly, even something as broad as that can be controversial. Specifically, it doesn’t speak at all about any non-human interests except insofar as humans express widely held beliefs to protect them.

I think one thing to add is that AIA researchers who hold more traditional moral beliefs (as opposed to wide moral circles and transhumanist beliefs) are probably less likely to believe that moral value spreading is worth much. The reason for this is obvious: if everyone around you holds, more or less, the same values that you do, then why change anyone’s mind? This may explain why many people dismiss the activity you proposed.

Matthew_Barnett 22 Feb 2018 5:07 UTC
8 points
0 ∶ 0
in reply to: adamaero’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment
I don’t think anyone here is suggesting supporting random CS grads financially. Although, they might endorse something like that indirectly by funding AI alignment research, which tends to attract CS grads.

I agree that simply because an asteroid collision would be devastating, it does not follow that we should necessarily focus on that work in particular. However, there are variables which I think you might be overlooking.

The reason why people are concerned with AI alignment is not necessarily because of the scope of the issue, but also the urgency and tractability of the problem. The urgency of the problem comes from the idea that advanced AI will probably be developed this century. The tractability of the problem comes from the idea that there exists a set of goals that we could in theory put into an AI goals that are congruent with ours—you might want to read up on the Orthogonality Thesis.

Furthermore, it is dangerous to assume that we should judge the effectiveness of certain activities merely based on prior evidence or results. There are some activities which are just infeasible to give post hoc judgements about—and this issue is one of them. The inherent nature of the problem is that we will probably only get about one chance to develop superintelligence—because if we fail, then we will all probably die or otherwise be permanently unable to alter its goals.

To give you an analogy, few would agree that because climate change is an unprecedented threat, it therefore follows that we should wait until after the damage has been done to assess the best ways of mitigating it. Unfortunately for issues that have global scope, it doesn’t look like we get a redo if things start going badly.

If you want to learn more about the research, I recommend reading Superintelligence by Nick Bostrom. The vast majority of AI alignment researchers are not worried about malevolent AI despite your statement. I mean this is in the kindest way possible, but if you really want to be sure that you’re on the right side of a debate, it’s worth understanding the best arguments against your position, not the worst.

Matthew_Barnett 23 Feb 2018 5:56 UTC
3 points
0 ∶ 0
in reply to: adamaero’s comment on: Why I prioritize moral circle expansion over artificial intelligence alignment

Please, what AIA organizations? MIRI?

Yes, MIRI is one. FHI is another.

That being said, I wish you would’ve examined the actual claims I presented. I did not claim AI researchers are worried about a malevolent AI.

You did, however, say “The theoretical threat of a malevolent strong AI would be immense. But that does not mean one has cause or a valid reason to support CS grad students financially.” I assumed you meant that you believed someone was giving an argument along the lines of “since malevolent AI is possible, then we should support CS grads.” If that is not what you meant, then I don’t see the relevance of mentioning malevolent AI.

Since you also stated that you had an issue with me not being charitable, I would reciprocate likewise. I agree that we should be charitable to each other’s opinions.

Having truthful views is not about winning debate. It’s about making sure that you hold good beliefs for good reasons, end of story. I encourage you to imagine this conversation not as a way to convince me that I’m wrong—but more of a case study about what the current arguments are, and whether they are valid. In the end, you don’t get points for winning an argument. You get points for actually holding correct views.

Therefore, it’s good to make sure that your beliefs actually hold weight under scrutiny. Not in a, “you can’t find the flaw after 10 minutes of self-sabotaged thinking” sort of way, but in a very deep understanding sort of way.

It is donating my income, as an individual that I take offense. People can fund whatever they want: A new planetary wing at a museum, research in robotics, research in CS, research in CS philosophy.

I agree people can fund whatever they want. It’s important to make a distinction between normative questions and factual ones. It’s true that people can fund whatever project they like; however, it’s also true that some projects have a high value from an impersonal utilitarian perspective. It is this latter category that I care about, which is why I want to find projects with particular high value. I believe that existential risk mitigation and AI alignment is among these projects, although I fully admit that I may be mistaken.

Although, Earning to Give does not follow. Thinking about and discussing the risks of strong AI does make sense, and we both seem to agree it is important.

If you agree that thinking about something is valuable, why not also agree that funding that thing is valuable. It seems you think that the field should just get a certain threshold of funding that allows certain people to think about the problem just enough—but not too much. I don’t a reason to believe that the field of AI alignment has reached that critical threshold. On the contrary, I believe the field is far from it at the moment.

Following the money, there is not a clear answer on which CS grad students are receiving it. Low or zero transparency. MIRI or no? Am I missing some public information?

I suppose when you make a donation to MIRI, it’s true that you can’t be certain about how they spend that money (although I might be wrong about this, I haven’t actually donated to MIRI). Generally though, funding an organization is about whether you think that their mission is neglected, and whether you think that further money would make a marginal impact in their cause area. This is no different than any other charity that EA aligned people endorse.

Second, what do you define as advanced AI? Before, I said strong AI. Is that what you mean? Is there some sort of AI in between? I’m not aware.

It might be confusing that there are all these terms for AI. To taboo the words “advanced AI”, “strong AI”, “AGI” or others—what I am worried about is an information processing system that can achieve broad success in cognitive tasks in a way that rivals or surpasses humans. I hope that makes it clear.

This is crucially where I split with AI safety. The theory is an idea of a belief about the far future. To claim that we’re close to developing strong AI is unfounded to me.

I’m not quite clear what you mean here. If you mean we are worried about AI in the far future, fine. But then in the next sentence you say that we’re worried about being close to strong AI. How can we simultaneously believe both. If AI is near then I care about the near-term future. If AI is not near, then I care about the long-term future. I do not claim either, however. I think it is important consideration even if it’s a long way off.

Neural networks do not seem to be (from my light research).

This is what I’m referring to when I talk about how important it is to really, truly understand something before developing an informed opinion about it. If you admit that you have only done light research, how can you be confident that you are right. Doing a bit of research might give you an edge for debate purposes, but we are talking about the future of life on Earth here. We really need to know the answers to these questions.

Perhaps a large rogue solar flair or the Yellowstone supervolcano. Or perhaps even a time travel analogy would suffice ~ time travel safety research. There is no tractability/solvability.

Lumping all existential risks in a single category and then asserting that there’s no tractability is a simplified approach. First what we need is the probability of any given existential risk occurring. For instance, if scientists discovered that the Yellowstone supervolcano was probably about to erupt sometime in the next few centuries, I’d definitely agree we should do research in that area, and we should fund that research as well. In fact, some research is being done in that area and I’m happy that it’s being done.

A belief in an idea about the future is a poor reason for claiming some sort of tractability for funding.

I’d agree with you if it was an idea asserted without evidence or reason. But there’s a whole load of arguments about why it is a tractable field, and how we can do things now—yes right now—about making the future safer. Ignorance of these arguments does not mean they do not exist.

Remember, ask yourself first what is true. Then form your opinion. Do not go the other way.

Matthew_Barnett 28 Mar 2019 21:57 UTC
2 points
0 ∶ 0
in reply to: AlexTheZoologist’s comment on: Reducing Wild Animal Suffering Literature Library: Introductory Materials, Philosophical & Empirical Foundations
Are you arguing against any particular finding? What specifically, do you disagree with? Which one of these articles would be rebuked by the academic community as a whole, and why? Until you can answer these questions, I don’t really understand your critique.

Matthew_Barnett 31 Mar 2019 0:47 UTC
4 points
0 ∶ 0
on: Altruistic action is dispassionate
I understand that you aren’t saying that altruism is completely unemotional, but I still want to emphasize the role that emotion plays. I do not distinguish too sharply between things that I want for personal reasons, and things that I want for altruistic concerns. Personally, when I learned about utility functions, it was a watershed moment for my understanding of ethics.
If you describe an agent as having a utility function, it means that all of its preferences are commensurate. To put it another way, the agent might want to have a cup of coffee and also want world peace. Importantly, the two preferences are the same type—I don’t distinguish between moral wants and non-moral wants.
Therefore, when I say that I am altruistic, I am not saying that it is my duty to be so. If I were to put my biases aside and dispassionately calculate the action with the highest utility, it is because I truly believe that being dispassionate is the best way to get what I want. I would do the same for actions which concern my own life, and feelings.
Splitting our motivation into two pieces, one personal, and one moral, seems like a remnant of our evolutionary past. It seems to me that people naturally believe in social norms, moral standards, duty, virtues and these don’t always align with what they personally want. I seek to dissolve this whole dichotomy: there is simply a world that I want to be in, and I am trying to do whatever is necessary to make that world the real one.

Matthew_Barnett 31 Mar 2019 2:08 UTC
2 points
0 ∶ 0
in reply to: toonalfrink’s comment on: Altruistic action is dispassionate
Yeah, there are many possible ways to frame this. I like the idea of a coherent agent, but that might just be the part of me capable of putting verbal thoughts on a forum page. In any case, over time I’ve experienced a shift from viewing preferences as different types which compete, to viewing preferences as all existing together in one coherent thread. Of course, my introspection is not perfect, but this is how I feel when I look inward to find what I really want.
I do not claim that this is what other people feel. However, to the extent which I find the idea pleasing, I certainly would like if people shared my view.

Matthew_Barnett 16 Sep 2019 18:52 UTC
4 points
0 ∶ 0
on: Does improving animal rights now improve the far future?
If it’s referring to making current humans more humane by changing their terminal values, then it’s not clear to me how this would occur. My understanding is that animal rights activities tend to spend their time showing people how bad conditions are, and I see no mechanism by which this would change people’s terminal values.
Plenty of people now think that slavery is bad now that it has been abolished. It is intuitively clear to me that when people lose the ability to rationalize something, then they will tend to be more careful before endorsing it as good (especially if it causes a bunch of suffering). Right now, few people explicitly care about animal suffering (besides maybe dogs and cats) -- but in a world where factory farming is remembered as a great crime of the past, I expect our attitudes to shift.

Matthew_Barnett 19 Oct 2019 4:57 UTC
9 points
0 ∶ 0
on: What are the best arguments for an exclusively hedonistic view of value?
I don’t have a fully hedonistic view, but I’m sympathetic toward one. I prefer using different words than “hedonism”, since hedonism has a bad connotation. I like to say that I care primarily about conscious experiences, where conscious experience refers to the common sense referent of “what it’s like to be me” or similar (for intuition pumps, read how Chalmers defines consciousness).
To me it comes down to two related notions:
1. I don’t see how non-conscious facts could possibly ever imply a tragedy. It’s a tragedy if someone gets hurt, but in what sense is something ever a tragedy if no one’s actually experiencing the badness?
2. Likewise, how could something ever be good if no one experiences it. What fact could I learn that would make me leap with joy, assuming the fact had no bearing on whether someone had a positive life or experience?
In practice, arguments against pure hedonism come down to pointing out a few things that are left out in the naive hedonistic view. These include: a lack of diversity of experiences, a lack of concern for truth, a lack of a coherent “adventure” that exists beyond the feeling of adventure. Fair enough, but I feel like these all could be bought at less than 1% of the price of regular hedonism. In other words, I think that I can still reasonably maintain that 99% of value comes from conscious experience and still keep these things.

Matthew_Barnett 19 Oct 2019 22:38 UTC
4 points
0 ∶ 0
on: The evolutionary argument against cognitive enhancement research is weak
I find the argument fairly weak for a number of reasons. Iodine supplementation seems to have worked great
Iodine supplementation isn’t enhancement; it’s more like fixing a broken component. If a machine has a broken part, fixing it might dramatically raise the performance of the machine, but that doesn’t tell you how easy it would be to improve a machine with no broken parts.

Matthew_Barnett 3 Nov 2019 5:59 UTC
10 points
0 ∶ 0
in reply to: Ofer’s comment on: Does 80,000 Hours focus too much on AI risk?
Also, I think that at least some researchers are less likely to discuss their estimates publicly if they’re leaning towards shorter timelines and a discontinuous takeoff
Could you explain more about why you think people who hold those views are more likely to be silent?

Matthew_Barnett 3 Nov 2019 19:33 UTC
7 points
0 ∶ 0
in reply to: Ofer’s comment on: Does 80,000 Hours focus too much on AI risk?
Presumably, people tend to avoid saying terrifying things.
I’m a bit skeptical of this statement, although I admit it could be true for some people. If anything I tend to think that people have a bias for exaggerating risk rather than the opposite, although I don’t have anything concrete to say either way.

Matthew_Barnett 17 Nov 2019 4:10 UTC
3 points
0 ∶ 0
in reply to: Ben Pace’s comment on: I’m Buck Shlegeris, I do research and outreach at MIRI, AMA
Apologies, aren’t we already in crunch time?
Are your referring to this comment from Eliezer Yudkowsky,
This is crunch time. This is crunch time for the entire human species. This is the hour before the final exam, we are trying to get as much studying done as possible, and it may be that you can’t make yourself feel that, for a decade, or 30 years on end or however long this crunch time lasts.

Matthew_Barnett 17 Nov 2019 4:35 UTC
3 points
0 ∶ 0
in reply to: Ben Pace’s comment on: I’m Buck Shlegeris, I do research and outreach at MIRI, AMA
I see. I asked only because I was confused why you asked “before crunch time” rather than leaving that part out.

Matthew_Barnett 19 Nov 2019 17:45 UTC
12 points
0 ∶ 0
in reply to: MikeJohnson’s comment on: I’m Buck Shlegeris, I do research and outreach at MIRI, AMA
Mike, while I appreciate the empirical predictions of the symmetry theory of valence, I have a deeper problem with QRI philosophy, and it makes me skeptical even if the predictions come to bear.
In physics, there are two distinctions we can make about our theories:
- Disputes over what we predict will happen.
- Disputes over the interpretation of experimental results.
The classic Many Worlds vs. Copenhagen is a dispute of the second kind, at least until someone can create an experiment which distinguishes the two. Another example of the second type of dispute is special relativity vs. Lorentz ether theory.
Typically, philosophers of science and most people who follow Lesswrong philosophy, will say that the way to resolve disputes of the second kind is to find out which interpretation is simplest. That’s one reason why most people follow Einstein’s special relativity over the Lorentz ether theory.
However, simplicity of an interpretation is often hard to measure. It’s made more complicated for two reasons,
- First, there’s no formal way of measuring simplicity even in principle in a way that is language independent.
- Second, there are ontological disputes about what type of theories we are even allowing to be under consideration.
The first case is usually not a big deal because we mostly can agree on the right language to frame our theories. The second case, however, plays a deep role in why I consider QRI philosophy to be likely incorrect.
Take, for example, the old dispute over whether physics is discrete or continuous. If you apply standard Solomonoff induction, then you will axiomatically assign 0 probability to physics being continuous.
It is in this sense that QRI philosophy takes an ontological step that I consider unjustified. In particular, QRI assumes that there simply is an ontologically primitive consciousness-stuff that exists. That is, it takes it as elementary that qualia exist, and then reasons about them as if they are first class objects in our ontology.
I have already talked to you in person why I reject this line of reasoning. I think that an illusionist perspective is adequate to explain our beliefs in why we believe in consciousness, without making any reference to consciousness as an ontological primitive. Furthermore, my basic ontological assumption is that physical entities, such as electrons, have mathematical properties, but not mental properties.
The idea that electrons can have both mathematical and mental properties (ie. panpsychism) is something I consider to be little more than property dualism, and has the same known issues as every property dualist theory that I have been acquainted with.
I hope that clears some things up about why I disagree with QRI philosophy. However, I definitely wouldn’t describe you as practicing crank philosophy, as that term is both loaded, and empirically false. I know you care a lot about critical reflection, debate, and standard scientific virtues, which immediately makes you unable to be a “crank” in my opinion.

Matthew_Barnett 20 Nov 2019 1:02 UTC
3 points
0 ∶ 0
in reply to: MikeJohnson’s comment on: I’m Buck Shlegeris, I do research and outreach at MIRI, AMA
Thanks Matthew! I agree issues of epistemology and metaphysics get very sticky very quickly when speaking of consciousness.
Agreed :).
My basic approach is ‘never argue metaphysics when you can argue physics’
My main claim was that by only arguing physics, I will never agree upon your theory because your theory assumes the existence of elementary stuff that I don’t believe in. Therefore, I don’t understand how this really helps.
Would you be prepared say the same about many worlds vs consciousness causes collapse theories? (Let’s assume that we have no experimental data which distinguishes the two theories).
One way to frame this is that at various points in time, it was completely reasonable to be a skeptic about modeling things like lightning, static, magnetic lodestones, and such, mathematically.
The problem with the analogy to magnetism and electricity is that fails to match the pattern of my argument. In order to incorporate magnetism into our mathematical theory of physics, we merely added more mathematical parts. In this, I see a fundamental difference between the approach you take and the approach taken by physicists when they admit the existence of new forces, or particles.
In particular, your theory of consciousness does not just do the equivalent of add a new force, or mathematical law that governs matter, or re-orient the geometry of the universe. It also posits that there is a dualism in physical stuff: that is, that matter can be identified as having both mathematical and mental properties.
Even if your theory did result in new predictions, I fail to see why I can’t just leave out the mental interpretation of it, and keep the mathematical bits for myself.
To put it another way, if you are saying that symmetry can be shown to be the same as valence, then I feel I can always provide an alternative explanation that leaves out valence as a first-class object in our ontology. If you are merely saying that symmetry is definitionally equivalent to valence, then your theory is vacuous because I can just delete that interpretation from my mathematical theory and emerge with equivalent predictions about the world.
And in practice, I would probably do so, because symmetry is not the kind of thing I think about when I worry about suffering.
I think metaphysical arguments change distressingly few peoples’ minds. Experiments and especially technology changes peoples’ minds. So that’s what our limited optimization energy is pointed at right now.
I agree that if you had made predictions that classical neuroscientists all agreed would never occur, and then proved them all wrong, then that would be striking evidence that I had made an error somewhere in my argument. But as it stands, I’m not convinced by your analogy to magnetism, or your strict approach towards talking about predictions rather than metaphysics.
(I may one day reply to your critique of FRI, as I see it as similarly flawed. But it is simply too long to get into right now.)

Matthew_Barnett 20 Nov 2019 1:53 UTC
3 points
0 ∶ 0
in reply to: MikeJohnson’s comment on: I’m Buck Shlegeris, I do research and outreach at MIRI, AMA
But- you won’t be able to copy our generator by doing that, the thing that created those novel predictions
I would think this might be our crux (other than perhaps the existence of qualia themselves). I imagine any predictions you produce can be adequately captured in a mathematical framework that makes no reference to qualia as ontologically primitive. And if I had such a framework, then I would have access to the generator, full stop. Adding qualia doesn’t make the generator any better—it just adds unnecessary mental stuff that isn’t actually doing anything for the theory.
I am not super confident in anything I said here, although that’s mostly because I have an outside view that tells me consciousness is hard to get right. My inside view tells me that I am probably correct, because I just don’t see how positing mental stuff that’s separate from mathematical law can add anything whatsoever to a physical theory.
I’m happy to talk more about this some day, perhaps in person. :)