SoerenMind

Karma: 595

SoerenMind Sep 9, 2020, 10:20 AM
2 points
0 ∶ 0
in reply to: Sanjay’s comment on: We’re (surprisingly) more positive about tackling bio risks: outcomes of a survey
Makes sense. I guess then the question is if the work of everyone except the x-risk focused NGOs helps reduce r x-risk much. I tend to think yes since much of pandemic preparedness also addresses the worst case scenarios. But that seems to be an open question.

SoerenMind Aug 27, 2020, 6:45 PM
8 points
0 ∶ 0
on: We’re (surprisingly) more positive about tackling bio risks: outcomes of a survey
Thanks, great analysis! Just registering that I still expect bio risk will be less neglected than in the past. The major consideration for me is institutional funding, due to its scale. Like you say:
We believe that an issue of the magnitude of COVID-19 will likely not be forgotten soon, and that funding for pandemic preparedness will likely be safe for much longer than in the aftermath of previous pandemics. In particular it may persist long enough to become institutionalised and therefore harder to cut.
Aside from future institutional funding, we also have to take the into account the current funding and new experience because they contribute to our cumulative knowledge and preparedness.

SoerenMind Jul 31, 2020, 4:37 PM
6 points
0 ∶ 0
on: The academic contribution to AI safety seems large
Important question, and nicely researched!
A caveat is that some essential subareas of safety may be neglected. This is not a problem when subareas substitute each other: e.g. debate substitutes for amplification so it’s okay if one of them is neglected. But there’s a problem when subareas complement each other: e.g. alignment complements robustness so we probably need to solve both. See also When causes multiply.
It’s ok when a subarea is neglected as long as there’s a substitute for it. But so far it seems that some areas are necessary components of AI safety (perhaps both inner and outer alignment are).

SoerenMind May 9, 2020, 7:08 PM
16 points
0 ∶ 0
on: How bad is coronavirus really?
This was also discussed on LessWrong:
https://www.lesswrong.com/posts/J8JvZxkABWwdXSH9d/is-the-coronavirus-the-most-important-thing-to-be-focusing

SoerenMind May 8, 2020, 7:06 PM
12 points
0 ∶ 0
on: Physical theories of consciousness reduce to panpsychism
Kudos btw for writing this. Consciousness is a topic where it can be really hard to make progress and I worry that people aren’t posting enough about it for fear of saying something wrong.

SoerenMind May 8, 2020, 6:59 PM
3 points
0 ∶ 0
in reply to: MichaelStJules’s comment on: Physical theories of consciousness reduce to panpsychism
I agree that physical theories of consciousness are pan psychist if they say that every recurrent net is conscious (or that everything that can be described as GWT is conscious). The main caveats for me are:
Does anyone really claim that every recurrent net is conscious? It seems so implausible. E.g. if I initialize my net with random parameters, it just computes garbage. Or if I have a net with 1 parameter it seems too simple. Or if the number of iterations is 1 (as you say), it’s just a trivial case of recurrence. Or if it doesn’t do any interesting task, such as prediction...
(Also, most recurrent nets in nature would be gerrymandered. I could imagine there are enough that aren’t though, such as potentially your examples).
NB, recurrence doesn’t necessarily imply recurrent processing (the term from recurrent processing theory). The ‘processing’ part could hide a bunch of complexity?

SoerenMind May 7, 2020, 11:43 AM
5 points
0 ∶ 0
on: Physical theories of consciousness reduce to panpsychism
I like your description of how complex physical processes like global attention / GWT to simple ones like feedforward nets.
But I don’t see how this implies that e.g. GWT reduces to panpsychism. E.g. to describe a recurrent net as a feedforward net you need a ridiculous number of parameters (with the same parameter values in each layer). So that doesn’t imply that the universe is full of recurrent nets (even if it were full of feedforward nets which it isn’t).
To draw a caricature of your argument as I understand it: It turns out computers can be reduced to logic gates. Therefore, everything is a computer.
Or another caricature: Recurrent nets are a special case of {any arrangement of atoms}. Therefore any arrangement of atoms is an RNN.
edit: missing word

SoerenMind Mar 27, 2020, 1:18 PM
4 points
0 ∶ 0
on: The Precipice is out today in US & Canada! Audiobook now available
Your link goes to the UK version. Here’s US:
https://smile.amazon.com/Precipice-Existential-Risk-Future-Humanity/dp/0316484911/ref=tmm_hrd_swatch_0?_encoding=UTF8&qid=&sr=

SoerenMind Mar 11, 2020, 9:47 AM
9 points
0 ∶ 0
in reply to: Lukas Finnveden’s comment on: EAF/FRI are now the Center on Long-Term Risk (CLR)
Just as a data point, “eye clear” took off for the conference ICLR so people seem to find the “clear” pronunciation intuitive.

SoerenMind Dec 29, 2019, 3:25 AM
13 points
0 ∶ 0
on: More info on EA Global admissions
Thanks for writing this. I don’t have a solution but I’m just registering that I would expect plenty of rejected applicants to feel alienated from the EA community despite this post.

SoerenMind Dec 13, 2019, 8:33 PM
7 points
0 ∶ 0
in reply to: Liam_Donovan’s comment on: Are we living at the most influential time in history?
It’s just an informal way to say that we’re probably typical observers. It’s named after Copernicus because he found that the Earth isn’t as special as people thought.

SoerenMind Oct 19, 2019, 1:17 PM
1 point
0 ∶ 0
on: Technical AGI safety research outside AI
Very nice list!

SoerenMind Oct 17, 2019, 10:20 AM
0 points
0 ∶ 0
on: Introducing Foretold.io: A New Open-Source Prediction Registry
Great work!!!

SoerenMind Oct 17, 2019, 10:19 AM
3 points
0 ∶ 0
on: The evolutionary argument against cognitive enhancement research is weak
Hmmm isn’t the argument still pretty broadly applicable and useful despite the exceptions?

SoerenMind Sep 29, 2019, 3:19 PM
8 points
0 ∶ 0
on: Is there a good place to find the “what we know so far” of the EA movement?
If you want a single source, I find the 80000 hours key ideas page and everything it links to quite comprehensive and well written.
What links here?
- Is there a good place to find the “what we know so far” of the EA movement? by Evan Rysdam (Sep 29, 2019, 8:42 AM; 25 points)

SoerenMind Sep 25, 2019, 11:43 PM
10 points
0 ∶ 0
on: Effective Altruism and Everyday Decisions
Like most commenters, I broadly agree with the empirical info here. It’s sort of obvious, but telling others things like “don’t go out of your way to use less plastic” or even just creating unnecessary waste in a social situation can be inconsiderate towards people’s sensibilities. Of course, this post advocates no such thing but I want to be sure nobody goes away thinking these things are necessarily OK.
(I was recently reminded of a CEA research article about how considerateness is even more important than most people think, and EAs should be especially careful because their behavior reflects on the whole community.)

SoerenMind Sep 23, 2019, 2:53 PM
1 point
0 ∶ 0
in reply to: Toby_Ord’s comment on: Are we living at the most influential time in history?
On second thoughts, I think it’s worth clarifying that my claim is still true even though yours is important in its own right. On Gott’s reasoning, P(high influence | world has 2^N times the # of people who’ve already lived) is still just 2^-N (that’s 2^-(N-1) if summed over all k>=N). As you said, these tiny probabilities are balanced out by asymptotically infinite impact.

I’ll write up a separate objection to that claim but first a clarifying question: Why do you call Gott’s conditional probability a prior? Isn’t it more of a likelihood? In my model it should be combined with a prior P(number of people the world has). The resulting posterior is then the prior for further enquiries.

SoerenMind Sep 19, 2019, 2:53 PM
7 points
0 ∶ 0
in reply to: Toby_Ord’s comment on: Are we living at the most influential time in history?
Interesting point!
The diverging series seems to be a version of the St Petersburg paradox, which has fooled me before. In the original version, you have a 2^-k chance of winning 2^k for every positive integer k, which leads to infinite expected payoff. One way in which it’s brittle is that, as you say, the payoff is quite limited if we have some upper bound on the size of the population. Two other mathematical ways are 1) if the payoff is just 1.99^k or 2) if it is 2^0.99k.

SoerenMind Sep 18, 2019, 1:32 PM
1 point
0 ∶ 0
in reply to: Toby_Ord’s comment on: Are we living at the most influential time in history?
If you’re just presenting a prior I agree that you’ve not conditioned on an observation “we’re very early”. But to the extent that your reasoning says there’s a non-trivial probability of [we have extremely high influence over a big future], you do condition on some observation of that kind. In fact, it would seem weird if any Copernican prior could give non-trivial mass to that proposition without an additional observation.
I continue my response here because the rest is more suitable as a higher-level comment.

SoerenMind Sep 18, 2019, 1:26 PM
6 points
0 ∶ 0
in reply to: Toby_Ord’s comment on: Are we living at the most influential time in history?
On your prior,
P(high influence) isn’t tiny. But if I understand correctly, that’s just because
P(high influence | short future) isn’t tiny whereas
P(high influence | long future) is still tiny. (I haven’t checked the math, correct me if I’m wrong).
So your argument doesn’t seems to save existential risk work. The only way to get a non-trivial P(high influence | long future) with your prior seems to be by conditioning on an additional observation “we’re extremely early”. As I argued here, that’s somewhat sketchy to do.
What links here?
- SoerenMind's comment on Are we living at the most influential time in history? by William_MacAskill (Sep 18, 2019, 1:32 PM; 1 point)