Theoretical Computer Science Msc student at the University of [Redacted] in the United Kingdom.
I’m an aspiring alignment theorist; my research vibes are descriptive formal theories of intelligent systems (and their safety properties) with a bias towards constructive theories.
I think it’s important that our theories of intelligent systems remain rooted in the characteristics of real world intelligent systems; we cannot develop adequate theory from the null string as input.
𝕮𝖎𝖓𝖊𝖗𝖆
Karma: 2,229
I prefer to just analyse and refute his concrete arguments on the object level.
I’m not a fan of engaging the person of the arguer instead of their arguments.
Granted, I don’t practice epistemic deference in regards to AI risk (so I’m not the target audience here), but I’m really not a fan of this kind of post. It rubs me the wrong way.
Challenging someone’s overall credibility instead of their concrete arguments feels like bad form and [logical rudeness] (https://www.lesswrong.com/posts/srge9MCLHSiwzaX6r/logical-rudeness).
I wish EAs did not engage in such behaviour and especially not with respect to other members of the community.
To expand on my complaints in the above comment.
I do not want an epistemic culture that finds it acceptable to challenge an individuals overall credibility in lieu of directly engaging with their arguments.
I think that’s unhealthy and contrary to collaborative knowledge growing.
Yudkowsky has laid out his arguments for doom at length. I don’t fully agree with those arguments (I believe he’s mistaken in 2 − 3 serious and important ways), but he has laid them out, and I can disagree on the object level with him because of that.
Given that the explicit arguments are present, I would prefer posts that engaged with and directly refuted the arguments if you found them flawed in some way.
I don’t like this direction of attacking his overall credibility.
Attacking someone’s credibility in lieu of their arguments feels like a severe epistemic transgression.
I am not convinced that the community is better for a norm that accepts such epistemic call out posts.
(I hadn’t seen this reply when I made my other reply).
What do you think of legitimising behaviour that calls out the credibility of other community members in the future?
I am worried about displacing the concrete object level arguments as the sole domain of engagement. A culture in which arguments cannot be allowed to stand by themselves. In which people have to be concerned about prior credibility, track record and legitimacy when formulating their arguments...
It feels like a worse epistemic culture.
Are “Bad People” Really Unwelcome in EA?
I think there are many EAs with “pure” motivations. I don’t know what the distribution of motivational purity is, but I don’t expect to be a modal EA.
I came via osmosis from the rat community (partly due to EA caring about AI safety and x-risk). I was never an altruistic person (I’m still not).
I wouldn’t have joined a movement focusing on improving lives for the global poor (I have donated to GiveWell’s Maximum Impact Fund, but that’s due to value drift after joining EA).
This is to say that I think that pure EAs exist, and I think that’s fine, and I think they should be encouraged.Being vegan
Frugal living, etc. are all fine IMO.
I’m just against using them as a purity tests. If the kind of people we want to recruit are people strongly committed to improving the world, then I don’t think those are strong (or even useful) signals.
I think ambition is a much stronger signal of someone who actually wants to make an impact than veganism/frugality/other moral fashions.
As long as we broadly agree on what a better world looks like (more flourishing, less suffering), then ambitious people seem valuable.
Even without strict moral alignment, we can pursue pareto optimal improvements on what we consider a brighter world?
Like most humans probably agree a lot more on what is moral than they disagree, and we can make the world better in ways that we both agree on?
I don’t think that e.g. not caring about animal welfare is that big an obstacle to cooperating with other EAs? I don’t want animals to suffer, and I wouldn’t hinder efforts to improve animal welfare. I’d just work on issues that are more important to me.Very compatible with “big tent” EA IMO.
We’re finally going mainstream, and that’s a good thing.
Huh. If I had a bright idea for AI Safety, I’d share it and expect to get status/credit for doing so.
The idea of hiding any bright alignment research ideas I came up with didn’t occur to me.
I’m under the impression that because of common sense morals (i.e. I wouldn’t deliberately sabotage to get the chance to play hero), selfishly motivated EAs like me don’t behave particularly different in common scenarios.
There are scenarios where my selfishness will be highlighted, but they’re very, very narrow states and unlikely to materialise in the real world (highly contrived and only in thought experiment land). In the real world, I don’t expect it to be relevant. Ditto for concerns about superrational behaviour. The kind of superrational coordination that’s possible for purely motivated EAs but isn’t possible with me is behaviour I don’t expect to actually manifest in the real world.
But that’s mostly relevant in small scale altruism? Like I wouldn’t give to beggars on the street. And I wouldn’t make great personal sacrifice (e.g. frugal living, donating the majority of my income to charity [I was donating 10% to GiveWell’s maximimum impact fund until a few months ago (Forex issues [I’m in Nigeria], now I’m unemployed)]) to improve the lives of others.
But I would (and did!) reorient my career to work on the most pressing challenges confronting humanity given my current/accessible skill set. I quit my job as a web developer, I’m going back to university for graduate study and plan to work on AI safety and digital minds.
My lack of altruism simply is not that relevant for trying to improve the condition for humanity.
What you’re missing is that I want to attain status/prestige/glory by having positive impact not through some other means.
It feels like you’re failing to grasp what that actually means?
My pursuit of status/prestige/glory is by trying to have the largest impact on a brighter future conditioning on being the person I am (with the skills and personality I have).
Personal impact on a brighter world.
I’m not a grant maker and don’t want to be.
I am not aware of any realistic scenario where I would act differently from someone who straightforwardly wanted to improve the world altruistically.
(The scenarios in which I would seem very contrived and unlikely to manifest in the real world.)
Could you describe a realistic scenario in which you think I’d act meaningfully different from an altruistic person in a way that would make me a worse employee/coworker?
I’m a rationalist.
I take scope sensitivity very seriously.
Impartiality. Maybe I’m more biased towards rats/EAs, but not in ways that seem likely to be decision relevant?
You could construct thought experiments in which I wouldn’t behave in an ideal utilitarian way, but for scenarios that actually manifest in the real world, I think I can be approximated as following some strain of preference utilitarianism?
I have a significantly consequentialist world view.
I am motivated by the vision of a much better world.
I am trying to create such a better world. I want to devote my career to that project.
I’m trying to optimise something like “expected positive impact on a brighter future conditional on being the person that I am with the skills available to/accessible for me”.
The ways I perceive that I differ from EAs is:
Embracing my desire for status/prestige/glory/honour
I’m not impartial to my own welfare/wellbeing/flourishing
I’m much less willing to undertake personal hardship (frugality, donating the majority of my income, etc.) and I think this is fine
I’m not (currently) vegan
I want to say that I’m not motivated by altruism. But people seem to be imagining behaviour/actions that I oppose/would not take and I do want to create a brighter future.
And I’m not sure how to explain why I want a much brighter future in a way that isn’t altruistic.
A much (immensely) better world is possible
We can make that happen
The “we should make that happen” feels like an obvious conclusion. Explaining the why draws blanks.
Rather than saying I’m not altruistic, I think it’s more accurate to say that I’m less willing to undertake significant personal hardship and I’m more partial to my own welfare/flourishing/wellbeing.
Maybe that makes me not EA, but I was under the impression that I was simply a non standard EA.
I think I’ve been defining “altruism” in an overly strict sense.
Rather than say I’m not altruistic, I mostly mean that:
I’m not impartial to my own welfare/wellbeing/flourishing
I’m much less willing to undertake personal hardship (frugality, donating the majority of my income, etc.) and I think this is fine
10% is not that big an ask (I can sacrifice that much personal comfort), but donating 50% or forgoing significant material comfort would be steps I would be unwilling to take.
(Reorienting my career doesn’t feel like a sacrifice because I’ll be able to have a larger positive impact through the career switch.)
I now think it was a mistake/misunderstanding to describe myself as non altruistic and believe that I was using an unusually high standard.
(That said, when I started the 10% thing, I did so under the impression that it was what the sacrifice I needed to make to gain acceptance in EA. Churches advocate a 10% tithe as well [which I didn’t pay because I wasn’t actually a Christian (I deconverted at 17 and open atheism is not safe, so I’ve hidden [and still hide] it)], but it did make me predisposed to putting up with that level of sacrifice [I’d faced a lot of social pressure to pay tithes at home, and I think I gave in once].
The 10% felt painful at first, but I eventually got used to it, and it became a source of pride. I could brag about how I was making the world a better place even with my meagre income.)
I plan to seek status/glory through making the world a better place.
That is, my desire for status/prestige/impact/glory is interpreted through an effective altruistic like framework.
“I want to move the world” transformed into “I want to make the world much better”.
“I want to have a large impact” became “I want to have a large impact on creating a brighter future”.
I joined the rationalist community at a really impressionable stage. My desire for impact/prestige/status, etc. persisted, but it was directed at making the world better.
I think the question determining your core philosophy would be which term you consider primary.
If this is not answered by the earlier statements, then it’s incoherent/inapplicable. I don’t want to have a large negative impact, and my desire for impact/prestige cannot be divorced from the context of “a much brighter world”.
For example if you view them as a means to an end of helping people and are willing to reject seeking them if someone convinces you they are significantly reducing your EV then that would reconcile the “A” part of EA.
My EV is personally making the world a brighter place.
I don’t think this is coherent either. I don’t view them as a means to an end of helping people.
But I don’t know how seeking status/glory by making the world a brighter place could possibly be reducing my expected value?
It feels incoherent/inapplicable.
A piece of advice I think younger people tend to need to hear is that you should be more willing to accept that “X is something I like and admire, and I am also not X” without having to then worry about your exact relationship to X or redefining X to include themselves (or looking for a different label Y). You are allowed to be aligned with EA but not be an EA and you might find this idea freeing (or I might be fighting the wrong fight here).
This is true, and if I’m not an EA, I’ll have to accept it. But it’s not yet clear to me that I’m just “very EA adjacent” as opposed to “fully EA”. And I do want to be an EA I think.
I might modify my values in that direction (why I said I’m not “yet” vegan as opposed to not vegan).
That sounds fair.
“You shouldn’t fund/patronise me or support my research” is probably a recommendation I’d be loathe to make. (Excluding cases where I’m already funded well enough that marginal funding is not that helpful.)
Selflessly rejecting all funding because I’m not the best bet for this particular project is probably something that I’d be unwilling to do.
(But in practice, I expect that probabilistic reasoning would recommend funding me anyways. I think having enough confidence to justify not funding a plausible pathway is unlikely before it’s too late.)
But yeah, I think this is an example of where selfishness would be an issue.
Thanks for the reply!
Thanks for making this case. More investment in China studies seems straightforwardly valuable.
(Still reading.)
So, I Want to Be a “Thinkfluencer”
Is promotion to the Frontpage automated? Do mods take an approve by default and then demote stuff they think is not worth promoting?
I’m used to the LessWrong approach of blogpost by default, and mods manually approve promotion to the front page. I’m not sure I want to make the decision for the community if my post should be on the front page. But I do want it to be promoted to the front page if the mods approve it.I feel like promotion to front page by default will disincentivise demotion to personal blog post if a post has high engagement irrespective of whether it’s otherwise frontpage material.
Why would posting mainly in these tiny communities be the best approach? First, I think these communities are already far more familiar with the topics you plan to publish on than the average reader. Second, they are – as I said – tiny.
The reports are for my “learning about the world” phase, not attempts at becoming a public intellectual.
As for why LW/EAF:
Feedback from my communities is more important to give me sustainable motivation than feedback from randoms
I’m more likely to get valuable feedback from these communities than others especially because they are more familiar with these ideas and have excellent epistemics
I don’t want to delay my report writing and such by adding the extra burden of setting up a blog
The rationalist/EA communities provide a natural audience for the reports
Feedback will likely be faster
I intend to start out writing for rats/EAs and rat/EA curious
That is when I shift from writing about stuff I’m learning about to giving more original takes
I may want to work for/or apply for funding from EA organisations, so having a history of useful writing would be helpful
Etc.
Second, they are – as I said – tiny. If you want to be a public intellectual, I think you should publish where public intellectuals generally publish. This is usually a combination of books, magazines, journals, and your own platforms (e.g. personal website/blog, social media etc.)
Eventually, I’ll do that. But I’ll start out a rationalist blogger before broadening my audience.
You could probably improve on your plan by making a much more in-depth analysis of what your exact goals are and what your exact audiences are. It seems to me a few steps are missing in this statement:
I believe such people provide considerable value to the world (and specifically to the project of improving the world).
What would probably be useful is, in a sense, a theory of change on how doing the things you want to do lead to the outcomes you want.
I think I’d want to eventually write articles/books directed at broader audiences/the intellectual public/people interested in improving the world. Well, I’m hoping to change the minds of important people I guess.
I want to help sell the following ideas:
The current state of the world is very suboptimal
Vastly better world states are possible
We can take actions that would make us significantly more likely to reach those vastly better states
We should do this
I’d like to paint concrete and coherent visions for a much brighter future (not concrete utopias, but general ways that we can make the world much better off)
Paretopian outcomes
I want to get people excited about such a future as something we should aspire to and work towards.
Here are things we can do to reach towards that future
I’d like to convince people positioned to have a large positive influence the world or to attain the leverage to have such an influence.
You are going to need those in the massively competitive landscape you aim for.
Yeah, probably. But they’re needed in general to improve the world, I think.
There is an ongoing “friend matching” campaign for GiveWell.
Anyone who donates through a friend link will have their donation matched up to $250. Please donate.
My friend link.