Software engineer, blogging and editing at /nickai/. PM me your fluid-g-increasing ideas. (Formerly President/VP of the EA Club at RIT (NY, USA)).
NicholasKross
[SEE NEW EDITS] No, *You* Need to Write Clearer
Quick Thoughts on A.I. Governance
Effective Altruism, Before the Memes Started
[Question] Is it crunch time yet? If so, who can help?
This font is coursing through my veins and increasing my IQ thank you
Reminder for many people in this thread:
“Having a small clique of young white STEM grads creates tons of obvious blindspots and groupthink in EA, which is bad.”
is not the same belief as
“The STEM/techie/quantitative/utilitarian/Pareto’s-rule/Bayesian/”cold” cluster-of-approaches to EA, is bad.”
You can believe both. You can believe neither. You can believe just the first one. You can believe the second one. They’re not the same belief.
I think the first one is probably true, but the second one is probably false.
Thinking the first belief is true, is nowhere near strong enough evidence to think the second one is also true.
A Quick List of Some Problems in AI Alignment As A Field
Possible upcoming LeftTube video about EA
Personal feelings: I thought Karnofsky was one of the good ones! He has opinions on AI safety, and I agree with most of them! Nooooooooooo!
Object-level: My mental model of the rationality community (and, thus, some of EA) is “lots of us are mentally weird people, which helps us do unusually good things like increasing our rationality, comprehending big problems, etc., but which also have predictable downsides.”
Given this, I’m pessimistic that, in our current setup, we’re able to attract the absolute “best and brightest and also most ethical and also most epistemically rigorous people” that exist on Earth.
Ignoring for a moment that it’s just hard to find people with all of those qualities combined… what about finding people with actual-top-percentile any of those things?
The most “ethical” (like professional-ethics, personal integrity, not “actually creates the most good consequences) people are probably doing some cached thing like “non-corrupt official” or “religious leader” or “activist”.
The most “bright” (like raw intelligence/cleverness/working-memory) people are probably doing some typical thing like “quantum physicist” or “galaxy-brained mathematician”.
The most “epistemically rigorous” people are writing blog posts, which may or may not even make enough money for them to do that full-time. If they’re not already part of the broader “community” (including forecasters and I guess some real-money traders), they might be an analyst tucked away in government or academia.
A broader-problem might be something like: promote EA --> some people join it --> the other competent people think “ah, EA has all those weird problems handled, so I can keep doing my normal job” --> EA doesn’t get the best and brightest.
Is principled mass-outreach possible, for AGI X-risk?
Devin’s reply/summary:
“Thanks for the comments. Sorry, I wrote a good deal of this stream of conscious so it isn’t really structured as an argument. More a way of me to connect some personal thoughts/experiences together in a hopefully productive way. I can see how that wouldn’t be super accessible. The basic argument embedded in it though is:
-
Effective Altruism, like many idealistic movements, started out taking critics very very seriously and trying to reach out to/be charitable to them as much as possible, which is a good thing
-
Effective Altruism, like most movements that grow older, is not quite like that anymore, it seems to respond with less frequency and generosity to critics than it used to, which is unfortunate but understandable
-
Understandable as it is, we should at least take a bit more notice of it if that’s the path we are going down because…
-
Many movements move on from here to ridiculing criticisms by treating common criticisms as though they were obviously, memeably false, and that everyone in the know gets that (I didn’t use examples, but the one most on my mind was the midwit meme format, which only requires the argument being ridiculed, your stated, undefended position, and some cartoons, to make it look like you’ve made a point). This is bad and we should be careful not to start doing it.”
-
Why I’m Not (Yet) A Full-Time Technical Alignment Researcher
We ought to have a new word, besides “steelmanning”, for “I think this idea is bad, but it made me think of another, much stronger idea that sounds similar, and I want to look at that idea now and ignore the first idea and probably whoever was advocating it”.
Learning Math in Time for Alignment
BOUNTY IDEA (also sent in the form): Exploring Human Value Codification.
Offered to a paper or study that demonstrates a mathematical (or otherwise engineering-ready) framework to measure human’s real preference-ordering directly. Basically a neuroscience experiment or proposal thereof.
End goal: Using this framework / results from experiment(s) done based on it, you can generate novel stimuli that seem similar to each other, and reliably predict which ones human subjects will prefer more. (Gradients of pleasure, of course, no harm being done). And, of course, the neuroscientific understanding of how this preference ordering came about.
Prize amount: $5-10k for the proposal, more to fund a real experiment, order of magnitude probably in the right ballpark.
Stop Using Discord as an Archive
[Question] [retracted] Discussion: Was SBF a naive utilitarian, or a sociopath?
It is more dramatic to break the curfew tho
Buy enough darkweb stimulants to move up a rank in League.
Strong agree, hope this gets into the print version (if it hasn’t already).