An undergrad at University of Maryland, College Park. Majoring in math.
After finishing The Sequences at the end of 9th grade, I started following the EA community, changing my career plans to AI alignment. If anyone would like to work with me on this, PM me!
I’m currently starting the EA group for the university of maryland, college park.
This was a particularly informative podcast, and you helped me get a better understanding of inner alignment issues, which I really appreciate.
To be clear I understand: the issue with inner alignment is that as an agent gets optimized for a reward/cost function on a training distribution, and to do well the agent needs to have a good enough world model to determine that it’s in or could be undergoing training, then if the training ends up creating an optimizer, it’s much more likely that that optimizer’s reward function is bad or a proxy, and if it’s sufficiently intelligent, it’ll reason that it should figure out what you want it to do, and do that. This is because there are many different bad reward functions an inner optimizer can have, but only one that you want it to actually have, and each of those bad reward functions will pretend to have the good one.
Although the badly-aligned agents seem like they’d at least be optimizing for proxies of what you actually want, as early (dumber) agents with unrelated utility functions wouldn’t do as well as alternative agents with approximately aligned utility functions.
Correct me on any mistakes please.
Also, because this depends on the agents being at least a little generally intelligent, I’m guessing there are no contemporary examples of such inner optimizers attempting deception.
I like the idea of having people with different opinions discuss their disagreements, but I don’t think they should be marketed as debates. That term doesn’t have positive connotations, and seems to imply that there will be a winner/loser. Even if there is no official winner/loser, it puts the audience and the participants in a zero-sum mentality.
I think something more like an adversarial collaboration would be healthier, and I like that term more because it’s not as loaded, and it’s more up front about what we actually want the participants to do.
If we successfully built this platform, would you consider using it? If your answer is “it depends”, what does it depend on?
I wouldn’t use it, since I don’t donate to campaigns, but I would certainly push all my more political friends and family members to use it.
That’s annoying. The formatting is fixed (I had to transfer over to the WYSIWYG editor, since the <br> solution for markdown didn’t work). I also don’t know how to transfer it over to a regular post. Thanks for telling me about these problems.
Ok. I’ll do that.
Oh, this seems fun! I’ll certainly be playtesting it in the coming days (it’s also been added to my wish-list).
Yes!!! Thank you for this! I absorb audio info much easier & quicker than text, so this will be very helpful.
Edit: also, now that you mention it, I could probably record forum posts myself as well, as there are likely many others like me. Do you want to partner up to coordinate on which posts to read & add a measure of social enforcement?
I don’t have a lot of time in the next few days, but I should be much freer after the 5th.
Nice! Have any preferences for what we use to coordinate together (Slack, Discord, Twitter, WhatsApp, …)? If not, then we can default to WhatsApp.
I am not on the EA global slack.
Doing so now.
For example, in the first month we launched them (July 2020), across the 3 different profiles, the detailed versions averaged 62% the number of downloads as the short versions, and the audio versions averaged 6% of the number of downloads of the short versions.
This changes my estimate of how useful the EA forum podcast will be, so thanks for sharing your experience.
Anchor sends messages to podcast platforms to get the podcast on them. They say this takes a few business days to complete. In the meantime, you can use Ben Schifman’s method.
We’ve talked in private, but I figure I should publicly thank you for your offer for help.
edit: this is the thank you.
Not entirely applicable to the discussion, but I just like talking about things like this and I finally found something tangentially related. Feel free to disregard.
The cold war wasn’t peacetime though… there was continuous fighting by both sides. The Americans and Chinese in Korea, the Americans in Vietnam, and the Russians in Afghanistan.
One can argue that these places don’t scale to the kind of military techniques and science that a World War 3 scenario would require. But this kind of war has never occurred with modern technology (specifically hydrogen bombs). How do we know that all of the ideas dreamed up by generals and military experts wouldn’t get tossed out the window the moment it was determined that they were inapplicable to a nuclear war?