Working independently on making AI systems reason safely about decision theory and acausal interactions, collaborating with Caspar Oesterheld and Emery Cooper.
I try to check the EA Forum very infrequently. If you wanna get in touch, you can use my admonymous link and leave your email address there, so I can reply to you! (You have to include contact details in the text field or else I can’t reply.)
You can also just send me thoughts and questions anonymously!
Hi! I’m looking for help with a project. If you’re interested or know someone who might be, it would be really great if you let me know/share. I’ll check the forum forum for dms.
Help with acausal research and get mentoring to learn about decision theory
Motivation: Caspar Oesterheld (inventor/discoverer of ECL/MSR), Emery Cooper and I are doing a project where we try to get LLMs to help us with our acausal research.
Our research is ultimately aimed at making future AIs acausally safe.
Project: In a first step, we are trying to train an LLM classifier that evaluates critiques of arguments. To do so, we need a large number of both good and bad arguments about decision theory (and other areas of Philosophy.)
How you’ll learn: If you would like to learn about decision theory, anthropics, open source game theory, …, we supply you with a curriculum. There’s a lot of leeway for what exactly you want to learn about. You go through the readings.
If you already know things and just want to test your ideas, you can optionally skip this step.
Your contribution: While doing your readings, you, write up critiques of arguments you read.
Bottom-line: We get to use your arguments/critiques for our projects and you get our feedback on them. (We have to read and label them for the project anyway.)
Logistics: Unfortunately, you’d be a volunteer. I might be able to pay you a small amount out-of-pocket, but it’s not going to be very much. Caspar and Em are both university employed and I am similar in means to an independent researcher. We are also all non-Americans based in the US which makes it harder for us to acquire money for projects and such for boring and annoying reasons.
Why are we good mentors: Caspar has dozens of publications on related topics. Em has a handful. And I have been around.
2. Be a saint and help with acausal research by doing tedious manual labor and getting little in return
We also need help with various grindy tasks that aren’t super helpful for learning, e.g. turning pdfs with equations etc. into sensible txts to feed to LLMs. If you’re motivated to help with that, we would be extremely grateful.
Yes, you’re totally right that I was just speaking about the range and not the expectation! That’s part of the reason why I said none of the points I made are decisive for working on s-risk. I was only providing arguments against the position that the range is symmetric, which I often see people take.
So, I don’t understand the double standard, where those subject to false allegations don’t enjoy anonymity, and those making the false allegations do get to enjoy anonymity.
I don’t think all people in the replies were arguing that Ben’s initial post was okay and deanonymizing Alice and or Chloe would be bad (which I think you would call a double standard, which I’m not commenting on right now). Some probably do but some probably think that Ben’s initial post was bad and that deanonymizing Alice and or Chloe would also be bad and that we shouldn’t try to correct one bad with another bad, which doesn’t look like a double standard to me.
Oh man, tough situation, and also congrats and this sounds awesome and admiration that you’re thinking about this question.
As other commenters, I don’t know the first thing about patents. Have you talked to a lawyer, including to figure out whether you can patent in a way that doesn’t involve the Tech Transfer office (so much)?
Not sure about the claim but possible! I certainly wouldn’t say no to something automatic. But I think if setting it yourself is easy enough, it would still get a bunch of the value! I think if the feature was implemented in a similar way to in-line commenting on LessWrong, where you just hover over the correct line and it offers you a bookmark-button that you just need to click, that would be low-friction enough for people like me to use it. (I think anything that’s two-click might be too much friction)
Within-post bookmarks
I often start posts but they are too long to read them in one go and the fact that I have to do that (or forever forget where I am) creates a big ugh field for me. Solution: Within-post bookmarks! I think it would be amazing if we could mark where we are in a post and the next time, we can just click to get ourselves back there!
(FWIW, I think that’s a major feature of printed media, which I have much less ugh feelings about reading. You can always put it away without it being annoying later on.)
If not about “really bad” stuff then feedback should be consensual as a norm—As a community I think we should want to be opting into feedback rather than assuming everyone wants it.
I agree with this. It seems quite hard to implement well unfortunately. Asking if someone wants to hear some (negative) feedback can make it really hard for the other person to say no and already does some of the damage, so in some sense already takes away from it being truly consensual. There probably is some way to do this skillfully but it seems hard/if there is a way that just works and is easy to apply, I don’t know it. That said, I think asking someone if there wanna hear some feedback and if now is a good time is usually better than nothing.
(That said, we might disagree on the details of in which cases non-consensual feedback is fine.)
People assume I want feedback a lot and frankly, I do, but some of it can be brutal. And I have pretty thick skin. I have been sad for days after EA feedback. I wouldn’t want other people to be treated like this without opting into it
Interesting. I don’t think I’ve made this experience (much—I had this kind of feedback once in 2019). Unless I just can’t think of it right now (very possible, I’m very forgetful and easily miss obvious things), I don’t think people give me much feedback at all. I wonder if some of the difference in what we emphasise comes from a difference in how people treat us based on demographics etc. (I’m a small woman while Nathan is a tall man. I think my conversation style also projects less perceived confidence than his. I would expect most people to expect me to be more sensitive.)
So maybe one unintuitive takeaway could be “offer marginally more feedback to people who most people based on a shallow impression wouldn’t think can take it; be marginally more careful with feedback to people who most people based on a shallow impression would think can take it.”
edit: Some context is that I wrote this post as a reaction to being frustrated over the years with concrete instances of people not sharing important negative feedback and finding it a bit crazy that people don’t do so (not in they are crazy but it’s crazy that the world works that way) - some of this is second-hand knowledge though.
Extreme nitpick, but you mean the optimizer’s curse, not the winner’s curse. The winner’s curse is that when you win an auction, you should expect to have overpaid (similar dynamics to the optimizer’s curse though.) The optimizer’s curse is really interesting and worth a read IMO! Doesn’t come from worries with p-hacking but applies even when the data you look at is fair and unbiased. IIRC due to the distribution of random errors but I could be wrong.