Working on forecasting infrastructure under grants from the EA Long-Term Future Fund and BERI. Those were counterfactually important for lots of stuff I write.
jacobjacob
Karma: 460
Inviting EA forecasters to AI forecasting platform
[Note: I double-checked with the moderators before posting this to ensure it was not too “marketingy”.]
Building a successful economy for collaborative cognitive work with high externalities
Thanks, that’s great to hear.
The prize has been going on for a while, which seems important, and I think the transparency of the Prize post is really important for making common knowledge of what kind of work there is demand for. So overall it’s pretty great.
The structure of feedback looks to me like: “here’s the object-level content of the post, and here are 2-3 reasons we liked it”. I think you could be more clear about what you want to incentivise. More precisely, the current structure doesn’t answer:
How strong were the reasons relative to each other? (e.g. maybe removing Reason A would make the person win 2nd prize instead of 1st, but removing Reason B might make them win no prize)
Were the reasons only jointly sufficient to merit the prize, or might accomplishing only one of them have worked?
What other properties did the post display, which did not merit the prize? For example, maybe prize-meriting posts tend to be quite long—even though length is not something you want to incentivise on the margin.
Why did the posts end up ordered the way they did? Beyond “the black-box voting process gave that verdict” :) Currently I don’t know why SHOW was judged as deserving 4x the prize money of “The Case for the Hotel”, for example.
Conversation on forecasting with Vaniver and Ozzie Gooen
I’m unfortunately only publishing the transcript at this time. The audio contains some sections that were edited out for privacy reasons.
Thanks, fixed!
AI Forecasting Dictionary (Forecasting infrastructure, part 1)
[Question] Could we solve this email mess if we all moved to paid emails?
I’m posting this as a first step towards collecting data. Poll is a good idea, thanks!
This was crossposted to LessWrong, replacing all the mentions of “EA” with “rationality”, mutatis mutandis.
AI Forecasting Resolution Council (Forecasting infrastructure, part 2)
AI Forecasting Question Database (Forecasting infrastructure, part 3)
[Part 2] Amplifying generalist research via forecasting – results from a preliminary exploration
[Part 1] Amplifying generalist research via forecasting – models of impact and challenges
A while back me and habryka put up a bounty for people to compile a systematic list of social movements and their fates, with some interesting results. You can find it here.
Why do you think this is better than encouraging people to join foretold.io as individuals? Do you think that we are lacking an institution or platform which helps individuals to get up to speed and interested in forecasting (so that they are good enough that foretold.io provides a positive experience)?
I’m not sure if the group should fully run the tournaments, as opposed to just training a local team, or having the group leader stay in some contact with tournament organisers.
Though I have an intuition that some support from a local group might make things better. A similar case might be sports. Even though young children might start skiing with their parents, they often eventually join local clubs. There they practice with a trainer and older children, and occasionally travel together to tournaments. Eventually some of the best skiers move on to more intense clubs with more dedicated training regimes.
Trying to cache out the intuition more concretely, some of the things the local group might provide are:
Teammates. For motivation, accountability and learning.
A lower threshold for entering.
Team leaders. Someone to organise the effort and who can take lots of schleps out of the process (e.g. when I did math competitions in high school I met some kids from the more successful schools, and they would have teachers who were more clued in to when the competitions happened and who would pitch it to new students, book rooms for participants and provide them with writing utensils, point them to resources to learn more, etc)
I don’t think this list is exhaustive.
do you think that these tournaments would be good signaling for students applying for future EA jobs?
Yes, I think they would be.
If there were a way to do this with those opinions laundered out, then I wouldn’t have a problem.
I interpret [1] you here as saying “if you press the button of ‘make people search for all their offensive and socially disapproved beliefs, and collect the responses in a single place’ you will inevitably have a bad time. There are complex reasons lots of beliefs have evolved to be socially punished, and tearing down those fences might be really terrible. Even worse, there are externalities such that one person saying something crazy is going to negatively effect *everyone* in the community, and one must be very careful when setting up systems that create such externalities. Importantly though, these costs aren’t intrinsically tied up with the benefits of this poll—you *can* have good ways of dispelling bubbles and encouraging important whistle-blowing, without opening a Pandora’s box of reputational hazards.”
1) Curious if this seems right to you?
2) More importantly, I’m curious about what concrete versions of this you would be fine with, or support?
Someone suggested:
a version with Forum users with >100 karma
Would that address your concerns? Is there anything else that would?
[1] This is to a large extent: “the most plausible version of something similar to what you’re saying, that I understand from my own position”, rather than than “something I’m very confident you actually belief”.
This set-up does seem like it could be exploitable in an adversarial manner… but my impression from reading the poll results, is that this is weak evidence against that actually being a failure mode—since it doesn’t seem to have happened.
I didn’t notice any attempts to frame a particular person multiple times. The cases where there were repeated criticism of some orgs seemed to plausibly come from different accounts, since they often offered different reasons for the criticism or seemed stylistically different.
Moreover, if asked beforehand about the outcomes of something that can be read as “an open invitation to anonymous trolling that will get read by a huge amount of people in the movement”… I would have expected to see things way, way worse than what I actually saw. In fact, I’ve seen many public and identifiable comments sections on Facebook, YouTube or Twitter that were much worse than this anonymous poll.
(I claim these things weakly based on having read through all the responses in the sheet. I didn’t analyse them in-depth with an eye to finding traces of adversarial action, and don’t expect my approach here would have caught more sophisticated attempts.)
When I and Tom came up with that, I don’t think we meant “belief” to be imbued with the usual philosophical connotations. Rather, we intended it to mean something like “action-guiding, introspectively accessible representation of a state of affairs existing independently of whether it is queried”.
When people ask me what I think about the world, I can often come up with lots of intelligent sounding answers—but it is unfortunately more rare that my actual actions, plans and normative evaluations are somehow suitably hooked up to, and crucially depend upon, those answers.