The difference between the most vs least spooky X-risks is way more than a 100X difference.
I think I would agree with this, if I had to put a number.
What I mean in my comment is, with this model, if you say okay letās pick a bigger n so that we see bigger differences in OOMs, then you are also introducing more points of failure in the estimation, and that effect dominates.
Do you have an a priori reason to discard this? Besides the conclusion being wacky, which is a good reason to discard a model anyways.
Clara Torres Latorre šø
Karma: 430
the OOM of variation in āground truthā come from alpha and n, not xmin
alpha, we could talk all day, but the model is not extremely sensitive to it
on the other hand, if you say letās have more OOMs in the possible values of ground truth, following the power law, that means jacking n up
and when you jack n up you have even more opportunities for errors to be crazy big, and this effect dominates (at least thatās what I read from the OP) and the curse becomes worse
now if we change alpha and n at the same time, idk
my honest opinion is that numbers are just one way to process information, and using them for this is so out of distribution that itās essentially meaningless (as it is when discussing p(doom) and stuff like that)
Ties in with the more meta-level fact that numbers are used a lot in EA/ārationalist spaces, even when there are kinds of uncertainty that donāt go along with them.
Iām sure people have written about this many times but donāt know who to cite.
Hi, could you share some info for reference:
Breakdown of expenses
How much did it cost (in time) to run the stuff
Schedule
Hi, I donāt love talking to GPT but:
I agree with you that there is a vibe that if you arenāt doing The Most Impactful Thing (TM), you donāt count. Thatās uncomfortable.
I disagree with your split between āpractitionersā and āNGO workersā. NGOs have practitioners, if they are delivering something.
I think urbanism (and policy in general) can be a high impact area, but you need to compare it to other aspects and policy. I havenāt run any numbers so I canāt know.
Not an answer but Rational Reminder (nerdy evidence-based finance podcast by Ben Felix and colleagues) interviewed Elie Hassenfeld (from GiveWell). Super interesting:
https://āārationalreminder.ca/āāpodcast/āā372
I find this quite useful to avoid the failure mode that worries me the most, namely, that once one derives a source of income from being a community builder, their incentives start to look like ādo whatever gets me renovatedā and less like ādo whatever I really believe is most impactfulā.
So separating the operations layer (professional), from the more ideological/āopinionable layer (not paid) seems like a very good idea to me.
The analogy with quakers having pastor and administrator roles separated is helpful to me to put words on why I have complicated feelings about EA paid CB roles, because they are in some sense both.
Iāve been thinking about this for a while since I read your post about it:
It would feel very weird and conflict-of-interest-y to fund someone in my local group to be a paid organizer, right now weāre a bunch of volunteers running it.
You say well stewarded meta-EA is very high impact. I agree in theory, but Iām not sure about how to know if something is well stewarded in practise.
Fantastic post, thank you for airing it.
Something that I can say at a personal level is: I find it very hard to trust a paid community builder, be it a priest, an amateur orchestra conductor or an EA community builder. I see conflicts of interest everywhere.
On the other hand, I think growing the movement makes sense, and having people dedicated to it makes sense, so if someone is doing it you want to measure how they are doing, etc
I find it very hard to reconcile.
By the way, I clicked the link (finally), and:
I donāt find Wiley anywhere
I donāt see a physical address anywhere
Meritpeer claims big numbers of users, but I didnāt find anyone talking about it (trustpilot, reddit, etc)
They have premium pricing to speed review up to 5 days or even 2 days
I wouldnāt even give it a chance, this arenāt a couple red flags, we talking November 1917 situation here.
Which makes me sad, because I really like the broader point of engaging with mainstream academia and playing ball, and Iāve been nerdsniped by a discussion about peer review and feel that Iām derailing the comment section.
I computed the time that it takes me * my salary, approx.
Ofc if I did this as a freelancer I would charge more.
Take this with a huge grain of salt, because thing vary enormously field to field.
10 days turnaround sounds too good to be true to me. If it was 6 months I would maybe give it a try for value of information.
But thereās more:
Price seems cheap: Last paper I reviewed took me 10-15 hours, so 300-500 + tax would be a reasonable price range and 150-250 sounds meagre. And you need to factor in admin, infra and costs that are not just paying the reviewer.
In my field, peer review is āpro bonoā but done during the āworking hoursā of people with public salaries mostly. And thereās the understanding that since we publish and someone else reviews, we also should review some. However, that means that the availability to review varies a lot depending on teaching, research, admin, etc, and we usually fit it in with low priority.
That means if you want someone to review a paper, even if they only need 2 days, the people that have the expertise usually have work to do and might not be willing to give up on next weekend.
About āwhat if I got paidā, itās complicated, bc it depends on if I really have the time (meaning, no plans on the only weekend in the 10 day window, and willing to work), but probably not.
And thereās also an ugh factor about getting paid for something that we usually do for free /ā as part of a salaried job. Iām not decided on it.
I agree with the general point of the post.
But I disagree with specifically using Wileyās services to obtain peer review [1].
I would be excited for people in EA sending their āresearchā pieces to mainstream academic journals more often.
[1] Iām a math researcher in academia. When I peer review an article, Iām usually granted 60 to 90 (sometimes more) days, depending on length. Adding admin, sometimes multiple times of revision, the time that editors need to find a suitable reviewer, missed deadlinesā¦ the whole process usually amounts to at least 6 months, and sometimes it takes years.
I donāt think 10 days is a reasonable timeline for peer review, and I find it even short to find a relevant and willing reviewer.
how big is the effect size?
Relatedly, I have met people doing āAI safetyā stuff without having impact in mind, and end up doing research on whatever shiny academic vaguely related topic without a clear theory of change, while still being intermingled with the EA crowd sociologically.
Some of them pay lip service to impact and write theories of change to get funds (as a pure mathematician, we do that in my line of work and I think I can recognize when you say there is impact to get the research you are obsessed with funded, and when you are actually trying to do good in the world).
Hey, thanks for your reply, I found it very informative.
Iām excited to read your sensitivity analysis writeup when you publish it.
Iām curious about how you model the time discounting more in detail:
As I read it right now, you pick six fixed multipliers for six fixed future time intervals. Am I understanding this correctly?
Under some assumptions that are common in EA that I donāt share such as the time of perils hypothesis, this would mean that, at least for some ways of computing, most utility is in the 500+ bucket.
And then the model would be very sensitive to people putting 0 vs any nonzero number for the 500+ years bucket. Is this the case?
Iām curious on why you canāt (as opposed to donāt want to or will not) discuss these on the EA Forum.
In my view, there are many points in the articles that could be interesting for discussion.
I think the devil is in the details. The principles are fine but what really matters is the operationalization. Hard to tell without more information about the program.
What do you mean by ātestedā? What outcomes are you measuring /ā or do you plan to do surveys...?