Linch

Karma: 4,972

“To see the world as it is, rather than as I wish it to be.”

I work for the EA research nonprofit Rethink Priorities. Despite my official title, I don’t really think of the stuff I do as “research.” In particular, when I think of the word “research”, I think of people who are expanding the frontiers of the world’s knowledge, whereas often I’m more interested in expanding the frontiers of my knowledge, and/or disseminating it to the relevant parties.

I’m also really interested in forecasting.

People may or may not also be interested in my comments on Metaculus and Twitter:

Metaculus: https://pandemic.metaculus.com/accounts/profile/112057/

Twitter: https://twitter.com/LinchZhang

Linch 5 Aug 2021 0:23 UTC
2 points
in reply to: Benjamin_Todd’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
I don’t think this addresses ofer’s objection if I understand it correctly (but then again the length of our back and forth comments is maybe strong evidence against me understanding the objection correctly!)

Linch 4 Aug 2021 6:44 UTC
17 points
on: This Can’t Go On
One reason we shouldn’t be too surprised that most scientific progress is recent, years-wise is that a very simple model of scientific progress might be that you need X people (or X smart, literate people etc) thinking for Y moments to have a Z% chance of coming up with a scientific insight.
And if you measure time anthropically rather than geologically, estimates suggest that >~20% of people who have ever lived were born after 1650. The fraction is likely higher if you measure by human-moments rather than number of humans (since dead infants don’t make many scientific insights), and higher still if you only count literate or otherwise knowledgeable humans with ample free time.

Linch 3 Aug 2021 18:18 UTC
4 points
in reply to: ofer’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
Though this conversation with you made me think about this some more and my above claim now seems to me too strong (I added an EDIT block).
I’m glad to cause an update! Hopefully it’s in the right direction! :)
Even ‘very high-status figures within longtermist EA’ can control a limited amount of funding, especially for requests that are speculative/weird/non-legible from the perspective of the relevant donors. I don’t know what’s the bar for “really good donation opportunities”, but the relevant thing here is to compare the EV of that $1M in the hands of Bostrom to the EV of that $1M in the hands of other longtermism aligned people.
Again, there are proxies you can look at, like what Carl Shulman donates to vs what actual winners of the donor lottery donates to. But maybe you don’t consider this much evidence, if you posit that Nick Bostrom specifically has unusually high discernment, specifically enough to donate to things in the band of activities that are “speculative/weird/non-legible” from the perspective of the relevant donors, but not speculative/weird/non-legible enough that the donor lottery administration won’t permit this.
I guess my rejoinder here is just an intuitive sense of disbelief? Several (say >=3?) orders of magnitude above 1 million gets you >1B, and as can be deduced in the figures in the post above, this is already well over the annual long-termist spending every year. If we believe that Nick Bostrom can literally accomplish much more good with 1 million than money allocated by the rest of the longtermist EA movement combined (including all money sent to FHI, where he works), isn’t this really wild? Also why aren’t we sending more money to Nick Bostrom to regrant?

(Though perhaps you came to the same conclusion by now).
Less importantly, you rely here on the assumption that being “a very high-status figure within longtermist EA” means you can influence a lot of funding, but the causal relationship may mostly be going in the other direction
I’m confused about what you’re saying here. P(B| do A) is not evidence against P(A|B), except in very rare circumstances.

Linch 3 Aug 2021 10:13 UTC
14 points
in reply to: ofer’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
My estimate is that the EV of that $1M, conditional on person X winning, is several orders of magnitude larger for X=[Nick Bostrom] than for almost any other value of X.
I doubt this is literally true fwiw. If Bostrom, a very high-status figure within longtermist EA, has really good donation opportunities to the tune of 1 million, I doubt it’d be unfunded. I also feel like there are similar analogous experiments made in the past where relatively low oversight grantmaking power was given to certain high-prestige longtermist EA figures( eg here and here). You can judge for yourself whether impact “several orders of magnitude higher” sounds right, personally I very much doubt it.
Why? Do you believe in something analogous to the efficient-market hypothesis for EA grantmaking
I meant “should” as a normative claim, not an empirical claim. Sorry if I miscommunicated.

Linch 3 Aug 2021 0:42 UTC
8 points
on: Linch’s Shortform
I’m pretty confused about the question of standards in EA. Specifically, how high should it be? How do we trade off extremely high evidential standards against quantity, either by asking people/ourselves to sacrifice quality for quantity or by scaling up the number of people doing work by accepting lower quality?

My current thinking:
1. There are clear, simple, robust-seeming arguments for why more quantity* is desirable, in far mode.
2. Deference to more senior EAs seems to point pretty heavily towards focusing on quality over quantity.

3. When I look at specific interventions/grant-making opportunities in near mode, I’m less convinced they are a good idea, and lean towards earlier high-quality work is necessary before scaling.
The conflict between the very different levels of considerations in #1 vs #2 and #3 makes me fairly confused about where the imbalance is, but still maybe worth considering further given just how huge a problem a potential imbalance could be (in either direction).
*Note that there was a bit of slippage in my phrasing, while at the frontiers there’s a clear quantity vs average quality tradeoff at the output level, the function that translates inputs to outputs does not necessarily mean increased quantity of inputs will result in decreased averaged quality. For example, research orgs can use more employees to focus on reviews, red-teaming, replications, etc of existing work, thus presumably increasing average research quality with increased quantity of inputs.

Linch 2 Aug 2021 21:56 UTC
4 points
in reply to: ofer’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
Some people may have a comparative advantage with respect to their ability to donate effectively such that the EV from their donation would be several orders of magnitude larger than the “average EV” from a donation of that amount.

This seems like a fairly surprising claim to me, do you have a real or hypothetical example in mind?
EDIT: Also I feel like in many such situations, such people should almost certainly become grantmakers!

Linch 2 Aug 2021 1:58 UTC
2 points
in reply to: AGB’s comment on: AGB’s Shortform
To answer your linguistic objection directly, I think one reason/intuition I have for not trusting probabilities much above 99% or much below 1% is that the empirical rates for the reference class of “fairly decent forecaster considers a novel well-defined question for some time, and then becomes inside-view utterly confident in the result” has a failure rate likely between 0.1% and 5%.
For me personally, I think the rate is slightly under 1%, including from misreading a question (eg forgetting the “not”) and not understanding the data source.
This isn’t decisive (I do indeed say things like giving <0.1% for direct human extinction from nuclear war or climate change this century) but is sort of a weak outside view argument for why anchoring on 1%-99% is not entirely absurd, even if we lived in an epistemic environment where basis points or 1-millionths probabilities are the default expressions of uncertainty.

Put another way, I think if the best research on how humans think of probabilities to date for novel well-defined problems is Expert Political Judgement where political experts’ “utter confidence” translates to a ~15% failure rate (and my personal anecdotal evidence lines up with the empirical results), I think I’d say something similar about 10-90% being range of “reasonable” probabilities even if we use percentage-point based language.

Linch 31 Jul 2021 10:20 UTC
3 points
in reply to: Charles He’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
Hmm yeah maybe(?) he just misspoke. I do think “clean meat” usually refers to in vitro lab grown meat rather than “all meat alternatives”, both within EA and more broadly, so if clean meat was a standin for PBM I’d stand by my assertion “may be easy to misinterpret for the typical reader”

FWIW I looked into PBM much less than clean meat but I would guess it would be overconfident to assume that replacing all (or most) slaughter-based meat via scaling up existing systems is inevitable and I would guess progress is at least somewhat amenable to philanthropic funding, though not necessarily on parity with top farmed animal welfare interventions like corporate campaigns.
Btw, I too find myself confused about this point by Benjamin_Todd and also am not sure exactly what’s going on here.
On the other hand, many animal welfare non-profits still seem more likely to say that if they had more funding, they’d hire a bunch more people, and salaries also seem fairly low. I’m not sure exactly what’s going on there.

Linch 31 Jul 2021 3:12 UTC
10 points
in reply to: Benjamin_Todd’s comment on: Is effective altruism growing? An update on the stock of funding vs. people
(For context I work in an org where currently >50% of researchers do subcause prioritization within animal welfare, though I’m not an animal welfare researcher myself. Speaking only for myself, just one person’s take, etc, etc)
Some quick personal thoughts on animal welfare funding vs talent constraints:
Naively I would guess that the median research hire in animal welfare for RP would contribute >$1 million/year of counterfactual value solely in terms of improving the quality and quantity of grantmaker decisions within animal welfare. For example, I would naively ascribe somewhat higher numbers for Neil’s EU work, or if quality of the moral weights work is improved by the equivalent of additional thinking of a median researcher-year.

(Note that this is fairly BOTEC and there are obvious biases for someone to think that their work and that of their coworkers is especially important).
Moreover, many animal welfare people are now focused on clean meat, which often doesn’t need philanthropic funding in the first place.
I think this is wrong or at least may be easy to misinterpret for the typical reader. The biggest bottleneck I see for clean meat is strategic clarity along the lines of “can we honestly and accurately have a coherent roadmap, enough to persuade funders and others that the research we’re currently doing is on the pathway to eventually making cost-competitive clean meat that will have widespread regulatory and consumer adoption” (A coherent and technically sound response to Humbird 2020 is necessary but not sufficient here).

But if you’re convinced that current work in clean meat is on the path to producing clean meat with the relevant desiderata, then in those worlds clean meat is much more bottlenecked by funding than by technical talent (compared to say research of malaria vaccines or universal viral sequencing). As a sanity check, all the cultured meat research in academia probably looks like <$10million/year(?), and maybe another !1.5 orders of magnitude more in industry, well over an order of magnitude less than the valuation of a single plant-based startup alone.

So I’d expect in worlds where clean meat research is tractable, I’d expect philanthropic funding to be significant in improving the field, eg by drawing people away from plant-based meats, biopharma, yeast germline manufacture, etc. I suspect (though I don’t have direct evidence) that many people in the latter two categories would take 30% pay cuts to work in something that’s technically interesting and with high potential altruistic value like clean meat, but not >75% pay cuts.

Linch 26 Jul 2021 9:59 UTC
2 points
in reply to: Arepo’s comment on: Part 3: Comparing agency organisational models
I might have misunderstood, but wouldn’t the same apply to the same customers applying for funding for tech work?
I think this applies somewhat for newer orgs and for very discerning/active donors. But my guess is not as much for more established orgs and busier/less discerning funders, because a funder can in theory look at the total inputs and outputs of an EA org and consider either average and marginal cost-benefit when deciding whether to fund them, rather than do sophisticated accounting on specific purchase decisions. So at least in a retrospective evaluation, rather than a line item consideration of whether past donations specifically spent on an improved website, better HR software, external recruiters, higher quality toilet paper etc, is worth it, I can just look at an org’s reported outputs and make a broad judgment of whether it’s worth the money (or other capital) that went in, and then further decide whether I trust the org’s leadership to make good decisions with extra $s. This means that customer EA orgs may be specifically mistaken about funding software work, but unless they make up for it in efficiency gains elsewhere, the total relatively low efficiency will become obvious eventually, and thus customer orgs are incentivized to maximize cost-benefits across the org.

Whereas if the EA org made a lot of use of high-quality labor implicitly or explicitly paid for elsewhere in the movement, this is harder to trace/account for as a funder.

Linch 26 Jul 2021 8:08 UTC
10 points
on: Part 1: EA tech work is inefficiently allocated & bad for technical career capital
I really like this idea! I’m particularly excited about having a new agency to do tech work for a fairly simple subset of the arguments you’ve mentioned:
1. The unmet demand for the work is there
  1. There appears to be a lot of high-EV software work that is currently undone within EA.
  2. There are probably many more software work that we haven’t identified because of not having the capacity to do them.
2. There are many talented software engineers within our movement, many of whom say they want to do direct work “someday.”
3. There appears to be a fair amount of money in our movement.
4. And yet the work is not done
5. Given that there’s a burning need in the market, it would be helpful to fill it.* An independent agency seems like a promising candidate to solve this “matching market” issue.
Personally, I find your arguments for having a independent tech agency and the advantages of an EA tech agency to be fairly compelling, and the argument for the agency to be primarily donor-funded rather than primarily consultant fee-funded or a hybrid system to be much weaker.
Like naively I’d have a picture of the initial tech agency to look like analogous to that of a VC funded startup where initial customers (EA orgs) are subsidized by VC (EA donor) money, until the tech agency “finds its feet” and can do a combination of being paid for services and funded for specific projects. (I gave more specific reasons for doubt in the relevant sections).
But nonetheless, don’t let my naysaying dissuade you! Given that the most likely route to making this tech agency a reality is through you (co)founding it, I think it’s much more important that the founder believes in the details of the product and the organization rather than a random Forum commentator does!
*A point that I’ve made privately a few times and should probably make into a full post is that a rational profit-maximizing firm should always increase investment until marginal cost equals marginal revenue, and the EA talent pipeline is nowhere near that.

Linch 26 Jul 2021 6:49 UTC
3 points
on: Part 3: Comparing agency organisational models
Re:
Incentives and feedback mechanisms
I’ve thought about this much less than you so take everything I say with a huge grain of salt, but I think I’m less worried about incentives and feedback mechanisms for the developers and more worried about incentives and feedback mechanisms for the “customers.” As you noted, the “something for nothing” angle is pretty perverse, but the real theoretical issue to me isn’t the problem with providing feedback on developers (which a customer realistically would give about things like features, UI, timeliness, and as you noted very rarely technical quality concerns), but having frank assessments of how valuable projects actually are, both ex ante and ex post.

By default if you’re offering free, high-quality labor to people, people are incentivized to exaggerate the value of their projects. Of course everybody batting for “team EA” lowers the incentive somewhat, but I would guess not enough.

Even if we imagine that this incentive does not exist (eg. because EA orgs are by stipulation always honest and deontologically opposed to exaggeration or motivated reasoning), I’d still expect having only goodwill and your own judgment (as opposed to goodwill, your own judgment, and $s) as the only check for impact to apply a strong selection effect on your customers, such that the customers who systematically overestimate the impact of (free) tech to their work will crowd out customers with more legitimate uses.
I suspect this issue to continue to exist even with paid EA work, as long as the value of your work exceeds the cost, but at least there is some sanity check on how skewed the incentives can get.

Linch 26 Jul 2021 6:23 UTC
2 points
on: Part 3: Comparing agency organisational models
More personal engagement with projects: Working on a project which you helped select as one of the highest value in the whole space seems extremely rewarding. Working on a project with an extra degree of removal—ie one that an overall high value org thinks is the highest value of their projects—seems much less so. (more on the possible tradeoffs of developer autonomy below)
I don’t understand this point. Are you assuming that the low-bono model is one where the developer has no input into which projects they work on? (eg because the company overall will always prioritize whichever orgs will pay them the most, or whichever org has the highest status). Naively it seems like having clients (as opposed to donors) should not limit developer autonomy much until the demand for tech work equals supply (which as you’ve illustrated will take a long time).

Linch 26 Jul 2021 5:55 UTC
2 points
in reply to: Miranda_Zhang’s comment on: Linch’s Shortform
Hello fellow Zhang!
Thanks! I don’t see nearly as much perfectionism (like none of the lyrics I can think of talks about rewriting things over and over), but I do think there’s an important element of obsession to Hamilton/Non-Stop, which I relate pretty hard to. Since I generate a lot of my expected impact from writing, and it’s quite hard to predict which of my ideas are the most useful/promising in advance, I do sometimes feel a bunch of internal pressure to write/think faster, produce more, etc, like a bit of a race against the clock to produce as much as I can (while maintaining quality standards).

So in the context of this song, I especially admire people (including the character Hamilton, some journalists, some bloggers, and coworkers) who manage to produce a lot of output without (much) sacrificing quality.

Linch 25 Jul 2021 9:23 UTC
5 points
in reply to: Arepo’s comment on: A Sequence Against Strong Longtermism
You can imagine that OP has limited opportunities or interest or time to improve, and can only focus on one thing. In that case I’d strongly encourage focusing on higher quality arguments over better style, as I usually find the lack of the former much more offputting than the latter.

Linch 25 Jul 2021 1:55 UTC
7 points
in reply to: Arepo’s comment on: A Sequence Against Strong Longtermism
For what it’s worth, I have the opposite reaction, and between the OP having higher quality arguments and less snark, would strongly prefer higher quality arguments.

Linch 25 Jul 2021 1:53 UTC
9 points
on: Narration: The case against “EA cause areas”
I have a mild preference for narrations to not show up on the front-page of the EA Forum, and instead eg be comments to the relevant posts, or bunded up together in a long intro sequence post each time.

I don’t know how unusual this preference is (eg I’m also like maybe in the fifth percentile of EAs for how many podcasts I listen to, for example)

Linch 25 Jul 2021 0:54 UTC
7 points
on: Linch’s Shortform
I know this is a really mainstream opinion, but I recently watched a recording of the musical Hamilton and I really liked it.

I think Hamilton (the character, not the historical figure which I know very little about) has many key flaws (most notably selfishness, pride, and misogyny(?)) but also virtues/attitudes that are useful to emulate.

I especially found the Non-stop song(lyrics) highly relatable/aspirational, at least for a subset of EA research that looks more like “reading lots and synthesize many thoughts quickly” and less like “think very deeply about a narrow research topic and come up with a fleeting novel insight.” Especially the refrain:
Why do you write like you’re running out of time?
Write day and night like you’re running out of time?
Every day you fight, like you’re running out of time
Keep on fighting, in the meantime-
And
Why do you always say what you believe?
Why do you always say what you believe?
Every proclamation guarantees
Free ammunition for your enemies (Awww!)
Why do you write like it’s going out of style? (Hey)
Write day and night like it’s going out of style? (Hey)
Every day you fight like it’s going out of style
Do what you do
And
How do you write like you’re running out of time? (Running out of time?)
Write day and night like you’re running out of time? (Running out of time?)
Every day you fight, like you’re running out of time
Like you’re running out of time
Are you running out of time? Awwww!
How do you write like tomorrow won’t arrive?
How do you write like you need it to survive?
How do you write every second you’re alive?
Every second you’re alive? Every second you’re alive?

Linch 25 Jul 2021 0:41 UTC
3 points
in reply to: eca’s comment on: Open Philanthropy is seeking proposals for outreach projects
I thought Decision Problem: Paperclips introduced a subset of AI risk arguments fairly well in gamified form, but I’m not aware of anybody where the game made them become interested enough in AGI alignment/risk/safety enough to work on it. Does anybody else on this forum have data/anecdata?

Linch 24 Jul 2021 22:53 UTC
8 points
on: Research into people’s willingness to change cause *areas*?
Do we have strong evidence that “average donors” even have “cause areas,” as an accurate/descriptively useful mapping of how they understand the world? My young and pre-EA self feels so distant from me that it’s barely worth mentioning, but I vaguely recall that teenage me donated to things as disparate as earthquake relief in Sichuan, local beggars, LGBT stuff and probably something something climate change.

I don’t think I ever consciously considered until several years later how dumb it was to a) donate to multiple things at the tiny amounts I was donating at the time and b) to have multiple cause areas of very varying cost-effectiveness and theories of change.