Derek Shiller

Karma: 996

Derek Shiller 22 Apr 2024 15:15 UTC
14 points
9 ∶ 0
in reply to: Henry Howard’s comment on: Announcing The New York Declaration on Animal Consciousness
I think it is valuable to have this stuff on record. If it isn’t recorded anywhere, then anyone who wants to reference this position in another academic work—even if it is the consensus within a field—is left presenting it in a way that makes it look like their personal opinion.

Derek Shiller 23 Feb 2024 15:19 UTC
5 points
0 ∶ 0
in reply to: calebp’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
Thanks for recording these thoughts!

Here are a few responses to the criticisms.

I think RP underrates the extent to which their default values will end up being the defaults for model users (particularly some of the users they most want to influence)

This is a fair criticism: we started this project with the plan of providing somewhat authoritative numbers but discovered this to be more difficult than we initially expected and instead opted to express significant skepticism about the default choices. Where there was controversy (for instance, in how many years forward we should look), we opted for middle-of-the-road choices. I agree that it would add a lot of value to get reasonable and well-thought-out defaults. Maybe the best way to approach controversy would be to opt for different sets of parameter defaults that users could toggle between based on what different people in the community think.

I found it difficult to provide very large numbers on future population per star—I think with current rates of economic and compute growth, the number of digital people could be extremely high very quickly.

The ability to try to represent digital people with populations per star was a last-minute choice. We originally just aimed for that parameter to represent human populations. (It isn’t even completely obvious to me that stars are the limiting factor on the number of digital people.) However, I also think these things don’t matter since the main aim of the project isn’t really affected by exactly how valuable x-risk projects are in expectation. If you think there may be large populations, the model is going to imply incredibly high rates of return on extinction risk work. Whether those are the obvious choice or not depends not on exactly how high the return, but on how you feel about the risk, and the risks won’t change with massively higher populations.

I think some x-risk interventions could plausibly have very long run effects on x-risk (e.g. by building an aligned super intelligence)

If you think we’ll likely have an aligned super-intelligence within 100 years, then you might try to model this by setting risks very low after the next century and treating your project as just a small boost on its eventual discovery. However, you might not think that either superaligned AI or extinction is inevitable. One thing we don’t try to do is model trajectory changes, and those seem potentially hugely significant, but also rather difficult to model with any degree of confidence.

The x-risk model seems to confuse existential risk and extinction risk (medium confidence—maybe this was explained somewhere, and I missed it)

We distinguish extinction risk from risks of sub-extinction catastrophes, but we don’t model any kind of as-bad-as-extinction risks.

Derek Shiller 10 Feb 2024 14:04 UTC
13 points
4 ∶ 0
on: How much pressure do you feel against externally expressing views which do not conform to those of your manager or organisation?
There is some nuance to the case that seems to get overlooked in the poll. I feel completely free to express opinions in a personal capacity that might be at odds with my employer, but I also feel that there are some things it would be inappropriate to say while carrying out my job without running it by them first. It seems like you’re interested in the latter feeling, but the poll is naturally interpreted as addressing the former.

Derek Shiller 8 Feb 2024 15:39 UTC
5 points
2 ∶ 1
in reply to: Remmelt’s comment on: Why I think it’s net harmful to do technical safety research at AGI labs
I think I agree that safety researchers should prefer not to take a purely ceremonial role at a big company if they have other good options, but I’m hesitant to conclude that no one should be willing to do it. I don’t think it is remotely obvious that safety research at big companies is ceremonial.

There are a few reasons why some people might opt for a ceremonial role:
1. It is good for some AI safety researchers to have access to what is going on at top labs, even if they can’t do anything about it. They can at least keep tabs on it and can use that experience later in their careers.
2. It seems bad to isolate capabilities researchers from safety concerns. I bet capabilities researchers would take safety concerns more seriously if they eat lunch every day with someone who is worried than if they only talk to each other.
3. If labs do engage in behavior that is flagrantly reckless, employees can act as whistleblowers. Non-employees can’t. Even if they can’t prevent a disaster, they can create a paper trail of internal concerns which could be valuable in the future.
4. Internal politics might change and it seems better to have people in place already thinking about these things.

Derek Shiller 7 Feb 2024 13:40 UTC
37 points
7 ∶ 0
on: Why I think it’s net harmful to do technical safety research at AGI labs
Do you think it would be better if no one who worked at OpenAI / Anthropic / Deepmind worked on safety? If those organizations devoted less of their budget to safety? (Or do you think we should want them to hire for those roles, but hire less capable or less worried people, so individuals should avoid potentially increasing the pool of talent from which they can hire?)

Derek Shiller 16 Jan 2024 14:37 UTC
3 points
1 ∶ 0
in reply to: NunoSempere’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview

Ditto for the “AI Misalignment Megaproject”: $8B+ expenditure to only have a 3% chance of success (?!), plus some other misc discounting factors. Seems like you could do better with $8B.

I think we’re somewhat bearish on the ability of money by itself to solve problems. The technical issues around alignment appear quite challenging, especially given the pace of development, so it isn’t clear that any amount of money will be able to solve them. If the issues are too easy on the other hand, then your investment of money is unlikely to be needed and so your expenditure isn’t going to reduce extinction risk.

Even if the technical issues are in the goldilocks spot of being solvable but not trivially so, the political challenges around getting those solutions adopted seem extremely daunting. There is a lot we don’t explicitly specify in these parameter settings: if the money is coming from a random billionaire unaffiliated with AI scene then it might be harder to get expertise and buy in then if it is coming from insiders or the federal government.

All that said, it is plausible to me that we should have a somewhat higher chance of having an impact coupled with a lower chance of a positive outcome. A few billion dollars is likely to shake things up even if the outcome isn’t what we hoped for.

Derek Shiller 16 Jan 2024 14:15 UTC
5 points
0 ∶ 1
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
I think you should put very little trust in the default parameters of the projects. It was our initial intention to create defaults that reflected the best evidence and expert opinion, but we had difficulty getting consensus on what these values should be and decided instead to explicitly stand back from the defaults. The parameter settings are adjustable to suit your views, and we encourage people to think about what those parameter settings should be and not take the defaults too seriously.

For readers’ context, AI safety technical research is 80,000 Hours’ top career path, whereas one could argue extinction risk from natural disasters is astronomically low.

The parameters allow you to control how far into the future you look and the outcomes include not just effects on the long-term future from the extinction / preservation of the species but also on the probabilities of near-term catastrophes that cause large numbers of death but don’t cause extinction. Depending on your settings, near-term catastrophes can dominate the expected value. For the default settings for natural disasters and bio-risk, much of the value of mitigation work (at least over the next 1000 years) comes from prevention of relatively small-scale disasters. I don’t see anything obviously wrong about this result and I expect that 80K’s outlook is based on a willingness to consider effects more than 1000 years in the future.

Derek Shiller 8 Jan 2024 20:04 UTC
3 points
0 ∶ 0
in reply to: david_reinstein’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
Thanks for your impressions. I think your concerns largely align with ours. The model should definitely be interpreted with caution, not just because of the correlations it leaves out, but because of the uncertainty with the inputs. For the things that the model leaves out, you’ve got to adjust its verdicts. I think that this is still very useful because it gives us a better baseline to update from.

As for where we get inputs from, Marcus might have more to say. However, I can speak to the history of the app. Previously, we were using a standard percentage improvement, e.g. a 10% increase in DALYs averted per $. Switching to allowing users to choose a specific target effectiveness number gave us more flexibility. I am not sure what made us think that the percentages we had previously set were reasonable, but I suspect it came from experience with similar projects.

Derek Shiller 23 Dec 2023 16:33 UTC
7 points
1 ∶ 1
on: AI Girlfriends Won’t Matter Much
I think that the real effect of chatbots won’t be better access to tailored porn, but rather constant artificial companionship. It won’t be AI girlfriends that are the threat, but AI friends. AI could be set up to be funnier, more loyal, more empathetic, and more available than human friends / partners. This seems like it could have significant effects on human psychology, for both the better and worse.

Derek Shiller 6 Dec 2023 19:32 UTC
3 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
I appreciate your attention to these details!

These values that we included in the CCM for these interventions should probably be treated as approximate and only accurate to roughly an order of magnitude. These actual numbers may be a bit dated and probably don’t fully reflect current thinking about the marginal value of GHD interventions. I’ll talk with the team about whether they should be updated, but note that this wasn’t a deliberate re-evaluation of past work.

That said, it important to keep in mind that there are disagreements about what different kinds of effects are worth, such as Open Philanthropy’s reassessment of cash transfers (to which both they and GiveWell pin their effectiveness evaluations). We can’t directly compare OP’s self-professed bar with GiveWell’s self-professed bar as if the units are interchangeable. This is a complexity that is not well represented in the CCM. The Worldview Investigations team has not tried to adjudicate such disagreements over GHD interventions.

Derek Shiller 5 Dec 2023 17:33 UTC
3 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
That is a good idea. We’ve considered similar ideas in the past. At present, the default parameters reflect best guesses of members of the team, but the process to generate them wasn’t always principled or systematic. I’d like to spend more time thinking about what these defaults should be and to provide public justifications for them. For the moment, you shouldn’t treat these values as authoritative.

Derek Shiller 30 Nov 2023 16:27 UTC
5 points
2 ∶ 0
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview

Is there any reason for you having decided to go for non-null probabilities of the interventions having no effect?

A zero effect reflects no difference in the value targeted by the intervention. For xrisk interventions, this means that no disaster was averted (even if the probability was changed). For animal welfare interventions, the welfare wasn’t changed by the intervention. Each intervention will have side effects that do matter, but those side effects will be hard to predict or occur on a much smaller scale. Non-profits pay salaries. Projects represent missed opportunity costs. Etc. Including them would add noise without meaningfully changing the results. We could use some scheme to flesh out these marginal effects, as you suggest, but it would take some care to figure out how to do so in a way that wasn’t arbitrary and potentially misleading. Do you see ways for this sort of change to be decision relevant?

It is also worth noting that assigning a large number of results to a single exact value makes certain computational shortcuts possible. More fine-grained assessments would only be feasible with fewer samples.

Less importanty, I also think the negative part of the effects distribution may have a different shape than the positive part. So the model would ideally allow one to specify not only the probability of the intervention being negative, but also the effects distribution conditional on the effect being negative (in the same way one can for the positive part).

Fair point. I agree that having separate settings would be more realistic. I’m not sure whether it would make a significant difference to the results to have the ability to set different shapes of positive and negative distributions, given the way these effects are sampled for an all-or-nothing verdict on whether the intervention makes a difference. However, there are costs to greater configurability, and we opted for less configurability here. Though I could see a reasonable person having gone the other way.

Derek Shiller 27 Nov 2023 13:21 UTC
3 points
1 ∶ 0
in reply to: Mo Putera’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
Based on the numbers, I’m guessing that this is a bug in which we’re showing the median DALYs averted per $1000 but describing it as the median cost per DALY. We’re planning to get rid of the cost per DALY averted and just stick with DALYs per $1000 to avoid future confusions.

Derek Shiller 22 Nov 2023 19:35 UTC
5 points
0 ∶ 0
in reply to: BrownHairedEevee’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
Thanks for this insightful comment. We’ve focused on capturing the sorts of value traditionally ascribed to each kind of intervention. For existential risk mitigation, this is additional life years lived. For animal welfare interventions, this is suffering averted. You’re right that there are surely other effects of these interventions. Existential risk mitigation and ghd interventions will have an effect on animals, for instance. Animal welfare interventions might contribute to moral circle expansion. Including these side effects is not just difficult, it adds a significant amount of uncertainty. The side effects we choose to model may determine the ultimate value we get out. The way we choose to model these side effects will add a lot of noise that makes the upshots of the model much more sensitive to our particular choices. This doesn’t mean that we think it’s okay to ignore these possible effects. Instead, we conceive of the model as a starting point for further thought, not a conclusive verdict on relative value assessments.

Similarly, for an animal welfare intervention such as a corporate cage-free campaign, the long-term effects would depend on how long the cage-free policy is expected to last, how well it’s enforced, etc. This would be undoubtedly complicated to model, but would help make these interventions easier to compare with traditionally “longtermist” interventions.

To some extent, these sorts of considerations can be included via existing parameters. There is a parameter to determine how long the intervention’s effects will last. I’ve been thinking of this as the length of time before the same policies would have been adopted, but you might think of this as the time at which companies renege on their commitments. We can also set a range of percentages of the population affected that represents the failure to follow through.

Derek Shiller 18 Nov 2023 17:31 UTC
1 point
0 ∶ 0
in reply to: garymm’s comment on: Notes on the risks and benefits of kidney donation
I don’t have any special insight. I would be surprised if there were aspects of donation that made the surgery especially likely to result in post-operative pain, so I would imagine that the prevalence of post-operative pain in general would give you some clue about how reliable this study is. That said, given what I’ve read, if there were subtle ways in which it significantly reduced quality of life, I wouldn’t be surprised if it wasn’t well publicized. It seems to me a good sign that the doctor mentioned the possibility of post-operative pain to you.

Derek Shiller 10 Nov 2023 20:00 UTC
8 points
1 ∶ 0
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
The issue is that our parameters can lead to different rates of cubic population growth. A 1% difference in the rate of cubic growth can lead to huge differences over 50,000 years. Ultimately, this means that if the right parameter values dictating population are sampled in a situation in which the effect of the intervention is backfires, the intervention might have an average negative value across all the samples. With high enough variance, the average sign will be determined by the sign of the most extreme value. If xrisk mitigation work backfires in ¹⁄₄ of cases, we might expect ¹⁄₄ of collections of samples to have a negative mean.

Derek Shiller 9 Nov 2023 20:37 UTC
3 points
0 ∶ 0
in reply to: Vasco Grilo’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
You’re right! That wasn’t particularly surprising in light of our moral weights. Thanks for clarifying: I did a poor job of separating the confirmations from the surprising results.

Derek Shiller 6 Nov 2023 16:22 UTC
10 points
2 ∶ 0
in reply to: OscarD’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
Thanks for your engagement and these insightful questions.

I consistently get an error message when I try to set the CI to 50% in the OpenPhil bar (and the URL is crazy long!)

That sounds like a bug. Thanks for reporting!

(The URL packs in all the settings, so you can send it to someone else—though I’m not sure this is working on the main page. To do this, it needs to be quite long.)

Why do we have probability distributions over values that are themselves probabilities? I feel like this still just boils down to a single probability in the end.

You’re right, it does. Generally, the aim here is just conceptual clarity. It can be harder to assess the combination of two probability assignments than those assignments individually.

Why do we sometimes use $/ D A L Y a n d s o m e t i m e s D A L Y s /$ ? It seems unnecessarily confusing.

Yeah. It has been a point of confusion within the team too. The reason for cost per DALY is that is a metric that is often used by people making allocation decisions. However, it isn’t a great representation for Monte Carlo simulations where a lot of outcomes involve no effect, because the cost per DALY is effectively infinite. This has some odd implications. For our purposes, DALYs per $1000 is a better representation. To try to accommodate both considerations, we include both values in different places.

OK, but what if life is worse than 0, surely we need a way to represent this as well? My vague memory from the moral weights series was that you assumed valence is symmetric about 0, so perhaps the more sensible unit would be the negative of the value of a fully content life.

The issue here is that interventions can affect different levels of suffering. For instance, a corporate campaign might include multiple asks that affect animals in different ways. We could have made the model more complicated by incorporating its effect on each different level. Instead, we simplified by ‘summarizing’ the impact with one level. We calibrated with research on the impact of similar afflictions in humans. You can represent a negative value just by choosing a higher number of hours than actually suffered. Think of it in terms of the amount of normal life that that suffering would balance out. If it is really bad, one hour of suffering might be as bad as weeks of normal life would be good.

The intervention is assumed to produce between 160 and 3.6K suffering-years per dollar (unweighted) condition on chickens being sentient.” This input seems unhelpfully coarse-grained, as it seems to hide a lot of the interesting steps and doesn’t tell me anything about how these numbers are estimated, and it is not the sort of thing I can intelligently just choose my own numbers for.

There is a balance between accuracy and model configurability. In some places, we want to include numbers that are based on other research that we thought was likely to be accurate, but where we couldn’t directly translate into the parameters of the model. I would like to convert those assessments into the terms of the model, maybe backtracking to see what parameters get a similar answer, but this wasn’t a priority.

In the small-scale biorisk project, I never seem to get more than about 1000 DALYs per $1000, even when I crank expansion speed to 0.9c and length of future to 1e8, and the annual extinction risk in era 4 to 1e-8. Why is this? Yes 150,000 is too few, but I thought I should at least see some large effect when I change key parameters by several OOMs. Not really sure what is going on here, I’ll be interested if you replicate this, and whether there is a bug or I am just misunderstanding something.

Our estimates include both calculations of catastrophic events and extinction. For the small-scale biorisk, the chance of a catastrophic event is relatively high, but the chance of extinction is low. I think you’re seeing the results of catastrophic events and no extinction events. When I up the probability of extinction to be higher, and include the far future, I see very large numbers. (E.g. https://bit.ly/ccm-bio-high-risk).

Derek Shiller 6 Nov 2023 15:32 UTC
10 points
2 ∶ 0
in reply to: OscarD’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
(1) Unfortunately, we didn’t record any predictions beforehand. It would be interesting to compare. That said, the process of constructing the model is instructive in thinking about how to frame the main cruxes, and I’m not sure what questions we would have thought were most important in advance.

(2) Monte Carlo methods have the advantage of flexibility. A direct analytic approach will work until it doesn’t, and then it won’t work at all. Running a lot of simulations is slower and has more variance, but it doesn’t constrain the kind of models you can develop. Models change over time, and we didn’t want to limit ourselves at the outset.

As for whether such an approach would work with the model we ended up with: perhaps, but I think it would have been very complicated. There are some aspects of the model that seem to me like they would be difficult to assess analytically—such as the breakdown of time until extinction across risk eras with and without the intervention, or the distinction between catastrophic and extinction-level risks.

We are currently working on incorporating some more direct approaches into our model where possible in order to make it more efficient.

Derek Shiller 3 Nov 2023 23:00 UTC
12 points
1 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
I believe Marcus and Peter will release something before long discussing how they actually think about prioritization decisions.