Larks

Karma: 11,633

Larks 20 Sep 2023 2:45 UTC
3 points
0 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Anthropic’s Responsible Scaling Policy & Long-Term Benefit Trust
Thanks for the explanation!

Larks 20 Sep 2023 2:20 UTC
5 points
0 ∶ 0
on: Anthropic’s Responsible Scaling Policy & Long-Term Benefit Trust
Thanks very much for sharing this!
The Anthropic board chose these initial Trustees after a year-long search and interview process to surface individuals who exhibit thoughtfulness, strong character, and a deep understanding of the risks, benefits, and trajectory of AI and its impacts on society. Trustees serve one-year terms and future Trustees will be elected by a vote of the Trustees.
I am curious about the decision for one-year terms. Given that the board chooses its own successors, and no-one on the board is full-time, I worry that the search for successors will end up occupying a huge fraction of their time. Also, non-overlapping terms might mean a loss of institutional memory. Did you consider a staggered setup?

Larks 18 Sep 2023 4:36 UTC
3 points
3 ∶ 1
on: Policy ideas for mitigating AI risk
Thanks for sharing this Thomas!
I would like to hear your thoughts on the centralisation of power this would cause. Historically it seems like multiple parties owning the means of production, and having wide latitude with how to employ this capital, has been a key driver of human progress and protector of liberty. In the event of TAI I worry that a powerful regulator like this would effectively centralise a huge degree of control over society and the economy in the hands of the regulator. For example, the regulator might adopt a definition of safety that incorporates political notions (e.g. what makes an output ‘toxic’), resulting in only ideologically compliant firms being are allowed to run large models, and hence giving these firms allied to the administration a major advantage over competitors. Note that this is not regulatory capture, and hence I don’t expect conflict of interest rules to resolve it.
Of course if the alternative is literally extinction then perhaps some people’s answer is (to some degree) ‘so be it’.

Larks 18 Sep 2023 3:00 UTC
16 points
2 ∶ 0
on: AI Pause Will Likely Backfire
Thanks very much for writing this very interesting piece!
The “AI safety winter” section argues that pre-2020, AI alignment researchers made little progress because they had no AI to work on aligning. But now that we have GPT-4 etc., I feel like we have a capabilities overhang, and it seems like there is plenty of AI alignment researchers to work on for the next 6 months or so? Then their work could be ‘tested’ by allowing some more algorithmic progress.

Larks 15 Sep 2023 18:17 UTC
7 points
2 ∶ 0
in reply to: alexherwix’s comment on: Radical Longtermism and the Seduction of Endless Growth: A Critique of William MacAskill’s ‘What We Owe the Future’
To me this feels like trying to bury voices one doesn’t want to hear, which may be helpful in the short run (less stress) but is probably not the best long term strategy (less understanding).
Time and attention are finite; I think a lot people people think they have spent a lot more time reading Torres and trying to give him the benefit of the doubt than they have given to almost anyone else, and a lot more than is deserved by the quality of the content.

Larks 15 Sep 2023 12:56 UTC
5 points
2 ∶ 1
on: Who should we interview for The 80,000 Hours Podcast?
Lawrence Newport to discuss his shockingly successful campaign to get the Bully XL dog banned in the UK.

Larks 15 Sep 2023 12:47 UTC
−2 points
0 ∶ 1
in reply to: alexherwix’s comment on: Radical Longtermism and the Seduction of Endless Growth: A Critique of William MacAskill’s ‘What We Owe the Future’
I didn’t downvote, but this article definitely would have warranted a downvote if it had been posted directly to the forum; if you think there are a few redeeming sections you should probably highlight them directly rather than asking people to sort through it all.

Larks 14 Sep 2023 20:32 UTC
2 points
0 ∶ 0
on: Is EA a well-intentioned version of ‘survival of the fittest’ (or largest)?
So to give a concrete example: less than 1% of the population worldwide have type 1 diabetes. Maybe your money would be better spent on type 2 diabetes, with 8% of the world’s population having that. Does this mean those with type 1 are unimportant or unworthy of funding?
Assuming it was equally tractable to make progress on both types of diabetes, and both currently received the same amount of funding, EAs would tend to say you should favour type 2 research. If enough people started to fund type 2, the cost-effectiveness of marginal dollars there would likely fall (because the low-hanging fruit were already picked), so work on type 1 would become more attractive on the margin. So basically, assuming equal tractability, if EA would likely target most but probably not all dollars be focused on the more numerous condition, and each individual should donate so as to help move the world towards this optimal equilibrium.
I also wouldn’t consider people with type 1 to be in any way morally inferior to those with type 2. There are just less of them, and all the extra people in the larger group matter.

Larks 14 Sep 2023 13:48 UTC
2 points
0 ∶ 0
in reply to: titotal’s comment on: Summary: High risk, low reward: A challenge to the astronomical value of existential risk mitigation
One paper should be about one thing, not try and debunk the entire x-risk argument all in one go, otherwise it would just become a sprawling, bloated mess. The important conclusion here is that the astronomical risk hypothesis relies on the time of perils hypothesis: previously i think a lot of people thought they were independent.
I think this (just arguing the Astronomical Risk Hypothesis relies on the Time of Perils Hypothesis (TOPH) pessimism, and leaving it at that) would be a very reasonable thing to do. But that’s not what this paper actually does:
- Pages 19 through 32 go through three arguments against TOPH. It’s just that what seems to me to be the strongest argument is relegated to a footnote. In your version of this paper, all these pages could be omitted.
- The conclusion acts as if TOPH has been refuted, rather than merely shown to be an important premise. If TOPH is true then pessimism about TOPH supports the astronomical risk hypothesis.
  - “One implication of this paper is that existential risk pessimism tends to favor the original statement of the demandingness problem. If existential risk is indeed high, then it is relatively less important to mitigate existential risk, and it will be also relatively less important to save resources for the future, since future gains are less likely to be realized. For discussion see Kagan (1984), Mulgan (2001) and Sobel (2007). That means it may indeed be better for consequentialists to direct their resources towards present people. This should be welcome news for the consequentialist, since it raises the possibility of avoiding a strengthened form of the demandingness objection to consequentialism.”
  - “We have seen that existential risk pessimism strongly reduces the value of existential risk mitigation.”
  - “But in general, the models of this paper suggest that existential risk mitigation may not be as important as many pessimists have taken it to be, and crucially that pessimism is a hindrance rather than a support to the case for existential risk mitigation. The case for existential risk mitigation is strongest under more optimistic assumptions about existential risk.”
- See also the abstract:
  - “It is natural to think that existential risk pessimism supports the astronomical value thesis. In this paper, I argue that precisely the opposite is true.”
- Or the conclusion from this post:
  - Thorstad concludes that it seems unlikely that we live in the time of perils. This implies that reducing existential risk is probably not overwhelmingly valuable and that the case for reducing existential risk is strongest when the risk is low.

Larks 14 Sep 2023 1:26 UTC
19 points
3 ∶ 1
on: Summary: High risk, low reward: A challenge to the astronomical value of existential risk mitigation
A much more powerful assumption is needed (one that combines all of these weaker assumptions).
Could you explain why the Time of Perils assumption is stronger? It seems to me to be consistent with rejecting the previous assumptions; for example, you could navigate the Time of Perils but have no impact on future years, especially if the risk there was already very low. Rather than being stronger, it just seems like a different model to me.
I was also disappointed to not see AGI discussed in relation to the Time of Perils. The only mention is this footnote:
One argument which I will not address is that the development of artificial intelligence
may bring an end to the time of perils, for example by putting human civilization under the
control of a single entity capable of managing existential risks (Bostrom 2014). The
response to this argument turns on a number of conceptual and empirical questions
surrounding artificial intelligence that are difficult to address in the space of a paper.
This seems very unsatisfactory to me, because as far as I can see (perhaps I am mistaken) AGI is the main reason most people believe in the Time of Perils hypothesis. Declaring it to be out of scope doesn’t mean you have struck a blow against Xrisk mitigation, it means you have erected a strawman.
On a much more mundane note, I found this paragraph very confusing:
At this point, the most helpful response would be to ask Ord for more details. We are
not told much about why we should expect humanity to grow in virtue or how this growth
could lead to a quick and substantial drop in existential risk. Without these details, it is
hard to place much stock in the appeal to civilizational virtue. But we may get some handle
on the prospects for Ord’s argument by thinking through some particular civilizational
virtues.
Doesn’t Toby work in the same office as GPI? If the most helpful thing to do would be to ask him for details… why not do that?

Larks 13 Sep 2023 1:03 UTC
6 points
2 ∶ 0
in reply to: Gemma Paterson’s comment on: John Green & Grassroots Activists Pressure Danaher To Drop Price of Tuberculosis Test
I agree that AMCs seem like a good idea!

Larks 12 Sep 2023 19:34 UTC
21 points
8 ∶ 1
in reply to: Gemma Paterson’s comment on: John Green & Grassroots Activists Pressure Danaher To Drop Price of Tuberculosis Test
When deciding whether to invest in something, there is always uncertainty about the outcome. If ex post successes will be met with demands for reduced prices, but ex post failures do not get compensation, the ex ante expected value is reduced and firms are less likely to invest, even if it’s still worthwhile for firms that have already invested to continue operating.
Note that the tax system is better designed than this, because unsuccessful ventures generate tax loses which are (roughly) symmetrical with the taxes paid on successful ventures.
The campaign is advocating that they reduce their profit margin which given the potential sales volumes will still likely make this a profitable outcome. I am pretty unconvinced that it would serious decision relevant factor.
I think it’s pretty clear that prices falling by 50% or 67%, and gross profits falling by 71-87%, is going to be an important decision factor for various things.

Larks 12 Sep 2023 17:59 UTC
37 points
8 ∶ 7
on: John Green & Grassroots Activists Pressure Danaher To Drop Price of Tuberculosis Test
This seems potentially pretty bad to me for Copenhagen Interpretation of Ethics reasons. Based on your description, it sounds like Danaher made a better product than anyone else and were already doing more (in this specific area) to help the global poor than anyone else. But rather than being praised them for this, they receive harsh criticism. Not only does this seem very unfair, I think it also contributes to the effect where people actively try to avoid dealing with poor people because they know doing so invites opprobrium, treatment they would avoid by making products exclusively for the well off.

Larks 11 Sep 2023 17:35 UTC
15 points
7 ∶ 0
in reply to: Ivan Burduk’s comment on: Ivan Burduk’s Quick takes
It’s cool they are doing this vote, but the actual process of voting is surprisingly painful! A 5 question text-response quiz for each vote, including asking for your email.

Larks 11 Sep 2023 16:51 UTC
−3 points
0 ∶ 0
in reply to: Nathan Young’s comment on: Nathan Young’s Shortform
This sounds like the sort of question you should email Richard to ask before you make blanket accusations.

Larks 10 Sep 2023 17:50 UTC
16 points
0 ∶ 0
in reply to: Nathan Young’s comment on: Nathan Young’s Shortform
Seems worth asking why we didn’t listen to top policy writers when they warned that we didn’t have good proposals.
What do you think of Thomas Larson’s bill? It seems pretty concrete to me, do you just think it is not good?

Larks 10 Sep 2023 17:39 UTC
34 points
17 ∶ 0
in reply to: Morpheus_Trinity’s comment on: Sharing Information About Nonlinear
is there not ample cause for the temporary suspension of individuals such as Emerson and Kat from participation in … forum activities
No, because if nothing else you need to give them time to respond.

Larks 9 Sep 2023 3:04 UTC
7 points
0 ∶ 0
in reply to: Joseph Lemien’s comment on: Joseph Lemien’s Shortform
I assumed it was functioning as a compound noun rather than a possessive. The word ‘Founders’ is modifying the type of Pledge, not claiming ownership of it.

Larks 9 Sep 2023 2:59 UTC
6 points
1 ∶ 0
on: Debate series: should we push for a pause on the development of AI?
Great idea, thanks for organizing!

Larks 8 Sep 2023 17:30 UTC
3 points
1 ∶ 0
in reply to: Rockwell’s comment on: Sharing Information About Nonlinear
I would have expected that becoming an employee in order to record videos and publish them would be a violation of the employment contracts. Certainly in my industry (finance) employees are not allowed to record sensitive information and publish it (though they can whistle-blow to the regulator if they want). Is there some rule or principle that makes it legal to break confidentiality clauses in these cases, or do farms just not bother to include them in contracts?
Additionally, my impression was that it often was illegal to fly drones over private property without permission, especially if you are invading privacy.