Joseph Bloom

• Joseph BloomSep 5, 2022, 7:42 AM

  1 point

  0 ∶ 0

  in reply to: freedomandutility’s comment on: EAs un­der­es­ti­mate un­cer­tainty in cause prioritisation

  Sorry for the slow reply.

  Talking about allocation of EA’s to cause areas.

  I agree that confidence intervals between x-risks are more likely to overlap. I haven’t really looked into super-volcanoes or asteroids and I think that’s because what I know about them currently doesn’t lead me to believe they’re worth working on over AI or Biosecurity.

  Possibly, a suitable algorithm would be to defer to/​check with prominent EA organisations like 80k to see if they are allocating 1 in every 100 or every 1000 EAs to rare but possibly important x-risks. Without a coordinated effort by a central body, I don’t see how you’d calibrate adequately (use a random number generator and if the number is less than some number, work on a neglected but possibly important cause?).

  My thoughts on EA allocation to cause areas have evolved quite a bit recently (partly due to talking 80k and others, mainly in biosecurity). I’ll probably write a post with my thoughts, but the bottom line is that, basically, the sentiment expressed here is correct and that it’s easier socially to have humility in the form of saying you have high uncertainty.
  
  Responding to the spirit of the original post, my general sense is that plenty of people are not highly uncertain about AI-related x-risk—you might have gotten that email from 80k titled “A huge update to our problem profile — why we care so much about AI risk”. That being said, they’re still using phrases like “we’re very uncertain”. Maybe the lack of uncertainty about some relevant facts is lower than their decision rule. For example, in the problem profile, they write:

  Overall, our current take is that AI development poses a bigger threat to humanity’s long-term flourishing than any other issue we know of.

  Different Views under Near-Termism

  If you don’t buy longtermism, you probably still care about x-risks, but your rejection of longtermism massively affects the relative importance of x-risks compared to nearterm problems, which affects cause prioritisation.

  This seems tempting to believe, but I think we should substantiate it. What current x-risks are not ranked higher than non-x-risks (or how much less of a lead do they have) relative to non-x-risks causes from a near-term perspective?

  I think this post proposes a somewhat detailed summary of how your views may change under a transformation from long-termist to near-termist. Scott says:

  Does Long-Termism Ever Come Up With Different Conclusions Than Thoughtful Short-Termism?

  I think yes, but pretty rarely, in ways that rarely affect real practice.

  His arguments here are convincing because I find an AGI event this century likely. If you didn’t, then you would disagree. Still, I think that even were AI not to have short timelines, other existential risks like engineered pandemics, super-volcanoes or asteroids might have milder only catastrophic variations, which near-termists would equally prioritise, leading to little practical variation in what people work on.

  Talking about different cultures and EA

  Similarly, I don’t expect diversity of thought to introduce entirely new causes to EA or lead to current causes being entirely abandoned, but I do expect it to affect cause prioritisation.

  I don’t entirely understand what East Asian cultures mean by balance /​ harmony so can’t tell how it would affect cause prioritisation, I just think there would be an effect.

  Can you reason out how “there would be an effect”?

• Joseph BloomAug 27, 2022, 11:00 PM

  3 points

  0 ∶ 0

  on: New Pod­cast: X-Risk Upskill

  Hi there,

  Interesting idea. I think there’s a lot of possible commentary or answers, so I’ll provide some quick thoughts based on ~1 month of reading/​upskilling in AI/​Bio-related X-risk, which I began since I received an FTX Future Fund Regrant.

  Does anyone have any advice before I start this project?

  Do experiments. These can inform your estimates of how valuable a podcast would be to others, how useful it would be to you and how much effort it would require. This post is a great experiment also, so kudos!

  In particular, are there any resources you recommend for teaching myself about machine learning, genomics, or politics?

  There are lots of different materials online for learning about these general topics. I would highly suggest you start with having a thorough understanding of the relevant x-risk cause areas without getting into technical details first, followed by learning about these technical topics if/​when they appear to be most appropriate.

  I’m interested in whether this particular piece of advice in the previous paragraph is contentious (with the other perspective being “go learn lots of general skills before getting more context on x-risks”). Still, I think that might be a costly approach involving spending lots of time learning extraneous detail with no apparent payoff.

  For AI:

  I think the best place to start is the Cambridge AGI Safety Fundamentals course (which has technical and governance variations). You don’t need a lot of Deep Learning expertise to do the course, and the materials are available online until they run one.

  For Bio:

  Tessa curated A Biosecurity and Biorisk Reading+ List, which covers several domains, including genomics.

  And are there any hidden risks I’m not considering that might make this idea worse than it seems?

  Other than not achieving your goals, or being costly, mitigated by starting small and doing experiments, the most significant potential risk is some information hazard. If you focus on pre-requisite skills, then info hazards might be less likely. There are dangers in being too careful around info hazards, so maybe the best action is to share podcasts with a small group of info hazard-aware community members first to check.

  Good luck! And please feel free to reach out if you’d like to discuss this further.

[Question] Are EAs in­ter­ested in us­ing or con­sum­ing more di­a­grams or data vi­su­al­i­sa­tions?

• Joseph BloomAug 24, 2022, 10:48 AM

  1 point

  0 ∶ 0

  in reply to: freedomandutility’s comment on: EAs un­der­es­ti­mate un­cer­tainty in cause prioritisation

  Thanks for clarifying.

  So I’m an example of someone in that position (I’m trying to work out how to contribute via direct work to a cause area) so I appreciate the opportunity to discuss the topic.

  Upon reflection, maybe the crux of my disagreement here is that I just don’t agree that the uncertainty is wide enough to effect the rankings (except in each tier) or to make the direct-work decision rule robust to personal fit.

  I think that X-risks have non-overlapping confidence intervals with non-x-risks because of the scale of the problem, and I don’t feel like this changes from a near-term perspective. Even small chances of major catastrophic events this century seen to dwarf other problems.

  80k’s second top priority areas are Nuclear security, Climate Change (extreme) and Improving Institutional decision making. For the first two, these seem to be associated with major catastrophe’s (maybe not x-risks) which also might be considered not to overlap with the next set of issues (factory farming/​global health).

  With respect to concerns that demographics might be heavily affecting cause prioritisation, I think it would be helpful to have specific examples of causes you think are under-estimated and the biases associated with them.

  For example, I’ve heard lots of different arguments that x-risks are concerning even if you don’t buy into long-termism. To a similar end, I can’t think of any causes that would be under-valued because of not caring adequately about balance/​harmony.

• Joseph BloomAug 24, 2022, 10:06 AM

  1 point

  0 ∶ 0

  on: Cause: biopharma R&D productivity

  Interesting post, I’m trying to understand it better. I think the cause area sounds good, but I don’t feel confident about the chance that there’s a huge amount of free-energy lying around (thinking in terms of Inadequate Equilibria). I feel like the heart of the argument is that “cultural” shift akin to what helped Space-X succeed could solve similar problems in Biopharma R&D.

  A detailed plan substantiating a trial EA project would help establish the tractability more substantially.

  Some specific points I’d like to clarify:

  1. What exactly are the “emergent properties of complex systems” mentioned in the neglectedness section? It sounds like perverse incentives, unexplained lack of translation of technological progress to profitability and maybe backfiring legislation?

  2. “The seeds of this are most likely to be found within younger, more innovative for-profit companies, so that is where EA should direct its resources.”—Can you point to any examples of such companies not succeeding for lack of funding? I think this could be a really strong point if you can point to examples of Space-X equivalents in biopharma that didn’t get off the ground for a lack of “a tonne of private capital and public subsidies”?

  3. “This essay advocates here a more distributed effort, one befitting the EA community.”—this sounds potentially very time-expensive. If money isn’t the limiting resource, EA community member time/​focus might be. What is your sense for the number of people/​amount of time it would take to test this hypothesis?

  I know a space-X engineer who also attributes culture to their success. I’m willing to accept this as a plausible catalyst for radical progress (although exactly what cultural hallmarks you mean might need to be specified).

• Joseph BloomAug 24, 2022, 4:52 AM

  14 points

  0 ∶ 0

  on: How to dis­close a new x-risk?

  Not a comprehensive answer but a few ideas. I don’t know of any existing documentation or organisation about how to do this.

  1. I think talking to people currently heavily involved in funding x-risk mitigation efforts is a good start. People with a proven track record of taking x-risks seriously are more likely to adequately consider the relevant concerns and assist by progressing the discussion and coming up with meaningful mitigation strategies. For example, you could email Nick Bostrom or someone at Open Philanthropy. I’ve heard Kevin Esvelt is someone with a track record or taking info-hazards seriously too.

  2. Maybe don’t go directly to super critical people in existing efforts. It’s possible that you should qualify your ideas first by talking to other experts (who you trust) in whichever domain is likely to know about those risks (although of course you’d want to avoid losing control of the narrative, such as by someone you tell overzealously raising alarm and damaging your credibility).

  There’s probably lots of specific reasoning that might be necessary based on the relevant risk (for example if it’s tied up with specific economic activity the way AI capabilities development is).

• Joseph BloomAug 24, 2022, 4:42 AM

  1 point

  0 ∶ 0

  on: EAs un­der­es­ti­mate un­cer­tainty in cause prioritisation

  I think if EAs better appreciated uncertainty when prioritising causes, people’s careers would span a wider range of cause areas.

  I’ve got a strong intuition that this is wrong, so I’m trying to think it through.

  To argue that EA’s underestimate uncertainty, you need to directly observe their uncertainty estimates (and have knowledge of the correct level of uncertainty to have). For example, if the community was homogenous and all assigned a 1% chance to Cause X being the most important issue (I’m deliberately trying not to deal with how to measure this) and there was a 99% chance of cause Y being the most important issue, then all individuals would choose to work on Cause Y. If the probabilities were 5% X and 95% you’d get the same outcome. This is because individuals are making single choices.

  Now, if there was a central body coordinating everyone’s efforts, in the first scenario, it still wouldn’t follow that 1% of people would get allocated to cause Y. Optimal allocation strategy aside, there isn’t this clean relationship between uncertainty and decision rules.

  I think 80 000 Hours could emphasise uncertainty more, but also that the EA community as a whole just needs to be more conscious of uncertainty in cause prioritisation.

  I think 80k is already very conscious of this (based on my general sense of 80k materials). Global priorities research is one of their 4 highest priorities areas and it’s precisely about having more confidence about what is the top priority.

  I think something that would help me understand better where you are coming from is to hear more about what you think the decision rules are for most individuals, how they are taking their uncertainty into account and more about precisely how gender/​culture interacts with cause area uncertainty in creating decisions.

• Joseph BloomAug 18, 2022, 7:13 AM

  1 point

  0 ∶ 0

  in reply to: Larks’s comment on: The Sur­vival and Flour­ish­ing Fund grant ap­pli­ca­tions open un­til Au­gust 23rd ($8m-$12m planned for dis­per­sal)

  A bit late here but I was looking into it and found this (https://​​survivalandflourishing.fund/​​s-process):

A pseudo math­e­mat­i­cal for­mu­la­tion of di­rect work choice be­tween two x-risks

• Joseph BloomAug 4, 2022, 4:31 AM

  2 points

  0 ∶ 0

  on: Who wants to be hired? (May-Septem­ber 2022)

  • Location: Melbourne, Australia

  • Remote: Yes

  • Willing to relocate: Yes, preferably UK.

  • Skills: Software Development (R, Python), Computational Biology (Proteomics), DevOps (CI/​CD, AWS, Azure), Data Science, Writing, Research

  • Résumé/​CV/​LinkedIn:

    • Joseph Bloom

    • https://​​www.linkedin.com/​​in/​​joseph-bloom1/​​

    • https://​​www.jbloomaus.com/​​JosephBloomCV.pdf

  • Email: joseph.i.bloom@gmail.com

  • Notes:

    • Cause Area: X-risk.

      • I suspect I might have the most traction likely in Bio (maybe early pandemic detection, could retrain for NGS, or co-found a project needing a bridge between bio and tech stacks).

      • Might also be useful in an alignment org (devOps, MLOps)

    • I’m on an FTX Future Fund Re-grant to find a role/​org. Lots of time to upskill/​retrain. Very willing if there’s a good argument for something adjacent to my skill set that is highly valuable.

    • Education: Bachelors Double degree in Computational Biology and Statistics and Stochastic Processes.

  • Available from and until: ASAP, for the foreseeable future.

  Questions, feedback, advice or suggestions? Feel free to email me or provide anonymously here: https://​​www.admonymous.co/​​jbloomaus.