JanB

Karma: 984

JanB 24 Sep 2022 7:10 UTC
81 points
39 ∶ 0
on: My Personal Story of Rejection from EA Global
I haven’t read the comments and this has probably been said many times already, but it doesn’t hurt saying it again:
From what I understand, you’ve taken significant action to make the world a better place. You work in a job that does considerable good directly, and you donate your large income to help animals. That makes you a total hero in my book :-)

JanB 1 Mar 2022 9:34 UTC
65 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Highly effective enhancement of productivity, health, and wellbeing for people in high-impact roles
Effective Altruism
When it comes to enhancement of productivity, health, and wellbeing, the EA community does not sufficiently utilise division of labour. Currently, community members need to obtain the relevant knowledge themselves and do related research, e.g. on health issues, themselves. We would like to see dedicated experts on these issues that offer optimal productivity, health, and wellbeing, as a service. As a vision, a person working in a high-impact role could book calls with highly trained nutrition specialists, exercise specialists, sleep specialists, personal coaches, mental trainers, GPs with sufficient time, and so on, increasing their work output by 50% while costing little time. This could involve innovative methods such as ML-enabled optimal experiment design to figure out which interventions work for each individual.
Note: Inspired by conversations with various people. I won’t name them here because I don’t want to ask for permission first, but will share the prize money with them if I win something.

JanB 1 Mar 2022 9:19 UTC
58 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Reducing gain-of-function research on potentially pandemic pathogens
Biorisk
Lab outbreaks and other lab accidents with infectious pathogens happen regularly. When such accidents happen in labs that work on gain-of-function research (on potentially pandemic pathogens), the outcome could be catastrophic. At the same time, the usefulness of gain-of-function research seems limited; for example, none of the major technological innovations that helped us fight COVID-19 (vaccines, testing, better treatment, infectious disease modelling) was enabled by gain-of-function research. We’d like to see projects that reduce the amount of gain-of-function research done in the world, for example by targeting coordination between journals or funding bodies, or developing safer alternatives to gain-of-function research.
Additional notes:
- There are many stakeholders In the research system (funders, journals, scientists, hosting institutions, hosting countries). I think the concentration of power is strongest in journals: there are only a few really high profile life-science journals(*). Currently, they do publish gain-of-function research. Getting high-profile journals to coordinate against publishing such research would strongly reduce incentives for academic researchers. There is plenty of precedent for collaboration between journals, such as
  - essentially all journals agreed to publish COVID research in Open Access (2016 Statement on Data Sharing in Public Health Emergencies)
  - essentially all journals agree to not publish research with human participants if there was no ethics committee approval for this research
  - essentially all journals agree on requiring authors to state their conflicts of interest
(*) Super roughly:
Tier 1: Science, Nature
Tier 2: Nature Medicine, Cell, (maybe some other Nature family journals)
Tier 3: Some more Nature family journals, Science Advances, PNAS, several others

-> If three publishers (Science, Nature Gropu, Cell Press) coordinated, this would cover all tier 1 and tier 2 journals, and a large part of tier 3 journals.

JanB 1 Mar 2022 9:16 UTC
51 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Cognitive enhancement research and development (nootropics, devices, …)
Values and Reflective Processes, Economic Growth
Improving people’s ability to think has many positive effects on innovation, reflection, and potentially individual happiness. We’d like to see more rigorous research on nootropics, devices that improve cognitive performance, and similar fields. This could target any aspect of thinking ability—such as long/short term memory, abstract reasoning, creativity—and any stage of the research and development pipeline—from wet lab research or engineering over testing in humans to product development.
Additional notes on cognitive enhancement research:
- Importance:
  - Sign of impact: You already seem to think that AI-based cognitive aids would be good from a longtermist perspective, so you will probably think that non-AI-based cognitive enhancement is also at least positive. (I personally think that’s somewhat likely but not obvious and would love to see more analysis on it).
  - Size of impact: AI-based cognitive enhancement is probably more promising right now. But non-AI-based cognitive enhancement is still pretty promising, there is some precedent (e.g. massive benefits from iodine supplementation), …
- Neglectedness: The research community on many areas of cognitive enhancement research is absolutely tiny. Even very low-hanging fruit doesn’t get picked (e.g. before my team started to work on it, nobody had tried to replicate this 20-year old paper, which found massive IQ increases).
- What funding could achieve: From talking to some people, this seems to be an issue of funding (medical funding bodies don’t usually cover enhancement).There are legions of life scientists/psychologists/clinicians who have the relevant skills for this type of research, so if funding was available, I’d expect a lot more of this reserach would happen

JanB 1 Mar 2022 9:53 UTC
30 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
AI alignment prize suggestion: Introduce AI Safety concepts into the ML community
Artificial Intelligence
Recently, there have been several papers published at top ML conferences that introduced concepts from the AI safety community into the broader ML community. Such papers often define a problem, explain why it matters, sometimes formalise it, often include extensive experiments to showcase the problem, sometimes include some initial suggestions for remedies. Such papers are useful in several ways: they popularise AI alignment concepts, pave the way for further research, and demonstrate that researchers can do alignment research while also publishing in top venues. A great example would be Optimal Policies Tend To Seek Power, published in NeurIPS. Future Fund could advertise prizes for any paper that gets published in a top ML/NLP/Computer Vision conference (from ML, that would be NeurIPS, ICML, and ICLR) and introduces a key concept of AI alignment.

JanB 7 Dec 2019 21:11 UTC
21 points
0 ∶ 0
in reply to: Manuel Allgaier’s comment on: A single person decides about funding for community builders world-wide
Thanks for bringing this to my attention, I modified the title and a respective part in the post.

I didn’t have the time to check in with CEA before writing the post so I had to choose between writing the post as is or not writing it at all. That’s why the first line says (in italics) “I’m not entirely sure that there is really no other official source for local group funding. Please correct me in the comments. ”
I think I could have predicted that this is not enough to keep people from walking away with a false impression so I think I should have chosen a different headline.

JanB 9 Apr 2022 8:40 UTC
17 points
0 ∶ 0
on: Why should we care about existential risk?
Here is my take on the value of extinction risk reduction, from some years ago: https://forum.effectivealtruism.org/posts/NfkEqssr7qDazTquW/the-expected-value-of-extinction-risk-reduction-is-positive
This posts also contains links to many other posts related to the topic.
Some other posts, that come to different conclusions:
https://forum.effectivealtruism.org/posts/BY8gXSpGijypbGitT/why-i-prioritize-moral-circle-expansion-over-artificial
https://forum.effectivealtruism.org/posts/RkPK8rWigSAybgGPe/a-longtermist-critique-of-the-expected-value-of-extinction-2
One final thing: Generally, I think people make a distinction between existential risk (roughly: permanent, irreversible, and drastic loss of value of the future) and extinction risk (extinction of humans), where extinction risk is just one type of existential risk.

JanB 1 Mar 2022 10:32 UTC
16 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
AI alignment prize suggestion: Improve our ability to evaluate (and provide training signal for) fuzzy tasks
Artificial Intelligence
There are many tasks that we want AI systems to do, for which performance cannot be evaluated automatically (and thus training signal provision is hard). If we don’t make progress on our ability to train systems for such tasks, we might end up in a world full of systems that optimise for that which is easy to measure, rather than what we actually want. One example of such a task is the evaluation of free-form text; there is currently no automated method to evaluate free-form text (with respect to criteria such as usefulness or correctness) that matches human evaluation. The Future Fund could offer prizes for work that takes a task for which the gold-standard of evaluation is humans, and demonstrates an automated evaluation method that matches human evaluation very closely (or work that demonstrates an automated evaluation method to be superior to human evaluation).
Note: This is crucially not the same as “training models to perform well on the task in question”. There are a number of technical reasons why what I suggest is easier. Intuitively, evaluating performance is often considerably easier than generating good performance. For example, I can watch a movie and say if it’s good, but I can’t make a good movie.

JanB 8 Sep 2021 9:00 UTC
16 points
0 ∶ 0
on: Should Grants Fund EA Projects Retrospectively?
I just quickly wanted to say that this seems related to impact certificates: https://forum.effectivealtruism.org/tag/certificate-of-impact
There have been a few forum posts on this topic, you can just search the forum (or google) for “impact certificate” and will probably find some interesting arguments.

JanB 2 Mar 2022 19:14 UTC
14 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Enlightenment at scale (provocative title :-) )
Values and Reflective Processes (?), X-risk (?)
A strong meditation practice promises enticing benefits to the meditator—less suffering, more control over ones attention and awareness, more insight, more equanimity. Brahmavihara practice promises the cultivation of loving-kindness, compassion, and empathetic joy. The world would be a much better place if everybody suffered less, had more equanimity, and felt strong compassion and empathy with other beings. But meditation is hard! Becoming a skilled meditator, and reaping these benefits, requires probably thousands of hours of dedicated practice. Most people will just not put in this amount of effort. But maybe it doesn’t need to be this way. The field of meditation teaching seems underdeveloped, and innovative methods that make use of technology (e.g. neurofeedback) seem largely unexplored. We are interested in supporting scalable solutions that bring the benefits of meditation to many people.
Note:
- I don’t actually know if meditation really has these benefits; this would needed to be established first (there should be quite some research on this by now). It seems plausible to me that meditation can be very beneficial. Several of my friends claim to have experienced significant benefits from meditation, and I think I can also point to tangible benefits in my own life.
- These innovation need not be directly related to meditation; for example, one could imagine development of an extremely safe and non-addictive pharmaceutical substance that would let people experience, say, strong compassion, and thus increase compassion in everyday life (see e.g. the use of MDMA in therapy).

JanB 3 Oct 2021 18:37 UTC
14 points
0 ∶ 0
on: Coaching: Reduce Struggle and Develop Talent
I personally have benefitted massively from coaching. E.g. I recently wrote this about one of my coaches:
“Paul is a truly excellent coach. I had 40 sessions with him over the course of 2 years. In these, I made transformational progress on topics as broad as motivation/procrastination, communication/teamwork/leadership, time and project management, and decision-making. Paul’s science-based and no-bullshit approach aims at long-term growth, not only fixing this week’s issues.”
Coaching increased my productivity a lot, but also helped me improve a lot in other areas, such as in cultivating a growth mindset and trust in myself. It’s hard to put numbers on this, but I wouldn’t be surprised if coaching (combined with “personal development” more broadly) increased my (expected) lifetime impact by 5x or so. Counting only the improvements in “efficiency/productivity” narrowly understood (roughly defined as “how well/fast can you do a given task + how long/regular can you work”), I’d say I probably became 2x as productive.
Because coaching was so extremely useful for me, I’m passionate about this topic and I’d be a massive fan of great (!) coaching being available to many people. And I’d also want to encourage many to try out coaching, probably with at least 2 different coaches.
I have also worked with coaches where the experience was merely good, in the realm of “coaching sessions tend to make my next week 10-20% more productive”. That is nice, of course, but far away from how useful great coaching can be.

JanB 6 Sep 2019 21:13 UTC
13 points
0 ∶ 0
in reply to: Kit’s comment on: Are we living at the most influential time in history?
How I see it:

Extinction risk reduction (and other type of “direct work”) affects all future generations similarly. If the most influential century is still to come, extinction risk reduction also affects the people alive during that century (by making sure they exist). Thus, extinction risk reduction has a “punting to future generations that live in hingey times” component. However, extinction risk reduction also affects all the unhingey future generations directly, and the effects are not primarily mediated through the people alive in the most influential centuries.

(Then, by definition, if ours is not a very hingey time, direct work is not a very promising strategy for punting. The effect on people alive during the “most influential times” has to be small by definition. If direct work did strongly enable the people living in the most influential century (e.g. by strongly increasing the chance that they come into existence), it would also enable many other generations a lot. This would imply that the present was quite hingey after all, in contradiction to the assumption that the present is unhingey.)

Punting strategies, in contrast, affect future generations primarly via their effect on the people alive in the most influential centuries.
What links here?
- MichaelA's comment on Problem areas beyond 80,000 Hours’ current priorities by Arden Koehler (23 Jun 2020 23:49 UTC; 12 points)

JanB 11 Jul 2022 11:10 UTC
10 points
0 ∶ 0
on: Questioning the Value of Extinction Risk Reduction
I’m excited about people thinking about this topic. It’s a pretty crucial assumption in the “EA longtermist space”, and relatively underexplored.

JanB 10 Mar 2022 20:44 UTC
10 points
0 ∶ 0
on: Samotsvety Nuclear Risk Forecasts — March 2022
Thanks so much for writing this! I expect this will be quite useful for many people.

I actually spent some time this week worrying a bit about a nuclear attack on the UK, bought some preparation stuff, figured out where I would seek shelter or when I’d move to the countryside, and so on. One key thing is that it’s just so hard to know which probability to assign. Is it 1%? Then I should GTFO! Is it 0.001% Then I shouldn’t worry at all.

JanB 1 Mar 2022 10:19 UTC
10 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
AI alignment prize suggestion: Demonstrate a true sandwiching project
Artificial Intelligence
Sandwiching projects are a concrete way for how to make progress on aligning narrowly superhuman models. They “sandwich” the model in between one set of humans which is less capable than it and another set of humans which is more capable than it at the fuzzy task in question, and b) figure out how to help the less-capable set of humans reproduce the judgments of the more-capable set of humans. For example, first fine-tune a coding model to write short functions solving simple puzzles using demonstrations and feedback collected from expert software engineers. Then try to match this performance using some process that can be implemented by people who don’t know how to code and/or couldn’t solve the puzzles themselves.
Importantly, there are many ways to attack a sandwiching project that are slightly cheating. The most challenging version of a sandwiching project would need to make sure that no information whatsoever from the more-capable set of humans is used in the training process. The Future Fund could offer prizes for demonstrations of sandwiching projects on various levels of impressiveness and generality of the employed method.

JanB 1 Mar 2022 9:39 UTC
10 points
0 ∶ 0
on: The Future Fund’s Project Ideas Competition
Refinement of project idea #22, Prediction Markets
Add: “In particular, we’d like to see prediction platforms that do all of the following three: use real money, are very easy to use, allow very easy creation of markets.

JanB 30 Apr 2022 20:00 UTC
9 points
0 ∶ 0
on: Big List of Cause Candidates: January 2021–March 2022 update
There has also been this post on cognitive enhancement research:

https://forum.effectivealtruism.org/posts/MojiqNw5MN6WMXETc/cause-profile-cognitive-enhancement-research-1

JanB 22 Sep 2019 23:40 UTC
9 points
0 ∶ 0
on: What should Founders Pledge research?
cognitive enhancement research

JanB 16 Dec 2018 22:21 UTC
9 points
0 ∶ 0
in reply to: Davidmanheim’s comment on: The expected value of extinction risk reduction is positive
Hi David, thanks for your comments.
1) This seems not to engage with the questions about short-term versus long-term prioritization and discount rates. I’d think that the implicit assumptions should be made clearer.
Yes, the article does not deal with considerations for and against caring about the long-term. This is discussed elsewhere. Instead, the article assumes that we care about the long-term (e.g. that we don’t discount the value of future lives strongly), and analyses what implications follow from that view.
We tried to make that explicit. E.g., the first point under “Moral assumptions” reads:
Throughout this article, we base our considerations on two assumptions:
1. That it morally matters what happens in the billions of years to come. From this very long-term view, making sure the future plays out well is a primary moral concern.
2) It doesn’t seem obvious to me that, given the universalist assumptions about the value of animal or other non-human species, the long term future is affected nearly as much by the presence or absence of humans. Depending on uncertainties about the Fermi hypothesis and the viability of non-human animals developing sentience over long time frames, this might greatly matter.
I think this point matters. Part 2.1 of the article deals with the implications of potential future non-human animal civilizations and extraterrestrials. I think the implications are somewhat complicated and depend quite a bit on your values, so I won’t try to summarize them here.
4) S-risks are plausibly more likely if moral development is outstripped by growth in technological power over relatively short time frames, and existential catastrophe has a comparatively limited downside.
We don’t try to argue for increasing the speed of technological progress.
Apart from that, it is not clear to me that extinction has “comparatively little downside” (compared to S-risks, you probably mean). It, of course, depends on your moral values. But even from a suffering-focused perspective, it may well be that we would—with more moral and empirical insight—come to realize that the universe is already filled with suffering. I personally would not be surprised if “S-risks by omission” (*) weighed pretty heavily in the overall calculus. This topic is discussed in part 2.2.
I don’t have anything useful to say regarding your point 3).
(*) term coined by Lukas Gloor, I think.

JanB 5 Oct 2022 15:27 UTC
8 points
0 ∶ 0
on: Why we’re not founding a human-data-for-alignment org
Thanks for investigating this and producing such an extremely thorough write-up, very useful!