brb243

• brb243Jul 22, 2022, 8:11 PM

  1 point

  0 ∶ 0

  on: Fix­ing bad in­cen­tives in EA

  What are the arguments/​evidence for low social recognition of work outside of EA orgs?

  Working for the government with an EA mindset should be recognized. Some other types of work outside of EA orgs are not well recognized but should be. EA-related opportunities in all non-EA-labeled orgs can be always considered alongside moving to EA-labeled orgs based on marginal value.

  For example, if someone works in an area as seemingly unrelated to EA as backend coding for a food delivery app, they can see if they can make an algorithm that makes vegan food more appealing, learn anything generalizable to AI safety that they can share with decisionmakers who would have otherwise not thought of the idea, gain customers by selling hunger banquet tickets, help the company sell their environmental impact through outcompeting electric scooter delivery by purchasing the much more cost-effective Founders Pledge environmental package in bulk, add some catchy discounts for healthy-food alternative to smoking for at-risk youth users, etc—plus donate to different projects which can address important issues—and compare that to their estimate of impact of switching to an EA org (e. g. full-time AI safety research or vegan food advocacy).

  for funders to estimate the amount of money they would have given to the organisation over a reasonably long period of time and provide that amount (potentially plus a bonus for honesty) to the board/​staff regardless.

  Do you think orgs do not bring some evidence to grantmakers in order to gain funding and this would resolve the issue? Depending on the jurisdiction, there may be laws associated with laying off employees, which include salary for several months to enable the person to find employment or government unemployment schemes. Do you think grantmakers make decisions based on perceived employee insecurity rather than cost-effectiveness? What are the decisionmaking processes that make it so that relatively cost-ineffective projects continue to be funded? Should employees of EA-related orgs that do not provide funding and government funding is not available be encouraged to have several months of savings around grant renewal decision times?

• brb243Jul 22, 2022, 6:50 PM

  3 points

  0 ∶ 0

  in reply to: Stefan_Schubert’s comment on: Fix­ing bad in­cen­tives in EA

  Prima facie, the norm against long-term projects and employment sounds quite ‘effectiveness/​efficiency-decreasing’ but it may just be a bias based on limited experience with this option.

  Long-term projects, if that is meant as funding renewal security, are not the norm in EA. Funding is renewed periodically, based on the most competitive opportunities at any time. Any lower marginal costs of established projects’ unit output is taken into account in funding new and existing ones.

  Long-term paid employment security is greater than that of projects. Organizations may prefer applicants who are willing to work for the org for a considerable time. This can be because the returns of training for that org and relationship-development aspect of some roles.

  A scheme where orgs cooperate in both skills training and relationship development can expedite innovation (skills can complement each other) and improve decisionmakers’ experiences (they are trusted in resolving problems based on various insights rather than one-sidedly ‘lobbied’ to make specific decisions).

  Non-EA orgs should also be involved, for the development of general skills that could be a suboptimal use of EA-related orgs’ time to train and of relationships that can be necessary for some EA-related projects.

• brb243Jul 22, 2022, 5:52 PM

  1 point

  0 ∶ 0

  on: Some­one should cre­ate an ‘EA Blinkist’ book sum­mary service

  1) Which books? There should be easily a 100 books related to EA (more and less broadly).

  2) Some of the books are thought stimulating (the value is readers’ contemplation about unanswered questions), some informative (present valuable info that is useful to problem solving), some directly motivational (get readers focus on an important problem), and some vaguely motivational (inspire people to do good by discussing other topics).

  I think summaries could be counterproductive for vaguely motivational books but could be also improving otherwise readers’ experience , because if one enjoys reading and realizing that they can do good, they can feel better or worse about it than if they skip this step and go straight into reviewing an option of doing good effectively.

  For directly motivational books, summaries should be the most valuable (people should be informed about important issues) but the summaries should be published only after various pressing issues are covered (for example, if only wild animal welfare books are summarized then people who do not want to read the entire text could focus on this area and make it so that AI safety has attention disproportionate to the marginal value/​need).

  Thought stimulating books could be actually discussed without (everyone) reading because that diversifies perspectives.

  Summarizing info that helps solve problems can be valuable to anyone who is resolving the issues.

  option of doing this on demand.

  I think EA orgs would enjoy it if that helps them in what they do. Some people may be interested in having different people read books and then summarize evidence and reasoning on some questions. Then, one person/​org can benefit from knowledge and thinking of multiple people and be thus more efficient.

• brb243Jul 22, 2022, 4:54 PM

  3 points

  1 ∶ 0

  in reply to: Amber Dawn’s comment on: Fel­low­ship al­ter­na­tive: idea syn­the­sis scenarios

  Thanks!

  One fun way could be maybe 2-hour block at events (4*30 mins). Virtual events are another option (more affordable than in-person, can take longer because attention is not so scarce). For a more chilled atmosphere, maybe a weekend retreat where a third of the program is this scenario and ideally multiple groups run it at the same time, to combine insights and meet more friends.

Fel­low­ship al­ter­na­tive: idea syn­the­sis scenarios

• brb243Jul 21, 2022, 9:12 PM

  4 points

  0 ∶ 0

  on: EA Fo­rum fea­ture sug­ges­tion thread

  ‘Commenting sprees’ - blocks of time where discussion with more immediate replies would be encouraged.

• brb243Jul 20, 2022, 11:43 PM

  1 point

  0 ∶ 0

  in reply to: Joseph’s comment on: Fel­low­ship al­ter­na­tive: Read­ing the EA Fo­rum (with dis­cus­sion ques­tions)

  for 6 or 7 of the 8 meetings it was just me and the facilitator on a video chat

  Wow, maybe the number of people in a group could be somewhat increased.

  The biggest benefit for me was the ability to bounce ideas around and have an instant response/​reply, shortening the feedback loop compared to simply reading and Googling around on my own.

  Ok, ah hah maybe ‘commenting sprees’ could be implemented in a otherwise this aspect can be difficult to imitate in a written form.

• brb243Jul 20, 2022, 11:28 PM

  5 points

  0 ∶ 0

  on: What might decor­ti­cate rats tell us about the dis­tri­bu­tion of con­scious­ness?

  my impression is that very few people have the comprehensive expertise to think well about these things.

  Do you mean people in general? Or, in EA/​neuroscience/​consciousness research, …

  In humans and other apes, the cerebral cortex is also widely believed to be largely responsible for generating our conscious experiences.

  Could you share any resources that suggest otherwise? It could be also interesting to seem them on a timeline.

  A human without a cerebral cortex is, for the most part, a living body with no mind.

  If ‘mind’ is interpreted as “awareness and intentional activity” and “body regulation and arousal” is not considered ‘living.’

  The behavior of decorticate rats is remarkably unaffected by extensive damage or removal of their cerebral cortex.

  So, it can be argued that healthy rats are unaware and unintentional and thus do not live?

  Various social changes
  Notably, inferior maternal care

  Is it that what makes us ‘a species with a prominent cortex’ is care (I mean the reverse of causality)? How does k- and r-selection relate to a species cortex properties (and to consciousness)?

  Difficulty with some complex cognitive tasks
  Inability to reason abstractly about locations
  Absence of food hoarding behavior

  Can individuals with no or relatively non-prominent cortex be really ‘present in the moment and place?’ Would then species with ‘lesser’ cortex than humans be much happier if they feel well in a place and are fed and sadder if they do not than humans do because humans ‘carry’ some memory and plan?

  require subtle experimental designs to detect.

  Out of curiosity, do you know of any designs or labs?

  the behavioral effects of decortication suggest that consciousness is either restricted to primates or else it is distributed fairly widely outside mammals.

  Only rats were studied so conclusions about various brains and nervous systems cannot be stated. Is it that this reasoning could suggest that primates would be more conscious than other species but not that these species would be non-conscious because cortex does affect rats’ behavior somewhat? Also, even if the decorticate rat behaves similarly as one with cortex, it can be that it is less conscious, for example cannot feel closeness with family as much on in specific ways?

  Alongside the lines of 4), it could be argued that humans have some consciousness in different parts of the brain, nervous system, cells, and other parts of the body, depending on the definition of consciousness. Thus, humans can ‘empathize’ with different species by focusing on that part/​activating it and deactivating others. For example, species that experience “arousal” but not “intentional activity” can be empathized with by focusing the former while seeking to block the latter.

  Are you aware of the Cambridge Declaration on Consciousness? What do you think about it?

• brb243Jul 20, 2022, 10:27 PM

  2 points

  0 ∶ 0

  on: Book a chat with an EA professional

  A critical reading is that this seems that most of the people would motivate chatters to take careers steps which may advance some of the projects that they or others have in mind.

  Is this the objective (definitely can be worthwhile), while other avenues should be found for discussing solutions to problems that these professionals focus on?

  The two people who so far commented about being willing to chat seem interested in brainstorming solutions (and unstructured chatting) in addition to sharing more one-sided advice.

  Before I noticed these comments I meant to suggest renaming the post to something like ‘Not sure about your next career steps and getting bored? Book a chat with an EA professional!’. With the comments, the expectations in the chat should be covered by the topics list.

• brb243Jul 20, 2022, 10:02 PM

  1 point

  0 ∶ 0

  on: If you’re un­happy, con­sider leaving

  These are just random and provocative thoughts.

  “well, the world sucks, but...”

  Maybe some of the people who you meet consider themselves with limited agency so express aggression and acknowledgement of the state of affairs? Or, they agree with the system that they live in, even if it allows some people to make profit (if the sentence continues ‘but what can you do, you need to make people [like themselves] pay high rent since then if you are [their job] you can benefit from [what they do]’). Or, they acknowledge the sentiment which is that one could be interested in leaving their situation but express inability or limited opportunities to do so (‘but it’s the same everywhere’). Or, they perceive a ‘threat’ of being deprived of their privilege if they focus on improving global issues (‘but it’s actually ok’/​‘but I also have issues that I need help with’/​‘but improving the situation would require a change the way I think which is challenging’).

  lots of people in the EA orbit are persistently unhappy

  Could you quantify or explain lots, persistently, and unhappy?

  And the solution to being persistently unhappy with a social arrangement or memeplex, usually, is to leave.

  Are there any risks associated with leaving a system with ambitions of doing the most good which is suboptimal?

  I am not an expert but is it that some therapies recommend re-interpretation of one’s situation, communication, and if there are no other things to consider which could make it better to stay, leaving?

  Famously, this often doesn’t occur to the person suffering for a long time, if ever, even if it looks like the obvious correct choice from the outside.

  One interpretation is that once a person is aware of global issues, others’ commitment to resolve them, their opportunities to participate (grants, jobs), and their marginal value in this effort (funding overhang), then they cannot leave the memeplex of doing the most good, even if they (temporarily) leave the community, because they are compelled by need.

  Another interpretation is that people who learn about EA will always think about impact to some extent, even if they leave EA. It is because the ideas make sense and they would feel somewhat bad not considering them since they learned it is ‘good’ to do so.

  Third way to think about one’s perspective on leaving EA is that people learn about EA, participate for a while, and while acknowledging that they could do a lot of good, consider that they would be happier if they left and fully focused on other things. They have the ‘approval’ and can reason that systemic change that would make it so that all people have better impact is needed and that there are enough brilliant people who actually choose to stay for whatever reason.

  People in all three categories can be happy or unhappy to different extents. One can suggest that keeping oneself physically healthy, having good relationships, being secure, treating and preventing any mental health issues, doing what one likes, and existing in an environment that one wants to be a part of can support one’s happiness. A person who ‘cannot leave’ can thus be very happy while a person who ‘leaves and forgets’ very unhappy and vice versa.

  Telling people to leave instead on supporting them with their issues can reduce the community’s aspect of happiness based on their valuation of the system that they live in ;)

  It can be argued that people who would be better of leaving should leave even if they were ‘great unique assets’ but that people should be supported with whatever they need, to a reasonable extent (e. g. taking advantage of the EA mindfulness program, Effective Self-Help, or working with some of the therapists and providers recommended by EA Mental Health Navigator or with the AI Safety Support Health Coach). Possibly, few people would argue that the relatively little resources spent on mental health support in the community are excessive. It could be great even if people use there ‘free’ resources and then leave the community, if they are thus happier. Of course, one can also take advantage of non-EA therapists and resources.

  What do I mean by leaving?

  I agree that non-EA friends can be fun. Even a person who is (momentarily) highly influenced by thinking about impact, who would probably feel bad about hanging out with, for example, oil investor inconsiderate about the environment, can be fine with most people, and even learn that issues are not so dark or bleak as could be prominent in some EA narratives (e. g. learning from an engineer that some software is quite ok).

  Why should one not apply for EA jobs? Is it to save themselves time which they might not have (e. g. would be sacrificing on social or financial)? Otherwise, applying for any jobs which are well compensated can make sense for financial needs.

  But there is no one understanding of self-worth in EA, just like there is no clear definition of EA. Should there be one understanding? For example, something which would take into account self-care as well as contribution by various means, with respect to one’s potential? Elliot Olds could be interested in this as he posted something about an impact list.

  I keep seeing people be nervous to like, seriously criticize EA, because their (aspirational) livelihoods depend on it.

  Hopefully their number is reducing? This piece to an extent sets an openness standard, the Future Fund has been gathering ideas, criticism, and ideas on how to gain criticism since the beginning, OPP is also asking what they can be better off funding, and Criticism is even contested in.

  Give the cool kids a few chances to let you in.

  There is no coolness scale ….......

  Has bursts of manic energy and gets excited about projects, but loses initiative when nobody really supports them.

  There was an article related to this. I hope EA content does not cause mania. Nothing on ‘memetic content’ was mentioned as a trigger.

  You might have less impact.

  Or you might not!

  It can be argued that one might not because what the EA community can offer for one’s choice of ‘high benevolence’ is nothing, besides perhaps the community. It can be better if a very large majority of people want to be in the community or are somewhat in control of and supported in their in-and-out vacillations.

  The people who seem to have had the greatest impact in history—Bourlag and Petrov, for instance—just sort of were in the right place at the right time with the right interests.

  Hm, just a hypothesis, what could be causing any mania in EA could be highlighting heroes who were at the right place at the right time with an interest in impact and narrating opportunities in EA in a similar way. This issue can be mitigated by suggesting that these individuals were parts of institutions that made it so that they made the actions and decisions they did. Plus, that there are many people unacknowledged for their contributions, due to generally, social structures which motivate people to work for them as they seek status. It is really cooperation that makes representatives succeed.

  For example,

  Petrov later indicated that the influences on his decision included that he had been told a US strike would be all-out, so five missiles seemed an illogical start … He felt that his civilian training helped him make the right decision. He said that his colleagues were all professional soldiers with purely military training and, following instructions, would have reported a missile launch if they had been on his shift. (Wikipedia)

  So, the individuals who made it so that the person who told Petrov about the likely extent of the strike and other people who he encountered during his training which improved his decisionmaking abilities, and possibly his superiors and their superiors who made it so that Petrov trusted that he is supported in independent decisionmaking can all be credited for this decision.

  But it’s interesting that inner core EA activities like reducing AI risk or community building at top universities just seem… weird and onanistic from the outside.

  One can suggest that this can be addressed. Narratives which make people seem unfriendly or weird can be reconsidered. For example, AI risk could be narrated as ameliorating human biases with respect to the law and improving human institutions based on biases detected by AI, but it would be a mistake to exclude the variety of other issues that AI safety addresses, such as the potential of and risks associated with an intergalactic expansion.

  An argument against discouraging ‘the general public’ from paying attention to some higher-risk issues, such as AI safety or biosecurity, is that this could increase risk. When structures that support positive development with increased public interest are not in place, it is better that EA core activities are just not discussed much in public, due to emotions about them. This could be rationalized and perhaps thus improve any negative emotions.

  Community building at top universities can be another example of a strategic consideration. It may be better to first include top problem solvers and only after people less good at it. A group started by non-top problem solvers could address issues relatively worse.

  That being said, you probably have some instinct of whether involvement with organized EA is making you unhappy.

  What do you mean by ‘organized’ EA? Are some resources/​events/​people more ‘organized’ than others? Could you share some examples?

  Those are bad signs.

  One can agree. Is it that the first two could be addressed by therapy focused on emotions while the latter two by reason?

• brb243Jul 20, 2022, 6:56 PM

  1 point

  0 ∶ 0

  in reply to: Donald Hobson’s comment on: Does the idea of AGI that benev­olently con­trol us ap­peal to EA folks?

  OK! You mean super-healthy as resilient to biological illnesses or perhaps processes (such as aging).

  Nanobots would probably work but mind uploading could be easier since biological bodies would not need to be kept up.

  While physical illness would not be possible in the digital world, mental health issues could occur. There should be a way to isolate only positive emotions. But, I still think that actions could be performed and emotions exhibited but nothing would be felt by entities that do not have structures similar to those in human brain that biologically/​chemically process emotions. Do you think that a silicon-based machine that incorporates specific chemical structures could be sentient?

  Ah, I think there is nothing beyond ‘healthy.’ Once one is unaffected by external and internal biological matters, they are healthy. Traditional physical competition would probably not make sense in the digital world. For example, high jump. But, humans could suffer digital viruses, which could be perhaps worse than the biological ones. But then, how would you differentiate a digital virus from an interaction, if both would change some aspects of the code or parameters?

• brb243Jul 20, 2022, 5:31 PM

  1 point

  0 ∶ 0

  in reply to: Joseph’s comment on: Fel­low­ship al­ter­na­tive: Read­ing the EA Fo­rum (with dis­cus­sion ques­tions)

  OK, but did the discussion Programs add something that you would not get from reading the Forum, such as nicer experience (or motivation to meet more people in person) or ideas on topics that could be also interesting to your colleagues? If so, then should there be an option without these aspects (e. g. for people less interested in more ‘dynamics navigation’ focused chats)?

  Also, it could be argued that the fraction of material that people who do not participate in a discussion-based program would not read can be crucial to people’s understanding of EA. But, early-on specialization of people can be optimal. For example, consider that a person interested in global development never reads any insect welfare texts. They think about massively scaling up insect farming to enable people escape poverty. They address skepticism from insect welfare researchers by assurance of positive welfare. Thus, the researchers are motivated to find a solution optimal for humans, and insects. If everyone read introductory texts from both (all) areas, it is possible that the thought that would lead to this mutually beneficial solution would not have been developed.

  Thanks! Yeah that makes sense.

• brb243Jul 20, 2022, 4:38 PM

  1 point

  0 ∶ 0

  in reply to: Algo_Law’s comment on: Fel­low­ship al­ter­na­tive: Read­ing the EA Fo­rum (with dis­cus­sion ques­tions)

  Yes! .. thank you. I think maybe there can be some organized page of summaries that people going though a ‘fellowship’ can update—so an aspect of the Wiki. Otherwise, just writing a comment or a comment on a comment can be a good way to demonstrate that one thought about the topics. Or, forming several narratives of the articles can be nice (the activity where anyone writes the next sentence).

  Thank you for pointing out the overlap. I can come up only with organization according to a vector space where the elements are the extent to which the article relates to specific topics but it would be nice to have something with better flow and with paths (with intersections) that would lead one to go for a bit at a time.

  A megathread would not solve the organization issue and could feel like the thoughts developed are not being utilized. Multiple smaller threads can be cool, but mostly for questions that are actually advanced by discussion or for those that can be interesting to get opinions on (not e. g. asking someone to rephrase main points). Stickied questions under tags may be a solution—also once a question is somewhat resolved or opinions at the time gathered, it can be replaced.

• brb243Jul 20, 2022, 4:10 PM

  1 point

  0 ∶ 0

  in reply to: AndreFerretti’s comment on: 300+ Flash­cards to Tackle Press­ing World Problems

  yes! that’s right I was off by a factor of 10 ..

  ok, as you wish.

Fel­low­ship al­ter­na­tive: Read­ing the EA Fo­rum (with dis­cus­sion ques­tions)

• brb243Jul 18, 2022, 11:04 AM

  1 point

  0 ∶ 0

  in reply to: Donald Hobson’s comment on: Does the idea of AGI that benev­olently con­trol us ap­peal to EA folks?

  Oh yeah, that makes sense. And if humans can’t imagine what super-healthy is then they need to defer to AGI—but should not misspecify what they meant ..

Dis­cus­sion Ques­tions for Books with Ex­pe­rienced Pro­fes­sion­als’ Audience

• brb243Jul 16, 2022, 10:48 PM

  2 points

  0 ∶ 0

  in reply to: Alexander_Berger’s comment on: A philo­soph­i­cal re­view of Open Philan­thropy’s Cause Pri­ori­ti­sa­tion Framework

  Since you define worldview as a “set of … beliefs that favor a certain kind of giving,” then it matters whether you understand income and health as “intrinsically [or] instrumentally valuable.” In the latter but not the former case, if you learn that income and health do not optimize for your desired end, you would change your giving.

  I am understanding investment recommendation implications as programs on education, relationship improvement, cooperation (with achievement outcomes), mental health, chronic pain reduction, happiness vs. life satisfaction research, conflict prevention and mitigation, companionship, employment, crime reduction, and democracy:

  and the objective list, where wellbeing consists in various objective goods such as knowledge, love, and achievement

  underweight invisible, ongoing misery (such as mental illness or chronic pain)

  the best thing for improving happiness may be different from the best thing for increasing life satisfaction. Investigating this requires extra work.

  other things affect our wellbeing too (such as war, loneliness, unemployment, crime, living in a democracy, etc.) and their value is not entirely reducible to effects on health or income.

  Divestment recommendations can be understood as bednets in Kenya, GiveDirectly transfers to some but not other members of large proportion extremely poor communities, and Centre for Pesticide Suicide Prevention:

  It’s worth pointing out that many of those whose lives are saved by the interventions that OP funds, such as anti-malaria bednets, will have a life satisfaction score below the neutral point, unless we set it at, or near to, ⁰⁄₁₀. IDinsight’s aforementioned beneficiary preferences survey has an SWB question and found those surveyed in Kenya had an average life satisfaction score of 2.3/​10.

  But OP (and others) tend to ignore fairness; the aim is just to do the most good.

  But, don’t happier people gain a greater benefit from an extra year of life than less happy people? If so, how can it be consistent to conclude we should account for quantity when assessing the value of saving lives, but not quality?

  I understand that you disengage from replies but I am interested in OP’s perspective on the 0-10 life satisfaction value at which you would invest into life satisfaction improving rather than family planning programs.

  I am also wondering about your definition of health and rationale for selecting the DALY metric to represent this state.

• brb243Jul 16, 2022, 9:50 PM

  1 point

  0 ∶ 0

  in reply to: Gabin’s comment on: Do EA folks want AGI at all?

  I am not saying that international institutions prove that they can 100% prevent human-made catastrophes but I think that they have the potential, if institutions are understood as the sets of norms that govern human behavior rather than large intergovernmental organizations, such as the UN.

  It may be technically easier but normatively more difficult for people to harm others, including decisionmakers causing existential catastrophes. For example, nuclear proliferation or bio weapons stockpiling was not extensively criticized by the public in the past, because people were having other issues and offering critical perspectives on decisionmaking was not institutionalized. Now, the public holds decisionmakers accountable to not using these weapons, by the ‘sentiment of disapproval.’ This is uniquely perceived by humans who act according to emotions.

  People can be manipulated by the internet to an extent. This considers their general ability to comprehend consequences and make own opinion based on different perspectives. For example, if people start seeing ads on Facebook about voting for a proliferation proponent that appeal to their aggression/​use biases to solicit fear and another ad shows the risks of proliferation/​war and explains the personal benefits of peace, then people will likely vote for peace.

  That makes sense: as an oversimplification, if AGI is trained to optimize for the expression ‘extreme pain’ then humans could learn to use the scale of ‘pain’ to denote pleasure. This would be an anti-alignment failure.

  That makes a lot of sense too: I think that one’s capacity to advance innovative objectives efficiently increases with improving subjective perception of the participants/​employees. For example, if there is a group of people who are beaten every time they disobey orders mandated to make a torture technology and another group that makes torture regulation and fosters positive cooperation and relationship norms, the former should think less innovatively and cooperate worse than the latter. So, the regulation should be better than the aggression.

  But, what if one ‘quality unit’ of the torture technology causes 100x more harm than one ‘quality unit’ of the regulatory technology can prevent. For instance, consider releasing an existing virus vs. preventing transmissions. Then, you need some institutional norms to prevent people from aggression. Ideally, there would be no so malevolent people: which can be either made by AI (e. g. recommendation algorithms for pleasure competitive with/​much better than hurting others which is not pleasant just traumatizing and possibly memorable) or by humans (e. g. programs for at-risk children).

  Yeah! That seems almost like an existential catastrophe if people are great (and) alive but actually suffering significantly. Considering that AI could just do that with one misspecification that somehow augments, AGI is risky. Humans would not develop in this way because they perceive so would stop doing what they dislike, if there is no ‘greater manipulative power’ which would compel them otherwise.

  I am optimistic about humans being able to develop an AGI that improves wellbeing better than what would have happened without such technology while keeping control over it to make adjustments if they perceive a decrease in wellbeing. But, if it is not necessary, then perhaps why take the risk.

• brb243Jul 16, 2022, 8:29 PM

  2 points

  0 ∶ 0

  in reply to: Donald Hobson’s comment on: Does the idea of AGI that benev­olently con­trol us ap­peal to EA folks?

  Yes, there are ways for this to go wrong. I’d not like to ingest nanobots which would be something like a worm infection but worse!

  But if the AI is actually benevolent, then it could be better than humans and ASI optimized for some of their objectives, working with human impulses (for example, offering food which looks the biggest but has a suboptimal nutrient ratio and disregards animal welfare or instead of social skills course making one addicted to a platform which makes people feel worse about others).

  AGI should prevent people from impulsive decisionmaking and foster rationality. It should be better, at least in the interim, than humans who could perpetuate some suboptimal characteristics. The issue is that maybe then humans would become practically AI without any apparent intervention.