Speaking from what I’ve personally seen, but it’s reasonable to assume it generalizes.
There’s an important pool of burned out knowledge workers, and one of the major causes is lack of value alignment, i.e. working for companies that only care about profits.
I think this cohort would be a good target for a campaign:
Effective giving can provide meaning for the money they make
Dedicating some time to take on voluntary challenges can help them with burnout (if it’s due to meaninglessness)
Tentatively and naively, I think this is accurate.
I’m wondering if there would be any way to target/access this population? If this campaigns existed, what action would it take? Some groups of people are relatively easy to access/target due to physical location or habits (college-aged people often congregate at/around college, vegan people often frequent specific websites or stores, etc.).
I imagine that someone much more knowledgeable about advertising/marketing than I am would have better ideas. All I can come up with off the top of my head is targeted social media advertisements: people who work at one of these several companies and who have recently searched for one of these few terms, etc.
Question: how to reconcile the fact that expected value is linear with preferences being possibly nonlinear?
Example: people are tipically willing to pay more than expected value for a small chance of a big benefit (lottery), or to remove a small chance of a big loss (insurance).
This example could be rejected as a “mental bias” or “irrational”. However, it is not obvious to me that linearity is a virtue, and even if it is, we are human and our subjective experience is not linear.
Thank you for pointing out log utility, I am aware of this model (and also other utility functions). Any reasonable utility function is concave (diminishing returns), which can explain insurance to some extent but not lotteries.
I could imagine that, for an altruistic actor, altruistic utility becomes “more linear” if it’s a linear combination of the utility functions of the recipients of help. This might be defensible, but it is not obvious for me unless that actor is utilitarian, at least in their altruistic actions.
(just speculating, would like to have other inputs)
I get the impression that sexy ideas get disproportionate attention, and that this may be contributing to the focus on AGI risk at the expense of AI risks coming from narrow AI. Here I mean AGI x-risk/s-risk vs narrow AI (+ possibly malevolent actors or coordination issues) x-risk/s-risk.
I worry about prioritising AGI when doing outreach because it may make the public dismiss the whole thing as a pipe dream. This happened to me a while ago.
My take is that I think there are strong arguments for why AI x-risk is overwhelmingly more important than narrow AI, and I think those arguments are the main reason why x-risk gets more attention among EAs.
Ah I see what you’re saying. I can’t recall seeing much discussion on this. My guess is that it would be hard to develop a non-superintelligent AI that poses an extinction risk but I haven’t really thought about it. It does sound like something that deserves some thought.
When people raise particular concerns about powerful AI, such as risks from synthetic biology, they often talk about them as risks from general AI, but they could come from narrow AI, too. For example some people have talked about the risk that narrow AI could be used by humans to develop dangerous engineered viruses.
My uninformed guess is that an automatic system doesn’t need to be superintelligent to create trouble, it only needs some specific abilities (depending on the kind of trouble).
For example, the machine doesn’t need to be agentic if there is a human agent deciding to make bad stuff happen.
So I think it would be an important point to discuss, and maybe someone has done it already.
Speaking from what I’ve personally seen, but it’s reasonable to assume it generalizes.
There’s an important pool of burned out knowledge workers, and one of the major causes is lack of value alignment, i.e. working for companies that only care about profits.
I think this cohort would be a good target for a campaign:
Effective giving can provide meaning for the money they make
Dedicating some time to take on voluntary challenges can help them with burnout (if it’s due to meaninglessness)
Tentatively and naively, I think this is accurate.
I’m wondering if there would be any way to target/access this population? If this campaigns existed, what action would it take? Some groups of people are relatively easy to access/target due to physical location or habits (college-aged people often congregate at/around college, vegan people often frequent specific websites or stores, etc.).
I imagine that someone much more knowledgeable about advertising/marketing than I am would have better ideas. All I can come up with off the top of my head is targeted social media advertisements: people who work at one of these several companies and who have recently searched for one of these few terms, etc.
Question: how to reconcile the fact that expected value is linear with preferences being possibly nonlinear?
Example: people are tipically willing to pay more than expected value for a small chance of a big benefit (lottery), or to remove a small chance of a big loss (insurance).
This example could be rejected as a “mental bias” or “irrational”. However, it is not obvious to me that linearity is a virtue, and even if it is, we are human and our subjective experience is not linear.
Look into logarithmic utility of money; there is some rich literature here
For an altruistic actor, money becomes more linear again, but I don’t have a quick reference here.
Thank you for pointing out log utility, I am aware of this model (and also other utility functions). Any reasonable utility function is concave (diminishing returns), which can explain insurance to some extent but not lotteries.
I could imagine that, for an altruistic actor, altruistic utility becomes “more linear” if it’s a linear combination of the utility functions of the recipients of help. This might be defensible, but it is not obvious for me unless that actor is utilitarian, at least in their altruistic actions.
(just speculating, would like to have other inputs)
I get the impression that sexy ideas get disproportionate attention, and that this may be contributing to the focus on AGI risk at the expense of AI risks coming from narrow AI. Here I mean AGI x-risk/s-risk vs narrow AI (+ possibly malevolent actors or coordination issues) x-risk/s-risk.
I worry about prioritising AGI when doing outreach because it may make the public dismiss the whole thing as a pipe dream. This happened to me a while ago.
My take is that I think there are strong arguments for why AI x-risk is overwhelmingly more important than narrow AI, and I think those arguments are the main reason why x-risk gets more attention among EAs.
Thank you for your comment. I edited my post for clarity. I was already thinking of x-risk or s-risk (both in AGI risk and in narrow AI risk).
Ah I see what you’re saying. I can’t recall seeing much discussion on this. My guess is that it would be hard to develop a non-superintelligent AI that poses an extinction risk but I haven’t really thought about it. It does sound like something that deserves some thought.
When people raise particular concerns about powerful AI, such as risks from synthetic biology, they often talk about them as risks from general AI, but they could come from narrow AI, too. For example some people have talked about the risk that narrow AI could be used by humans to develop dangerous engineered viruses.
My uninformed guess is that an automatic system doesn’t need to be superintelligent to create trouble, it only needs some specific abilities (depending on the kind of trouble).
For example, the machine doesn’t need to be agentic if there is a human agent deciding to make bad stuff happen.
So I think it would be an important point to discuss, and maybe someone has done it already.