Just a quick response on the CEA’s groups team end.
...
Institutions like Facebook, Mckinsey, and Goldman spend ~ $1 million per school per year at the institutions they recruit from trying to pull students into lucrative careers that probably at best have a neutral impact on the world.
I’m surprised to see CEA making such a strong claim. I think we should have strong priors against this stance, and I don’t think I’ve seen CEA publish conclusive evidence in the opposite direction.
Firstly, note that these three companies come from very different sectors of the economy and do very different things.
Secondly, even if you assign high credence to the problems with these firms, it seems like there is a fair bit of uncertainty in each case, and you are proposing a quite harsh upper bound - ‘probably at best neutral’.
Thirdly, each of these are (broadly) free market firms, who exist only because they are able to persuade people to continue using their services. It’s always possible that they are systematically mistaken, and that CEA really does understand social network advertising, management consulting, trading and banking better than these customers… but I think our prior should be a little more modest than this. Usually when people want to buy something it is because they want that thing and think it will be useful for them.
Finally, there are in fact for each of these firms a bunch of concrete benefits they provide. Rarely do I see these explicitly weighed in the calculus against the problems:
Facebook allows people to keep in touch with friends and relatives, to share their thoughts and news about their lives, and meet like-minded new friends. Certainly I have personally made many new friends over facebook, and engaged in many good discussions. It also allows advertisers to show their products to the people who are most likely to appreciate them, saving others from having their time wasted with irrelevant ads.
McKinsey provides advice and allows for the diffusion of best practices from leading firms to others in the economy. They can also help management overcome internal veto players and other opposition to change by helping supply credibility to decisions. For some types of consulting (though a little different to what McKinsey mainly does) we even have RCTs showing that they improve productive efficiency.
Goldman’s trading arm provides a wide range of services to market participants, like research, prime brokerage and market making, that are necessary to help keep markets efficient. They also provide investment banking services, allowing companies and governments to raise money to finance projects, and retail banking, giving ordinary people higher interest rates than they’d get from their legacy banks.
It’s possible that these has been some explicit analysis of these firms to support your very strong statement. I searched on the forum for ‘McKinsey’ to try to find it, but at least the first page or so of results were generally positive references—e.g. people quoting their work on climate change, or positively referencing how they would address a problem. 80k does have an old article with some cursory analysis of the harms of finance, but the analysis is seriously flawed, and it doesn’t cover Management Consulting or Social Networks at all.
Curious if you disagree with Jessica’s key claim, which is “McKinsey << EA for impact”? I agree Jessica is overstating the case for “McKinsey ⇐ 0″, but seems like best-case for McKinsey is still order(s) of magnitude less impact than EA.
Subpoints:
Current market incentives don’t address large risk-externalities well, or appropriately weight the well-being of very poor people, animals, or the entire future.
McKinsey for earn-to-learn/give could theoretically be justified, but that doesn’t contradict Jessica’s point of spending money to get EAs
Most students require a justification for anyone charitable spending significant amounts of money on movement building & competing with McKinsey reads favorably
Agree we should usually avoid saying poorly-justified things when it’s not a necessary feature of the argument, as it could turn off smart people who would otherwise agree.
Sorry, I was trying to get a quick response to this post and I made a stronger claim than I intended. I was trying to say that I think that EA careers are doing much more good than the ones mentioned on average and so spending money is a good bet here. I wasn’t intending to make a definitive judgment about the overall social impact of those other careers, though I know my wording suggests that. I also generally want to note that this element was a personal claim and not necessarily a CEA endorsed one.
This was a great comment and thoughtful reply and the top comment was great too.
Looking at the other threads generated from the top comment, it looks like tiny turns of phrase in that top comment, produced (unreasonably) large amounts of discussion.
I think we all learned a valuable lesson about the importance of clarity and precision when commenting on the EA forum.
Thirdly, each of these are (broadly) free market firms, who exist only because they are able to persuade people to continue using their services. It’s always possible that they are systematically mistaken, and that CEA really does understand social network advertising, management consulting, trading and banking better than these customers… but I think our prior should be a little more modest than this. Usually when people want to buy something it is because they want that thing and think it will be useful for them.
I consider this to be a pretty weak argument, so it doesn’t contribute much to my priors, which although weak (and so the particulars of a company matter much more), are probably centered near neutral on net welfare effects (in the short to medium term). I think a large share of goods people buy and things they do are harmful to themselves or others before even considering the loss of income/time as a result, or worse for them than the things they compete with. It’s enough that I wouldn’t have a prior strongly in favour of what profitable companies are doing being good for us. Here are reasons pushing towards neutral or negative impacts:
A lot of goods are mostly for signaling, especially signaling wealth, which often has negative externalities and I’d guess little positive value for the individual. Brand name versions of things, clothing, jewelry, cars.
Many modern ways people spend their time (enabled by profitable companies) have probably made us less active, more indoor-bound, less close with others, and less pursuant of meaning and meaningful goals, which may conflict with people’s reflective preferences, as well as generally be bad for health, mental health and other measures of wellbeing. Basically a lot of the things we do on our computers and phones.
Many things are stimulating and addictive, and companies are optimizing for want, not welfare. Want and welfare can come apart when we optimize for want. So we get cigarettes, addictive video games, junk food, algorithms optimizing for clicks when we’d be better off stepping away from the internet or doing more substantial things online, and lots of salt, sugar and calories in our foods.
Media companies may optimize for revenue over accurate reporting. This includes outrage, playing to our fears, demonizing and polarization.
Some companies make us want their stuff for fear of missing out or social pressure, so it can be closer to coercion than providing a valuable opportunity.
I’d guess relatively little is spent on advertisement for things that we have good evidence for improving our welfare, because most of those things are hard to profit from: basic healthy foods, exercise (although there are certainly exercise products and programs that get advertised, but less so just gym memberships, joining sports leagues, running outside), just spending more time with your friends and family (in cheap ways, although travel and amusement parks are advertised), pursuing meaning or meaningful goals, helping others (even charity ads are relatively rare). So, advertisement seems to push us towards things that are worse for us than the alternatives we’d have gone with. To capitalize on the things that do make us substantially better off, companies may sell us more expensive versions that aren’t (much) better or things to go with them that don’t substantially help.
I’d expect a lot of hedonic adaptation for many goods and services, but not mental health (almost by definition), physical pain and to a lesser extent general health and mobility, which are worsened by a lot of the things companies provide, directly or indirectly by competing with the things that are better for health.
Company valuations don’t usually substantially reflect their externalities, and shorting companies is riskier and more costly than buying and holding shares, so this biases markets towards positively valuing companies even if their overall value for the world is negative.
There are often negative externalities on nonhuman animals in particular, although the overall effects on nonhuman animals may be complicated when you also consider the effects on wild animals.
I do think it’s plausible McKinsey and Goldman have done and do more good than harm for humans in the short term, based on the arguments you give, but I don’t have a strong view either way. It could depend largely on whether raising people’s consumption levels makes them better off overall (and how much) in the places where people are most affected by these companies. Measures of well-being do seem to positively correlate with income/wealth/consumption at the individual level, and I’d guess also at the aggregate level for developing countries, but I’d guess not for developed countries, or at best weakly so. There are negative externalities for increasing an individual’s income on others’ life satisfaction, although it’s possible a large share is due to rescaling, not actually thinking your life is worse absolutely than otherwise. See:
Based on GiveDirectly in Kenya. They had multiple measures of wellbeing, but negative effects were only observed for life satisfaction for non-recipient households of cash transfers in the same village. See Table A5.
This table from Veenhoven, R. (2019). The Origins of Happiness: The Science of Well-Being over the Life Course., reproduced in this post.
Some companies may also contribute to relative inequality or even counterfactually make the median or poor person absolutely poorer through their political activities.
The categories of things I’m optimistic about for human welfare in the short to medium term are:
Things that save us time, so we can spend more time on things that actually make us better off.
Things that improve or protect our health (including mental health).
Things that make us (feel) safer/more secure (physically, financially, etc.).
Things that make us more confident, but without substantially net negative externalities (negative externalities may come from positional goods, costly signaling, peer pressure).
Things that help us make better decisions, without important negative effects.
I’m neutral to optimistic about these (possibly neutral because they just replace cheaper versions of themselves that would be just as good):
In-person activities with friends/family.
Things for hobbies or projects.
Restaurants.
I’m about neutral and pretty uncertain about screen-based entertainment (TV, movies, video games), and recreational substances that aren’t extremely addictive or harmful (alcohol, marijuana).
There are also a lot of externalities that act at least equally on humans, like carbon emissions, promotion of ethnic violence, or erosion of privacy. Those are all examples off the top of my head for Facebook specifically.
I upvoted Larks’ comment, but like you I think this particular argument, “people buy from these firms”, is weak.
But surely, the spirit of the original comment is correct too.
No matter which worldview you have, the value of a top leader moving into EA is overwhelmingly larger than the the social value of the same leader “rowing” in these companies.
Also, at the risk of getting into politics (and really your standard internet argument) gesturing at “free market” is really complicated. You don’t need to take the view of Matt Stoller or something to notice that the benefits of these companies can be provided by other actors. The success of these companies and their resources that allow recruitment with 7 figure campus centres probably has a root source different than pure social value.
The implication that this statement requires CEA to have a strong model of these companies seems unfair. Several senior EAs, who we won’t consider activists or ideological, have deep experiences in these or similar companies. They have opinions that are consistent with the parent comment’s statement. (Being too explicit here has downsides.)
I think the main crux here is that even if Jessica/CEA agrees that the sign of the impact is positive, it still falls in the neutral bracket because on the CEA worldview the impact is roughly negligible relative to the programs that they are excited about.
If you disagree with this maybe you agree with the weaker claim of the impact being comparatively negligible weighted by the resources these companies consume? (there’s some kind of nuance to ‘consuming resources’ in profitable companies, but I guess this is more gesturing at a leaving value on the table framing as opposed to just is the organisation locally net negative or positive.
Do you think people are better off overall than otherwise because of Facebook (and social media generally)? You may have made important connections on Facebook, but many people probably invest less in each connection and have shallower relationships because of social media, and my guess is that mental health is generally worse because of social media (I think there was an RCT on getting people to quit social media, and I wouldn’t be surprised if there were multiple studies. I don’t have them offhand). I’d guess social media is basically addictive for a lot of people, so people often aren’t making well-informed decisions about how much to use, and it’s easy for it to be net negative despite widespread use. People joining social media pressures others to join, too, making it more costly to not be on it, so FB creates a problem (induces fear of missing out) and offers a solution to it. Cancel culture, bubbles/echo chambers, the spread of misinformation, and polarization may also be aggravated by social media.
That being said, maybe FB was really important for the growth of the EA community. I mostly got into EA through FB initially, although it’s not where I was first exposed to EA. If we think the EA community is important enough, then this plausibly dominates. And, of course, it’s where Open Phil’s funding came from, but that seems to be historical luck, not really anything special about Facebook, except the growth of its market cap.
On the other hand, FB accelerated the development of AI capabilities, e.g. PyTorch was primarily built by FB. But maybe we should also consider this to be only weakly related to FB’s role in social media, and more related to the fact that it’s just a large tech company.
There are also multiple counterfactuals we could consider: no Facebook + people spend less time on social media, and no Facebook + people spend about as much time on social media (possibly on one similar to FB, or whatever other options there are now). In the first case, I think it’s hard to make a balanced argument for FB being robustly net positive. In the second case, the impact is closer to 0, from either direction, and it’s harder to evaluate its sign. Then there’s the counterfactual impact of FB getting a more productive hire, or one who is otherwise more valued by FB.
I think McKinsey and Goldman would have other firms step into their spaces if they weren’t around.
I don’t think this is persuasive. I think most actions people take either increase or decrease x-risk, and you should start with a ~50% prior for which side of neutrality a specific action is on (though not clearly true; see discussion here). I agree there’s some commonsensical notions that economic growth is good, including for the LT future, but I personally find arguments in the opposite direction to be slightly stronger. Your own comment to an earlier post is one interesting item on the list of arguments I’d muster in that direction.
Ahh, interesting argument! I wasn’t thinking about the argument that these firms might (e.g.) slightly accelerate economic growth, which might then cause an increase in x-risk (if safety is not equivalently accelerated). In general I feel sufficiently unclear about such considerations—like maybe literally 50:50 equipoise is a reasonable prior—that I am loath to let them overwhelm a more concrete short-term impact story in our cost-benefit analysis, in the absence of a clear causal link to a long run impact in the opposite direction, as you suggest in the article.
In this case I think my argument still goes through, because the claim I’m objecting to is so strong—that there is in some sense a >50% probability that every reasonable scenario has all three firms being negative.
I’m surprised to see CEA making such a strong claim. I think we should have strong priors against this stance, and I don’t think I’ve seen CEA publish conclusive evidence in the opposite direction.
Firstly, note that these three companies come from very different sectors of the economy and do very different things.
Secondly, even if you assign high credence to the problems with these firms, it seems like there is a fair bit of uncertainty in each case, and you are proposing a quite harsh upper bound - ‘probably at best neutral’.
Thirdly, each of these are (broadly) free market firms, who exist only because they are able to persuade people to continue using their services. It’s always possible that they are systematically mistaken, and that CEA really does understand social network advertising, management consulting, trading and banking better than these customers… but I think our prior should be a little more modest than this. Usually when people want to buy something it is because they want that thing and think it will be useful for them.
Finally, there are in fact for each of these firms a bunch of concrete benefits they provide. Rarely do I see these explicitly weighed in the calculus against the problems:
Facebook allows people to keep in touch with friends and relatives, to share their thoughts and news about their lives, and meet like-minded new friends. Certainly I have personally made many new friends over facebook, and engaged in many good discussions. It also allows advertisers to show their products to the people who are most likely to appreciate them, saving others from having their time wasted with irrelevant ads.
McKinsey provides advice and allows for the diffusion of best practices from leading firms to others in the economy. They can also help management overcome internal veto players and other opposition to change by helping supply credibility to decisions. For some types of consulting (though a little different to what McKinsey mainly does) we even have RCTs showing that they improve productive efficiency.
Goldman’s trading arm provides a wide range of services to market participants, like research, prime brokerage and market making, that are necessary to help keep markets efficient. They also provide investment banking services, allowing companies and governments to raise money to finance projects, and retail banking, giving ordinary people higher interest rates than they’d get from their legacy banks.
It’s possible that these has been some explicit analysis of these firms to support your very strong statement. I searched on the forum for ‘McKinsey’ to try to find it, but at least the first page or so of results were generally positive references—e.g. people quoting their work on climate change, or positively referencing how they would address a problem. 80k does have an old article with some cursory analysis of the harms of finance, but the analysis is seriously flawed, and it doesn’t cover Management Consulting or Social Networks at all.
Curious if you disagree with Jessica’s key claim, which is “McKinsey << EA for impact”? I agree Jessica is overstating the case for “McKinsey ⇐ 0″, but seems like best-case for McKinsey is still order(s) of magnitude less impact than EA.
Subpoints:
Current market incentives don’t address large risk-externalities well, or appropriately weight the well-being of very poor people, animals, or the entire future.
McKinsey for earn-to-learn/give could theoretically be justified, but that doesn’t contradict Jessica’s point of spending money to get EAs
Most students require a justification for anyone charitable spending significant amounts of money on movement building & competing with McKinsey reads favorably
Agree we should usually avoid saying poorly-justified things when it’s not a necessary feature of the argument, as it could turn off smart people who would otherwise agree.
Sorry, I was trying to get a quick response to this post and I made a stronger claim than I intended. I was trying to say that I think that EA careers are doing much more good than the ones mentioned on average and so spending money is a good bet here. I wasn’t intending to make a definitive judgment about the overall social impact of those other careers, though I know my wording suggests that. I also generally want to note that this element was a personal claim and not necessarily a CEA endorsed one.
This was a great comment and thoughtful reply and the top comment was great too.
Looking at the other threads generated from the top comment, it looks like tiny turns of phrase in that top comment, produced (unreasonably) large amounts of discussion.
I think we all learned a valuable lesson about the importance of clarity and precision when commenting on the EA forum.
FYI I would have upvoted this if not for the final paragraph
I consider this to be a pretty weak argument, so it doesn’t contribute much to my priors, which although weak (and so the particulars of a company matter much more), are probably centered near neutral on net welfare effects (in the short to medium term). I think a large share of goods people buy and things they do are harmful to themselves or others before even considering the loss of income/time as a result, or worse for them than the things they compete with. It’s enough that I wouldn’t have a prior strongly in favour of what profitable companies are doing being good for us. Here are reasons pushing towards neutral or negative impacts:
A lot of goods are mostly for signaling, especially signaling wealth, which often has negative externalities and I’d guess little positive value for the individual. Brand name versions of things, clothing, jewelry, cars.
Many modern ways people spend their time (enabled by profitable companies) have probably made us less active, more indoor-bound, less close with others, and less pursuant of meaning and meaningful goals, which may conflict with people’s reflective preferences, as well as generally be bad for health, mental health and other measures of wellbeing. Basically a lot of the things we do on our computers and phones.
Many things are stimulating and addictive, and companies are optimizing for want, not welfare. Want and welfare can come apart when we optimize for want. So we get cigarettes, addictive video games, junk food, algorithms optimizing for clicks when we’d be better off stepping away from the internet or doing more substantial things online, and lots of salt, sugar and calories in our foods.
Media companies may optimize for revenue over accurate reporting. This includes outrage, playing to our fears, demonizing and polarization.
Some companies make us want their stuff for fear of missing out or social pressure, so it can be closer to coercion than providing a valuable opportunity.
I’d guess relatively little is spent on advertisement for things that we have good evidence for improving our welfare, because most of those things are hard to profit from: basic healthy foods, exercise (although there are certainly exercise products and programs that get advertised, but less so just gym memberships, joining sports leagues, running outside), just spending more time with your friends and family (in cheap ways, although travel and amusement parks are advertised), pursuing meaning or meaningful goals, helping others (even charity ads are relatively rare). So, advertisement seems to push us towards things that are worse for us than the alternatives we’d have gone with. To capitalize on the things that do make us substantially better off, companies may sell us more expensive versions that aren’t (much) better or things to go with them that don’t substantially help.
I’d expect a lot of hedonic adaptation for many goods and services, but not mental health (almost by definition), physical pain and to a lesser extent general health and mobility, which are worsened by a lot of the things companies provide, directly or indirectly by competing with the things that are better for health.
Company valuations don’t usually substantially reflect their externalities, and shorting companies is riskier and more costly than buying and holding shares, so this biases markets towards positively valuing companies even if their overall value for the world is negative.
There are often negative externalities on nonhuman animals in particular, although the overall effects on nonhuman animals may be complicated when you also consider the effects on wild animals.
I do think it’s plausible McKinsey and Goldman have done and do more good than harm for humans in the short term, based on the arguments you give, but I don’t have a strong view either way. It could depend largely on whether raising people’s consumption levels makes them better off overall (and how much) in the places where people are most affected by these companies. Measures of well-being do seem to positively correlate with income/wealth/consumption at the individual level, and I’d guess also at the aggregate level for developing countries, but I’d guess not for developed countries, or at best weakly so. There are negative externalities for increasing an individual’s income on others’ life satisfaction, although it’s possible a large share is due to rescaling, not actually thinking your life is worse absolutely than otherwise. See:
Haushofer, J., Reisinger, J., & Shapiro, J. (2019). Is your gain my pain? Effects of relative income and inequality on psychological well-being.
Based on GiveDirectly in Kenya. They had multiple measures of wellbeing, but negative effects were only observed for life satisfaction for non-recipient households of cash transfers in the same village. See Table A5.
This table from Veenhoven, R. (2019). The Origins of Happiness: The Science of Well-Being over the Life Course., reproduced in this post.
This graph, reproduced in this post.
Other writing on the Easterlin Paradox.
Some companies may also contribute to relative inequality or even counterfactually make the median or poor person absolutely poorer through their political activities.
The categories of things I’m optimistic about for human welfare in the short to medium term are:
Things that save us time, so we can spend more time on things that actually make us better off.
Things that improve or protect our health (including mental health).
Things that make us (feel) safer/more secure (physically, financially, etc.).
Things that make us more confident, but without substantially net negative externalities (negative externalities may come from positional goods, costly signaling, peer pressure).
Things that help us make better decisions, without important negative effects.
I’m neutral to optimistic about these (possibly neutral because they just replace cheaper versions of themselves that would be just as good):
In-person activities with friends/family.
Things for hobbies or projects.
Restaurants.
I’m about neutral and pretty uncertain about screen-based entertainment (TV, movies, video games), and recreational substances that aren’t extremely addictive or harmful (alcohol, marijuana).
I’m pessimistic about:
Social media.
Status-signaling goods/positional goods/luxuries.
Processed foods.
Cigarettes.
There are also a lot of externalities that act at least equally on humans, like carbon emissions, promotion of ethnic violence, or erosion of privacy. Those are all examples off the top of my head for Facebook specifically.
I upvoted Larks’ comment, but like you I think this particular argument, “people buy from these firms”, is weak.
Ok. Lark’s response seems correct.
But surely, the spirit of the original comment is correct too.
No matter which worldview you have, the value of a top leader moving into EA is overwhelmingly larger than the the social value of the same leader “rowing” in these companies.
Also, at the risk of getting into politics (and really your standard internet argument) gesturing at “free market” is really complicated. You don’t need to take the view of Matt Stoller or something to notice that the benefits of these companies can be provided by other actors. The success of these companies and their resources that allow recruitment with 7 figure campus centres probably has a root source different than pure social value.
The implication that this statement requires CEA to have a strong model of these companies seems unfair. Several senior EAs, who we won’t consider activists or ideological, have deep experiences in these or similar companies. They have opinions that are consistent with the parent comment’s statement. (Being too explicit here has downsides.)
I think the main crux here is that even if Jessica/CEA agrees that the sign of the impact is positive, it still falls in the neutral bracket because on the CEA worldview the impact is roughly negligible relative to the programs that they are excited about.
If you disagree with this maybe you agree with the weaker claim of the impact being comparatively negligible weighted by the resources these companies consume? (there’s some kind of nuance to ‘consuming resources’ in profitable companies, but I guess this is more gesturing at a leaving value on the table framing as opposed to just is the organisation locally net negative or positive.
Do you think people are better off overall than otherwise because of Facebook (and social media generally)? You may have made important connections on Facebook, but many people probably invest less in each connection and have shallower relationships because of social media, and my guess is that mental health is generally worse because of social media (I think there was an RCT on getting people to quit social media, and I wouldn’t be surprised if there were multiple studies. I don’t have them offhand). I’d guess social media is basically addictive for a lot of people, so people often aren’t making well-informed decisions about how much to use, and it’s easy for it to be net negative despite widespread use. People joining social media pressures others to join, too, making it more costly to not be on it, so FB creates a problem (induces fear of missing out) and offers a solution to it. Cancel culture, bubbles/echo chambers, the spread of misinformation, and polarization may also be aggravated by social media.
That being said, maybe FB was really important for the growth of the EA community. I mostly got into EA through FB initially, although it’s not where I was first exposed to EA. If we think the EA community is important enough, then this plausibly dominates. And, of course, it’s where Open Phil’s funding came from, but that seems to be historical luck, not really anything special about Facebook, except the growth of its market cap.
On the other hand, FB accelerated the development of AI capabilities, e.g. PyTorch was primarily built by FB. But maybe we should also consider this to be only weakly related to FB’s role in social media, and more related to the fact that it’s just a large tech company.
There are also multiple counterfactuals we could consider: no Facebook + people spend less time on social media, and no Facebook + people spend about as much time on social media (possibly on one similar to FB, or whatever other options there are now). In the first case, I think it’s hard to make a balanced argument for FB being robustly net positive. In the second case, the impact is closer to 0, from either direction, and it’s harder to evaluate its sign. Then there’s the counterfactual impact of FB getting a more productive hire, or one who is otherwise more valued by FB.
I think McKinsey and Goldman would have other firms step into their spaces if they weren’t around.
I don’t think this is persuasive. I think most actions people take either increase or decrease x-risk, and you should start with a ~50% prior for which side of neutrality a specific action is on (though not clearly true; see discussion here). I agree there’s some commonsensical notions that economic growth is good, including for the LT future, but I personally find arguments in the opposite direction to be slightly stronger. Your own comment to an earlier post is one interesting item on the list of arguments I’d muster in that direction.
Ahh, interesting argument! I wasn’t thinking about the argument that these firms might (e.g.) slightly accelerate economic growth, which might then cause an increase in x-risk (if safety is not equivalently accelerated). In general I feel sufficiently unclear about such considerations—like maybe literally 50:50 equipoise is a reasonable prior—that I am loath to let them overwhelm a more concrete short-term impact story in our cost-benefit analysis, in the absence of a clear causal link to a long run impact in the opposite direction, as you suggest in the article.
In this case I think my argument still goes through, because the claim I’m objecting to is so strong—that there is in some sense a >50% probability that every reasonable scenario has all three firms being negative.