Founder and CEO of TRACE, an AI evaluations startup.
Founder and Chair of the Board of Effective Altruism Latvia
Board Member and Treasurer of Dzīvnieku brīvība (“Animal Freedom”)
10+ year experience in positions related to finance and economics
Artūrs Kaņepājs
Karma: 123
Thanks. A general point is that I stripped the model of all nonessential elements (such as non-labor inputs, multiple goods, flexible prices, gradual automation with some firms remaining nonautomated, intl trade) to drive home the basic point that automation does not necessarily lead to an increase in output. That the interests of the firm owners are aligned with the output they get, not that of the total economy. One parallel (non-generous to firm owners) is to a dictator who may wish to increase grip of power at a huge cost to their country.
Now, if workers can find new jobs, possibly even in other industries, this is not a problem. This is the default argument and, at least over timelines that span generations, empirical observation. But this no longer holds when there are no other jobs, i.e. under fool automation. I am now not sure if the “full economy-wide automation” idea was clear in the post, maybe I should clarify it...
It does not seem that Perfect Competition alone would influence the result: the firms that automate would outcompete those that don’t. Also, it does not seem like constant per-unit of output costs (e.g. oil) would change much. Semi-fixed or fixed costs (cars, computers) could have more complex effects, probably dependent on the parametrization. I agree the model can be extended in a number of ways, this could be one.
Thanks, added an example which I hope clarifies things. In the example, taxi firm owners go ahead with the automation if they become slightly better off as a result, even if it nullifies the output for everyone else.
The example refers to a single firm. To bring this even closer to reality, the situation could be modelled with multiple firms that decide to automate simultaneously, but with fewer available rides, e.g. due to longer average time of the rides. I haven’t done the modelling explicitly but think the basic result would be the same. I.e. firm owners automate even if this leads to lower aggregate output.
Thanks, added an example.
A very interesting and fresh (at least to my mind) take, thanks again! I also think “Pause AI” is a simple ask, hard to misinterpret. In contrast, “Align AI”, “Regulate AI”, Govern, Develop Responsibly and others don’t have such advantages. Resonates with asks for a “ban” when campaigning for animals, as opposed to welfare improvements.
I do fear however that inappropriate execution can alienate supporters. Over the last several years when I told someone that I was advocating a fur farming ban, often the first reply was that they don’t support “our” tacticsm, namely—spilling paint on fur coats and letting animals out of their cages, which is not something my organisation ever did. And that’s from generally neutral or sympathetic acquaintances.
The common theme here is a Victim—either the one with a ruined fur coat, or the farmers. For AI the situation is better: the most salient Victims to my mind are a few megarich labs (assuming that the AI Pause applies to the most advanced models/capabilities). It would seem important to stress that products people already use will not be affected (to avoid loss aversion like with meat); and a limited effect on small businesses with open source solutions.
P.S. I am broadly aware about the potential of nonviolent action & that PETA is competent. But do worry that the backlash can be sizeable and lasting enough to make the expected impact negative.
Insightful stats! They also show
1) attitudes in Europe close to those in the US. My hunch is that in the EU there could be comparable or even more support for “Pause AI”, because of the absence of top AI labs.
2) A correlation with factors such as GDP and freedom of speech. Not sure which effect dominates and what to make of it. But censorship in China surely won’t help advocacy efforts.
So the stats make me more hopeful for advocacy impact also in EU & UK. But less so China, which is a relevant player (mixed recent messages on that with the chip advances & economic slowdown).
Excellent post!
Thank you David, upvoted. Coming from a small country with one big city and a small community, I read this with Global vs National in mind, as opposed to National vs City EA groups. I still think it’s probably useful for new engagement and retention to have some minimum regular online as well as physical activities (e.g. at least quarterly meetups). Though there are some ongoing and semi-fixed costs, like IT infrastructure and database maintenance. Any specific words of caution you have w.r.t. relating what you wrote to Global vs (small) National?
Thank you, very informative! Impressive numbers for the new HEAs. Suggests there could be significant untapped potential in other Baltic countries, which is inspiring.
Though the “Active members” number has remained roughly the same throughout 2022. Do you have any explanation why is that? Is there some sort of saturation? Would be interesting to know other year-on-year numbers also e.g. for HEAs, if you have them.
P.S. Coming the top mushroom hunter nation I respectfully nod about the mushroom scientists fact.
Yes Russia had convinced others and FSB had convinced Putin that it’s military was much better than it actually was; a key reason why the advances stalled and probably also why Putin launched the war.
But specifically about underestimating Ukraine’s chances, I think the “agency” did impact outcomes a lot. The willingness and ability by society to decide and agree on what’s best for the country and act accordingly is roughly what I mean by “agency” in this context.
Had Zelensky accepted offers to flee and had UA society and military accepted the outside views in the first days of the war, then the RU military could have advanced relatively easily. Even in the poor condition that it was in. But resistance had a huge backing from Ukrainians, that is why Zelensky’s popularity soared from 27% to 80-90% when he declined offers to flee. Seems likely to me that Putin did not expect that, expected a large part of population to welcome his soldiers are liberators from the unpopular government.
https://iwpr.net/global-voices/zelenskys-approval-ratings-soar-amid-war
https://ratinggroup.ua/research/ukraine/obschenacionalnyy_opros_ukraina_v_usloviyah_voyny_26-27_fevralya_2022_goda.html
Writing from Latvia here. One thing I’ve noticed is the extent to which outside observers underestimate the agency of smaller countries: either Ukraine, or newer NATO members. The interests and the resolve of the newer NATO members to join NATO to defend themselves is still neglected in the opinions of some prominent public intellectuals. The term “NATO expansion” is misleading, “NATO enlargement” is better.
One example how, I suspect, underestimation of agency did lead to wrong predictions: many predicted that the leadership of Ukraine, and the cities of Kyiv and even Lviv would quickly fall. Did not happen. (Admittedly I did not quantify predictions at the time and still feel quite ignorant about the many factors involved.)
https://www.metaculus.com/questions/9743/zelenskyy-remains-president-of-ua-by-2023/
https://www.metaculus.com/questions/9939/kyiv-to-fall-to-russian-forces-by-april-2022/
https://www.metaculus.com/questions/9899/russian-troops-in-lviv-in-2022/
Excellent news! Global cooperation is exactly what’s needed. Not only to in fact address the problem, but also as a counterpoint to one of the pillars of accelerationism i.e. the view that global cooperation on AI is impossible
Note: Venue has been changed to “Gravity Hall” in the same building.
Reassuring to read that RP treated the pledged or anticipated crypto donations with caution, and about the crisis management exercises / stress tests. Perhaps others organisations can learn. Thanks.
If FTX leadership had refused, they should have refused to run the FTX Foundation and made it public that FTX leadership had refused the audit. Then, EA leaders should have discouraged major EA organizations from taking money from the FTX Foundation and promoted a culture of looking down on anyone who took money from the Foundation.
To continue thinking it through: the above seems like a theoretical sequence of outcomes that would never in fact materialize. More likely FTX leadership would have known ahead of time and wouldn’t have offered funding in the first place.
I think it’s useful to think about what useful actions would have been. But what really matters is—how to act going forward. IMHO any ad hoc decision by FTX founders to request audit for one funder but not another seems problematic. Can be influenced by conflicts of interest, private relations, and a general lack of competence/standards about such situations. Ideally I think there would be a published list of requirements, including audit/governance requirements, to which donors should adhere.
Then again, donors & appropriate levels of audit scrutiny probably vary widely, so it would not be easy to specify the details needed. I guess much can be learned from the KYC/AML (know you client/anti money laundering) practices in banking. Also, some industries can be ruled out completely (I’m not of the opinion that crypto should, but not far from it anymore). An [old] example of an exclusion list for a bank:
https://www.ebrd.com/downloads/about/sustainability/Environmental_and_Social_Exclusion_and_Referral_Lists_15092008.pdf
I do think EA is above treating this as a black swan event. Fraud in unregulated finance (crypto even more so) even if at least initially guided by good (no to speak of naively utilitarian) intentions is to be expected. Most people did not expect this to happen with SBF/FTX, but some did. There’s a lot of potential to learn from this and make the movement more resilient against future cases of funder’s fraud via guidelines, practices. E.g. clarifying that dirty money won’t work towards achieving EA aims. And that EA credibility should not be lent to dubious practices.
Other than that I agree with the gist of this post & comment but it’s also important to gradually update views. Upvoted the comment of John_Maxwel
Thanks for laying this out. May help many to figure out what to do.
My view about a red line at the moment is that a charity should stop accepting donations once it’s know that a source is fraudulent. To avoid fraudsters deriving benefit from being associated with the charity. However, when the donations are in the past, there is only a very weak, if any, PR benefit for the fraudsters at hand. In that situation an important (and in EA context maybe the main) harm is that keeping the money by charity will encourage future naive consequentialists. So this indirect effect on one side, and the immediate harm to the causes on the other. It’s a tough call, glad I don’t have to make it.
It would be great if the current discussion leads to a clear precedent and it being clearly stated in EA charity policies what will be done in case of funder’s fraud. This would give the clear signals to the naive consequentialists and (in case of fraud) make decisions for charities easier. Actually I bet there are well described precedents and some guidelines available, is anyone aware of such?
My impression is that these concerns are in practice pretty much recognized, which is behind the EA focus on extinction or similar permanent changes, i.e. absorbing states. Forecast precision becomes irrelevant after entering an absorbing state, and so becomes “diminution” and “washing out” (“option unawareness” still seems relevant).