Garrison

• Garrison 5 Dec 2024 17:44 UTC

  4 points

  0 ∶ 0

  in reply to: Sarah Cheng’s comment on: Is the AI Dooms­day Nar­ra­tive the Product of a Big Tech Con­spir­acy?

  Thanks Sarah!

• Garrison 5 Dec 2024 17:44 UTC

  7 points

  0 ∶ 0

  in reply to: titotal’s comment on: Is the AI Dooms­day Nar­ra­tive the Product of a Big Tech Con­spir­acy?

  Bengio and Hinton are the two most-cited researchers alive. Ilya Sutskever is the 3rd most cited AI researcher, and though he’s not on that paper, the superalignment intro blog post from OpenAI says this, “Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue.” LeCun is probably the top AI researcher who’s not worried about controlling a superintelligence (4th in total citations after Sutskever).

  This is obviously a semantics disagreement, but I stand by the original claim. Note that I’m not saying that all the top AI researchers are worried about x-risk.

  In regards to your overall point, it does not rebuts the idea that some people have been cynically exploiting AI fears for their own gain. I mean, remember that OpenAI was founded as an AI safety organisation. The actions of Sam Altman seem entirely consistent with someone hyping X-risk in order to get funding and support for OpenAI, then pivoting to downplaying risk as soon as ditching safety gets more profit. I doubt this applies to all people or even the majority, but it does seem like it’s happened at least once.

  I largely agree with this and alluded to this possibility here:

  If AI companies ever needed to rely on doomsday fears to lure investors and engineers, they definitely don’t anymore.

  I might write a separate piece on the best evidence for the hype argument, which OpenAI I think has been the biggest winner of. My guess is that Altman actually did believe what he was saying about AI risk back in 2015. Superintelligence came out the year before, and it’s not a surprising view for him to have given what else we know about him.

  I’d also guess that Altman and Elon are two of the people most associated with the x-risk story, which has been the biggest driver of skepticism about it.

  There’s also been more recent evidence of him ditching x-risk fears now that it seems convenient. From a recent Fox News interview:

  Interviewer: “A lot of people who don’t understand AI, and I would put myself in that category, have got a basic understanding, but they worry about AI becoming sentient, about it making autonomous decisions, about it telling humans you’re no longer in charge?”

  Altman: “It doesn’t seem to me to be where things are heading…is it conscious or not will not be the right question, it will be how complex of a task can it do on its own?”

  Interviewer: “What about when the tool gets smarter than we are? Or the tool decides to take over?”

  Altman: “I think tools in many senses are already smarter than we are. I think that the internet is smarter than you or I, the internet knows a lot of things. In fact, society itself is vastly smarter and more capable than any one person. I think we’re already good at working with tools, institutions, structures, whatever you want to call it, that are vastly more capable than one person and as long as we have a reasonably level playing field where one person or one company has vastly more power than anybody else, I think we know how to deal with that.”

Is the AI Dooms­day Nar­ra­tive the Product of a Big Tech Con­spir­acy?

• Garrison 28 Nov 2024 19:25 UTC

  5 points

  0 ∶ 0

  in reply to: 1a3orn’s comment on: Gw­ern on cre­at­ing your own AI race and China’s Fast Fol­lower strat­egy.

  Hsu clarified his position on my thread here:

  “Clarifications:

  1. The mafia tendencies (careerist groups working together out of self-interest and not to advance science itself) are present in the West as well these days. In fact the term was first used in this way by Italian academics.

  2. They’re not against big breakthroughs in PRC, esp. obvious ones. The bureaucracy bases promotions, raises, etc. on metrics like publications in top journals, cititations, … However there are very obvious wins that they will go after in a coordinated way—including AI, semiconductors, new energy tech, etc.

  3. I could be described as a China hawk in that I’ve been pointing to a US-China competition as unavoidable for over a decade. But I think I have more realistic views about what is happening in PRC than most China hawks. I also try to focus on simple descriptive analysis rather than getting distracted by normative midwit stuff.

  4. There is coordinated planning btw govt and industry in PRC to stay at the frontier in AI/​AGI/​ASI. They are less susceptible to “visionaries” (ie grifters) so you’ll find fewer doomers or singularitarians, etc. Certainly not in the top govt positions. The quiet confidence I mentioned extends to AI, not just semiconductors and other key technologies.”

• Garrison 21 Nov 2024 4:13 UTC

  8 points

  2 ∶ 0

  in reply to: Seth Herd’s comment on: China Hawks are Man­u­fac­tur­ing an AI Arms Race

  Pasted from LW:

  Hey Seth, appreciate the detailed engagement. I don’t think the 2017 report is the best way to understand what China’s intentions are WRT to AI, but there was nothing in the report to support Helberg’s claim to Reuters. I also cite multiple other sources discussing more recent developments (with the caveat in the piece that they should be taken with a grain of salt). I think the fact that this commission was not able to find evidence for the “China is racing to AGI” claim is actually pretty convincing evidence in itself. I’m very interested in better understanding China’s intentions here and plan to deep dive into it over the next few months, but I didn’t want to wait until I could exhaustively search for the evidence that the report should have offered while an extremely dangerous and unsupported narrative takes off.

  I also really don’t get the error pushback. These really were less technical errors than basic factual errors and incoherent statements. They speak to a sloppiness that should affect how seriously the report should be taken. I’m not one to gatekeep ai expertise, but idt it’s too much to expect a congressional commission with a top recommendation to commence in a militaristic AI arms race to have SOMEONE read a draft who knows that chatgpt-3 isn’t a thing.

China Hawks are Man­u­fac­tur­ing an AI Arms Race

• Garrison 15 Nov 2024 18:55 UTC

  2 points

  0 ∶ 0

  in reply to: Camille’s comment on: Is Deep Learn­ing Ac­tu­ally Hit­ting a Wall? Eval­u­at­ing Ilya Sutskever’s Re­cent Claims

  Thanks Camille! Glad you found it useful.

• Garrison 14 Nov 2024 16:15 UTC

  6 points

  0 ∶ 0

  in reply to: PabloAMC 🔸’s comment on: Is Deep Learn­ing Ac­tu­ally Hit­ting a Wall? Eval­u­at­ing Ilya Sutskever’s Re­cent Claims

  Yeah, I got some pushback on Twitter on this point. I now agree that it’s not a great analogy. My thinking was that we technically know how to build a quantum computer, but not one that is economically viable (which requires technical problems to be solved and for the thing to be scalable/​not too expensive). Feels like a not all squares are rectangles, but all rectangles are squares thing. Like quantum computing ISN’T economically viable but that’s not the main problem with it right now.

• Garrison 13 Nov 2024 19:33 UTC

  3 points

  0 ∶ 0

  in reply to: Andy Morgan 🔸’s comment on: Is Deep Learn­ing Ac­tu­ally Hit­ting a Wall? Eval­u­at­ing Ilya Sutskever’s Re­cent Claims

  Thanks so much Andy! Hope you enjoy :)

Is Deep Learn­ing Ac­tu­ally Hit­ting a Wall? Eval­u­at­ing Ilya Sutskever’s Re­cent Claims

• Garrison 25 Oct 2024 18:34 UTC

  2 points

  0 ∶ 0

  in reply to: Larks’s comment on: Miles Brundage re­signed from OpenAI, and his AGI readi­ness team was disbanded

  thank you!

Miles Brundage re­signed from OpenAI, and his AGI readi­ness team was disbanded

The Tech In­dus­try is the Biggest Blocker to Mean­ingful AI Safety Regulations

My ar­ti­cle in The Na­tion — Cal­ifor­nia’s AI Safety Bill Is a Mask-Off Mo­ment for the Industry

• Garrison 17 Jul 2024 16:27 UTC

  2 points

  5 ∶ 4

  on: Against Aschen­bren­ner: How ‘Si­tu­a­tional Aware­ness’ con­structs a nar­ra­tive that un­der­mines safety and threat­ens humanity

  “With Islamic terrorism, these involved mass surveillance and detention without trial.”

  I think Islamist terrorism would be more accurate and less inflammatory.

• Garrison 17 Jul 2024 16:26 UTC

  5 points

  1 ∶ 0

  on: Against Aschen­bren­ner: How ‘Si­tu­a­tional Aware­ness’ con­structs a nar­ra­tive that un­der­mines safety and threat­ens humanity

  BTW, this link (Buzan, Wæver and de Wilde, 1998) goes to a PaperPile citation that’s not publicly accessible.

• Garrison 8 Jul 2024 16:47 UTC

  2 points

  0 ∶ 0

  in reply to: Mayowa Osibodu’s comment on: Pod­cast with Yoshua Ben­gio on Why AI Labs are “Play­ing Dice with Hu­man­ity’s Fu­ture”

  I think building AI systems with some level of autonomy/​agency would make them much more useful, provided they are still aligned with the interests of their users/​creators. There’s already evidence that companies are moving in this direction based on the business case: https://​​jacobin.com/​​2024/​​01/​​can-humanity-survive-ai#:~:text=Further%2C%20academics%20and,is%20pretty%20good.%E2%80%9D

  This isn’t exactly the same as self-interest, though. I think a better analogy for this might be human domestication of animals for agriculture. It’s not in the self-interest of a factory farmed chicken to be on a factory farm, but humans have power over which animals exist so we’ll make sure there are lots of animals who serve our interests. AI systems will be selected for to the extent they serve the interests of people making and buying them.

  RE international development: competition between states undercut arguments for domestic safety regulations/​practices. These are exacerbated by beliefs that international rivals will behave less safely/​responsibly, but you don’t actually need to believe that to justify cutting corners domestically. If China or Russia built an AGI that was totally safe in the sense that it is aligned with its creators interests, that would be seen as a big threat by the US govt.

  If you think that building AGI is extremely dangerous no matter who does it, then having more well-resourced players in the space increases the overall risk.

• Garrison 27 Jun 2024 21:31 UTC

  14 points

  5 ∶ 3

  on: Re­sponses to Man­i­fest are not about Manifest

  People can and should read whoever and whatever they want! But who a conference chooses to platform/​invite reflects on the values of the conference organizers, and any funders and communities adjacent to that conference.

  Ultimately, I think that almost all of us would agree that it would be bad for a group we’re associated with to platform/​invite open Nazis. I.e. almost no one is an absolutist on this issue. If you agree, then you’re not in principle opposed to exlcuding people based on the content of their beliefs, so the question just becomes: where do you draw the line? (This is not a claim that anyone at Manifest actually qualifies as an open Nazi, more just a reductio to illustrate the point.)

  Answering this question requires looking at the actual specifics: what views do people hold? Were those views legible to the event organizers? I fear that a lot of the discourse is getting bogged down in euphemism, abstraction, and appeals to “truth-seeking,” when the debate is actually: what kind of people and worldviews do we give status to and what effects does that have on related communities.

  If you think that EA adjacent orgs/​venues should platform open Nazis, as long as they use similar jargon, then I simply disagree with you, but at least you’re being consistent.

• Garrison 19 Jun 2024 21:00 UTC

  37 points

  19 ∶ 1

  in reply to: jacobjacob’s comment on: My ex­pe­rience at the con­tro­ver­sial Man­i­fest 2024

  My mistake on the guardian US distinction but to call it a “small newspaper” is wildly off base, and for anyone interacting with the piece on social media, the distinction is not legible.

  Candidly, I think you’re taking this topic too personally to reason clearly. I think any reasonable person evaluating the online discussion surrounding manifest would see it as “controversial.” Even if you completely excluded the guardian article, this post, Austin’s, and the deluge of comments would be enough to show that.

  It’s also no longer feeling like a productive conversation and distracts from the object level questions.

• Garrison 19 Jun 2024 20:50 UTC

  14 points

  4 ∶ 6

  in reply to: richard_ngo’s comment on: Why so many “racists” at Man­i­fest?

  What prominent left wing thinkers exhibited anti semitism recently?