Jonas Hallgren

• Jonas Hallgren 4 Mar 2025 13:35 UTC

  1 point

  0 ∶ 0

  on: Where are all the deep­fakes?

  I guess a random thought I have here is that you would probably want video and you would probably want it to be pretty spammable so you have many shots at it. Looking at twitter we already see like a large amounts of bots around commenting on things which is like a text deepfake.
  
  Like I can see in a year or so when SORA is good enough that creating a short form stabel video is easy we will see a lot more manipulation of voters through various social media through deepfakes.
  
  (I don’t think the tech is easy enough to use yet for it to be painless to do it even though it is possible. I spent a couple of hours trying to set this up for a showcase once and you had to do some fine-tuning and training stuff, there was no plug and play which is probably a bottleneck for now.)

• Jonas Hallgren 28 Feb 2025 10:01 UTC

  3 points

  0 ∶ 0

  in reply to: Linch’s comment on: Linch’s Shortform

  FWIW, I find that if you analyze places where we’ve successfully aligned things in the past (social systems or biology etc.) you find that the 1th and 2nd types of alignment really don’t break down in that way.
  
  After doing Agent Foundations for a while I’m just really against the alignment frame and I’m personally hoping that more research in direction will happen so that we get more evidence that other types of solutions are needed. (e.g alignment of complex systems such as has happened in biology and social systems in the past)

• Jonas Hallgren 13 Feb 2025 10:03 UTC

  1 point

  0 ∶ 2

  in reply to: Matthew_Barnett’s comment on: Matthew_Bar­nett’s Shortform

  FWIW, I completely agree with what you’re saying here and I think that if you seriously go into consciousness research and especially for what we westerners more label as a sense of self rather than anything else it quickly becomes infeasible to hold a position that the way we’re taking AI development, e.g towards AI agents will not lead to AIs having self-models.

  For all matters and purposes this encompasses most theories of physicalist or non-dual theories of consciousness which are the only feasible ones unless you want to bite some really sour apples.

  There’s a classic “what are we getting wrong” question in EA and I think it’s extremely likely that we will look back in 10 years and say, “wow, what are we doing here?”.

  I think it’s a lot better to think of systemic alignment and look at properties that we want for the general collective intelligences that we’re engaging in such as our information networks or our institutional decision making procedures and think of how we can optimise these for resillience and truth-seeking. If certain AIs deserve moral patienthood then that truth will naturally arise from such structures.

  (hot take) Individual AI alignment might honestly be counter-productive towards this view.

• Jonas Hallgren 9 Feb 2025 10:50 UTC

  2 points

  0 ∶ 0

  on: FIRE and do­ing good

  I’m not a career councellor so take everything with a grain of salt but you did publically post this asking for unsolicited advice, so here you go!
  
  So, more directly if you’re thinking of EA as a community that needs specific skills and you’re wondering what to do, your people management skills, strategy & general leadership skills are likely to be high in demand from other organisations: https://​​forum.effectivealtruism.org/​​posts/​​LoGBdHoovs4GxeBbF/​​meta-coordination-forum-2024-talent-need-survey
  
  Someone else mentioned that enjoyment can be highly organisation specific and even specific to the stage of the organisation.

  My thought is something like:

  1. Take a year off and commit yourself to only doing exploration during the year, try out working in different organisations at different scales, maybe more early stage maybe later stage, I’m sure you got some knowledge on what is best here.

     1. Here’s a fun book that mentions optimal exploration exploitation, I think of this a lot when it comes to my own life, it might be useful:

        1. https://​​www.goodreads.com/​​book/​​show/​​25666050-algorithms-to-live-by

     2. I thought this book was pretty good for a very specific strategy of quick career role exploration and how you can go about doing that:

        1. https://​​www.goodreads.com/​​book/​​show/​​26046333-designing-your-life

  2. Think about what roles that you can leverage your strategic people management & leadership skills whilst still enjoying the work? If you really want to do more coding then a CTO or similar role somewhere probably makes a lot of sense.

     1. Maybe you could work at a deep tech company?

     2. Maybe early stage startups is something you enjoy more, maybe you’re more of a zero to one type of person?

     3. Figure out what it exactly is that you don’t enjoy, you might be surprised, you might not be.

  3. Test, test, test. If you’ve found yourself able to do this in the past you have a lot of clout to be able to do it again, it is a lot easier for an executive to get investment again, know what people to hire, etc.

  4. I know a bunch of people who have felt similar things to what you’re doing in this moment, specifically people in executive managerial roles. The pattern that I see from everyone is that take a break (shocker!) and then it really varies when it comes to how fast they get back into it again.

     1. Maybe there are specific mental health things you can improve that makes you 20% more effective at listening that can really help at the next thing?

     2. I like to think of it as them decompressing and learning the lessons from the past very focused period before getting back at it again.

  Those are some random thoughts, best of luck to you!

• Jonas Hallgren 20 Jan 2025 21:56 UTC

  9 points

  2 ∶ 1

  on: What are we do­ing about the EA Fo­rum? (Jan 2025)

  So I’ll just give some reporting on a vibe I’ve been feeling on the forum.

  I feel a lot more comfortable posting on LessWrong compared to the EA forum because it feels like there’s a lot more moral outrage here? Like if I go back 3 or 4 years I felt that the forum was a lot more open to discussing and exploring new ideas. There’s been some controversies recently around meat-eater problem stuff and similar and I can’t help but just feel uncomfortable posting stuff with how people have started to react?

  I like the different debate weeks as I think it sets up a specific context to create more content which is quite great. Maybe it’s a vibe thing, maybe it’s something else but I feel that the virtue of open-hearted truthseeking is missing compared to a couple of years back and it makes me want to avoid posting.

  I do believe that the post standard should be lowered at least a bit and for things to be more exploratory again. So uhhhm, more events that invite more community writing and engagement?

• Jonas Hallgren 24 Dec 2024 15:53 UTC

  4 points

  0 ∶ 1

  on: GiveWell may have made 1 billion dol­lars of harm­ful grants, and Am­bi­tious Im­pact in­cu­bated 8 harm­ful or­gani­sa­tions via in­creas­ing fac­tory-farm­ing?

  I want to preface that I don’t have a strong opinion here, just some curiosity and a question.

  If we are focusing on second order effects wouldn’t it make sense to bring up something like moral circle expansion and its relation to ethical and sustainable living over time as well?

  From a long-term perspective, I see one of the major effects of global health being better decision making through moral circle expansion.

  My question to you is then what time period you’re optimising for? Does this matter for the argument?

• Jonas Hallgren 19 Dec 2024 8:53 UTC

  6 points

  0 ∶ 0

  in reply to: David Thorstad’s comment on: Blog up­date: Reflec­tive altruism

  Thank you for that substantive response, I really appreciate it! It was also very nice that you mentioned the Turner et.al definitions, I wasn’t expecting that.
  
  (Maybe write a post on that? There’s a comment that mentions uptake from major players in the EA ecosystem and maybe if you acknowledge you understand the arguments they would be more sympathetic? Just a quick thought but it might be worth engaging there a bit more?)
  
  I just wanted to clarify some of the points I was trying to make yesterday as I do realise that they didn’t all get across as I wanted them to.

  I completely agree with you on the advancing progress point, I personally am quite against it from a “general”-level, I do not believe that we will be able to counterfactually change the “rowing” speed that much in the grand scheme of things. I also believe that is the conclusion of Toby’s posts if I remember correctly. Toby was rather stating that existential risk reduction is worth a lot compared to any progress that we might be able to make. “Steering” away from the bad stuff is worth more. (That’s the implicit claim from the modelling even though he’s as epistemically humble as you philosophers always are (which is commendable!).)

  Now for the power-seeking stuff. I appreciate your careful reasoning about these things and I see what you mean in that there’s no threat model from that claim in itself. If we say that the classical way it is construed is something that is equivalent to minimizing free energy, this is a tautological statement and doesn’t help for existential risk.

  I think I can agree with you that we’re not clear enough about the existential risk angle to have a clearly defined goal for what to do. I do think there’s an argument there but that we have to be quite clear with how we’re defining it for it to make foundational sense. A question that arises is if in the process of working on it we get more clarity about what it fundamentally is, similar to a startup figuring out what they’re doing along the way? It might still be worth the resources from a unknown unknown perspective and institutional practices shifting perspective if that makes sense? TAI is such a big thing and it will only happen once so spending those resources on relatively shaky foundations might still make sense?

  I’m, however, not sure that this is the case and Wei Dai for example has an entire agenda about “metaphilosophy” where the claim is that we’re too philosophically confused to make sense of alignment. In general, I would agree that ensuring the philosophical and mathematical basis is very important to coordinate the field and it is something I’ve been thinking about for a while.

  I personally am trying to import ideas from existing fields that deal with generally intelligent agents in biology and cognitive science such as Active Inference and Computational Biology into the mix to see how TAI will affect society. If we see smaller branches of science as specific offshoots of philosophy then I think the places with the most rigorous thinking on the foundations are the ones that have dealt with it for a long time. I’ve found a lot of interesting models about misalignment in these areas that I think can be transported into the AI Safety frame.

  I really appreciate the deconstructive approach that you have to the intellectual foundations of the field. I do believe that there are alternatives to the classic risk story but you have to some extent break down the flaws in the existing arguments in order to advocate for new arguments.

  Finally, where I think these threat models come from are arguments similar to the ones in What Failure Looks Like from Paul Christiano and the going out with a wimper idea. This is also explored in Yuval Noah Harari’s books Nexus and Homo Deus. This threat model is more similar to the authoritian capture idea compared to something like a runaway intelligence explosion.

  I’m looking forward to more work in this area from you!

• Jonas Hallgren 18 Dec 2024 12:45 UTC

  21 points

  1 ∶ 0

  on: Blog up­date: Reflec­tive altruism

  Thank you for this post David!
  
  I’ve from time to time engaged with my friends in discussion about your criticisms of longtermism and some existential risk calculations. I found that this summary post of your work and interaction calrifies my perspective on the general “inclination” that you have in engaging with the ideas, one that seems like a productive one!

  Sometimes, I felt that it didn’t engage with some of the core underlying claims of longtermism and exisential risk which did annoy me.

  I want to respect the underlying time spend assymmetry of the following question as I feel I could make myself less ignorant if I had the time to spend which I feel I currently do not have. But what are your thoughts on Toby Ord’s perspective and posts on existential risk?: https://​​forum.effectivealtruism.org/​​posts/​​XKeQbizpDP45CYcYc/​​on-the-value-of-advancing-progress
  and:
  https://​​forum.effectivealtruism.org/​​posts/​​hh7bgsDzP6rKZ5bbW/​​robust-longterm-comparisons
  
  I felt that some of the arguments where about discount rates and that they didn’t really make that much moral sense to me, neither did person-affecting views. I have hard time seeing the arguments for them and maybe that’s just the crux of the matter.
  
  The following will be unfair to say as I haven’t spent the time required to fully understand your models but I sometimes feel that there are deeper underlying assumptions and questions that you pass by in your arguments.
  
  I will be going to a domain I know well, AI Safety. For example, I agree with the power-seeking arguments not being fully true, especially not the early papers yet it doesn’t engage with later follow up work such as:
  https://​​arxiv.org/​​pdf/​​2303.16200
  
  Finally I believe that for the power-seeking claim, there’s a large amount of evidence for power-seeking within real world systems. For me it seems an overstep to reject power-seeking due to MIRI work?
  
  You can redefine power-seeking itself as minimizing free energy which is in itself a theory of predictive processing or Active Inference and that has showed to have remarkable predictive capacity for saying useful things about systems that are alive. Yes a specific interpretation of power-seeking may not hold true but for me it is throwing the baby out with the bathwater.
  
  I would love to hear your thoughts here and I’m looking forward to more good-faith discussions! (this is not sarcasm but I’m genuinely happy that you’re engaging with good faith arguments!)
  
  Edit: I do want to clarify that I do not believe that any AI system will converge towards instrumental goals and that it does make sense to question the foundations of the AI Safety assumptions and that I appluade you for doing so. It is rather a question of how much it will do so and under what conditions, in what systems it will do so. (I also made the language less combative)

• Jonas Hallgren 18 Dec 2024 9:58 UTC

  8 points

  0 ∶ 0

  on: There is No EA Sort­ing Hat

  I’ve found a lot of my EA friends falling into this decision paralysis so thank you for this post, I will link this to them!

• Jonas Hallgren 24 Nov 2024 18:06 UTC

  1 point

  0 ∶ 0

  in reply to: jessica_mccurdy🔸’s comment on: How I Im­proved my Wellbeing

  I just did different combinations of the sleep supplements, you still get the confounder effects but it removes some of the cross-correlation. So Glycine 3 days, no magnesium followed by magnesium 3 days, no glycine e.t.c. It’s not necessarily going to give you a high accuracy but you can see if it works or not and a rough effect size

• Jonas Hallgren 22 Nov 2024 11:03 UTC

  3 points

  0 ∶ 0

  in reply to: huw’s comment on: How I Im­proved my Wellbeing

  I use bearable for 3 months at a time to get a picture of what is currently working. You can track effect sizes of supplements in sleep quality for example if you also have a way of tracking your sleep.
  
  Funnily enough, I noticed there were a bunch of ⁸⁰⁄₂₀ stuff in my day through using bearable. I found doing a cold shower, loving kindness meditation in the morning and getting sunlight in the morning were like a difference of 30% in energy and enjoyment so I now do these religiously and it has worked wonders. (I really like bearable for these sorts of experiments.)

• Jonas Hallgren 21 Oct 2024 14:55 UTC

  3 points

  0 ∶ 0

  in reply to: Neel Nanda’s comment on: How much I’m pay­ing for AI pro­duc­tivity soft­ware (and the fu­ture of AI use)

  Sorry for not noticing the comment earlier!
  
  Here’s the Claude distillation based on my reasoning on why to use it:
  
  Reclaim is useful because it lets you assign different priorities to tasks and meetings, automatically scheduling recurring meetings to fit your existing commitments while protecting time for important activities.
  
  For example, you can set exercising three times per week as a priority 3 task, which will override priority 2 meetings, ensuring those exercise timeblocks can’t be scheduled over. It also automatically books recurrent meetings so they fit into your existing schedule, like for team members or mentors/​mentees.
  
  This significantly reduces the time and effort spent on scheduling, as you can easily add new commitments without overlapping more important tasks. The main advantage is the ability to set varying priorities for different tasks, which streamlines the process of planning weekly and monthly calls, resulting in almost no overhead for meeting planning and making it simple to accommodate additional commitments without conflicting with higher-priority tasks..

• Jonas Hallgren 11 Oct 2024 18:15 UTC

  1 point

  0 ∶ 0

  on: How much I’m pay­ing for AI pro­duc­tivity soft­ware (and the fu­ture of AI use)

  Thanks Jacques! I was looking for an upgrade to some of my LLM tools. I was looking for some IDEs and I’ll check that out.

  The only tip I’ve got is using reclaim.ai instead of calendly for automatic meeting scheduling, it slaps.

• Jonas Hallgren 9 Oct 2024 10:12 UTC

  2 points

  0 ∶ 0

  in reply to: crunk004’s comment on: What does the sys­tems per­spec­tive say about effec­tive in­ter­ven­tions?

  Thanks! That post adresses what I was pointing at a lot better than I did in mine.
  
  I can see from your response that I didn’t get across my point as well as I wanted to but I appreciate the answer none the less!
  
  It was more a question of what leads to the better long-term consequences rather than combining them.

[Question] What does the sys­tems per­spec­tive say about effec­tive in­ter­ven­tions?

• Jonas Hallgren 8 Oct 2024 9:12 UTC

  3 points

  0 ∶ 1

  on: Dis­cus­sion thread: An­i­mal Welfare vs. Global Health De­bate Week

  It seems plausible animals have moral patienthood and so the scale of the problem is larger for animals whilst also having higher tractability. At the same time, you have cascading effects of economic development into better decision making. As a longtermist, this makes me very uncertain on where to focus resources. I will therefore put myself centrally to signal my high uncertainty.

• Jonas Hallgren 18 Sep 2024 7:28 UTC

  3 points

  1 ∶ 0

  in reply to: Arepo’s comment on: Eval­u­a­tions from Man­i­fund’s EA Com­mu­nity Choice initiative

  I think that still makes sense under my model of a younger and less tractable field?
  
  Experience comes partly from the field being viable for a longer period of time since there can be a lot more people who have worked in that area in the past.

  The well-described steps and concrete near-term goals can be described as a lack of easy tractability?

  I’m not saying that it isn’t the case that the proposals in longtermism are worse today but rather that it will probably look different in 10 years? A question that pops up for me is about how great the proposals and applications were in the beginning of animal welfare as a field. I’m sure it was worse in terms of legibility of the people involved and the clarity of the plans.(If anyone has any light to shed on this, that would be great!)

  Maybe there’s some sort of effect where the more money and talent a field gets the better the applications get. To get there you first have to have people spend on more exploratory causes though? I feel like there should be anecdata from grantmakers on this.

• Jonas Hallgren 16 Sep 2024 18:19 UTC

  3 points

  0 ∶ 0

  on: Eval­u­a­tions from Man­i­fund’s EA Com­mu­nity Choice initiative

  I enjoyed the post and I thought the platform for collective action looked quite cool.

  I also want to mention that I think tractability is just generally a really hard thing for longtermism. It’s also a newer field and so on expectation I think you should just believe that the projects will look worse than in animal welfare. I don’t think there’s any need for psychoanalysis of the people in the space even though it has its fair share of wackos.

• Jonas Hallgren 8 Jul 2024 15:56 UTC

  6 points

  1 ∶ 0

  on: Mak­ing AI Welfare an EA pri­or­ity re­quires jus­tifi­ca­tions that have not been given

  Great point, I did not think of the specific claim of 5% when thinking of the scale but rather whether more effort should be spent in general.

  My brain basically did a motte and baily on me emotionally when it comes to this question so I appreciate you pointing that out!

  It also seems like you’re mostly critiquing the tractability of the claim and not the underlying scale nor neglectedness?

  It kind of gives me some GPR vibes as for why it’s useful to do right now and that dependent on initial results either less or more resources should be spent?

• Jonas Hallgren 7 Jul 2024 19:26 UTC

  4 points

  0 ∶ 0

  on: Re­think Pri­ori­ties’ Digi­tal Con­scious­ness Pro­ject Announcement

  Super exciting!
  
  I just wanted to share a random perspective here: Would it be useful to model sentience alongside consciousness itself?

  If you read Daniel Dennett’s book Kinds of Minds or take some of the Integrated Information Theory stuff seriously, you will arrive at this view of a field of consciousness. This view is similar to Philip Goff’s or to more Eastern traditions such as Buddhism.

  Also, even in theories like Global Workspace Theory, the amount of localised information at a point in time matters alongside the type of information processing that you have.

  I’m not a consciousness researcher or anything, but I thought it would be interesting to share. I wish I had better links to research here and there, but if you look at Dennett, Philip Goff, IIT or Eastern views of consciousness, you will surely find some interesting stuff.