TFD

• TFDApr 24, 2025, 11:54 PM

  3 points

  1 ∶ 0

  in reply to: Yarrow’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  My personal take is that there are pretty reasonable arguments that what we have seen in AI/​ML since 2015 suggests AI will be a big deal. I like the way I have seen Yoshua Bengio talk about it “over the next few years, or a few decades”. I share the view that either of those possibilities are reasonable. People who are highly confident that something like AGI is going to arrive over the next few years are more confident in this than I am, but I think that view is within the bounds of reasonable interpretation of the evidence. I think it is also with-in the bounds of reasonable to have the opposite view, that something like AGI is most likely further than a few years away.

  Don’t believe me? Talk to me again in 5 years and send me a fruit basket. (Or just kick the can down the road and say AGI is coming in 2035...)

  I think this is a healthy attitude and that I think is worth appreciating. We may get answers to these questions over the next few years. That seems pretty positive to me. We will be able to resolve some of these disagreements productively by observing what happens. I hope people who have different views now keep this in mind and that the environment is still in a good place for people who disagree now to work together in the future if some of these disagreements get resolved.

  I will offer the ea forum internet-points equivalent of a fruit basket to anyone who would like one in the future if we disagree now and in the future they are proven right and I am proven wrong.

  I think part of the sociological problem is that people are just way too polite about how crazy this all is and how awful the intellectual practices of effective altruists have been on this topic.

  Can you saw what view it is you think is crazy? It seems quite reasonable to me to think that AI is going to be a massive deal and therefore that it would be highly useful to influence how it goes. On other other hand, I think people often over-estimate the robustness of the arguments for any given strategy for how to actually do that influencing. In other words, its reasonable to prioritize AI, but people’s AI takes are often very over-confident.

• TFDApr 24, 2025, 11:30 PM

  −1 points

  0 ∶ 1

  in reply to: Yarrow’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  I appreciate your comment.

  It seems clear that if Jaime had different views about the risk-reward of hypothetical 21st century AGI, nobody would be complaining about him loving his family.

  I do think this is substantially correct, but I also want to acknowledge that these can be difficult subjects to navigate. I think anyone has done anything wrong, I’m sure I myself have done something similar to this many times. But I do think its worth trying to understand where the central points of disagreement lie, and I think this really is the central disagreement.

  On the question of changing EA attitudes towards AI over the years, although I personally think AI will be a big deal, could be dangerous, and those issues are worth of significant attention, I also can certainly see reasons why people might disagree and why those people would have reasonable grievances with decisions by certain EA people and organizations.

  An idea that I have pondered for a while about EA is a theory about which “boundaries” a community emphasizes. Although I’ve only ever interacted with EA by reading related content online, my perception is that EA really emphasizes the boundary around the EA community itself, while de-emphasizing the boundaries around individual people or organizations. The issues around Epoch I think demonstrate this. The feeling of betrayal comes from viewing “the community” as central. I think a lot of other cultures that place more emphasize on those other boundaries might react differently. For example, at most companies I have worked at, although certainly they would never be happy to see an employee leave, they wouldn’t view moving to another job as a betrayal, even if an employee went to work for a direct competitor. I personally think placing more emphasis on orgs/​individuals rather than the community as a whole could have some benefits, such as with the issue you raise about how to navigate changing views on AI.

  Although emphasizing “the community” might seem like its ideal for cooperation, I think it can actually harm cooperation in the presence of substantial disagreements, because it generates dynamics like what is going on here. People feel like they can’t cooperate with people across the disagreement. We will probably see some of these disagreements resolved over the next few years as AI progresses. I for one hope that even if I am wrong I can take any necessary corrections on-board and still work with people who I disagreed with to make positive contributions. Likewise, I hope that if I am right, people who I disagreed with still feel like they can work with me despite that.

  As a side note, it’s also strange to me that people are treating the founding of Mechanize as if it has a realistic chance to accelerate AGI progress more than a negligible amount — enough of a chance of enough of an acceleration to be genuinely concerning. AI startups are created all the time. Some of them state wildly ambitious goals, like Mechanize. They typically fail to achieve these goals. The startup Vicarious comes to mind.

  I admit I had a similar thought, but I am of two minds about it. On the one hand, I think intentions do matter. I think it is reasonable to point out if you think someone is making a mistake, even if you think ultimately that mistake is unlikely to have a substantial impact because the person is unlikely to succeed in what they are trying to do.

  On the other hand, I do think the degree of the reaction and the way that people are generalizing seems like people are almost pricing in the idea that the actions in question have already had a huge impact. So I do wonder if people are kind of over-updating on this specific case for similar reasons to what you mention.

• TFDApr 20, 2025, 2:51 PM

  2 points

  0 ∶ 0

  in reply to: Rebecca’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  Although I haven’t thought deeply about the issue you raise you could definitely be correct, and I think they are reasonable things to discuss. But I don’t see their relevance to my arguments above. The quote you reference is itself discussing a quote from Sevilla that analyzes a specific hypothetical. I don’t necessarily think Sevilla had the issues you raise in mind when we was addressing that hypothetical. I don’t think his point was that based on forecasts of life extension technology he had determined that acceleration was the optimal approach in light of his weighing of 1 year-olds vs 50 year-olds. I think his point is more similar to what I mention above about current vs future people. I took a look at more of the X discussion, including the part where that quote comes from, and I think it is pretty consistent with this view (although of course others may disagree). Maybe he should factor in the things you mention, but to the extent his quote is being used to determine his views, I don’t think the issues you raise are relevant unless he was considering them when he made the statement. On the other hand, I think discussing those things could be useful in other, more object level discussions. That’s kind of what I was getting at here:

  I think, at bottom, the problem is that Sevilla makes mistake in his analysis and/​or decision-making about AI. His statements aren’t norm-violating, they are just incorrect (at least some of them are, in my opinion). I think its worth having clarity about what the actual “problem” is.

  I know I’ve been commenting here a lot, and I understand my style may seem confrontational and abrasive in some cases. I also don’t want to ruin people’s day with my self-important rants, so, having said my piece, I’ll drop the discussion for now and let you get on with other things.

  (although it you would like to response you are of course welcome, I just mean to say I won’t continue the back-and-forth after, so as not to create a pressure to keep responding.)

• TFDApr 20, 2025, 11:44 AM

  1 point

  0 ∶ 0

  in reply to: Rebecca’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  Prioritising young people often makes sense from an impartial welfare standpoint

  Sure, I think you can make a reasonable argument for that, but if someone disagreed with that, would you say they lack impartiality? To me it seems like something that is up for debate, within the “margin-of-error” of what is meant by impartiality. Two EAs could come down on different sides of that issue and still be in good standing in the community, and wouldn’t be considered to not believe in the general principle of impartiality. Likewise, I think we can interpret Jeff Kaufman’s argument above as expressing a similar view about an individual’s loved-ones. It is within the “margin-of-error” of impartiality to still have a higher degree of concern for loved-ones, even if that might not be living up to the platonic ideal of impartiality.

  My point in bringing this up is, the exact reason why the statement in question is bad seems to be shifting a bit over the conversation. Is the core reason that Sevilla’s statement is objectionable really that it might up-weight people in a certain age group?

• TFDApr 20, 2025, 11:35 AM

  4 points

  1 ∶ 1

  in reply to: Rebecca’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  When I click the link I see three posts that go Sevilla, Lifland, Sevilla. I based my comments above on those. I haven’t read through all the other replies by others or posts responding to them. If there is context in those or else where that is relevant I’m open to changing my mind based on that.

  He repeatedly emphasises that it’s about his literal friends, family and self, and hypothetical moderate but difficult trade offs with the welfare of others.

  Can you say what statements lead you to this conclusion? For example, you quote him saying something I haven’t seen, perhaps part of the thread I didn’t read.

  “But I want to be clear that even if you convinced me somehow that the risk that AI is ultimately bad for the world goes from 15% to 1% if we wait 100 years I would not personally take that deal. If it reduced the chances by a factor of 100 I would consider it seriously. But 100 years has a huge personal cost to me, as all else equal it would likely imply everyone I know [italics mine] being dead. To be clear I don’t think this is the choice we are facing or we are likely to face.“

  To me, this seems to confirm what I said above:

  Based on my read of the thread, the comment was in response to a question about benefiting people sooner rather than later. This is why I say it reduces to an existing-person-effecting view (which, at least as far as I am aware, is not an unacceptable position to hold in EA). The question is functionally about current vs future people, not literally Sevilla’s friends and family specifically.

  Yes, Sevilla is motivated specifically by considerations about those he loves, and yes, there is a trade-off, but that trade-off is really about current vs future people. People who aren’t longtermists for example would also implicate this same trade-off. I don’t think Sevilla would be getting the same reaction here if he just said he isn’t a longtermist. Because of the nature of the available actions, the interests of Sevilla’s loved-ones is aligned with those of current people (but not necessarily future people). The reason why “everyone [he] know[s]” will be dead is because everyone will be dead, in that scenario.

  You might think that having loved-ones as a core motivation above other people is inherently a problem. I think this is answered above by Jeff Kaufman:

  I don’t think impartiality to the extent of not caring more about the people one loves is a core value for very many EAs? Yes, it’s pretty central to EA that most people are excessively partial, but I don’t recall ever seeing someone advocate full impartiality.

  I agree with this statement. Therefore my view is that simply stating that you’re more motivated by consequences to your loved-ones is not, in and of itself, a violation of a core EA idea.

  Jason offers a refinement of this view. Perhaps what Kaufman says is true, but what if there is a more specific objection?

  There are a number of jobs and roles that expect your actions in a professional capacity to be impartial in the sense of not favoring your loved ones over others. For instance, a politician should not give any more weight to the effects of proposed legislation on their own mother than the effect on any other constituent.

  Perhaps the issue is not necessarily that Sevilla has the motivation itself, but that his role comes with a specific conflict-of-interest-like duty, which the statement suggests he is violating. My response was addressing this argument. I claim that the duty isn’t as broad as Jason seems to imply:

  It seems like the view expressed reduces to an existing-person-effecting view. Is their any plausible mechanism by which an action by Epoch is supposed to impact Sevilla’s friends/​relatives specifically? I seriously doubt it. The only plausible mechanism would be that AI goes well instead of poorly, which would benefit all existing people. This makes the politician comparison, as stated, dis-analogousness. Would you say that if a politician said their motivation to become a politician was to make a better world for their children, for example, that would somehow violate their duties? Seems like a lot of politicians might have issue if that were the case.

  Does a politician who votes for a bill and states they are doing so to “make a better world for their children”, violate a conflict-of-interest duty? Jason’s argument seems to suggest they would. Let’s assume they are being genuine, they really are significantly motivated by care for their children, more than for a random citizen. They apply more weight to the impact of the legislation on their children then to others, violating Jason’s proposed criteria.

  Yet I don’t think we would view such statements as disqualifying for a politician. The reason is that the mechanism by which they benefit their children really only operates by also helping everyone else. Most legislation won’t have any different impact on their children compared to any other person. So while the statement nominally suggests a conflict-of-interest, in practice the politicians incentives are aligned, the only way that voting for this legislation helps their children is that it helps everyone, and that includes their children. If the legislation plausibly did have a specific impact on their child (for example impacting an industry their child works in), then that really could be a conflict-of-interest. My claim is there needs to be some greater specificity for a conflict to exist. Sevilla’s case is more like the first case than the second, or at least that is my claim:

  Is their any plausible mechanism by which an action by Epoch is supposed to impact Sevilla’s friends/​relatives specifically? I seriously doubt it. The only plausible mechanism would be that AI goes well instead of poorly, which would benefit all existing people.

  So, what has Sevilla done wrong? My analysis is this. It isn’t simply that he is more motivated to help his loved-ones (Kaufman argument). Nor is it something like a conflict-of-interest (my argument). In another comment on this thread I said this:

  People can do a bad thing because they are just wrong in their analysis of a situation or their decision-making.

  I think, at bottom, the problem is that Sevilla makes mistake in his analysis and/​or decision-making about AI. His statements aren’t norm-violating, they are just incorrect (at least some of them are, in my opinion). I think its worth having clarity about what the actual “problem” is.

• TFDApr 20, 2025, 2:14 AM

  1 point

  0 ∶ 1

  in reply to: Jason’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  the prior generation of my family has passed on, I had young children

  This seems to suggest that you think the politicians “making the world better for my children” statement would then also be problematic. Do you agree with that?

  I’ll be honest, this argument seems a bit too clever. Is the underlying problem with the statement really that it implies a set of motivations that might slightly up-weight a certain age group? One of the comments speaks of “core values” for EA. Is that really a core value? I’m pretty sure I recall reading an argument by McAskill about how actually we should more heavily weight young people in various ways (I think it was voting), for example. I serious doubt most EAs could claim that they literally are distributionally exact in weighting all morally relevant entities in every decision they make. I think the “core value” that exists probably isn’t really this demanding, although I could be wrong.

• TFDApr 20, 2025, 1:52 AM

  4 points

  0 ∶ 2

  in reply to: Rebecca’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  But there’s no trade off between personal and impartial preferences there. That seems to me to be quite different from saying you’re prioritising eg your parents and grandparents getting to have extended lifespans over other people’s children’s wellbeing.

  I can see why you would interpret it this way given the context, but I read the statement differently. Based on my read of the thread, the comment was in response to a question about benefiting people sooner rather than later. This is why I say it reduces to an existing-person-effecting view (which, at least as far as I am aware, is not an unacceptable position to hold in EA). The question is functionally about current vs future people, not literally Sevilla’s friends and family specifically. I think this matches the “making the world better for your children” idea. You can channel a love of friends and family into an altruistic impulse, so long as there isn’t some specific conflict-of-interest where you’re benefiting them specifically. I think the statement in question is consistent with that.

  The discussion also isn’t about the effects of Epoch’s specific work, so I’m a bit confused by your argument relying on that.

  I’m bringing this up because I think its implausible that anything that is being discussed here has some specific relevance to Sevilla’s friends and family as individuals (in support of my point above). In other words, due to the nature of the actions being taken

  there’s no trade off between personal and impartial preferences there

  In what way are any concrete actions that are relevant here prioritizing Sevilla’s family over other people’s children? Although I can see how it might initially seem that way I don’t think that’s what the statement was intended to communicate.

• TFDApr 20, 2025, 1:05 AM

  9 points

  2 ∶ 0

  on: An In­tro­duc­tion to the Prob­lem of Authority

  I think there are two main appeals to libertarianism. One “practical libertarianism” is based on a belief that on current margins moving towards less government would be beneficial. I’m sympathetic to this position and I think one can hold it for purely consequentialist reasons.

  The above argument is “philosophical libertarianism”. I’m not so convinced by these arguments.

  On these theories, for example, you wouldn’t be justified in stealing a loaf of bread even to save a starving person’s life.

  I think the reason a lot of libertarian theories bite this bullet is that failing to do so seems to be abandoning the alleged reasons for libertarianism. For example, the article argues that one of the common sense reasons why other concepts of government fail is that they seem to imply the government can do things that normal people can’t do, like take people’s property. But we have now just said that actually, it is allowed for normal people, not just governments, to take people’s property if their is a good reason. We could go through an entire sequence of hypotheticals, like can you take money from someone to buy bread if you’re starving? If you aren’t starving but someone else is, can you take money from someone else to buy bread to give it to the starving person? The upshot of that sequence is that if you don’t bite the bullet, then there’s no limiting principle. You’re basically saying redistribution is in fact allowed.

  Likewise for the lifeboat example. I don’t see how any principled approach to libertarianism can give the answer that you can threaten the rest of the passengers. That’s literal coercion! If you then engage in the insane terminological gerrymandering where that counts as “self-defense” then so does so much other stuff. Is it “self-defense” if you force a doctor to give you a surgery you can’t afford? None of the desired libertarian conclusions would follow. Especially given that libertarianism also needs to justify why private property is a thing, and most justification I have seen go back to freedom of a person’s labor by way of homesteading. Giving the answer the author seems to give for the lifeboat hypo seems like a massive problem for libertarianism.

  This doesn’t support a content-independent entitlement to enforce the state’s rules, nor a content-independent obligation to obey.

  I’m not sure why there is a requirement that a theory of government be content-independent. This seems like an arbitrary requirement the author has imposed on theories they don’t favor. Kind of by definition, a consequentialism wouldn’t support a “content-independent” position? But they could still support government based on an expectation about the distribution of government actions they expect to actually be realized. They could also support something like an-cap, for a consequentialist it seems like a modeling/​empirical question (and kind of collapse back to practical libertarianism potentially).

  I think there is essentially a moral hypothetical no-free-lunch theorem. No principled moral theory can exist that matches “common sense” intuitions on all hypotheticals. Although I’m open to practical libertarianism, nothing here seems convincing that philosophical libertarianism is the “least bad” option.

• TFDApr 20, 2025, 12:35 AM

  6 points

  0 ∶ 0

  in reply to: Jason’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  How common is it for such repayments to occur, and what do you think would be the standard for the level of clarity of the commitment, and who does that commitment would have to be to? For example, is there a case that 80k hours should refund payments in light of their pivot to focus on AI? I know there are differences, their funder could support the move etc., but in the spirit of the thing, where is the line here?

  Editing to add: One of my interests in this topic is that EA/​rationalists seem to have some standards/​views that diverge somewhat from what I would characterize as more “mainstream” approaches to these kinds of things. Re-reading the OP, I noticed a detail I initially missed:

  Habryka wonders whether payment would have had to be given to Epoch for use of their benchmarks suite.

  to me this does seem like it implicates a more mainstream view of a potential conflict-of-interest.

• TFDApr 20, 2025, 12:14 AM

  2 points

  0 ∶ 0

  in reply to: MichaelDickens’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  it suggests the concern is an object level one, not a meta one. the underlying “vibe” I am getting from a lot of these discussions is that the people in question have somehow betrayed EA/​the community/​something else. That is a meta concern, one of norms. You could “betray” the community even if you are on the AI deceleration side things. If the people in question or Epoch made a specific commitment that they violated, that would be a “meta” issue, and would be one regardless of their “side” on the deceleration question. Perhaps they did do such a thing, but I haven’t seen convincing information suggesting this. I think that really the main explanatory variable here is in fact what “side” this suggests they are on. If that is the case, I think it is worth having clarity about it. People can do a bad thing because they are just wrong in their analysis of a situation or their decision-making. That doesn’t mean their actions constitute a betrayal.

• TFDApr 20, 2025, 12:03 AM

  11 points

  5 ∶ 2

  in reply to: Jason’s comment on: Epoch AI alumni launch Mech­a­nize to “au­to­mate the whole econ­omy”

  It seems like the view expressed reduces to an existing-person-effecting view. Is their any plausible mechanism by which an action by Epoch is supposed to impact Sevilla’s friends/​relatives specifically? I seriously doubt it. The only plausible mechanism would be that AI goes well instead of poorly, which would benefit all existing people. This makes the politician comparison, as stated, dis-analogousness. Would you say that if a politician said their motivation to become a politician was to make a better world for their children, for example, that would somehow violate their duties? Seems like a lot of politicians might have issue if that were the case.

  I think this suggests a risk that the real infraction here is honestly stating the consideration about friends and family. Is it really the case that no-one leading AI safety orgs that are aiming for deceleration are motivated, at least partly, by the desire to protect their own friends and family from the consequences of AI going poorly? I will confess that is a big part of my own reasons from being interested in this topic. I would be very surprised if the standard being suggested here was really as ubiquitous as these comments suggest.

• TFDApr 15, 2025, 6:20 PM

  3 points

  0 ∶ 0

  on: AISN #51: AI Frontiers

  A group of former OpenAI employees filed a proposed amicus brief in support of Musk’s lawsuit on the future of OpenAI’s for-profit transition. Meanwhile, OpenAI countersued Elon Musk.

  Crossposting my comment from lesswrong.

  I think this is the first time that the charter has been significantly highlighted in this case. My own personal view is the charter is one of the worst documents for OpenAI (therefore good for Musk), and having their own employees stating that it was emphasized a lot and treated as binding are very bad facts for OpenAI and associated defendants. The timeline for all this stuff isn’t 100% clear to me, so I can imagine there being issues with whether the charter was timed such that it is relevant for Musk’s own reliance, but the vibes of this for OpenAI are horrendous. Also raises an interesting possibility of whether the “merge-and-assist” part of the charter might be enforceable.

  The docket seems to indicate that Eugene Volokh is representing the ex-OpenAI amici (in addition to Lawrence Lessig). To my understanding Volokh is a first amendment expert and has also done work on transparency in courts. The motion for leave to file also indicate that OpenAI isn’t necessarily on board with the brief being filed. I wonder if they are possibly going to argue their ex-employees shouldn’t be allowed to do what their doing (perhaps trying to enforce NDAs?), and Volokh is perhaps planning to weigh in on that issue?

• TFDApr 14, 2025, 10:21 PM

  4 points

  0 ∶ 0

  on: How to make sense of wrong­ness if you’re a consequentialist

  But there’s a third sense of wrong that’s slightly more slippery, which utilitarians can gladly affirm. This is about who merits significant criticism and ostracization for their behavior. You, by not giving to charity, have not acted optimally. Ted Bundy, by killing people, has also not acted optimally. Yet there’s a categorical difference between your action and Ted’s. Ted should be thrown in jail—you shouldn’t. Ted is significantly and abnormally blameworthy while you are not.

  If someone is really, truly going “full consequentialist” shouldn’t they believe that criticism or ostracism is merited based on the consequences of doing those things? So if Ted Bundy’s murder elasticity with regard to your criticism is weak and your actions don’t contribute much to deterrence, he actually doesn’t deserve much criticism. Whereas if someone’s donation elasticity with regard to your criticizing/​ostracizing is high then they would be worthy of criticism/​ostracism.

  Its definitely possible that there are consequentialist reasons that things might turn out the way you say, but it seems kinda convenient that these match up with certain deontological intuitions (prohibition on murder vs no charity obligation).

  I also agree with Richard that what determines how blameworthy a person is has a lot to do with the extent of their exertion of willpower for the sake of morality. If you have ten trillion dollars, while giving a million dollars to charity is a very good thing to do, you don’t get many virtue points as it requires almost no sacrifice from you. In contrast, if you’re poor but still donate, because doing so is psychologically taxing, you get virtue points. This is one reason why vegans get virtue points.

  I agree with this on an intuitive level, but to me it suggests that I’m not fully a consequentialist, but rather that my views are kind of a messy mix of different moral theories, such that virtues and degree of deontologic violations all get thrown into the equation.

The limits of black-box eval­u­a­tions: two hypotheticals

A differ­ent take on the Musk v OpenAI pre­limi­nary in­junc­tion order

• TFDMar 11, 2025, 1:23 PM

  1 point

  0 ∶ 0

  in reply to: Jason’s comment on: Emer­gency pod: Judge plants a le­gal time bomb un­der OpenAI (with Rose Chan Loui)

  I expect the court would either reject or ignore an amicus filing at this stage in the process.

  Worth noting that the court has already accepted two amicus briefs on the preliminary injunction, one by Encode Justice and the other by the Delaware AG.

  To the extent that the amicus brief was about a legally peripheral issue—like AI as a GCR—it would likely be read by a law clerk (a bright recent graduate) who would tell the judge something like “This foundation submitted an amicus brief arguing that AI may go rogue and kill us all. Doesn’t seem relevant to the issues in this case.”

  Although this is of course speculation, I wonder if this is the type of reaction that the Encode Justice brief (and possible future ones like it), might have received. Reading the brief I could definitely see how it might come across as kind of trying to deputize the court into a policy question, while not really hitting upon issues that are hugely relevant to the case actually in front of the court.

  *Edited to fix a typo