tobycrisford 🔸

Karma: 521

tobycrisford 🔸 22 Dec 2024 11:11 UTC
1 point
0 ∶ 0
in reply to: Hawk.Yang 🔸’s comment on: o3
It might be fair to say that the o3 improvements are something fundamentally different to simple scaling, and that Chollet is still correct in his ‘LLMs will not simply scale to AGI’ prediction. I didn’t mean in my comment to suggest he was wrong about that.
I could imagine someone criticizing him for exaggerating how far away we were from coming up with the necessary new ideas, given the o3 results, but I’m not so interested in the debate about exactly how right or wrong the predictions of this one person were.
The interesting thing for me is: whether he was wrong, or whether he was right but o3 does represent a fundamentally different kind of model, the upshot for how seriously we should take o3 seems the same! It feels like a pretty big deal!
He could have reacted to this news by criticizing the way that o3 achieved its results. He already said in the Dwarkesh Patel interview that someone beating ARC wouldn’t necessarily imply progress towards general intelligence if the way they achieved it went against the spirit of the task. When I clicked the link in this post, I thought it likely I was about to read an argument along those lines. But that’s not what I got. Instead he was acknowledging that this was important progress.
I’m by no means an expert, but timelines in the 2030s still seems pretty close to me! I’d have thought, based on arguments from people like Chollet, that we might be a bit further off than that (although only with the low confidence of a layperson trying to interpret the competing predictions of experts who seem to radically disagree with each other).
Given all the problems you mention, and the high costs still involved in running this on simple tasks, I agree it still seems many years away. But previously I’d have put a fairly significant probability on AGI not being possible this century (as well as assigning a significant probability to it happening very soon, basically ending up highly uncertain). But it feels like these results make the idea that AGI is still 100 years away seem much less plausible than it was before.

tobycrisford 🔸 21 Dec 2024 7:00 UTC
14 points
1 ∶ 4
on: o3
The ARC performance is a huge update for me.
I’ve previously found Francois Chollet’s arguments that LLMs are unlikely to scale to AGI pretty convincing. Mainly because he had created an until-now unbeaten benchmark to back those arguments up.
But reading his linked write-up, he describes this as “not merely an incremental improvement, but a genuine breakthrough”. He does not admit he was wrong, but instead paints o3 as something fundamentally different to previous LLM-based AIs, which for the purpose of assessing the significance of o3, amounts to the same thing!

tobycrisford 🔸 18 Dec 2024 13:41 UTC
12 points
1 ∶ 0
on: Fund Causes Open Phil Underfunds (Instead of Your Most Preferred Causes)
I think the presentation of this argument here misses some important considerations:
- The way that you want us to act with respect to OP is already the way that OP is trying to act with respect to the rest of the world.
- The same considerations that lead OP to choose not to allocate all their funds to the highest expected value cause should also be relevant for individual donors, and could legitimately mean that they should diversify as well. There seems to be an inconsistency in saying these considerations are valid for OP but not for individuals.
- Even if you are a pure marginal EV maximizer (you think these considerations are invalid for OP and for invidiauls), OP’s donations won’t always be relevant to your donation decisions, and if they are, it is the absolute amounts donated, rather than the percentages, that are relevant.
The way that you want us to act with respect to OP is already the way that OP is trying to act with respect to the rest of the world
EAs don’t fund the most important causes, based purely on scale (otherwise tonnes of things EAs ignore would score highly, e.g. vaccination programs in rich countries). A core part of EA is looking for causes which are neglected. We look for the areas that are receiving the least funding relative to what they would receive in our ideal world, because these are likely to be the areas where our donations will have the highest marginal impact.
This is the reply to people who argue “oh you want local charities to disappear and to send all the money to malaria nets”. The reply is: “No! In my ideal world, malaria nets would quickly attract all the funding they need. Then there would still be plenty of money left over for other things. But I think I should look at the world I actually live in, recognize that malaria nets are outrageously underfunded, and give all my resources there.”
So in a sense, the argument you are making here isn’t anything new. You are just saying we should try to act towards other EAs in a similar way to how EAs as a group act towards the rest of the world. And I don’t disagree with this. But I think we should go all the way. I think we should treat other EAs in the same way that we treat the rest of the world. If I understand your argument correctly, you are trying to draw a distinction between the EA community and everyone else.
The same considerations that lead OP to choose not to allocate all their funds to the highest expected value cause should also be relevant for individual donors
OP do not allocate all of their funding to the ‘best’ cause. Even if OP were a pure EV maximizer, they might have valid reasons not to do this, because they have such a big budget. It may be that diminishing marginal returns mean that the ‘best’ cause stops being the best once OP have given a certain level of funds to it, at which point they should switch to funding another cause instead.
But my impression is that this is not OP’s reason for donating to multiple causes (or at least not their only reason). They are not purely trying to maximize expected value, or at least not in a naive first order way. One reason to diversify might be donor risk aversion, like you mention (e.g. you want to maximize EV while bounding the risk that you have no positive impact at all), and there are plenty of other considerations that might come into it too, e.g. sense of duty to a certain cause, reputation, belief in unquantifiable uncertainty and impossibility of making certain cause comparisons etc
But if these considerations are valid for OP then they should also be relevant for individual donors. For example, if an individual donor wants to bound the risk that they have no impact, then that might well mean not donating everything to the cause they think is most underfunded by OP. It would only make sense to do this if they had a weird type of risk aversion where they want to bound the risk that the EA community as a whole has no positive impact, but are unconcerned about their own donations’ risk. This seems very arbitrary! Either they should care about the risk for their own donations, and should diversify, or they should be concerned with all of humanity’s donations, in which case OP should not be diversifying either!
Pure EV maximizers don’t care about percentages anyway
You could bite the bullet and say that neither OP nor individual donors should be diversifying their donations (except when faced with diminishing marginal utility). For these individual donors, they should be donating everything to one cause (and probably one charity unless they have a lot to give!) But even for these donors, it’s not which causes OP underfund that really matters. It’s what causes all of humanity underfund. So it is not the percentages of OP’s funding allocation that matter, it’s the absolute value.
If OP are a relatively small player in a cause area (global health..?) then their donation decisions are unlikely to be especially relevant to the individual donor. If they thought global health was the top cause before OP donations were taken into account, it probably still will be afterwards. But if OP are a relatively big player (animal welfare..?) then their donations are more relevant, due to diminishing marginal utility. But it is the absolute amount of funding they are moving, not the percentages, which will determine this.

tobycrisford 🔸 17 Dec 2024 8:06 UTC
1 point
0 ∶ 0
in reply to: Sam Tucker’s comment on: Why our goal should be to end factory farming within our lifetime, and why that goal is realistically achievable
I’ve been vegan for 11 years, and to me the growth felt faster in the first 5 years than it did in the second. This could easily just be due to my changing life circumstances (first 5 years as a student and living with other vegans), but that’s my personal anecdotal evidence. Recently it also seems like all the vegan restaurants have been closing in my city (Manchester, UK) although hopefully(?) that is more to do with the economic situation than with a decline in veganism.
The link you’ve shared on the proportion of the population identifying as vegan is encouraging, but I’m finding it hard to figure out the data source for their graph. I’m sure I saw some data shared by someone on the EA forum recently that suggested the growth of veganism had been stagnating recently, but not sure how to find that now!
This seems like a really important question though and I’d love to read an in-depth analysis of what the answer is likely to be.

tobycrisford 🔸 10 Dec 2024 13:27 UTC
3 points
0 ∶ 0
on: Are fishing and agriculture bad for animals?
This is a fascinating analysis, but if I understand it correctly, you are estimating the impact of fishing and agriculture on average wild animal wellbeing (which you estimate by its effect on the death rate), not total wellbeing, as the first sentence of your post states. Is that correct?
This seems important, as I don’t think there are many who would defend the idea that average welfare is what matters in population ethics? So I’m not sure how important the considerations you point out are. The change in population size seems like it’s going to be the much more important effect here.
It also doesn’t seem obvious to me that we should be able to estimate the impact of fishing or agriculture on average welfare purely by its impact on the death rate. Aren’t there lots of other ways they could impact wild animal welfare too (e.g. by changing the cause of death for wild-animals)?

tobycrisford 🔸 4 Dec 2024 13:19 UTC
1 point
0 ∶ 0
on: Why our goal should be to end factory farming within our lifetime, and why that goal is realistically achievable
I hope you are right, but is there evidence that veganism is growing exponentially?

tobycrisford 🔸 3 Dec 2024 17:42 UTC
2 points
0 ∶ 0
on: [Linkpost] Will MacAskill coauthors new book titled “An Introduction to Utilitarianism”, with Richard Chappell and Darius Meissner
Same content on bluesky, for those avoiding twitter now: https://bsky.app/profile/wdmacaskill.bsky.social/post/3lcdlb4lbdk2m

tobycrisford 🔸 15 Nov 2024 17:36 UTC
3 points
0 ∶ 0
on: The Case for Quantum Technologies
I think this is a fascinating area, and the problems you’ve highlighted seem like important problems. I find it hard to believe it’s a cause area EAs should focus on though.
As you explain, the clearest threat is the impact on cryptography, but it doesn’t seem likely to me that that problem is neglected. There are huge incentives for governments and companies to solve that problem, and I think they are probably already doing lots of work on it..?

tobycrisford 🔸 12 Nov 2024 7:52 UTC
1 point
0 ∶ 0
on: How much (more) data do we need to claim extreme cost-effectiveness?
A question jumped out at me when reading these results. I should caveat this by emphasizing that I am very much not an expert in this kind of evaluation and this question may be naive.
Is there any seasonal effect on mortality in Malawi? If so, is it ok for the pre-intervention period to be 12-months while the post-intervention period is 18-months?

tobycrisford 🔸 10 Nov 2024 11:29 UTC
1 point
0 ∶ 0
in reply to: VettedCauses’s comment on: A Critique of Animal Charity Evaluators (ACE)
This is very helpful, thanks!

tobycrisford 🔸 8 Nov 2024 17:59 UTC
12 points
0 ∶ 0
on: A Critique of Animal Charity Evaluators (ACE)
If you’re correct in the linked analysis, this sounds like a really important limitation in ACE’s methodology, and I’m very glad you’ve shared this!
In case anyone else has the same confusion as me when reading your summary: I think there is nothing wrong with calculating a charity’s cost effectiveness by taking the weighted sum of the cost-effectiveness of all of their interventions (weighted by share of total funding that intervention receives). This should mathematically be the same as (Total Impact / Total cost), and so should indeed go up if their spending on a particular intervention goes down (while achieving the same impact).
The (claimed) cause of the problem is just that ACE’s cost-effectiveness estimate does not go up by anywhere near as much as it should when the cost of an intervention is reduced, leading the cost-effectiveness of the charity as a whole to actually change in the wrong direction when doing the above weighted sum!
If this is true it sounds pretty bad. Would be interested to read a response from them.
Of course, the other thing that could be going on here, is that average cost-effectiveness is not the same as cost-effectiveness on the margin, which is presumably what ACE should care about. Though I don’t see why an intervention representing a smaller share of a charity’s expenditure should automatically mean that this is not where extra dollars would be allocated. The two things seem independent to me.

tobycrisford 🔸 31 Oct 2024 18:31 UTC
1 point
0 ∶ 0
on: Marginal Revolution: Effective Altruists and Finance Theory
I would be very interested to read a summary of what Tyler Cowen means by all this!
I know it was left as an exercise for the reader, but if someone wants to do the work for me it would be appreciated :)

tobycrisford 🔸 31 Oct 2024 17:49 UTC
1 point
0 ∶ 0
in reply to: jenny_kudymowa’s comment on: Fungal diseases: Health burden, neglectedness, and potential interventions
Makes sense, thank you for the reply and clarification!

tobycrisford 🔸 21 Oct 2024 6:36 UTC
1 point
1 ∶ 0
on: Fungal diseases: Health burden, neglectedness, and potential interventions
This is a fascinating summary!
I have a bit of a nitpicky question on the use of the phrase ‘confidence intervals’ throughout the report. Are these really supposed to be interpreted as confidence intervals? Rather than the Bayesian alternative, ‘credible intervals’..?
My understanding was that the phrase ‘confidence interval’ has a very particular and subtle definition, coming from frequentist statistics:
- 80% Confidence Interval: For any possible value of the unknown parameter, there is an 80% chance that your data-collection and estimation process would produce an interval which contained that value.
- 80% Credible interval: Given the data you actually have, there is an 80% chance that the unknown parameter is contained in the interval.
From my reading of the estimation procedure, it sounds a lot more like these CIs are supposed to be interpreted as the latter rather than the former? Or is that wrong?
Appreciate this is a bit of a pedantic question, that the same terms can have different definitions in different fields, and that discussions about the definitions of terms aren’t the most interesting discussions to have anyway. But the term jumped out at me when reading and so thought I would ask the question!

tobycrisford 🔸 17 Oct 2024 16:32 UTC
3 points
0 ∶ 0
on: AI data gaps could lead to ongoing Animal Suffering
This is a really interesting post, and I appreciate how clearly it is laid out. Thank you for sharing it! But I’m not sure I agree with it, particularly the way that everything is pinned to the imminent arrival of AGI.
Firstly, the two assumptions you spell out in your introduction, that AGI is likely only a few years away, and that it will most likely come from scaled up and refined versions of moden LLMs, are both much more controversial than you suggest (I think)! (Although I’m not confident they are false either)
But even if we accept those assumptions, the third big assumption here is that we can alter a superintelligent AGI’s values in a predictable and straightforward way by just adding in some synethetic training data which expresses the views we like, when building some of its component LLMs. This seems like a strange idea to me!
If we removed some concept from the training data completely, or introduced a new concept that had never appeared otherwise, then I can imagine that having some impact on the AGI’s behaviour. But if all kinds of content are included in significant quantities anyway, then i find it hard to get my head around the inclusion of additional carefully chosen synthetic data having this kind of effect. I guess it clashes with my understanding of what a superintelligent AGI means, to think that its behaviour could be altered with such simple manipulation.
I think an important aspect of this is that even if AGI does come from scaling up and refining LLMs, it is not going to just be a LLM in a straightforward definition of that term (i.e. something that communicates by generating each word with a single forward pass through a neural network). At the very least it must also have some sort of hidden internal monologue where it does chain of thought reasoning, and stores memories, etc.
But I don’t know much about AI alignment, so would be very interested to read and understand more about the reasoning behind this third assumption.
All that said, even ignoring AGI, LLMs are likely going to be used more and more in people’s every day lives over the next few years, so training them to express kinder views towards animal seems like a potentially worthwhile goal anyway. I don’t think AGI needs to come into it!

tobycrisford 🔸 24 Sep 2024 7:47 UTC
3 points
0 ∶ 0
in reply to: MichaelStJules’s comment on: Utilitarianism and the replaceability of desires and attachments
I agree that we can imagine a similar scenario where your identity is changed to a much lesser degree. But I’m still not convinced that we can straightforwardly apply the Platinum rule to such a scenario.
If your subjective wellbeing is increased after taking the pill, then one of the preferences that must be changed is your preference not to take the pill. This means that when we try to apply the Platinum rule: “treat others as they would have us treat them”, we are naturally led to ask: “as they would have us treat them when?” If their preference to have taken the pill after taking it is stronger than their preference not to take the pill before taking it, the Platinum rule becomes less straightforward.
I can imagine two ways of clarifying the rule here, to explain why forcing someone to take the pill would be wrong, which you already allude to in your post:
- We should treat others as they would have us treat them at the time we are making the decision. But this would imply that if someone’s preferences are about to naturally, predictably, change for the rest of their life, then we should disregard that when trying to decide what is best for them, and only consider what they want right now. This seems much more controversial than the original statement of the rule.
- We should treat others as they would have us treat them, considering the preferences they would have over their lifetime if we did not act. But this would imply that if someone was about to eat the pill by accident, thinking it was just a sweet, and we knew it was against their current wishes, then we should not try to stop them or warn them. This would create a very odd action/inaction distinction. Again, this seems much more controversial than the original statement of the rule.
In the post you say the Platinum rule might be the most important thing for a moral theory to get right, and I think I agree with you on this. It is something that seems so natural and obvious that I want to take it as a kind of axiom. But neither of these two extensions to it feel this obvious any more. They both seem very controversial.
I think the rule only properly makes sense when applied to a person-moment, rather than to a whole person throughout their life. If this is true, then I think my original objection still applies. We aren’t dealing with a situation where we can apply the platinum rule in isolation. Instead, we have just another utilitarian trade-off between the welfare of one (set of) person(-moments) and another.

tobycrisford 🔸 23 Sep 2024 7:32 UTC
3 points
0 ∶ 0
on: Utilitarianism and the replaceability of desires and attachments
This was a really thought-provoking read, thank you!
I think I agree with Richard Chappell’s comment that: “the more you manipulate my values, the less the future person is me”.
In this particular case, if I take the pill, my preferences, dispositions, and attitudes are being completely transformed in an instant. These are a huge part of what makes me who I am, so I think that after taking this pill I would become a completely different person, in a very literal sense. It would be a new person who had access to all of my memories, but it would not be me.
From this point of view, there is no essential difference between this thought experiment, and the common objection to total utilitarianism where you consider killing one person and replacing them with someone new, so that total well-being is increased.
This is still a troubling thought experiment of course, but I think it does weaken the strength of your appeal to the Platinum rule? We are no longer talking about treating a person differently to how they would want to be treated, in isolation. We just have another utilitarian thought experiment where we are considering harming person X in order to benefit a different person Y.
And I think my response to both thought experiments is the same. Killing a person who does not want to be killed, or changing the preferences of someone who does not want them changed, does a huge amount of harm (at least on a preference-satisfaction version of utilitarianism), so the assumption in these thought experiments that overall preference satisfaction is nevertheless increased is doing a lot of work, more work than it might appear at first.

tobycrisford 🔸 21 Sep 2024 19:41 UTC
1 point
1 ∶ 0
on: Does the Self-Sampling Assumption Imply Clairvoyance?
I really like this thought experiment, thank you for sharing!
Personally, I agree with you, and I think the answer to your headline question is: yes! Your reasoning makes sense to me anyway. (At least if we don’t combine the Self-Sampling Assumption with another assumption like the Self-Indication Assumption as well).
I think that your example is essentially equivalent to the Doomsday argument, or the Adam+Eve paradox, see here: https://anthropic-principle.com/preprints/cau/paradoxes But I like that your thought experiment really isolates the key problem and puts precise numbers on it!

tobycrisford 🔸 9 Sep 2024 16:58 UTC
15 points
2 ∶ 0
in reply to: Habryka’s comment on: Dispelling the Anthropic Shadow
I haven’t digested the full paper yet, but based on the summary pasted below, this is precisely the claim I was trying to argue for in the “Against Anthropic Shadow” post of mine that you have linked.
It looks like this claim has been fleshed out in a lot more detail here though, and I’m looking forward to reading it properly!
In the post you linked I also went on quite a long digression trying to figure out if it was possible to rescue Anthropic Shadow by appealing to the fact that there might be large numbers of other worlds containing life (this plausibly weakens the strength of evidence provided by A, which may then stop the cancellation in C). I decided it technically was possible, but only if you take a strange approach to anthropic reasoning, with a strange and difficult-to-define observer reference class.
Possibly focusing so much on this digression was a mistake though, since the summary above is really pointing to the important flaw in the original argument!

tobycrisford 🔸 4 Sep 2024 16:42 UTC
2 points
0 ∶ 0
in reply to: Oisín Considine’s comment on: What are the best charities working in vegan outreach?
This is a fantastic answer, thank you!