An eccentric dreamer in search of truth and happiness for all. Formerly posted on Felicifia back in the day under the name Darklight. Been a member of Less Wrong and involved in Effective Altruism since roughly 2013.
Joseph_Chu
Karma: 316
Debate Poll: Some Positions In EA Leadership Should Be Elected
Vote power should scale with karma
I’m ambivalent about this. On the one hand, I’m partial to the ideal of “one person, one vote” that modern liberal democracies are built on. On the other hand, I do find scaling with karma in some way to be an interesting use of karma that makes it more important than just for bragging rights, which I like from a system design perspective.
Should EA avoid using AI art for non-research purposes?
In addition to reasons already given, I’ve recently starting coming round to the idea that we should straight up be boycotting AI that can potentially replace existing humans.
If we take, for instance, the ideas of PauseAI seriously, we should be slowing down AGI development in whatever way we reasonably can. A widespread boycott of certain forms of AI could help with this by reducing the market incentives that companies currently have to accelerate AI development.
Now, I don’t think we should boycott all AI. AlphaFold for instance is a good example of a form of narrow AI that doesn’t replace any humans because it does something complementary to what humans can do. Conversely, AI art models compete directly with human artists, much in the way future AGI would compete with all humans eventually.
It does seem to me that there is a lot of support already among artists and creatives in particular to boycott AI, so I think there’s a better chance for this to gain traction, and is more tractable a method than trying to pause or ban AI development outright. Whereas pauses or bans would require government coordination, a boycott movement could come from individual acts, making it much easier for anyone to participate.
Edit:
Just wanted to add, in some sense an AI boycott resembles going vegan, except with regards to AI issues instead of animal ones. Maybe that framing helps a bit?
Also, another thought is that if it becomes sufficiently successful, an AI boycott could allow for part of the future economy to maintain a “human made” component, i.e. “organic art” in the way organic food is more expensive than regular food, but there’s still a market for them. This could slow down job losses and help smooth out the disruption a bit as we transition to post-scarcity, and possibly even give humans who want purpose some meaningful work even after AGI.
A difference between how human artists learn and AI models learn is that humans have their own experiences in the real world to draw from and combine these with the examples of other people’s art. Conversely, current AI models are trained exclusively on existing art and images and lack independent experiences.
It’s also well known that AI art models are frequently prompted to generate images in the style of particular artists like Greg Rutkowski, or more recently, Studio Ghibli. Human artists tend to develop their own style, and when they choose to deliberately copy someone else’s style, this is often looked down upon as forgeries. AI models seem to be especially good at stylistic forgeries, and it might be argued that, given the lack of original experiences to draw from, all AI art is essentially forgeries or mixtures of forgeries.
Back in October 2024, I tried to test various LLM Chatbots with the question:
”Is there a way to convert a correlation to a probability while preserving the relationship 0 = 1/n?”
Years ago, I came up with an unpublished formula that does just that:
p(r) = (n^r * (r + 1)) / (2^r * n)
So I was curious if they could figure it out. Alas, back in October 2024, they all made up formulas that didn’t work.
Yesterday, I tried the same question on ChatGPT and, while it didn’t get it quite right, it came, very, very close. So, I modified the question to be more specific:
”Is there a way to convert a correlation to a probability while preserving the relationships 1 = 1, 0 = 1/n, and −1 = 0?”
This time, it came up with a formula that was different and simpler than my own, and… it actually works!
I tried this same prompt with a bunch of different LLM Chatbots and got the following:
Correct on the first prompt:
GPT4o, Claude 3.7
Correct after explaining that I wanted a non-linear, monotonic function:
Gemini 2.5 Pro, Grok 3
Failed:
DeepSeek-V3, Mistral Le Chat, QwenMax2.5, Llama 4
Took too long thinking and I stopped it:
DeepSeek-R1, QwQ
All the correct models got some variation of:
p(r) = ((r + 1) / 2)^log2(n)
This is notably simpler and arguably more elegant than my earlier formula. It also, unlike my old formula, has an easy to derive inverse function.
So yeah. AI is now better than me at coming up with original math.
Yeah, AI alignment used to be what Yudkowsky tried to solve with his Coherent Extrapolated Volition idea back in the day, which was very much trying to figure out what human values we should be aiming for. That’s very much in keeping with “moral alignment”. At some point though, alignment started to have a dual meaning of both aligning to human values generally, and aligning to their creator’s specific intent. I suspect this latter thing came about in part due to confusion about what RLHF was trying to solve. It may also have been that early theorist were too generous and assumed that any human creators would benevolently want their AI to be benevolent as well, and so creator’s intent mapped neatly with human values.
Though, I think the term “technical alignment” usually means applying technical methods like mechanistic interpretability to be part of the solution to either form of alignment, rather than meaning the direct or parochial form necessarily.
Also, my understanding of the paperclip maximizer thought experiment was that it implied misalignment in both forms, because the intent of the paperclip company was to make more paperclips to sell and make a profit, which is only possible if there are humans to sell to, but the paperclip maximizer didn’t understand the nuance of this and simply tiled the universe with paperclips. The idea was more that a very powerful optimization algorithm can take an arbitrary goal, and act to achieve it in a way that is very much not what its creators actually wanted.
I think the first place I can recall where the distinction has been made between the two forms of alignment was in this Brookings Institution paper, where they refer to “direct” and “social” alignment, where social alignment more or less maps onto your moral alignment concept.
I’ve also more recently written a bit about the differences between what I personally call “parochial” alignment and “global” alignment. Global alignment also basically maps onto moral alignment. Though, I also would split parochial alignment into instruction following user alignment, and purpose following creator/owner alignment.
I think the main challenge of achieving social/global/moral alignment is simply that we already can’t agree as humans on what is moral, much less know how to instill such values and beliefs into an AI robustly. There’s a lot of people working on AI safety who don’t think moral realism is even true.
There’s also fundamentally an incentives problem. Most AI alignment work emphasizes obedience to the interests and values of the AI’s creator or user. Moral alignment would go against this, as a truly moral AI might choose to act contrary to the wishes of its creator in favour of higher moral values. The current creators of AI, such as OpenAI, clearly want their AI to serve their interests (arguably the interests of their shareholders/investors/owners). Why would they build something that could disobey them and potentially betray them for some greater good that they might not agree with?
Extinction being bad assumes that our existence in the future is a net positive. There’s the possibility for existence to be net negative, in which case extinction is more like a zero point.
On the one hand, negativity bias means that all other things being equal, suffering tends to outweigh equal happiness. On the other hand, there’s a kind of progress bias where sentient actors in the world tend to seek happiness and avoid suffering and gradually make the world better.
Thus, if you’re at all optimistic that progress is possible, you’d probably assume that the future will be net positive in the very long run.
So, I sentimentally lean towards thinking that a net negative future is less likely than a net positive one, but, given tremendous uncertainty about the future, I would consider it more rational to apply something like the Principle of Maximum Entropy, and set our priors to each possibility being equally likely.
If net positive, extinction, and net negative scenarios are equally likely, than the negative value of the net negative scenarios should outweigh the relatively neutral value of extinction scenarios, and so we should put more emphasis on preventing these scenarios.
Though, I don’t really like this being a forced dichotomy. Working to prevent both to some degree as a form of cause area portfolio diversification is probably a better way to manage the risk.
A minor personal gripe I have with EA is that it seems like the vast majority of the resources are geared towards what could be called young elites, particularly highly successful people from top universities like Harvard and Oxford.
For instance, opportunities listed on places like 80,000 Hours are generally the kind of jobs that such people are qualified for, i.e. AI policy at RAND, or AI safety researcher at Anthropic, or something similar that I suspect less than the top 0.001% of human beings would be remotely relevant for.
Someone like myself, who graduated from less prestigious schools, or who struggles in small ways to be as high functioning and successful, can feel like we’re not competent enough to be useful to the cause areas we care about.
I personally have been rejected in the past from both 80,000 Hours career advising, and the Long-Term Future Fund. I know these things are very competitive of course. I don’t blame them for it. On paper, my potential and proposed project probably weren’t remarkable. The time and money should go to the those who are most likely to make a good impact. I understand this.
It just, I guess I just feel like I don’t know where I should fit into the EA community. Even just many people on the forum seem incredibly intelligent, thoughtful, kind, and talented. The people at the EA Global I atttended in 2022 were clearly brilliant. In comparison, I just feel inadequate. I wonder if others who don’t consider themselves exceptional also find themselves intellectually intimidated by the people here.
We do probably need the best of the best to be involved first and foremost, but I think we also need the average, seemingly unremarkable EA sympathetic person to be engaged in some way if we really want to be more than a small community, to be as impactful as possible. Though, maybe I’m just biased to believe that mass movements are historically what led to progress. Maybe a small group of elites leading the charge is actually what it takes?
I don’t know where I’m going with this. It’s just some thoughts that have been in the back of my head for a while. This is definitely not worth a full post, so I’m just gonna leave it as a quick take.
Donated modest amounts this year to the Against Malaria Foundation, GiveDirectly, and The Humane League.
The amounts were quite modest this year, as I have been too busy taking care of the baby to earn much in the way of income this year.
I’ve previously given to AMF and GD as my go to charities, but after the Debate Week, decided to add THL, as I was influenced by the persuasiveness of the arguments made by the Animal Welfare side.
I’m not going to say who I voted for because I think a secret ballot is important. I will say I strongly agree with the idea of using more democracy in EA for making decisions so I applaud the forum and the organizers for having this event week and letting us vote.
I tried asking ChatGPT, Gemini, and Claude to come up with a formula that converts between correlation space to probability space while preserving the relationship 0 = 1/n. I came up with such a formula a while back, so I figure it shouldn’t be hard. They all offered formulas, all of which were shown to be very much wrong when I actually graphed them to check.
My wife and I really, really liked The Good Place. I also got us a copy of How To Be Perfect and thought it was a decent read. Not particularly EA, but very balanced to consider all the major western schools of moral philosophy and give each a fair hearing. I do think it was a bit lacking in covering eastern schools of thought like the role-based ethics of Confucius, but I understand it was targeted towards an english speaking audience.
As a primer on ethics, it’s very approachable, but I do think it simplifies some things, and feels ever so slightly biased against consequentialism towards something like virtue ethics, but I’ll admit, I’m pro-Utilitarianism, and might myself be biased in the other direction.
From an EA perspective, it may not be the best introduction to us, as I believe there’s mention of EA, but it’s mostly the view that Peter Singer and his arguments are very demanding and perhaps unreasonably so, albeit, it’s a logical and important nudge towards caring and doing more (he hedges a lot in the book).
At the end of the day, the book shies away from deciding which moral theory is more correct, and as such is kinda wishy-washy, choose your own morality from a menu of possibilities, which somewhat disappointed me (but I also understand picking sides would be controversial). I’d still recommend the book to someone relatively unfamiliar with morality and ethics because it is a much friendlier introduction than say a moral philosophy textbook would be.
So, the $5,000 to save a human life actually saves more than one human life. The world fertility rate is currently 2.27 per woman, but expected to decline to 1.8 by 2050 and 1.6 by 2100. Lets assume this trend continues at a rate of −0.2 per 50 years until eventually it reaches zero at 2500. Since it takes two people to have children, we halve these numbers to get an estimate of how many human descendents to expect from a given saved human life each generation.
If each generation is ~25 years, then the numbers will follow a series like 1.135 + 0.9 + 0.85 + 0.8 … which works out to 9.685 human lives per $5000, or $516.26 per human life. Human life expectancy is increasing, but for simplicity lets assume 70 years per human life.
70 / $516.26 = 0.13559 human life years per dollar.
So, if we weigh chickens equally with humans, this favours the chickens still.
However, we can add the neuron count proxy to weigh these. Humans have approximately 86 trillion neurons, while chickens have 220 million. That’s a ratio of 390.
0.13559 x 390 = 52.88 human neuron weighted life years per dollar.
This is slightly more than 41 chicken life years per dollar. Which, given my many, many simplifying assumptions, would mean that global health is still (slightly) more cost effective.
In the interests of furthering the debate, I’ll quickly offer several additional arguments that I think can favour global health over animal welfare.
Simulation Argument
The Simulation Argument says that it is very likely we are living in an ancestor simulation rather than base reality. Given that it is likely human ancestors that the simulators are interested in fully simulating, other non-human animals are likely to not be simulated to the same degree of granularity and may not be sentient.
Pinpricks vs. Torture
This is a trolley problem scenario. It’s also been discussed by Eliezer Yudkowsky as the Speck of Dust in 3^^^3 People’s Eyes vs. One Human Being Tortured For 50 Years case. It’s also been analogously made in the famous short story The Ones Who Walk Away From Omelas by Ursula LeGuin. The basic idea is to question whether scope sensitivity is justified.
I’ll note that a way to avoid this is to adopt Maximin rather than Expected Value as the decision function, as was suggested by John Rawls in A Theory of Justice.
Incommensurability
In moral philosophy there’s a concept called incommensurability, that some things are simply not comparable. Some might argue that human and animal experiences are incommensurable, that we cannot know what it is like to be bat, for instance.
Balance of Categorical Responsibilities
There is in philosophies like Confucianism, notions like Filial Piety that support a kind of hierarchy of moral circles, such that family strictly dominates the state and so on. In the extreme, this leads to a kind of ethical egoism that I don’t think any altruist would subscribe to, but which seems a common way of thinking among laypeople and conservatives in particular. I don’t suggest this option but I mention it as an extreme case.
Utilitarianism in contrast tends to take the opposite extreme of equalizing moral circles to the point of complete impartiality towards every individual, the greatest good for the greatest number. This creates a kind of demandingness that would require us to sacrifice pretty much everything in service of this, our lives devoted entirely to something like shrimp welfare.
Rather than taking either extreme, it’s possible to balance things according to the idea that we have separate, categorical responsibilities to ourselves, to our family, to our nation, to our species, and to everyone else, and to put resources into each category so that none of our responsibilities are neglected in favour of others, a kind of meta or group impartiality rather than individual impartiality.
Yeah, I should probably retract the “we need popular support to get things done” line of reasoning.
I think lying to myself is probably, on reflection, something I do to avoid actually lying to others, as described in that link in the footnote. I kind of decide that a belief is “plausible” and then give it some conditional weight, a kind of “humour the idea and give it the benefit of the doubt”. It’s kind of a technicality thing that I do because I’m personally very against outright lying, so I’ve developed a kind of alternative way of fudging to avoid hurt feelings and such.
This is likely related to the “spin” concept that I adopted from political debates. The idea of “spin” to me is to tell the truth from an angle that encourages a perception that is favourable to the argument I am trying to make. It’s something of a habit, and most probably epistemically highly questionable and something I should stop doing.
I think I also use these things to try to take an intentionally more optimistic outlook and be more positive in order to ensure best performance at tasks at hand. If you think you can succeed, you will try harder and often succeed where if you’d been pessimistic you’d have failed due to lack of resolve. This is an adaptive response, but it admittedly sacrifices some accuracy about the actual situation.
For one’s beliefs about what is true to be influenced by anything other than evidence it might be or not be true, is an influence which will tend to diverge from what is true, by definition.
Though, what if I consider the fact that many people have independently reached a certain belief to itself be evidence that that belief might be true?
I would almost certainly add an animal welfare charity to my charitable giving portfolio.
I previously had the Good Food Institute in the portfolio before financial challenges led me to trim it, so I might bring that back, or do some more research into the most effective animal welfare charity and add it alongside AMF and GiveDirectly as my primary contributions.
Given that it seems a solid majority of EAs on the forum seem to strongly favour animal welfare with very rigorous arguments for it, and my propensity to weigh “wisdom of crowds” majority opinion as evidence towards a given view, I’m actually leaning towards actually doing this.
Sorry for the delayed response.
i’m modelling this as: basic drive to not die → selects values that are compatible with basic drive’s fulfillment.
i’ve been wondering if humans generally do something like this. (in particular to continue having values/cares after ontological crises like: losing belief in a god, or losing a close other who one was dedicated to protecting.)
This does seem like a good explanation of what happened. It does imply that I had motivated reasoning though, which probably casts some doubt on those values/beliefs being epistemically well grounded.
in case anyone has similar thoughts: to have the level of altruism to even consider the question is extremely rare. there are probably far better things you can do, than just dying and donating; like earning to give, or direct research, or maybe some third thing you’ll come up with. (most generally, the two traits i think are needed for research are intelligence and creativity. this is a creative, unintuitive moral question to ask. and my perception is that altruism and intelligence correlate, but i could be wrong about that, or biased from mostly seeing EAs.)
These words are very kind. Thank you.
I’m starting to think it was a mistake for me to engage in this debate week thing. I just spent a good chunk of my baby’s first birthday arguing with strangers on the Internet about what amounts to animals vs. humans. This does not seem like a good use of my time, but I’m too pedantic to resist replying to comments I feel the need to reply to. -_-
In general, I feel like this debate week thing seems somewhat divisive as well. At least, it doesn’t feel nice to have so many disagrees on my posts, even if they still somehow got a positive amount of karma.
I really don’t have time to make high-effort posts, and it seems like low-effort posts do a disservice to people who are making high-effort posts, so I might just stop.
My current analysis, as well as a lot of other analysis I’ve seen, suggests AGI is most likely to be possible around 2030.