Formalism
Coherentist theory of epistemic justification
Correspondence theory of truth
Consequentialism
FCCC
Karma: 126
When designing a system, you give it certain goals to satisfy. A good example of this done well is voting theory. People come up with apparently desirable properties, such the Smith criterion, and then demonstrate mathematically that certain voting methods succeed or fail the criterion. Some desirable goals cannot be achieved simultaneously (an example of this is Arrow’s impossiblility theorem).
Lotteries give every ticket has an equal chance. And if each person has one ticket, this implies each person has an equal chance. But this goal is in conflict with more important goals. I would guess that lotteries are almost never the best mechanism. Where they improve situations is for already bad mechanisms. But in that case, I’d look further for even better systems.
If people fill in the free-text box in the survey, this is essentially the same as sending an email. If I disagree with the fund’s decisions, I can send them my reasons why. If my reasons aren’t any good, the fund can see that, and ignore me; if I have good reasons, the fund should (hopefully) be swayed.
Votes without the free-text box filled in can’t signal whether the voter’s justifications are valid or not. Opinions have differing levels of information backing them up. An “unpopular” decision might be supported by everyone who knows what they’re talking about; a “popular” decision might be considered to be bad by every informed person.
My idea of EA’s essential beliefs are:
Some possible timelines are much better than others
What “feels” like the best action often won’t result in anything close to the best possible timeline
In such situations, it’s better to disregard our feelings and go with the actions that get us closer to the best timeline.
This doesn’t commit you to a particular moral philosophy. You can rank timelines by whatever aspects you want: Your moral rule can tell you to only consider your own actions, and disregard their effects on the behaviour of other people’s actions. I could consider such a person to be an effective altruist, even though they’d be a non-consequentialist. While I think it’s fair to say that, after the above beliefs, consequentialism is fairly core to EA, I think the whole EA community could switch away from consequentialism without having to rebrand itself.
The critique targets effective altruists’ tendency to focus on single actions and their proximate consequences and, more specifically, to focus on simple interventions that reduce suffering in the short term.
But she also says EA has a “god’s eye moral epistemology”. This seems contradictory. Even if we suppose that most EAs focus on proximate consequences, that’s not a fundamental failing of the philosophy, it’s a failed application of it. If many fail to accurately implement the philosophy, it doesn’t imply the philosophy bad[1]: There’s a difference between a “criterion of right” and a “decision procedure”. Many EAs are longtermists who essentially use entire timelines as the unit of moral analysis. This is clearly is not focused on “proximate consequences”. That’s more the domain of non-consequentialists (e.g. “Are my actions directly harming anyone?”).
The article’s an incoherent mess, even ignoring the Communist nonsense at the end.
- ↩︎
This is in contrast with a policies being bad because no one can implement them with the desired consequences.
It happens in philosophy sometimes too: “Saving your wife over 10 strangers is morally required because...” Can’t we just say that we aren’t moral angels? It’s not hypocritical to say the best thing is to do is save the 10 strangers, and then not do it (unless you also claim to be morally perfect). Same thing here. You can treat yourself well even if it’s not the best moral thing to do. You can value non-moral things.
I think you’re conflating moral value with value in general. People value their pets, but this has nothing to do with the pet’s instrumental moral value.
So a relevant question is “Are you allowed to trade off moral value for non-moral value?” To me, morality ranks (probability distributions of) timelines by moral preference. Morally better is morally better, but nothing is required of you. There’s no “demandingness”. I don’t buy into the notions of “morally permissible” or “morally required”: These lines in the sand seem like sociological observations (e.g. whether people are morally repulsed by certain actions in the current time and place) rather than normative truths.
I do think having more focus on moral value is beneficial, not just because it’s moral, but because it endures. If you help a lot of people, that’s something you’ll value until you die. Whereas if I put a bunch of my time into playing chess, maybe I’ll consider that to be a waste of time at some point in the future. There’s other things, like enjoying relationships with your family, that also aren’t anywhere close to the most moral thing you could be doing, but you’ll probably continue to value.
You’re allowed to value things that aren’t about serving the world.
Hey Bob, good post. I’ve had the same thought (i.e. the unit of moral analysis is timelines, or probability distributions of timelines) with different formalism
The trolley problem gives you a choice between two timelines (). Each timeline can be represented as the set containing all statements that are true within that timeline. This representation can neatly state whether something is true within a given timeline or not: “You pull the lever” , and “You pull the lever” . Timelines contain statements that are combined as well as statements that are atomized. For example, since “You pull the lever”, “The five live”, and “The one dies” are all elements of , you can string these into a larger statement that is also in : “You pull the lever, and the five live, and the one dies”. Therefore, each timeline contains a very large statement that uniquely identifies it within any finite subset of . However, timelines won’t be our unit of analysis because the statements they contain have no subjective empirical uncertainty.
This uncertainty can be incorporated by using a probability distribution of timelines, which we’ll call a forecast (). Though there is no uncertainty in the trolley problem, we could still represent it as a choice between two forecasts: guarantees (the pull-the-lever timeline) and guarantees (the no-action timeline). Since each timeline contains a statement that uniquely identifies it, each forecast can, like timelines, be represented as a set of statements. Each statement within a forecast is an empirical prediction. For example, would contain “The five live with a credence of 1”. So, the trolley problem reveals that you either morally prefer (denoted as ), prefer (denoted as ), or you believe that both forecasts are morally equivalent (denoted as ).
I watched those videos you linked. I don’t judge you for feeling that way.
Did you convert anyone to veganism? If people did get converted, maybe there were even more effective ways to do so. Or maybe anger was the most effective way; I don’t know. But if not, your own subjective experience was worse (by feeling contempt), other people felt worse, and fewer animals were helped. Anger might be justified but, assuming there was some better way to convert people, you’d be unintentionally prioritizing emotions ahead of helping the animals.
Another thing to keep in mind: When we train particular physical actions, we get better at repeating that action. Athletes sometimes repeat complex, trained actions before they have any time to consciously decide to act. I assume the same thing happens with our emotions: If we feel a particular way repeatedly, we’re more likely to feel that way in future, maybe even when it’s not warranted.
We can be motivated to do something good for the world in lots of different ways. Helping people by solving problems gives my life meaning and I enjoy doing it. No negative emotions needed.
“writing down stylized models of the world and solving for the optimal thing for EAs to do in them”
I think this is one of the most important things we can be doing. Maybe even the most important since it covers such a wide area and so much government policy is so far from optimal.
you just solve for the policy … that maximizes your objective function, whatever that may be.
I don’t think that’s right. I’ve written about what it means for a system to do “the optimal thing” and the answer cannot be that a single policy maximizes your objective function:
Societies need many distinct systems: a transport system, a school system, etc. These systems cannot be justified if they are amoral, so they must serve morality. Each system cannot, however, achieve the best moral outcome on its own: If your transport system doesn’t cure cancer, it probably isn’t doing everything you want; if it does cure cancer, it isn’t just a “transport” system...
Unless by policy, you mean “the entirety of what government does”, then yes. But given that you’re going to consider one area at a time, and you’re “only including all the levers between which you’re considering”, you could reach a local optimum rather than a truly ideal end state. The way I like to think about it is “How would a system for prisons (for example) be in the best possible future?” This is not necessarily going to be the system that does the greatest good at the margin when constrained to the domain you’re considering (though they often are). Rather than think about a system maximizing your objective function, it’s better to think of systems as satisfying goals that are aligned with your objective function.
And bits describe proportional changes in the number of possibilities, not absolute changes...
And similarly, the 3.3 bits that take you from 100 possibilities to 10 are the same amount of information as the 3.3 bits that take you from 10 possibilities to 1. In each case you’re reducing the number of possibilities by a factor of 10.
Ahhh. Thanks for clearing that up for me. Looking at the entropy formula, that makes sense and I get the same answer as you for each digit (3.3). If I understand, I incorrectly conflated “information” with “value of information”.
I think this is better parsed as diminishing marginal returns to information.
How does this account for the leftmost digit giving the most information, rather than the rightmost digit (or indeed any digit between them)?
per-thousandths does not have double the information of per-cents, but 50% more
Let’s say I give you $1 + $ where is either 0, $0.1, $0.2 … or $0.9. (Note $1 is analogous to 1%, and is equivalent adding a decimal place. I.e. per-thousandths vs per-cents.) The average value of , given a uniform distribution, is $0.45. Thus, against $1, adds almost half the original value, i.e. $0.45/$1 (45%). But what if I instead gave you $99 + $? $0.45 is less than 1% of the value of $99.
The leftmost digit is more valuable because it corresponds to a greater place value (so the magnitude of the value difference between places is going to be dependent on the numeric base you use). I don’t know information theory, so I’m not sure how to calculate the value of the first two digits compared to the third, but I don’t think per-thousandths has 50% more information than per-cents.
From a bayesian perspective there is no particular reason why you have to provide more evidence if you provide credences
This statement is just incorrect.
Does this match your view?
Basically, yeah.
But I do think it’s a mistake to update your credence based off someone else’s credence without knowing their argument and without knowing whether they’re calibrated. We typically don’t know the latter, so I don’t know why people are giving credences without supporting arguments. It’s fine to have a credence without evidence, but why are people publicising such credences?
But you say invalid meta-arguments, and then give the example “people make logic mistakes so you might have too”. That example seems perfectly valid, just often not very useful.
My definition of an invalid argument contains “arguments that don’t reliably differentiate between good and bad arguments”. “1+1=2″ is also a correct statement, but that doesn’t make it a valid response to any given argument. Arguments need to have relevancy. I dunno, I could be using “invalid” incorrectly here.
And I’d also say that that example meta-argument could sometimes be useful.
Yes, if someone believed that having a logical argument is a guarantee, and they’ve never had one of their logical arguments have a surprising flaw, it would be valid to point that out. That’s fair. But (as you seem to agree with) the best way to do this is to actually point to the flaw in the specific argument they’ve made. And since most people who are proficient with logic already know that logic arguments can be unsound, it’s not useful to reiterate that point to them.
Also, isn’t your comment primarily meta-arguments of a somewhat similar nature to “people make logic mistakes so you might have too”?
It is, but as I said, “Some meta-arguments are valid”. (I can describe how I delineate between valid and invalid meta-arguments if you wish.)
Describing that as pseudo-superforecasting feels unnecessarily pejorative.
Ah sorry, I didn’t mean to offend. If they were superforecasters, their credence alone would update mine. But they’re probably not, so I don’t understand why they give their credence without a supporting argument.
Did you mean “some ideas that are probably correct and very important”?
The set of things I give 100% credence is very, very small (i.e. claims that are true even if I’m a brain in a vat). I could say “There’s probably a table in front of me”, which is technically more correct than saying that there definitely is, but it doesn’t seem valuable to qualify every statement like that.
Why am I confident in moral uncertainty? People do update their morality over time, which means that they were wrong at some point (i.e. there is demonstrably moral uncertainty), or the definition of “correct” changes and nobody is ever wrong. I think “nobody is ever wrong” is highly unlikely, especially because you can point to logical contradictions in people’s moral beliefs (not just unintuitive conclusions). At that point, it’s not worth mentioning the uncertainty I have.
I definitely don’t think EAs are perfect, but they do seem above-average in their tendency to have true beliefs and update appropriately on evidence, across a wide range of domains.
Yeah, I’m too focused on the errors. I’ll concede your point: Some proportion of EAs are here because they correctly evaluated the arguments. So they’re going to bump up the average, even outside of EA’s central ideas. My reference classes here were all the groups that have correct central ideas, and yet are very poor reasoners outside of their domain. My experience with EAs is too limited to support my initial claim.
It’s almost irrelevant, people still should provide their supporting argument of their credence, otherwise evidence can get “double counted” (and there’s “flow on” effects where the first person who updates another person’s credence has a significant effect on the overall credence of the population). For example, say I have arguments A and B supporting my 90% credence on something. And you have arguments A, B and C supporting your 80% credence on something. And neither of us post our reasoning; we just post our credences. It’s a mistake for you to then say “I’ll update my credence a few percent because FCCC might have other evidence.” For this reason, providing supporting arguments is a net benefit, irrespective of EA’s accuracy of forecasts.
The two statements are pretty similar in verbalized terms (and each falls under loose interpretations of what “pretty sure” means in common language), but ought to have drastically different implications for behavior!
Yes you’re right. But I’m making a distinction between people’s own credences and their ability to update the credences of other people. As far as changing the opinion of the reader, when someone says “I haven’t thought much about it”, it should be an indicator to not update your own credence by very much at all.
I basically think EA and associated communities would be better off to have more precise credences, and be accountable for them
I fully agree. My problem is that this is not the current state of affairs for the majority of Forum users, in which case, I have no reason to update my credences because an uncalibrated random person says they’re 90% confident without providing any reasoning that justifies their position. All I’m asking for is for people to provide a good argument along with their credence.
I agree that EAs put superforecasters and superforecasting techniques on a pedestal, more than is warranted.
I think that they should be emulated. But superforcasters have reasoning to justify their credences. They break problems down into components that they’re more confident in estimating. This is good practice. Providing a credence without any supporting argument, is not.
I’m not sure how you think that’s what I said. Here’s what I actually said:
A superforecaster’s credence can shift my credence significantly...
If the credence of a random person has any value to my own credence, it’s very low...
The evidence someone provides is far more important than someone’s credence (unless you know the person is highly calibrated and precise)...
[credences are] how people should think...
if you’re going to post your credence, provide some evidence so that you can update other people’s credences too.
I thought I was fairly clear about what my position is. Credences have internal value (you should generate your own credence). Superforecasters’ credences have external value (their credence should update yours). Uncalibrated random people’s credences don’t have much external value (they shouldn’t shift your credence much). And an argument for your credence should always be given.
I never said vague words are valuable, and in fact I think the opposite.
This is an empirical question. Again, what is the reference class for people providing opinions without having evidence? We could look at all of the unsupported credences on the forum and see how accurate they turned out to be. My guess is that they’re of very little value, for all the reasons I gave in previous comments.
you are concretely making the point that it’s additionally bad for them to give explicit credences!
I demonstrated a situation where a credence without evidence is harmful:
If we have different credences and the set of things I’ve considered is a strict subset of yours, you might update your credence because you mistakenly think I’ve considered something you haven’t.
The only way we can avoid such a situation is either by providing a supporting argument for our credences, OR not updating our credences in light of other people’s unsupported credences.
Yes, in most cases if somebody has important information that an event has XY% probability of occurring, I’d usually pay a lot more to know what X is than what Y is.
As you should, but Greg is still correct in saying that Y should be provided.
Regarding the bits of information, I think he’s wrong because I’d assume information should be independent of the numeric base you use. So I think Y provides 10% of the information of X. (If you were using base 4 numbers, you’d throw away 25%, etc.)
But again, there’s no point in throwing away that 10%.
I agree. Rounding has always been ridiculous to me. Methodologically, “Make your best guess given the evidence, then round” makes no sense. As long as your estimates are better than random chance, it’s strictly less reliable than just “Make your best guess given the evidence”.
Credences about credences confuse me a lot (is there infinite recursion here? I.e. credences about credences about credences...). My previous thoughts have been give a credence range or to size a bet (e.g. “I’d bet $50 out of my $X of wealth at a Y odds”). I like both your solutions (e.g. “if I thought about it for an hour...”). I’d like to see an argument that shows there’s an optimal method for representing the uncertainty of a credence. I wouldn’t be surprised if someone has the answer and I’m just unaware of it.
I’ve thought about the coin’s 50% probability before. Given a lack of information about the initial forces on the coin, there exists an optimal model to use. And we have reasons to believe a 50-50 model is that model (given our physics models, simulate a billion coin flips with a random distribution of initial forces). This is why I like your “If I thought about it more” model. If I thought about the coin flip more, I’d still guess 49%-51% (depending on the specific coin, of course).
I mean, very frequently it’s useful to just know what someone’s credence is. That’s often an order of magnitude cheaper to provide, and often is itself quite a bit of evidence.
I agree, but only if they’re a reliable forecaster. A superforecaster’s credence can shift my credence significantly. It’s possible that their credences are based off a lot of information that shifts their own credence by 1%. In that case, it’s not practical for them to provide all the evidence, and you are right.
But most people are poor forecasters (and sometimes they explicitly state they have no supporting evidence other than their intuition), so I see no reason to update my credence just because someone I don’t know is confident. If the credence of a random person has any value to my own credence, it’s very low.
This is like saying that all statements of opinions or expressions of feelings are bad, unless they are accompanied with evidence, which seems like it would massively worsen communication.
That would depend on the question. Sometimes we’re interested in feelings for their own sake. That’s perfectly legitimate because the actual evidence we’re wanting is the data about their feelings. But if someone’s giving their feelings about whether there are an infinite number of primes, it doesn’t update my credences at all.
I think opinions without any supporting argument worsen discourse. Imagine a group of people thoughtfully discussing evidence, then someone comes in, states their feelings without any evidence, and then leaves. That shouldn’t be taken seriously. Increasing the proportion of those people only makes it worse.
Bayesians should want higher-quality evidence. Isn’t self-reported data is unreliable? And that’s when the person was there when the event happened. So what is the reference class for people providing opinions without having evidence? It’s almost certainly even more unreliable. If someone has an argument for their credence, they should usually give that argument; if they don’t have an argument, I’m not sure why they’re adding to the conversation.
I’m not saying we need to provide peer-reviewed articles. I just want to see some line of reasoning demonstrating why you came to the conclusion you made, so that everyone can examine your assumptions and inferences. If we have different credences and the set of things I’ve considered is a strict subset of yours, you might update your credence because you mistakenly think I’ve considered something you haven’t.
There are mechanisms that aggregate distributed knowledge, such as free-market pricing.
Not with 100 percent accuracy, but that’s not the right question. We want to know whether it can be done better than chance. Someone can lack knowledge and be biased and still reliably do better than random (try playing chess against a computer that plays uniformly random moves).
Wouldn’t the “efficient-policy hypothesis” imply that lotteries are worse than the existing systems? I don’t think you really believe this. Are our systems better than most hypothetical systems? Usually, but this doesn’t mean there’s no low-hanging fruit. There’s plenty of good policy ideas that are well-known and haven’t been implemented, such as 100 percent land-value taxes.
Let’s take a subset of the research funding problem: How can we decide what to fund for research about prisoner rehabilitation? I’ve suggested a mechanism that would do this.