Formalism
Coherentist theory of epistemic justification
Correspondence theory of truth
Consequentialism
FCCC
Karma: 126
But you say invalid meta-arguments, and then give the example “people make logic mistakes so you might have too”. That example seems perfectly valid, just often not very useful.
My definition of an invalid argument contains “arguments that don’t reliably differentiate between good and bad arguments”. “1+1=2″ is also a correct statement, but that doesn’t make it a valid response to any given argument. Arguments need to have relevancy. I dunno, I could be using “invalid” incorrectly here.
And I’d also say that that example meta-argument could sometimes be useful.
Yes, if someone believed that having a logical argument is a guarantee, and they’ve never had one of their logical arguments have a surprising flaw, it would be valid to point that out. That’s fair. But (as you seem to agree with) the best way to do this is to actually point to the flaw in the specific argument they’ve made. And since most people who are proficient with logic already know that logic arguments can be unsound, it’s not useful to reiterate that point to them.
Also, isn’t your comment primarily meta-arguments of a somewhat similar nature to “people make logic mistakes so you might have too”?
It is, but as I said, “Some meta-arguments are valid”. (I can describe how I delineate between valid and invalid meta-arguments if you wish.)
Describing that as pseudo-superforecasting feels unnecessarily pejorative.
Ah sorry, I didn’t mean to offend. If they were superforecasters, their credence alone would update mine. But they’re probably not, so I don’t understand why they give their credence without a supporting argument.
Did you mean “some ideas that are probably correct and very important”?
The set of things I give 100% credence is very, very small (i.e. claims that are true even if I’m a brain in a vat). I could say “There’s probably a table in front of me”, which is technically more correct than saying that there definitely is, but it doesn’t seem valuable to qualify every statement like that.
Why am I confident in moral uncertainty? People do update their morality over time, which means that they were wrong at some point (i.e. there is demonstrably moral uncertainty), or the definition of “correct” changes and nobody is ever wrong. I think “nobody is ever wrong” is highly unlikely, especially because you can point to logical contradictions in people’s moral beliefs (not just unintuitive conclusions). At that point, it’s not worth mentioning the uncertainty I have.
I definitely don’t think EAs are perfect, but they do seem above-average in their tendency to have true beliefs and update appropriately on evidence, across a wide range of domains.
Yeah, I’m too focused on the errors. I’ll concede your point: Some proportion of EAs are here because they correctly evaluated the arguments. So they’re going to bump up the average, even outside of EA’s central ideas. My reference classes here were all the groups that have correct central ideas, and yet are very poor reasoners outside of their domain. My experience with EAs is too limited to support my initial claim.
It’s almost irrelevant, people still should provide their supporting argument of their credence, otherwise evidence can get “double counted” (and there’s “flow on” effects where the first person who updates another person’s credence has a significant effect on the overall credence of the population). For example, say I have arguments A and B supporting my 90% credence on something. And you have arguments A, B and C supporting your 80% credence on something. And neither of us post our reasoning; we just post our credences. It’s a mistake for you to then say “I’ll update my credence a few percent because FCCC might have other evidence.” For this reason, providing supporting arguments is a net benefit, irrespective of EA’s accuracy of forecasts.
The two statements are pretty similar in verbalized terms (and each falls under loose interpretations of what “pretty sure” means in common language), but ought to have drastically different implications for behavior!
Yes you’re right. But I’m making a distinction between people’s own credences and their ability to update the credences of other people. As far as changing the opinion of the reader, when someone says “I haven’t thought much about it”, it should be an indicator to not update your own credence by very much at all.
I basically think EA and associated communities would be better off to have more precise credences, and be accountable for them
I fully agree. My problem is that this is not the current state of affairs for the majority of Forum users, in which case, I have no reason to update my credences because an uncalibrated random person says they’re 90% confident without providing any reasoning that justifies their position. All I’m asking for is for people to provide a good argument along with their credence.
I agree that EAs put superforecasters and superforecasting techniques on a pedestal, more than is warranted.
I think that they should be emulated. But superforcasters have reasoning to justify their credences. They break problems down into components that they’re more confident in estimating. This is good practice. Providing a credence without any supporting argument, is not.
I’m not sure how you think that’s what I said. Here’s what I actually said:
A superforecaster’s credence can shift my credence significantly...
If the credence of a random person has any value to my own credence, it’s very low...
The evidence someone provides is far more important than someone’s credence (unless you know the person is highly calibrated and precise)...
[credences are] how people should think...
if you’re going to post your credence, provide some evidence so that you can update other people’s credences too.
I thought I was fairly clear about what my position is. Credences have internal value (you should generate your own credence). Superforecasters’ credences have external value (their credence should update yours). Uncalibrated random people’s credences don’t have much external value (they shouldn’t shift your credence much). And an argument for your credence should always be given.
I never said vague words are valuable, and in fact I think the opposite.
This is an empirical question. Again, what is the reference class for people providing opinions without having evidence? We could look at all of the unsupported credences on the forum and see how accurate they turned out to be. My guess is that they’re of very little value, for all the reasons I gave in previous comments.
you are concretely making the point that it’s additionally bad for them to give explicit credences!
I demonstrated a situation where a credence without evidence is harmful:
If we have different credences and the set of things I’ve considered is a strict subset of yours, you might update your credence because you mistakenly think I’ve considered something you haven’t.
The only way we can avoid such a situation is either by providing a supporting argument for our credences, OR not updating our credences in light of other people’s unsupported credences.
Yes, in most cases if somebody has important information that an event has XY% probability of occurring, I’d usually pay a lot more to know what X is than what Y is.
As you should, but Greg is still correct in saying that Y should be provided.
Regarding the bits of information, I think he’s wrong because I’d assume information should be independent of the numeric base you use. So I think Y provides 10% of the information of X. (If you were using base 4 numbers, you’d throw away 25%, etc.)
But again, there’s no point in throwing away that 10%.
I agree. Rounding has always been ridiculous to me. Methodologically, “Make your best guess given the evidence, then round” makes no sense. As long as your estimates are better than random chance, it’s strictly less reliable than just “Make your best guess given the evidence”.
Credences about credences confuse me a lot (is there infinite recursion here? I.e. credences about credences about credences...). My previous thoughts have been give a credence range or to size a bet (e.g. “I’d bet $50 out of my $X of wealth at a Y odds”). I like both your solutions (e.g. “if I thought about it for an hour...”). I’d like to see an argument that shows there’s an optimal method for representing the uncertainty of a credence. I wouldn’t be surprised if someone has the answer and I’m just unaware of it.
I’ve thought about the coin’s 50% probability before. Given a lack of information about the initial forces on the coin, there exists an optimal model to use. And we have reasons to believe a 50-50 model is that model (given our physics models, simulate a billion coin flips with a random distribution of initial forces). This is why I like your “If I thought about it more” model. If I thought about the coin flip more, I’d still guess 49%-51% (depending on the specific coin, of course).
I mean, very frequently it’s useful to just know what someone’s credence is. That’s often an order of magnitude cheaper to provide, and often is itself quite a bit of evidence.
I agree, but only if they’re a reliable forecaster. A superforecaster’s credence can shift my credence significantly. It’s possible that their credences are based off a lot of information that shifts their own credence by 1%. In that case, it’s not practical for them to provide all the evidence, and you are right.
But most people are poor forecasters (and sometimes they explicitly state they have no supporting evidence other than their intuition), so I see no reason to update my credence just because someone I don’t know is confident. If the credence of a random person has any value to my own credence, it’s very low.
This is like saying that all statements of opinions or expressions of feelings are bad, unless they are accompanied with evidence, which seems like it would massively worsen communication.
That would depend on the question. Sometimes we’re interested in feelings for their own sake. That’s perfectly legitimate because the actual evidence we’re wanting is the data about their feelings. But if someone’s giving their feelings about whether there are an infinite number of primes, it doesn’t update my credences at all.
I think opinions without any supporting argument worsen discourse. Imagine a group of people thoughtfully discussing evidence, then someone comes in, states their feelings without any evidence, and then leaves. That shouldn’t be taken seriously. Increasing the proportion of those people only makes it worse.
Bayesians should want higher-quality evidence. Isn’t self-reported data is unreliable? And that’s when the person was there when the event happened. So what is the reference class for people providing opinions without having evidence? It’s almost certainly even more unreliable. If someone has an argument for their credence, they should usually give that argument; if they don’t have an argument, I’m not sure why they’re adding to the conversation.
I’m not saying we need to provide peer-reviewed articles. I just want to see some line of reasoning demonstrating why you came to the conclusion you made, so that everyone can examine your assumptions and inferences. If we have different credences and the set of things I’ve considered is a strict subset of yours, you might update your credence because you mistakenly think I’ve considered something you haven’t.
You seem to have switched from the claim that EAs often report their credences without articulating the evidence on which those credences rest, to the claim that EAs often lack evidence for the credences they report.
Habryka seems to be talking about people who have evidence and are just not stating it, so we might be talking past one another. I said in my first comment “There’s also a lot of pseudo-superforcasting … without any evidence backing up those credences.” I didn’t say “without stating any evidence backing up those credences.” This is not a guess on my part. I’ve seen comments where they say explicitly that the credence they’re giving is a first impression, and not something well thought out. It’s fine for them to have a credence, but why should anyone care what your credence is if it’s just a first impression?
See Greg Lewis’s recent post; I’m not sure if you disagree.
I completely agree with him. Imprecision should be stated and significant figures are a dumb way to do it. But if someone said “I haven’t thought about this at all, but I’m pretty sure it’s true”, is that really all that much worse than providing your uninformed prior and saying you haven’t really thought about it?
From a bayesian perspective there is no particular reason why you have to provide more evidence if you provide credences
Sure there is: By communicating, we’re trying to update one another’s credences. You’re not going to be very successful in doing so if you provide a credence without supporting evidence. The evidence someone provides is far more important than someone’s credence (unless you know the person is highly calibrated and precise). If you have a credence that you keep to yourself, then yes, there’s no need for supporting evidence.
if only to avoid problems of ambiguous language.
Ambiguous statements are bad, 100%, but so are clear, baseless statements.
As you say, people can legitimately have credences about anything. It’s how people should think. But if you’re going to post your credence, provide some evidence so that you can update other people’s credences too.
EA epistemology is weaker than expected.
I’d say nearly everyone’s ability to determine an argument’s strength is very weak. On the Forum, invalid meta-arguments* are pretty common, such as “people make logic mistakes so you might have too”, rather than actually identifying the weaknesses in an argument. There’s also a lot of pseudo-superforcasting, like “I have 80% confidence in this”, without any evidence backing up those credences. This seems to me like people are imitating sound arguments without actually understanding how they work. Effective altruists have centred around some ideas that are correct (longtermism, moral uncertainty, etc.), but outside of that, I’d say we’re just as wrong as anyone else.
*Some meta-arguments are valid, like discussions on logical grounding of particular methodologies, e.g. “Falsification works because of the law of contraposition, which follows from the definition of logical implication”.
Yeah, that’s right. The problem with my toy model is that it assumes that funds can actually estimate their optimal bid, which would need to be an exact prediction of the their future returns at an exact time, which is not possible. Allowing bids to reference a single, agreed-upon global index reduces the problem to a prediction of costs, which is much easier for the funds. And in the long run, returns can’t be higher the return of the global index, so it should maximize long-run returns.
However, most (?) indices are made by committees, which I don’t like, so I wanted to see other people’s ideas for making this workable. (But index committees are established and seem to work well, so relying on them is less risky than setting up a brand-new committee as proposed in that government report.)
Yeah, it’s definitely flawed. I was more thinking that the bids could be made as a difference between an index (probably a global one). So the profit-maximizing bids for the funds would be the index return (whatever it happens to be) minus their expected costs. And then you have large underwriters of the firms, who make sure that the fund’s processes are sound. What I’d like is everyone to be in Vanguard/Blackrock, but there should be some mechanism for others to overthrow them if someone can match the index at a lower cost.
Caught red handed. I’d been thinking about this idea for a while and was trying to get the maths to work last night, so I had my prison/immigration idea next to me for reference.
I like this idea; we should have many more second-price auctions out there. Do you have any further references about it?
Thanks. I’m not the best person to ask about auctions. For people looking for an introduction, this video is pretty good. If anyone’s got a good textbook, I’d be interested.
Maximizing the Long-Run Returns of Retirement Savings
Ah yes, I have mentioned in other comments about regulation keeping private prisons in check too. I should have restated it here. I am in favour of checks and balances, which is why my goal for the system contains ”...within the limits of the law”. I agree with almost everything you say here (I’d keep some public prisons until confident that the market is mature enough to handle all cases better than the public system, but I wouldn’t implement your 10-year loan).
Human rights laws. Etc.
Yep, I’m all for that. One thing that people are missing is that the goal of a system kind of… “compresses” the space of things you want to happen. That compression is lossy. You want that goal to lose as few things as possible, but you will lose some things. To fix that, you will need some regulation to make sure the system works in the important edge cases.
prisons may not have a strong incentive to care about the welfare of the prisoners whilst they are in the prison
This is incorrect. They do have a strong incentive, since the contract comes into effect immediately after the auction: If a crime happens in their prison, the prison has to pay the government. The resulting problem is that prisons have an incentive to hide these crimes. So I recommended that prisons be outfitted with cameras and microphones that are externally audited.
Basically everyone was convinced by the theory of small loans to those too poor to receive finance.
I was against microfinance, but I also don’t know how they justified the idea. I think empirical evidence should be used to undermine certain assumptions of models, such as “People will only take out a loan if it’s in their own interests”. Empirically, that’s clearly not always the case (e.g. people go bankrupt from credit cards), and any model that relies on that statement being true may fail because of it. A theoretical argument with bad assumptions is a bad argument.
“Any prison system that does so [works] will look similar to mine, e.g. prisons would need to get paid when convicts pay tax.”
Blanket statements like this – suggesting your idea or similar is the ONLY way prisons can work still concerns me and makes me think that you value theoretical data too highly compared to empirical data.
That’s not what I said. I said “Most systems can work well in the short run”, but systems that don’t internalize externalities are brittle (“For a system to work well and be robust”). Internalizing externalities has a very strong evidence base (theoretical and empirical). If anyone can show me a system that internalizes externalities and doesn’t look like my proposal, I will concede.
I still think it could help the case to think about how a pilot prison could be made to produce useful data.
I think we agree that you want “cheap and informative tests”. Some of the data you’re talking about already exists (income tax data, welfare payments, healthcare subsidies), which is cheap (because it already exists) and informative (because it’s close to the data prisons would need to operate effectively).
Social impact bonds for prisons are already in place, and that system is similar to mine in some respects (though it has poor, ad hoc justification, and so the system should fail in predictable ways). You’re right about me not being interested those systems. Social impact bonds are probably the best reference class we have for my proposal. But if they failed, I wouldn’t update my beliefs very much since the theory for the impact bonds is so poor.
Hmm. Actually, you’re right, you can make a small trial work.
1. Randomly select enough inmates to fill US prisons. Randomly put 50 percent of those inmates in set A, and the other inmates in set B.
2. Allow any prison to bid on the contracts of set B. The participating prisons have to bid on all of those inmates’ contracts.
3. Select the prisons in such a way that those prisons are filled, and that the sum of the bids is maximized. Because all prisons are bidding, you get a thick market, and the assumptions of my argument should hold (you may have to compensate prisons who make reasonable bids).
4. Select random prisons to take on set A. Does this introduce selection bias? Yes, and that’s exactly the point. In my proposal, the best prisons self-select by making higher bids (criterion 1).
5. Observe.
I’m interested to hear what you think.
Theoretical reasons are great but actual evidence [is] more important.
Good theoretical evidence is “actual evidence”. No amount of empirical evidence is up to the task of proving there are an infinite number of primes. Our theoretical argument showing there are an infinite number of primes is the strongest form of evidence that can be given.
That’s not to say I think my argument is airtight, however. My argument could probably be made with more realistic assumptions (alternatively, more realistic assumptions might show my proposed system is fundamentally mistaken). My model also just describes an end state, rather than provide any help on how to build this market from scratch (implementing the system overnight would produce a market that’s too thin and inefficient).
Theory can go wrong if the assumptions are false or the inferences are invalid. Both of these errors happen all the time of course, and in subtle ways too, so I agree that empirical evidence is important. But even with data, you need a model to properly interpret it. Data without a model only tells you what was measured, which usually isn’t that informative. No matter what the numbers are, no one can say the UK prison system is better than that of the US without some assumptions. And no one can get data on a system that hasn’t been tried before (depending on the reference classes available to you).
Consider international development. The effective altruism community has been saying for years (and backing up these claims) that in development you cannot just do things that theoretically sound like they will work (like building schools) but you need to do things that have empirical evidence of working well.
Can you show me a theoretical model of school building that would convince me that it would work when it would, in fact, fail? I don’t think I would be convinced. (How can you be sure the teachers are good? How can you be sure good teachers will stay? How can you be sure students will show up?) You can’t bundle all theory in the same basket (otherwise I could point to mathematics and say theory is always sound). Whether a theory is good evidence hinges on whether the assumptions are close enough to reality.
People are very very good at persuading themselves in what they believe [...]
Be wary of the risk of motivated reasoning
[But your] claim might be true and you might have good evidence for it
The process of building the argument changed what I believed the system should be. I have no dog in this fight. I claimed “If the UK system currently works well, I suspect that you have good regulators who are manually handling the shortcomings of the underlying system” for several reasons:
it is true of so many systems I’ve seen, including all of the prison systems I’ve seen (reference class)
For a system to work well and be robust, externalities need to be internalized. Any prison system that does so will look similar to mine, e.g. prisons would need to get paid when convicts pay tax. You would have mentioned if the UK system did so.
The data I’ve seen on UK prisons is about 50% recidivism, which I don’t think would be the case under a functional system.
Don’t under-value evidence, you might miss things. An underplayed strength of your case for fixing private prisons is that the solution you suggest is testable. A single pilot prison could be run and data collected and lessons learned. To some degree this could even be done by an committed entrepreneur with minimal government support.
A pilot prison wouldn’t work because it wouldn’t have competitive bidding. I did mention in Aaron Gertler’s comment what data should be collated prior to implementation. I’m all for looking at the relevant data.
If one [UK public prisons or UK private prisons are] clearly failing it motivates change in the other.
That doesn’t seem like a good system.
The bidding process and the actualisation of losses (tied to real social interests) keep the prisons in check.
Thanks for the kind comment.
My guess is that the US would be the best place to start (a thick “market”, poor outcomes), but I’m talking about prison systems in general.
I’m not familiar with the UK system, but I haven’t heard of any prison system with a solid theoretical grounding. Theory is required because we want to compare a proposed system to all other possible systems and conclude that our proposal is the best. You want theoretical reasons to believe that your system will perform well, and that good performance will endure.
Most systems can work well in the short run, but that doesn’t mean they’re good. For example, if I were ruled by a good king, I still wouldn’t want to have a monarchy in place. If the UK system currently works well, I suspect that you have good regulators who are manually handling the shortcomings of the underlying system.
I emailed Robin Hanson about my immigration idea in 2018. His post was in 2019. But to be fair, he came up with futarchy well before I started working on policy.
pay annual dividends proportional to these numbers
Doing things in proportion (rather than selling the full value) undervalues the impact of good forecasts. Since making forecasts has a cost, proportional payment (where the proportionality constant is not equal to 1) would generate inefficient outcomes: Imagine the contribution of the immigrant is $100 and it costs $80 to make the forecast, then paying forecasters anything less than 80% will cause poor outcomes.
Basically, yeah.
But I do think it’s a mistake to update your credence based off someone else’s credence without knowing their argument and without knowing whether they’re calibrated. We typically don’t know the latter, so I don’t know why people are giving credences without supporting arguments. It’s fine to have a credence without evidence, but why are people publicising such credences?