In case you know this off-hand or it’s easy for you to get or point me in the right direction, do you know how they established SBF’s intent to misuse the billions in customer funds? What I got from Googling this didn’t seem very convincing, but I didn’t read court documents directly. (See also https://forum.effectivealtruism.org/posts/ggkiDAZowmuzqZrnX/is-anyone-else-still-confused-about-what-exactly-happened-at )
MichaelStJules
I suspect he meant something like an improvement of 40 percentage points along the normalized 0-100% scale, so with a scale of −10 to 10, this would be adding 8 to their welfare: 8=40%*(10-(-10)).
(Or it could just be 40 percentage points along the normalized negative part of the welfare scale, so +4 on a scale of −10 to 10.)
A few years late, but this was an interesting read.
I have a few related thoughts:
I’ve tried to characterize types of subjective welfare that seem important to me here. The tl;dr is that they’re subjective appearances of good, bad, better, worse, reasons, mattering. Effectively conscious evaluations of some kind, or particular dispositions for those. That includes the types of preferences we would normally want to worry about, but not merely behavioural preferences in entirely unconscious beings or even conscious beings who would make no such conscious evaluations. But consciousness, conscious evaluation, pleasure, suffering, etc. could be vague and apparently merely behavioural preferences could be borderline cases.
On preferences referring to preferences, you can also get similar issues like two people caring more about the other’s preferences than their own “personal” preferences, both positively like people who love each other, or negatively, like people who hate each other. You can get a system of equations, but the solution can go off to infinity or not make any sense at all. There’s some other writing on this in Bergstrom, 1989, Bergstrom, 1999, Vadasz, 2005, Yann, 2005 and Dave and Dodds, 2012.
Still, I wonder if we can get around issues from preferences referring to preferences by considering what preferences can actually be about in a way that actually makes sense physically. Preferences have to be realized physically themselves, too, after all. If the result isn’t logically coherent, then maybe the preference is actually just impossible to hold? But maybe you still have problems when you idealize? And maybe indeterminacy or vagueness is okay. And since a preference is realized in a finite system with finitely many possible states (unless you idealize, or consider unbounded or infinite time), then the degrees of satisfaction they can take are also bounded.
When ice seemed like it could have turned out to be something other than the solid phase of water, we would be comparing the options based on the common facts — the evidence or data — the different possibilities were supposed to explain. And then by finding out that ice is water, you learn that there is much more water in the world, because you would then also have to count all the ice on top of all the liquid water.[13] If your moral theory took water to be intrinsically good and more of it to be better, this would be good news (all else equal).
Suppose we measure amounts by mass. The gram was in fact originally defined as the mass of one cubic centimetre of pure water at 0 °C.[1] We could imagine having defined the gram as the mass of one cubic centimetre of liquid water, but using water that isn’t necessarily pure, not fixing the temperature or using a not fully fixed measure for the centimetre. This introduces uncertainty about the measure of mass itself, and we’d later revise the definition as we understood more, but we could still use it in the meantime. We’d also aim to roughly match the original definition: the revised mass of one cubic centimetre of water shouldn’t be too different from 1 gram under the new definition.
This is similar to what I say we’d do with consciousness: we define it first relative to human first-person experiences and measure relative to them, but revise the concept and measure with further understanding. We should also aim to make conservative revisions and roughly preserve the value in our references, human first-person experiences.
- ^
In French:
Gramme, le poids absolu d’un volume d’eau pure égal au cube de la centième partie du mètre , et à la température de la glace fondante.
- ^
However, this has only proved to be useful to make predictions in the observable universe, so extending it to the entire universe would not be empirically justifiable.
Useful so far! The problem of induction applies to all of our predictions based on past observations. Everything could be totally different in the future. Why think the laws of physics or observations will be similar tomorrow, but very different outside our observable universe? It seems like essentially the same problem to me.
As a result, I get the impression the hypothesis of an infinite universe is not falsibiable, such that it cannot meaningly be true or false.
Why then assume it’s finite rather than infinite or possibly either?
What if you’re in a short-lived simulation that started 1 second ago and will end in 1 second, and all of your memories are constructed? It’s also unfalsifiable that you aren’t. So, the common sense view is not meaningfully true or false, either.
My claim is that at least generally speaking, and I think actually always, theories that are under consideration only predict these relative differences and not the absolute amounts.
(...)
I have some ideas about why this is[1], but mainly I can’t think of any examples where this is not the case. If you can think of any then please tell me as that would at least partially invalidate this scale invariance thing (which would be good).
I think what matters here is less whether they predict absolute amounts, but which ones can be put on common scales. If everything could be put on the same common scale, then we would predict values relative to that common scale, and could treat the common scale like an absolute one. But scale invariance would still depend on you using that scale in a scale-invariant way with your moral theory.
I do doubt all theories can be put on one common scale together this way, but I suspect we can find common scales across some subsets of theories at a time. I think there usually is no foundational common scale between any pair of theories, but I’m open to the possibility in some cases, e.g. across approaches for counting conscious subsystems, causal vs evidential decision theory (MacAskill et al., 2019), in some pairs of person-affecting vs total utilitarian views (Riedener, 2019, also discussed in my section here). This is because the theories seem to recognize the same central and foundational reasons, but just find that they apply differently or in different numbers. You can still value those reasons identically across theories. So, it seems like they’re using the same scale (all else equal), but differently.
I’m not sure, though. And maybe there are multiple plausible common scales for a given set of theories, but this could mean two envelopes problem between those common scales, not between the specific theories themselves.
And I agree that there probably isn’t a shared foundational common scale across all theories of consciousness, welfare and moral weights (as I discuss here).
I think you would also say that theories don’t need to predict this overall scale parameter because we can always fix it based on our observations of absolute utility
Ya, that’s roughly my position, and more precisely that we can construct common scales based on our first-person observations of utility, although with the caveat that in fact these observations don’t uniquely determine the scale, so we still end up with multiple first-person observation-based common scales.
this is the bit of maths that I’m not clear on yet, but I do currently think this is not true (i.e. the scale parameter does matter still, especially when you have a prior reason to think there would be a difference between the theories).
Do you think we generally have the same problem for other phenomena, like how much water there is across theories of the nature of water or the strength of gravity as we moved from the Newtonian picture to general relativity? So, we shouldn’t treat theories of water as using a common scale, or theories of gravity as using a common scale? Again, maybe you end up with multiple common scales for water, and multiple for gravity, but the point is that we still can make some intertheoretic comparisons, even if vague/underdetermined, based on the observations the theories are meant to explain, rather than say nothing about hiw they relate.
In these cases, including consciousness, water and gravity, it seems like we first care about the observations, and then we theorize about them, or else we wouldn’t bother theorizing about them at all. So we do some (fairly) theory-neutral valuing.
When Hormel employees and other associated people gave $500k to an end-of-life care charity—a donation which is part of Lewis’s data—I don’t think this was a secret scheme to increase beef consumption.
Ya, I wouldn’t want to count that. I didn’t check what the data included.
People who work in agriculture aren’t some sort of evil caricature who only donate money to oppose animal protection; a lot of their donations are probably motivated by the same concerns that motivate everyone else.
I agree. I think if the money is coming through an interest/industry group or company, not just from an employee or farmer, then it’s probably usually lobbying for that interest/industry group or company or otherwise to promote the shared interests of that group. Contributions from individuals could be more motivated by political identity and other issues than just protecting or promoting whatever industry they work in.
Vegans could donate to an animal protection group, like HSUS, to lobby on their behalf. That should make it clear why they’re donating.
Gradations of moral weight
I doubt that was to support animal protection, though.
At least, it seems SBF lied or misled about Alameda having privileged access, because Alameda could borrow and go badly into the negative without posting adequate collateral and without liquidation, and this was something only Alameda was allowed to do, and was intentional by design. This seems like fraud, but doing this wouldn’t imply Alameda would borrow customers’ funds without consent and violate FTX’s terms of service, which seems like the bigger problem at the centre of the case.
Also, it seems their insurance fund numbers were fake and overinflated.
https://www.citationneeded.news/the-fraud-was-in-the-code/
I haven’t followed the case that closely and there’s a good chance I’m missing something, but it’s not obvious to me that they intended to allow Alameda to borrow customer funds that weren’t explicitly and consensually offered for borrowing on FTX (according to FTX’s own terms of service). However, I’m not sure what happened to allow Alameda to borrow such funds.
By design, only assets explicitly and consensually in a pool for lending (or identical assets with at most the same total per asset, e.g. separately USD, Bitcoin, etc.) should be up for being borrowed. You shouldn’t let customers borrow more than is being consensually offered by customers.[1] That would violate FTX’s terms of service. It also seems like an obvious thing to code.
But they didn’t ensure this, so what could have happened? Some possibilities:
They just assumed the amounts consensually available for lending would always match or exceed the amounts being borrowed without actually checking or ensuring this by design, separately by asset type (USD, Bitcoin, etc..). As long as more was never borrowed, they would not violate their terms of service. That’s a bad design, but plausibly not fraud. But then they allowed Alameda to borrow more, and Alameda borrowed so much it dipped into customer funds without consent. They could have done this without knowing it, so again plausibly not fraud. Or, maybe they did do it knowingly, so it would be fraud. But I’m not sure the evidence presented supports intent beyond a reasonable doubt.
They assumed they had enough net assets to cover customer assets, even if they had to sell different assets from what customers thought they held. Or, they might have assumed they’d be able to cover whatever users would want to withdraw at a time, even if it meant not actually holding at least the same assets in the same or greater amounts, e.g. the same amount of USD or more, the same amount of Bitcoin or more, and so on. In either case, if they didn’t care that they would not actually hold the same assets separately in the same or greater respective amounts (e.g. separately enough USD, enough Bitcoin, etc.) than what the customers retained rights to, this would be against FTX’s terms of service, and it would seem they never really intended to honor their own terms of service, which looks like fraud.
- ^
Which assets are actually borrowed and lent don’t need to match exactly. If A wants to lend Bitcoin and B wants to borrow USD, FTX could take A’s Bitcoin, sell it for USD and then lend the USD to B. That would be risky in case the Bitcoin price increased, but A and B could assume this risk or FTX could use an insurance fund or otherwise disperse the risk across funds opted into lending/borrowing, depending on the terms of service. This needn’t dip into other customer funds without consent. I don’t know if FTX did this.
This doesn’t necessarily totally eliminate all risk aversion, because the outcomes of actions can also be substantially correlated across correlated agents for various reasons, e.g. correlated agents will tend to be biased in the same directions, the difficulty of AI alignment is correlated across the multiverse, the probability of consciousness and moral weights of similar moral patients will be correlated across the multiverse, etc.. So, you could only apply the LLN or CLT after conditioning separately on the different possible values of such common factors to aggregate the conditional expected value across the multiverse, and then you recombine.
On cluelessness: if you have complex cluelessness as deep uncertainty about your expected value conditional on the possibility of acausal influence, then it seems likely you should still have complex cluelessness as deep uncertainty all-things-considered, because deep uncertainty will be infectious, at least if its range is higher (or incomparable) than that assuming acausal influence is impossible.
For example, suppose
10% to acausal influence, and expected value of some action conditional on it of −5*10^50 to 10^51, a range due to deep uncertainty.
90% to no acausal influence, and expected value of 10^20 conditional on it.
Then the unconditional expected effects are still roughly −5*10^49 to 10^50, assuming the obvious intertheoretic comparisons between causal and acausal decision theories from MacAskill et al., 2021, and so deeply uncertain. If you don’t make intertheoretic comparisons, then you could still be deeply uncertain, but it could depend on how exactly you treat normative uncertainty.
If you instead use precise probabilities even with acausal influence and the obvious intertheoretic comparisons, then it would be epistemically suspicious if the expected value conditional on acausal influence were ~0 and didn’t dominate the expected value without acausal influence. One little piece of evidence biasing you one way or another gets multiplied across the (possibly infinite) multiverse under acausal influence.[1]
- ^
Maybe the expected value is also infinite without acausal influence, too, but a reasonable approach to infinite aggregation would probably find acausal influence to dominate, anyway.
I would add that acausal influence is not only not Pascalian, but that it can make other things that may seem locally Pascalian or at least quite unlikely to make a positive local difference — like lottery tickets, voting and maybe an individual’s own x-risk reduction work — become reasonably likely to make a large difference across a multiverse, because of variants of the Law of Large Numbers or Central Limit Theorem. This can practically limit risk aversion. See Wilkinson, 2022 (EA Forum post).
(EDITED)
I didn’t refer to ignorance of the law. The point is that if you don’t know you took something without paying, it’s not theft. Theft requires intent.
https://www.law.cornell.edu/wex/theft
A jury can find someone guilty of theft (or fraud) without adequate evidence of intent, but that would be a misapplication of the law and arguably wrongful conviction.
If you find out later you took something without paying and make no attempt to return or repay or don’t intend to do so, it might be theft, because then you intend to keep what you’ve taken. I don’t know. If you’re unable to pay it back for whatever reason already at the time of finding out (because of losses or spending), I don’t know how that would be treated, but probably less harshly, and maybe just under civil law, with a debt repayment plan or forfeiture of future assets, not criminal conviction.
Either way, fraud definitely requires intent.
Ah, I misunderstood.
Still, I think they did violate their own terms of service and effectively misappropriated/misused customers’ funds, whether or not it was intentional/fraud/criminal.
If I tell you I’ll hold your assets and won’t loan them out or trade with them, and don’t take reasonable steps to ensure that and instead actually accidentally loan them out and trade with them, then I’ve probably done something wrong.
It may not be criminal without intent, though, and I just owe you your assets (or equivalent value) back. Accidentally walking out of a store with an item you didn’t pay for isn’t a crime, although you’re still liable for returning it/repayment.
I just read the Wikipedia page on the case and didn’t see compelling evidence of intent, at least not beyond a reasonable doubt https://en.m.wikipedia.org/wiki/United_States_v._Bankman-Fried
Also Googling “sbf prove intent” (without quotes) didn’t turn up anything very compelling, at least for the first handful of results, in my view.
There’s a lot here, so I’ll respond to what seems to be most cruxy to me.
Another way to get this intuition is to imagine an unfeeling robot that derives the concept of utility from some combination of interviewing moral patients and constructing a first principles theory[2]. It could even get the correct theory, and derive that e.g. breaking your arm is 10 times as bad as stubbing your toe. It would still be in the dark about how bad these things are in absolute terms though.
I agree with this, but I don’t think this is our epistemic position, because we can understand all value relative to our own experiences. (See also a thread about an unfeeling moral agent here.)
My claim is that the epistemic position of all the different theories of welfare are effectively that of this robot. And as a result of this, observing any absolute amount of welfare (utility) under theory A shouldn’t update you as to what the amount would be under theory B, because both theories were consistent with any absolute amount of welfare to begin with. In fact they were “maximally uncertain” about the absolute amount, no amount should be any more or less of a surprise under either theory.
I agree that directly observing the value of a toe stub, say, under hedonism might not tell you much or anything about its absolute value under non-hedonistic theories of welfare.[1]
However, I think we can say more under variants of closer precise theories. I think you can fix the badness of a specific toe stub across many precise theories. But then also separately fix the badness of a papercut and many other things under the same theories. This is because some theories are meant to explain the same things, and it’s those things to which we’re assigning value, not directly to the theories themselves. See this section of my post. And those things in practice are human welfare (or yours specifically), and so we can just take the (accessed) human-relative stances.
You illustrate with neuron count theories, and I would in fact say we should fix human welfare across those theories (under hedonism, say, and perhaps separately for different reference point welfare states), so evidence about absolute value under one hedonistic neuron count theory would be evidence about absolute value under other hedonistic theories.
I suspect conscious subsystems don’t necessarily generate a two envelopes problem; you just need to calculate the expected number of subsystems and their expected aggregate welfare relative to accessed human welfare. But it might depend on which versions of conscious subsystems we’re considering.
For predictions of chicken sentience, I’d say to take expectations relative to human welfare (separately with different reference point welfare states).
- ^
I’d add a caveat that evidence about relative value under one theory can be evidence under another. If you find out that a toe stub is less bad than expected relative to other things under hedonism, then the same evidence would typically support that it’s less bad for desires and belief-like preferences than you expected relative to the same other things, too.
- ^
I agree it is unprincipled, and I strongly endorse expected value maximisation in princple, but maybe using the geometric mean is still a good method in practice?
I would want to know more about what our actual targets should plausibly be before making any such claim. I’m not sure we can infer much from your examples. Maybe an analogy is that we’re aggregating predictions of different perspectives, though?
I think the welfare range outputted by any given model should always be positive.
Other animals could fail to be conscious, and so have welfare ranges of 0.
The funds Alameda otherwise borrowed from FTX came entirely from the margin lending program, which was permissible under the Terms Of Service.
Source? My impression is that FTX and Alameda used the same bank account(s) and they (intentionally or unintentionally) let Alameda use FTX customer funds without consent, beyond just margin lending and against the Terms of Service.
Is not enough organizations really the problem? For technical AI safety research, at least, I hear research management capacity is a bottleneck. A new technical AI safety org would compete with the others over the same potential research managers.
Another issue could be that few interventions seem net positive (maybe things have changed since that comment 3 years ago).