I’m working on impact markets – markets to trade nonexcludable goods. (My profile.)
I have a conversation menu and a Calendly for you to pick from!
If you’re also interested in less directly optimific things – such as climbing around and on top of boulders or amateurish musings on psychology – then you may enjoy some of the posts I don’t cross-post from my blog, Impartial Priorities.
Pronouns: Ideally she or they. I also still go by Denis and Telofy in various venues.
Even scandal-prone individuals can’t survive in a vacuum. (You may be thinking of sandals, not scandals?)
We have sympathies towards both movements, and consider ourselves to take the middle path. We race forward and accelerate as quickly as possible while mentioning safety.
Mentioning safety is a waste of resources that you could direct toward attaching propulsion to asteroids to get them here faster.
In fact, asteroids will inevitably hit earth earlier or later, and if they kill humanity, clearly they are superior to humanity. The true masters of our future lightcone are the asteroids. That which can be destroyed by asteroids ought to be destroyed by asteroids.
True progress is in speeding the inevitable. Resistance is futile.
This post is also a great info hazard. It risks causing impostors with sub-146 IQs (2009 LW survey) to feel adequate!
That’s a good point. Time discounting (a “time premium” for the past) has also made me very concerned about the welfare of Boltzmann brains in the first moments after the Big Bang. It might seem intractable to still improve their welfare, but if there is any nonzero chance, the EV will be overwhelming!
But I also see the value in longtermism because if these Boltzmann brains had positive welfare, it’ll be even more phenomenally positive from the vantage points of our descendants millions of years from now!
There is a Swiss canton Appenzell Innerrhoden (AI). Maybe we can hide there and trick the AI into thinking it already invaded it?
Scandals don’t just happen in the vacuum. You need to create the right conditions for them. So I suggest:
We spread concern about the riskiness of all altruistic action so that conscientious people (who are often not sufficiently scandal-prone) self-select out of powerful positions and open them up to people with more scandal potential.
We encourage more scathing ad-hom attacks on leadership so that those who take any criticism to heart self-select out of leadership roles.
We make these positions more attractive to scandal-prone people by abandoning cost-effectiveness analyses and instead base strategy and grantmaking on vibes and relationships.
We further improve the cushiness of these positions by centralizing power and funding around them to thwart criticism and prevent Hayekian diversity and experimentation.
We build stronger relationships with powerful, unscrupulous people and companies by, e.g., helping them with their hiring.
We emphasize in-person networking and move the most valuable networks to some of the most expensive spots in the world. That way access to the network comes with even greater dependency on centralized funding, making it easier to control.
[Meta: I’m not claiming anyone is doing these things on purpose! It would be nice, though, if more people were trying to counter these risk factors for scandals and generally bad epistemics.]
Great summary!
You probably base “Even though this use of funds was unintentional and sounds extremely sketchy, FTX’s general counsel testified that FTX’s terms of service did not prohibit it” on:
The government didn’t want to focus you on that. Why? Again, the only witness who said he had read the terms of service was Can Sun, the general counsel who had helped to draft it. Even though he was very careful in what he told you, he admitted that nowhere do the terms of service contain language that prevents FTX from loaning customer fiat deposits to Alameda or anyone else.
Can Sun didn’t think so. (Unless I misunderstand something.) He said that there was the margin lending program that did allow that but that had a few hundred million USD in it, so by far not enough to explain Alameda’s borrowing. He didn’t think that FTX or Alameda could’ve borrowed from capital outside the margin lending program because it was owned by the customers.
So I think what the defense lawyer is trying to do here is to say that the ToS did not explicitly prohibit such borrowing, but he omits that the borrowing is still implicitly prohibited just like it is generally prohibited to borrow other people’s fund without their permission.
Yep, makes a lot of sense!
Yep, failing fast is nice! So you were just skeptical on priors because any one new thing is unlikely to succeed?
Yep, that makes a lot of sense. I’ve done donation forwarding for < 10 projects once, and it was already quite time-consuming!
I’ve (so far) read the first post and love it! But when I was working full-time on trying to improve grantmaking in EA (with GiveWiki, aggregating the wisdom of donors), you mostly advised me against it. (And I am actually mostly back to ETG now.) Was that because you weren’t convinced by GiveWiki’s approach to decentralizing grantmaking or because you saw little hope of it succeeding? Or something else? (I mean, please answer from your current perspective; no need to try to remember last summer.)
Oh, great! I interpreted “This is an offer to donate this amount to the project on the condition that it eventually becomes active” to mean that the project might not become active for any number of reasons, only one of them being the funding goal.
Oh, brilliant! USDC would also be my top choice. But I’m basically paying into a DAF, and so can’t get a refund if this project doesn’t succeed, right? That would have a high cost in option value since I don’t know whether my second-best donation opportunity will be on Manifund. Is there a way to donate to Rethink Priorities or the Center on Long-Term Risk through Manifund? That would lower-bound the cost in option value.
From what I’ve learned about Shapley values so far, this seems to mirror my takeaway.
Nice! To be sure, I want to put an emphasis on any kind of attribution being an unnecessary step in most cases rather than on the infeasibility of computing it.
There is complex cluelessness, nonlinearity from perturbations at perhaps even the molecular level, and a lot of moral uncertainty (because even though I think that evidential cooperation in large worlds can perhaps guide us toward solving ethics, that’ll take enormous research efforts to actually make progress on), so infeasibility is already the bread and butter of EA. In the end we’ll find a way to 80⁄20 it (or maybe −80/20 it, as you point out, and we’ll never know) to not end up paralyzed. I’ve many times just run through mental “simulations” of what I think would’ve happened if any subset of people on my team had not been around, so this 80/20ing is also possible for Shapley values.
If you do retroactive public goods funding, it’s important that the collaborators can, up front, trust that the rewards they’ll receive will be allocated justly, so being able to pay them out in proportion to the Shapley value would be great. But as altruists, we’re only concerned with rewards to the point where we don’t have to worry about our own finances anymore. What we really care about is the impact, and for that it’s not relevant to calculate any attribution.
I do not understand this point but would like to (since the stance I developed in the original post went more in the direction of “EAs are too individualist”).
I might be typical-minding EAs here (based on me and my friends) but my impression is that a lot of EAs are from lefty circles that are very optimistic about the ability of a whole civilization to cooperate and maximize some sort of well-being average. We’ve then just turned to neglectedness as our coordination mechanism rather than long, well-structured meetings, consensus voting, living together and other such classic coordination tools. In theory (or with flexible resources, dominant assurance contracts, and impact markets) that should work fine. Resources pour into campaigns that are deemed relatively neglected until they are not, at which point the resources can go to the new most neglected thing. Eventually nothing will be neglected anymore.
So it seems to me that the spirit is the same one of cooperativeness, community, and collective action. Just the tool we use to coordinate is a new one.
But some 99.9% (total guess) of the population are more individualist than that (well, I’ve only ever lived in WEIRD cultures, so I’m in an obvious bubble). They don’t think in terms of civilizations thriving or succumbing to infighting but in terms of the standing of their family in society or even just their own. (I’m excluding people in poverty here – almost anyone, including most altruists, well behave selfishly when they are in dire straits.)
Shapley values are useful for startups or similar enterprises that have a set goal that everyone works toward. The degree to which they work toward it is a fixed attribute of the collaborator. The core is more about trying to find an attribution split that sets just the right incentives to maximize the number of people who are interested in collaborating in the first place. (I think I’m getting this backwards, but something of this sort. It’s been too long since I research these things.)
If someone is very community- and collective-action-minded, they’ll have the tacit assumption that everyone is working towards the good of the whole community, and they’re just wondering how they can best contribute to that. That’s how I see most EAs.
If someone is very individualistic, they’ll want to travel 10 countries, have 2 kids, drive a car that can accelerate real fast, and get their brain frozen when they die. They’ll have no tacit assumptions about any kind of greater community or their civilization and never think about collective action. But if they did, their question would be what’s in it for them, and if there is something in it for them, if they can conspire with a smaller set of collaborators to get more of it. They’ll turn to cooperative game theory, crunch the numbers, and then pick out just the right co-conspiritors to form a subcoalition.
So that’s the intuition behind that overly terse remark in my last message. ^.^
Off topic: I have a badly structured, hastily written post where I argue that it’s not optimal for EAs to focus maximally on the one thing where they can contribute most (AI safety, animal rights, etc.) and neglect everything else but that it’s probably better to cooperate with all other efforts in their immediate environment that they endorse to at least the extent to which the median person in the environment cooperates with them. Or else we’re all (slightly) sabotaging each other all the time and get less change in aggregate. I feel like mainstream altruists (a small percentage of the population) do this better than some EAs, and it seems conceptually similar to individualism.
Very glad to read that, thank you for deciding to add that piece to your comment :)!
Awww! :-D
Great work; I hope you’ll succeed with the fundraiser! Do you have blockchain-based donation options like, e.g., Rethink Priorities offers? Ideally one of the major Ethereum layer 2s or Solana? Ty!
Shapley values are a great tool for divvying up attribution in a way that feels intuitively just, but I think for prioritization they are usually an unnecessary complication. In most cases you can only guess what they might be because you can’t mentally simulate the counterfactual worlds reliably, and your set of collaborators contains billions of potentially relevant actors. But as EAs we can “just” choose whatever action will bring about the world history with the greatest value regardless of any impact attribution to ourselves or anyone.
I like the Shapley value and think it would make similar recommendations, but it adds another layer of infeasability (and arbitrariness) on top of an already infeasably complex optimization problem without adding any value.
Then again many of us are strongly motivated by “number go up,” so Shapley values are probably helpful for self-motivation. :-3
(I think if EAs were more individualist, “the core” from cooperative game theory would be more popular than the Shapley value.)
Oh, and we get so caught up in the object-level here that we tend to fail to give praise for great posts: Great work writing this up! When I saw it, it reminded me of Brian Tomasik’s important article on the same topic, and sure enough, you linked it right before the intro! I’m always delighted when someone does their research so well that whatever random spontaneous associations I (as a random reader) have are already cited in the article!
I use “impact” to mean “net impact,” basically.
An output could be a piece of forest that is protected from logging. An outcome is some amount of CO2 converted into O2 that wouldn’t otherwise. But also a different piece of forest getting logged that wouldn’t otherwise. And a bunch of r-strategist animals dying of parasites, starvation, and predation who would otherwise not have been born. Some impact on the workers who now have to travel further to log trees. And much more.
The attempt to trade off all of these effects (perhaps using an open source repository of composeable probabilistic models like Squiggle) is what results in an impact estimate.
Frankie made a nice explainer video for that!
What a market does, idealizing egregiously, is that people with special knowledge or insight invest into things early: Thus less informed people (some of whom have more capital) can watch the valuations, and invest into projects with high and increasing valuations or some other valuation-based marker of quality. A process of price discovery.
AngelList, for example, facilitates that. They have a no-action letter from the SEC (and the startups on AngelList have at least a registration D I imagine), so they didn’t have to register as a broker-dealer to be allowed to match startups to investors. I think they have some funds that are led by seasoned investors, and then the newbie investors can follow the seasoned ones by investing into their funds. Or some mechanism of that sort.
We’re probably not getting a no-action letter, and we don’t have the money yet to start the legal process to get our impact credits registered with the CFTC. So instead we recognized that in the above example investors are treating valuations basically like scores. So we’re just using scores for now. (Some rich people say money is just for keeping score. We’re not rich, so we use scores directly.)
The big advantage of actual scores (rather than using monetary valuations like scores) is that it’s legally easy. The disadvantage is that we can’t pitch GiveWiki to profit-oriented investors.
So unlike AngelList, we’re not giving profit-oriented investors the ability to follow more knowledgeable profit-oriented investors, but we’re allowing donors/grantmakers to follow more knowledgeable donors/grantmakers. (One day, with the blessing of the CFTC, we can hopefully lift that limitation.)
We usually frame this as a process of three phases:
Implement the equivalent of price discovery with a score. (The current state of GiveWiki.)
Pay out a play money currency according to the score.
Turn the play money currency into a real impact credit that can be sold for dollars (with the blessing of the CFTC).
Thanks for putting the Exotic Tofu Project on my screen! I also like all the others.
We (me and my cofounder) run yet another “impact certificates” project. We started out with straightforward impact certificates, but the legal hurdles for us and for the certificate issuers turned out too high and possibly (for us) insurmountable, at least in the US.
We instead turned to the system that works for carbon credits. These are not so much traded on the level of the certificate or impact claim but instead there are validators that confirm that the impact has happened according to certain standards and then pay out the impact credits (or carbon credits) associated with that standard.
That system seems more promising to us as it has all the advantages of impact certificate markets but also the advantage that one party (e.g., us) can do the legal battle in the US once for this impact credit (and can even rely on the precedent of carbon credits), and thereby pave the ground for all the other market participants that come after and don’t have to worry about the legalities anymore. There are already a number non-EA organizations that are working toward a similar vision.
Even outside such restrictive jurisdictions as the US, this system has the advantage that it allows for deeper liquidity on the impact credit markets (compared to the auctions for individual impact certificates). But the US is an important market for EA and AI safety, so we couldn’t just ignore it even if it hadn’t been for this added benefit.
We’ve started bootstrapping this system with GiveWiki in January of last year. But over the course of the year we’ve found it very hard to find anyone who wanted to use the system as a donor/grantmaker. Most of the grantmakers we were in touch with had lost their funding in Nov. 2022; others wanted to wait until the system is mature; and many smaller donors had no trouble finding great funding gaps without our help.
We will keep the platform running, but we’ll probably have wait for the next phase of funding overhang when there are more grantmakers and they actually have trouble finding their funding gaps.
(H/t to Dony for linking this thread to me!)
Only half a person per sandal I think!