# Impact Prizes as an alternative to Certificates of Impact

Epistemic state: quite uncertain

TLDR Example

An EA donor puts up a $50k prize for dis­tri­bu­tion in 2022. In 2022, sev­eral pro­jects that have started since 2019 ap­ply. Their net EA im­pacts are es­ti­mated, and these es­ti­mates (vs. the to­tal value es­ti­mate of all sub­mis­sions) are even­tu­ally used to give them cor­re­spond­ing pro­por­tional amounts of the ​$50k.

Back in 2019, sev­eral pro­jects sell “rights” to their prize, and these get sold around. It’s ex­pected that $1M in es­ti­mated to­tal value will ap­ply, so the mar­ket value of the claim of ev­ery$10 of es­ti­mated im­pact is ​$0.50. One pro­ject sets up an es­ti­ma­tion ser­vice where they pub­li­cly es­ti­mate the even­tual eval­u­a­tion of ev­ery pro­ject, to help make the mar­ket more effi­cient, with the goal of them­selves get­ting part of the prize. # Im­pact Prizes I re­ally like the goal of Cer­tifi­cates of Im­pact, but per­son­ally find them sub­op­ti­mal in prac­tice. I think Im­pact Prizes pre­sent an in­ter­est­ing al­ter­na­tive. It’s also pos­si­ble Cer­tifi­cates of Im­pact could be used with Im­pact Prizes to gain the ad­van­tages of both down the road. The most ba­sic defi­ni­tion of Im­pact Prizes is some­thing like, “Dec­la­ra­tions and fulfill­ment of prizes aimed at pub­lic benefit.” Such a defi­ni­tion would ap­ply to many ex­ist­ing char­ity prizes. They’ve re­cently been used with suc­cess on LessWrong in the in­ter­ac­tions of the AI Align­ment Prize. I think these get more in­ter­est­ing with some ex­tra less-ex­plored fea­tures. # Pos­si­ble Features ## Tokenization If one group has an ex­pec­ta­tion of mak­ing$2,000 of prize money from a fu­ture Im­pact Prize, they should be able to sell that claim to a 3rd party. This should be re­ally sim­ple, and that 3rd party should be able to eas­ily re­sell it.

We can call “parts” of this claim “to­kens.”[1]

Sup­pose there’s a sin­gle $10,000 prize, to be awarded in 2020, and a spe­cific group has a 20% chance of win­ning that prize. Then that group has an ex­pected value of$2,000 of prize money. That group cre­ates 100 to­kens rep­re­sent­ing 100% of the claim to that prize. They sell 50 to­kens for $1,000. Later, they do win the$10,000 prize. How­ever, be­cause they only own 50% of the to­kens, they only get $5,000. The other$5,000 goes to the pre­vi­ous share pur­chaser.

## Pro­por­tional Prizes

If only one prize was given, then share pur­chasers would only be in­ter­ested in pro­jects that have chances of be­ing the top sub­mit­ted pro­ject. This seems sub­op­ti­mal.

Imag­ine in­stead that once the prize eval­u­a­tion ses­sion be­gins, ev­ery sin­gle pro­ject is nu­mer­i­cally eval­u­ated for im­pact. Then each one gets a re­ward in pro­por­tion to the im­pact that the pro­ject was rated as hav­ing.

Example

In­stead of pre­sent­ing $10,000 for “all pro­jects”, it may make more sense to di­vide this pool to en­courage a few ar­eas. For in­stance, it may be com­mon prac­tice to ear­mark 20% for sup­port and eval­u­a­tion. The idea of this would be to en­courage some peo­ple to “do good” by do­ing things that would help the prize. Some things to help could in­clude set­ting up a Pre­dic­tion Tour­na­ment to es­tab­lish com­mon knowl­edge of prize ex­pec­ta­tions, or web tools to make pur­chas­ing and sel­l­ing more ac­cessible. Be­cause some prizes would go to­wards efforts to help the prize sys­tem, this could lead to a minor prize-value-pro­mo­tion econ­omy. As stated above, some peo­ple could set up pre­dic­tion sys­tems, and other peo­ple could make pre­dic­tions of prize out­comes on them. Others may act as po­lice, de­tect­ing and re­port­ing on bad ac­tors. Users could in­ves­ti­gate not only bad be­hav­ior for prizes but also good be­hav­ior. In many tour­na­ment sys­tems groups be­come quite com­pet­i­tive and in­di­rect ser­vices like ed­u­ca­tion or col­lab­o­ra­tion can be un­der­val­ued. If there could be a lot of value in some of these ar­eas, then it should be ev­i­dently valuable when peo­ple point that out. The pres­ence of some mo­ti­vated ac­tors ac­tively in­ves­ti­gat­ing and pro­mot­ing over­looked ac­tivity would hope­fully lead to more of that ac­tivity. ## Deal­ing With Mul­ti­ple Prizes One dis­ad­van­tage of Im­pact Prizes, com­pared to Cer­tifi­cates of Im­pact, is that they could get com­pli­cated when there are sev­eral differ­ent prizes by differ­ent donors. A naive im­ple­men­ta­tion of Im­pact Prizes could de­mand a unique to­ken mint­ing per pro­ject per prize, which would make things very messy. Any given pro­ject may have dozens of to­kens to worry about and trade, and many ex­changes may be be­tween clusters of to­kens at a time. A sim­pler setup would look some­thing more like Cer­tifi­cates of Im­pact. Only one to­ken is made per pro­ject, but that to­ken can be used for all Im­pact Prizes. Per­haps there would be a few com­mon stan­dards of to­kens if Im­pact Prizes with differ­ent pa­ram­e­ters. Example Say in 50% of to­kens of a pro­ject are sold for$2,000, and later that pro­ject wins ​$5,000 from an Im­pact Prize. With the case of shared to­kens, this to­ken-holder could ex­pect to pos­si­bly win even more money later on from other Im­pact Prizes as well. A re­lated tech­nique could just be that fu­ture donors of­ten donate to ex­ist­ing Im­pact Prizes in­stead of cre­at­ing new ones. This would mean that Im­pact Prizes would be lower-bound (the ex­ist­ing cash pool) but not sim­ply up­per-bound (it’s not clear how much more money will be added). It’s pos­si­ble that Cer­tifi­cates of Im­pact could work effec­tively as one of these to­ken stan­dards. ## Risks and Insurance One bias that these sys­tems may cre­ate is that ac­tors may be mo­ti­vated to max­i­mize up­side risk, but may not care about min­i­miz­ing down­side risks. As long as Im­pact Prizes can only give out money (rather than de­mand money), than the low­est one should ex­pect from a highly risky out­come is zero. One way to get around this would be with for­mal in­surance sys­tems. All pro­jects that cre­ate to­kens could be re­quired to pur­chase in­surance upon pro­ject for­ma­tion. When it comes to eval­u­a­tion time, the Im­pact Prize could re­quest that the in­surer pay­out for any pro­jects that are eval­u­ated to be net-nega­tive. It’s not ob­vi­ous how they should strike a bal­ance be­tween charg­ing for the en­tire cost or for the pro­por­tional cost. In the case of mul­ti­ple prizes, per­haps dam­ages should be han­dled out­side the prize sys­tem. # Challenges ## Le­gal Implications I think that to­k­enized Im­pact Prize sys­tems in par­tic­u­lar may be quite legally com­pli­cated. Cor­po­rate stock sys­tems come with lots of rules, in part be­cause there’s been an es­tab­lished record of peo­ple ma­nipu­lat­ing them in shady ways for per­sonal gain. If so­phis­ti­cated fi­nan­cial in­stru­ments like short­ing be­came pos­si­ble, challenges could arise that would nor­mally be ad­dressed by cor­po­rate law. For in­stance, in­sider trad­ing is reg­u­lated, in part, to pre­vent cor­po­rate em­ploy­ees from tak­ing rel­a­tively sim­ple ac­tions to short their own stocks and then pur­posely cause bad things to hap­pen. If an Im­pact Prize sys­tem was es­tab­lished, it would have to ei­ther work within the cur­rent le­gal in­fras­truc­ture, like stock, or out­side of it. Both come with dis­ad­van­tages. It’s pos­si­ble that only ac­cred­ited in­vestors would be able to pur­chase Im­pact Prize to­kens, though this may be a fine first step. Th­ese con­sid­er­a­tions would re­ally need to be eval­u­ated by an ac­tual at­tor­ney. I sug­gest any­one con­sid­er­ing do­ing this at scale hire an at­tor­ney first. That said, similar prob­lems would come up with Cer­tifi­cates of Im­pact if they was done to a similar scale. They also may be ad­e­quately ad­dressed by ex­ist­ing cryp­tocur­rency To­ken pro­jects. ## Eval­u­a­tion Costs The fi­nal prize eval­u­a­tions could be quite costly to pro­duce. A few meth­ods above could help, but sig­nifi­cant costs would re­main. I feel like there’s prob­a­bly clever ways of think­ing about this to in­cen­tivize ev­ery­one to max­i­mize to­tal value. For ex­am­ple, per­haps the eval­u­a­tion cost comes out of each pro­ject’s value, in­cen­tiviz­ing pro­jects not to ap­ply if that to­tal would be be­low zero, and in­cen­tiviz­ing them to make the eval­u­a­tion easy. ## Open­ness Costs My cur­rent model is that there a lot of in­cen­tives to not make most kinds of eval­u­a­tions pub­lic. Per­haps the best com­par­i­son is prizes that are given out based on rubrics, though here most of the re­sults of most of those rubrics are not made pub­lic. The Im­pact Prize eval­u­a­tions may be con­tro­ver­sial and are likely to be at least some­what mi­s­un­der­stood. Public eval­u­a­tions may re­ally re­quire a com­mu­nity that is quite epistem­i­cally ma­ture. Con­tro­versy could cre­ate li­a­bil­ity. If a Twit­ter war or similar gets started, it’s pos­si­ble there could be enough anger for any prize to be can­celed, or at least to stop fu­ture prizes. ## Technology For such a sys­tem to work well, a de­cent of work may be needed both on tech­ni­cal tool­ing and in im­ple­men­ta­tion cre­ativity. ## Cul­tural Risks If Im­pact Prizes took off, I could imag­ine some ac­tors draw­ing into the ecosys­tem who only mo­ti­vated by mak­ing prof­its. The to­ken sys­tem may be looked at as a form of gam­bling (some­what similar to the stock mar­ket) and may lead to some gam­bling ten­den­cies. I think there may be some sig­nifi­cant down­sides here, but es­ti­mate that the up­sides will be higher. (but this should be tested!) It could ob­vi­ously be partly com­bat­ted us­ing some of the tech­niques men­tioned above. # Com­par­i­son to Cer­tifi­cates of Impact A Philo­soph­i­cal Comparison Per­haps the main philo­soph­i­cal differ­ence be­tween Im­pact Prize To­kens and Cer­tifi­cates of Im­pact is that Cer­tifi­cates of Im­pact, ac­cord­ing to Paul Chris­ti­ano, are sup­posed to rep­re­sent causal re­spon­si­bil­ity. As he writes, Allo­cat­ing cer­tifi­cates re­quires ex­plicit and trans­par­ent al­lo­ca­tion of causal re­spon­si­bil­ity, both within teams and be­tween teams and donors. I per­son­ally find the causal re­spon­si­bil­ity bit un­in­tu­itive, and don’t ex­pect a much larger com­mu­nity (es­pe­cially out­side the EA sphere) to ac­cept it. Im­pact Prize to­kens would be de­cou­pled from this idea. A Ra­tio Comparison I be­lieve Cer­tifi­cates of Im­pact are sup­posed to be priced at their ex­pected rates of im­pact, so$1 worth of cer­tifi­cates means $1 of coun­ter­fac­tual im­pact. I think this will prove some­what in­flex­ible. I ques­tion the mar­ket vi­a­bil­ity of a$1 to $1 peg. If the de­mand is much less than what is nec­es­sary to cre­ate a$1 to $1 peg, then I would ex­pect this to re­sult in an illiquid mar­ket. That said, of course a$1 to $1 would make things very sim­ple if it works. A vari­able ra­tio could be fairly con­fus­ing and could re­quire so­phis­ti­cated pur­chasers (well, ones that could do two mul­ti­pli­ca­tions.) Complexity Per­haps the main challenge to Im­pact Prizes as I dis­cuss them is their ad­di­tional com­plex­ity, com­pared to Cer­tifi­cates of Im­pact. It may re­quire set­ting up a prize in ad­vance, and then when that hap­pens ei­ther do­ing a bunch of eval­u­a­tions or figur­ing out clever ways of de­creas­ing that bur­den. # Fur­ther Work The fea­ture space is quite large. I’d like it to be larger. I’d be cu­ri­ous to hear other ideas for fea­tures or to mod­ify the above fea­tures. One area I’m par­tic­u­larly in­ter­ested in is how best to struc­ture the open­ness of eval­u­a­tions. I think that the “Open­ness Cost” is very sig­nifi­cant, and it would be nice to be able to re­duce it while still main­tain­ing much of the benefits of the eval­u­a­tions. [1] There’s much about the Blockchain world I don’t like, but they have used to­kens ex­ten­sively for this spe­cific pur­pose. I don’t want to use the word “shares” be­cause these parts will not have any vot­ing rights, and legally there are other im­por­tant dis­tinc­tions. Spe­cial thanks to Ryan Carey for dis­cussing the con­cept, pro­vid­ing writ­ing feed­back, and sug­gest­ing I use the name “Im­pact Prizes” in­stead of some­thing more ob­tuse. • Great post; I had been think­ing about writ­ing some­thing very similar. In many ways I think you have ac­tu­ally un­der­stated the po­ten­tial of the idea. Ad­di­tion­ally I think it ad­dresses some of the con­cerns Owen raised last time. Eval­u­a­tion Costs The fi­nal prize eval­u­a­tions could be quite costly to pro­duce. I ac­tu­ally think the fi­nal eval­u­a­tions might be cheaper than the sta­tus quo. At the mo­ment OpenPhil (or who­ever) has to do two things: 1) Judge how good an out­come is. 2) Judge how likely differ­ent out­comes are. With this plan, 2) has been (par­tially) out­sourced to the mar­ket, leav­ing them with just 1). Cul­tural Risks If Im­pact Prizes took off, I could imag­ine some ac­tors draw­ing into the ecosys­tem who only mo­ti­vated by mak­ing prof­its. This is not a bug, this is a fea­ture! There is a very large pool of peo­ple will­ing to pre­dict ar­bi­trary out­comes in re­turn for money, that we have thus far only very in­di­rectly been tap­ping into. In gen­eral bring­ing in more traders im­proves the effi­ciency of a mar­ket. Even if you add noisy traders, their pres­ence im­proves the in­cen­tives for ‘smart money’ to par­ti­ci­pate. I think it’s un­likely we’d reach the scale re­quired for ac­tual hedge funds to get in­volved, but I do think it’s plau­si­ble we could get a lot of hedge fund guys par­ti­ci­pat­ing in their spare time. Le­gal Implications In terms of le­gal sta­tus, one op­tion I’ve been think­ing about would be copy­ing Pre­dic­tIt. If we have to pay taxes ev­ery time a cer­tifi­cate is trans­ferred, the trans­ac­tion costs will be pro­hibitive. I am quite wor­ried it will be hard to make this work within US law un­for­tu­nately, which is not very friendly to this sort of ex­per­i­men­ta­tion. At the same time, given the SEC’s at­ti­tude to­wards non-com­pli­ant se­cu­rity is­suance, I would not want to op­er­ate out­side it! Quick other thoughts One is­sue with the idea is it is hard for OpenPhil to add more promised fund­ing later, be­cause the ini­tial in­vest­ment will already have been com­mit­ted at some fixed level. e.g. If OpenPhil ini­tially promise$10m, and then later bump it to $20m, pro­jects that have already sold their to­kens can­not ex­pand to take ad­van­tage of this in­crease, so it is effec­tively pure wind­fall with no in­cen­tive effect. A pos­si­ble solu­tion would be co­horts; we promise$10m in 2022 for pro­jects started in 2019, and then later add an­other $12m, paid in 2023, for 2020 pro­jects. • Thanks! Some quick re­sponses to parts. Le­gal Implications I think copy­ing Pre­dic­tIt would be pretty messy. I’m cu­ri­ous about the fea­si­bil­ity of us­ing crypto, similar to Crip­toKit­ties. It seems like sev­eral crypto groups did es­sen­tially a similar thing, and per­haps in the medium-term, they will be rec­og­nized as be­ing le­gal in the United States. “If OpenPhil ini­tially promise$10m, and then later bump it to $20m, pro­jects that have already sold their to­kens can­not ex­pand to take ad­van­tage of this in­crease, so it is effec­tively pure wind­fall with no in­cen­tive effect. ” I agree co­horts are one solu­tion. Another is­sue though, is that if the au­di­ence thought there was a de­cent chance of a bump, then the (“40% chance there would be a$10mil bump”) would be fac­tored into the price.

• Is there a post­mortem some­where on Cer­tifi­cates of Im­pact & challenges they faced when im­ple­ment­ing?

• I think I might have been the sec­ond largest pur­chaser of the cer­tifi­cates. My ex­pe­rience was that we didn’t at­tract the re­ally high qual­ity pro­jects I’d want, and those we did see had very high reser­va­tion prices from the sel­l­ers, per­haps due to the en­dow­ment effect. I sus­pect sel­l­ers might say that they didn’t see enough buy­ers. Pos­si­bly we just had a chicken-and-egg prob­lem, com­bined with ev­ery­one in­volved be­ing kind of busy.

• Not that I know of. Paul has a lot of stuff go­ing on, for one thing. :)

I think some peo­ple are still ex­cited about Cer­tifi­cates of Im­pact though.

• My im­pres­sion is that no­body has made it their job (and spent at least a month and prefer­ably a year or two) to make Cer­tifi­cates of Im­pact work. i.e. money is real be­cause hu­mans have agreed to be­lieve it’s real, and be­cause there’s a lot of good in­fras­truc­ture that helps it work. If Cer­tifi­cates of Im­pact (or Prizes) are to be real some­one needs to ac­tu­ally build a thing and hype it con­tin­u­ously. So far it doesn’t feel like it’s been tried.

• I’d gen­er­ally agree with that.

Hon­estly, the tech­ni­cal in­fras­truc­ture for Cer­tifi­cates of Im­pact would be very similar to that for Im­pact Prizes as I dis­cuss them above. I think both would be re­ally in­ter­est­ing to test at larger scales.

Im­pact Prizes may need less hype though, but may be more difficult to scale.

• Hmm, I think they need about the same amount of hype. I do think Im­pact Prizes aren’t any harder to scale – Cer­tifi­cates of Im­pact already de­pend on some­thing like Im­pact Prizes even­tu­ally ex­ist­ing.

Ac­tu­ally, I think of Im­pact Prizes as “a pre­cise for­mu­la­tion of how one might scale the hype and money nec­es­sary for Cer­tifi­cates to work.”

• That makes sense to me. When I said “harder to scale”, I mean harder to “put a bunch on top of each other”. In some ways it’s not as el­e­gant.

Agreed that Im­pact Prizes are one ways that Cer­tifi­cates of Im­pact could work long-term. Like, one group places \$100k of Im­pact Prizes for 2030, where it will only be used to pur­chase Cer­tifi­cates of Im­pact.

• This seems very re­lated to so­cial im­pact bonds: “So­cial Im­pact Bonds are a type of bond, but not the most com­mon type. While they op­er­ate over a fixed pe­riod of time, they do not offer a fixed rate of re­turn. Re­pay­ment to in­vestors is con­tin­gent upon speci­fied so­cial out­comes be­ing achieved.”

• So the prize money gets paid out in 2022, in the tl;dr ex­am­ple? (I’m a lit­tle un­clear about that from my quick read.)

This means that the Im­pact Prize wouldn’t help teams fund their work dur­ing the 2019-22 pe­riod. Am I un­der­stand­ing that cor­rectly?

• Part of the point is that, al­though the prize isn’t awarded un­til 2022, you can still sell your rights to the prize in 2019, to some­one who pre­dicts that you will win the prize in 2022.

• Got it. So this would go some­thing like:

• There’s a prize!

• I’m go­ing to do X, which I think will win the prize!

• Do you want to buy my rights to the prize, once I win it af­ter do­ing X ?

Seems like this will se­lect for sales & per­sua­sion abil­ity (which could be an im­por­tant qual­ity for suc­cess­fully ex­e­cut­ing pro­jects).

• Yep.

I imag­ine it would se­lect in part for sales & per­sua­sion, but not more than for other prizes (where you need to do the same for the judges). The mid­dle­men would fo­cus on the fi­nan­cial mo­tive, so I’d ex­pect them to be rel­a­tively sane.

I would re­ally de­sire the eval­u­a­tions and pre­dic­tions/​es­ti­ma­tions to be re­ally good, in or­der to make sure peo­ple fo­cus on the right things.

• I think this idea is similar to al­ice.si (see https://​​al­ice.si/​​ or for more de­tail https://​​github.com/​​al­ice-si/​​whitepa­per/​​blob/​​mas­ter/​​Alice%20white%20pa­per%20-%20FV%200.9.pdf)

I know the founder of al­ice.si (not very well, but we’ve met up a cou­ple of times).

(Note that al­ice is on the blockchain and I’m not con­vinced there’s much benefit apart from the fact that some peo­ple don’t trust char­i­ties and the blockchain might help with that)

Also, I haven’t read this very care­fully, so apolo­gies if the two ideas are not as similar as I think