Math Pedantic/Computer Science Honours interested in Cause Prioritization. Currently working with QURI on Squiggle and Pedant.
SamNolan
Karma: 796
That’s true! could easily be something other than 1.5. In London, it was found to be 1.5, in 20 OECD countries, it was found to be about 1.4. James Snowden assumes 1.59.
I could but don’t represent eta with actual uncertainty! This could be an improvement.
Now that I’ve realised this, I will remove the entire baseline consumption consideration. As projecting forward I assume GiveDirectly will just get better at selecting poor households to counteract the fact that they should be richer. Thanks for pointing this out!
Oh no, I’ve missed this consideration! I’ll definitely fix this as soon as possible.
Would love to! I’m in communication to set up an EA Funds grant to continue building these for other GiveWell charities. I’d also like to do this with ACE! but I’ll need to communicate with them about it.
Quantifying Uncertainty in GiveWell’s GiveDirectly Cost-Effectiveness Analysis
Hey Neil,
How is this different from EA CoLabs? This team is working to connect people with projects and need as much help as they can help as they can get. Would it be worth joining them over starting a new project?
Maybe, your work there is definitely interesting.
However, I don’t fully understand your project. Is it possible to refine a Cost Effectiveness Analysis from this? I’d probably need to see a worked example of your methodology before being convinced it could work.
Hello Michael!
Yes, I’ve heard of Idris (I don’t know it, but I’m a fan, I’m looking into Coq for this project). I’m also already a massive fan of your work on CEAs, I believe I emailed you about it a while back.
I’m not sure I agree with you about the DSL implementation issue. You seem to be mainly citing development difficulties, whereas I would think that doing this may put a stop to some interesting features. It would definitely restrict the amount of applications. For instance, I’m fully considering Pedant to be simply a serialization format for Causal. Which would be difficult to do if it was embedded within an existing language.
Making a language server that checks for dimensional errors would be very difficult to do in a non-custom language. It may just be possible in a language like Coq or Idris, but I think Coq and Idris are not particularly user friendly, in the sense that someone with no programming background could just “pick them up”.
I may be interested in writing your CEAs into Pedant in the future, because I find them very impressive!
Hopefully Pedant ends up pretty much being a continuation and completion of Squiggle, that’s the dream anyway. Basically Squiggle plus more abstraction features, and more development time poured into it.
Causal is amazing, and if I could introduce Causal into this mix, this would save a lot of my time in developing, and I would be massively appreciative. It would likely help enable many of the things I’m trying to do.
I definitely was considering adding some form of exporting feature to Pedant at some point. I’m not sure that it’s within the current scope/roadmap of Pedant, but maybe at some point in the future!
Thanks for your considerations!
Yes, I agree. I can very much add tuple style function application, and it will probably be more intuitive if I do so. It’s just that the theory works out a lot easier if I do Haskell style functions.
It seems to be a priority however. I’ve added an issue for it.
The web interface should be able to write pedant code without actually installing Pedant. Needing to install custom software is definitely a barrier.
Pedant, a type checker for Cost Effectiveness Analysis
Thanks for pointing that out! I just fixed it up.
Have your community notified of new EA Jobs
A list of technical EA projects
For Improving Infrastructure around epistemics and forecasting, Ozzie or Nuno would likely be the best to answer this, so here I’m just trying to put myself in their mind. These ideas are a mixture of mine + a discussion with Ozzie.
I would say a clear opportunity would be to investigate looking into writing prediction functions, rather than just predictions. Say for instance “If SpaceX has a press release about an innovation to be released before 2025, then I estimate SpaceX to become a trillion dollar company 5 years earlier”. Having such a fidelity makes it possible to understand the best forecasting techniques better and aids in computer systems being able to answer these types of questions. As for as I know, this doesn’t exist.
As a side note, I think this type of forecasting platform would be awesome for policy evaluation. “If this policy is implemented in X way I predict that the policy will create a decrease in the unemployment rate by Y%”. The applications of the proper application of this idea are endless.
Another would be creating a platform that allows you to properly calibrate parameters for a Cost Effectiveness Calculation using forecasting, or evaluate outcomes of business decisions using forecasting.
I’m not a pro in this area, but that’s currently what I see.
For Improving Infrastructure around Cost Effectiveness Analysis, my current project is pedant.
Pedant is a math DSL that’s designed to make it easier to write cost effectiveness analysis. It checks the calculations for things like dimensional violations, and hopefully in the future allows you to calculate with uncertainties and explore cost effectiveness calculations more graphically.
I wouldn’t say that there are people who are asking for cost effectiveness analysis, and more that they simply aren’t done or are of low quality to large amounts of EA causes. For instance, even GiveWell’s work that we consider to be the gold standard does not properly account for uncertainty in parameters (although Cole Haus has done so in the forum), there is controversy around the accuracy of ALLFED’s guesstimate Cost Effectiveness Model, which may be systematically optimistic about their parameters, and these are some of the best ones out there! I don’t believe ACE uses explicit cost effectiveness calculations, let alone smaller EA organisations. In conversions with Ozzie and Michael Aird I believe that they seem to share a similar sentiment.
I mainly just assumed that this problem could be because these calculations are quite difficult to do, take a lot of time, and can be very difficult to get right. So as a developer I just thought tooling. I’m not particularly creative.
I would be interested in collaborators. Help I would need includes:
Actual coding of pedant. Mainly you would need to be familiar with Haskell to the extent that you can code basic parsers.
Feedback on development. I’m always on the lookout for people who can tell me when I’m wrong and should work on something else. I’ve currently got two sources of feedback. I would be more than willing to have a third.
Testing and usage. I’d love to see someone use pedant to do a variety of cost effectiveness analysis just to see what types of features are most needed in the language. I’ve currently got a CEA for GiveDirectly and the Against Malaria Foundation, and would appreciate help on writing the rest of GiveWell’s charities out, and maybe even other calculations such as ALLFED’s CEAs or Nuno’s Shallow Evaluations of Longtermist Organizations. You need a lot of patience to do this, it does take a while, and you are doing really simple transformation from one format to another.
Documentation and recommendations. It would be lovely to get a list of recommendations and best practices for writing cost effectiveness calculations based on what say GiveWell has done. Currently, the only documentation for pedant is the README file on the main page.
Really, if you or anyone else is interested, probably best to just contact me directly.
“having people sell products where all proceeds go to charity” is different from simply earning to give as it uses this fact to market to a buyer. The idea is that I may be more willing to purchase a second hand book from someone else if I know that the proceeds go to an effective charity (although I find that this is a surprisingly weak motivator, in my experience people don’t purchase things even if they know the money goes to an effective charity...).
I run a bookstore to this end that is currently not that successful, that I really want to see become a larger thing. Although this is likely mainly because I’m not that good at running shopify stores.
https://altruisticbook.net/
I have a friend who’s interested in much more ambitious ideas than this.
Thank you so much for the post! I might communicate it as:
People are asking the question “How much money do you have to donate to get an expected value of 1 unit of good” Which could be formulated as:
E(good(x))=1
where x is the amount you donate and good(x) is the amount of utility you get out of it.
In most cases, this is linear, so: good(x)=goodcost∗x. And E(goodcostx)=1.
Solving for x in this case gets x=E(goodcost)−1, but the mistake is to solve it and get x=E(costgood).
Please correct me if this is a bad way to formulate the problem! Can’t wait to see your future work as well