PhD student at Aberdeen University studying Bayesian reasoning
Interested in practical exercises and theoretical considerations related to causal inference, forecasting and prioritization.
Jsevillamol
Karma: 1,271
A bottom-up approach for improving public decision making
Improving the Public Management of Global Catastrophic Risks in Spain
I have received a private comment asking about the role of civil protection during COVID.
This is what I answered:
About COVID, we asked everyone about their role on it.Basically the picture I got is: public health is delegated to the ministry of health, and in particular pandemics are seen as the business of the Centre of Sanitary Alerts (CCAES) The CCAES does have an early warning system, but we could not talk to them and we don’t know how the systems reacted to COVID.
In the civil prot side, they basically did nothing until the govn declared an alarm state. It is not clear if the activation was urged by the CCAES, civil protection or if it came from up top. After the alarm state was declared, civil prot organized decentralised committees at the state, autonomy and municipal level to monitor the situation and discuss next steps. These committees incorporated public health experts, civil prot servants and other public and private figures with relevant expertise.
The military “forces” under command of civil protection (the UME) were mobilized, but they lacked expertise on dealing with pandemics and basically were ordered to help disinfect hospitals and other public places, even after it became obvious that the pandemic was airborne and that disinfecting helped little.
In general I think that the civil prot response was good and prompt once they were ordered to act. In particular, I think the decentralised response involving several expertise was as good as could have been given their lack of preparation.
But they weren’t able to anticipate and react to early signs and their own forces were unprepared for a pandemic. It would also have helped to have mapped out in advance who were relevant experts to consult at every level of the system
Echoing everyone else, thank you for all your hard work.
I do not exaggerate when I say you are the best forum moderator I have ever seen. I am really impressed with your availability, creativity and kindness. You have driven the culture of this website to a whole new level, and inspired me and I bet many others to write better content.
Good luck at OpenPhil!
I think that this kind of criticism is really useful, and I am glad it was written.
That being said, there is something about this post that really rubbed me the wrong way. This is a shame, because the topic is very pertinent and deserves an in-depth discussion—how do opportunities for funding today compare to opportunities for funding 5 or 10 years from now?
Let me try to give my best shot at a more thoughtful critique.
In the midst of a global pandemic that pushed 150 million people into extreme poverty while billionaires’ wealth grew by $5.5 trillion, it’s odd timing.
This is pretty much an unrelated remark. While I agree that this reflects inequality and a sad arrangement of the world, it has very little to do with whether GiveWell should or not be patient.
I am pointing this out because this is one of things that made me icky about the post, even though it is a great post otherwise.
The team at GiveWell know all these numbers of course, and they would likely agree that influencing more money is better. But their decision here tries to optimize 0.1% of U.S. charitable giving in isolation from the other 99.9%; when, in reality, growing that 99.9% and allocating it better will mean a lot more for our world than asking those donors to hold out for possible silver bullets down the road.
The argument here is that focusing on fundraising is probably a better use of GiveWell’s resources than being patient with their money. But this assumes that fundraising trades off against patience, which I believe is false. GiveWell can focus on fundraising AND be patient.
I assume that the crux here is that GiveDirectly believes that spending more money now would have a good publicity effect, that would promote philanthropy and raise the total amount of donations overall.
I would change my mind if this was the case, but I don’t see this as obvious.Since 2018, we have asked GiveWell to fully engage with this study and others, but they have opted not to, citing capacity constraints. Until they do, they may be underestimating the effects of cash transfers by 2.6x and overstating the benefits of waiting by the same amount.
This is the strongest argument in this document. If GiveWell has not taken into account these second hand effects and those would increase the effectiveness by x2 that might push them over the edge of effectiveness that would justify giving more now.
On the other hand, I am unsure of how much this affects the math on whether to expect better giving opportunities in the future. In fact, it could counterintuitively mean that good spending opportunities are easier to find than we expected, which could incentive waiting.
We’re skeptical about GiveWell’s hopes to find much more without a more fundamental re-evaluation of the scale of the sector, the opportunities available, and their role within it. We hope we’re wrong about that, but if we’re not it’s another reason to move faster now, rather than hold out for better opportunities down the road.
I agree with your position—if I believed that GiveWell has run out of low hanging fruit then this definitely is a good argument in favour of spending more now instead of waiting.
On the other hand, GiveWell is incredibly young. And they seem to be capacity constrained enough that they cannot afford to look into very relevant studies on cost-effectiveness like the one you mention in the post. I would be quite surprised if there weren’t better funding opportunities than the ones already identified.
So it seems prudent to expect that better opportunities will be identified later.
However, people living in poverty have good reasons to answer this table’s impossible questions in a variety of different ways. They may also have good reasons to differ with GiveWell on what else matters, let alone which interventions will do the most for the things that matter most to them. Many of these potential differences of opinion are highly relevant to whether or not GiveWell should save funds and give further down the road.
I absolutely agree with this. And I commend GiveDirectly on its efforts to incorporate the voice of their recipients in their processes—I think its exemplary and something to be imitated.
I’d love to see GiveWell expand their work on this area, and to hire a more diverse team to help integrate many valuable perspectives.
I know very little about the chances of finding better funding opportunities in the future and of GiveWell and GiveDirectly. So please definitely take all of this with a grain of salt.
My TL;DR of the post is that GiveWell might be understimating direct giving and needs to do more work on listening to charity beneficiaries to learn how to help them best. It was also argued that GiveWell is unlikely to find better funding opportunities in the future, but I didn’t find those arguments particularly convincing.
Thank you to GiveDirectly for taking the time to engage with the community. Highlighting these problems is definitely something I would like to happen more often, so we stand a better chance at fixing them.
Takeaways from our interviews of Spanish Civil Protection servants
I do think so!
It’s hard to contest that change across many dimensions has been accelerating.
And it would make sense that this accelerating change makes parental advice less applicable, and thus parents less influential overall.
The A,B,C example you came up with is certainly a strike against average log odds and in favor of average probs.
I have though more about this. I now believe that this invariance property is not reasonable—aggregating outcomes is (surprisingly) not a natural operation in Bayesian reasoning. So I do not think this is a strike agains log-odd pooling.
Why did you end up being turned off by EA?
It’s hard to pinpoint but I think it’s somehting along the lines of a) the messaging didn’t match my perceived self-image (“I am not an altruist”), b) they seemed weirdly fanatical (“donating 10% of my money seems crazy weird”) and c) I was not impressed with the people I interacted with (concretely the people from eg the rationality community seemed comparatively more thoughful and to be working on cooler things).
I am unsure of whether I would have changed my mind had I interacted more with the community at that time—I think the quality of discussion has improved a lot since then.
> How did it end up being crucial for your long-term engagement?After the MIRI Summer Fellows I started organizing a community in Spain (primarily about rationality, though some of the other people involved were self-identifying Effective Altruists and we also organized events about that. I also participated in several more Rationality and Effective Altruism events.
I kept talking to Effective Altruists regularly, and eventually became convinced that they were working on cool things and that it was a community I wanted to be a part of.
Retreats are awesome!
It was the MIRI Summer Fellows in 2015. For full disclosure it was not about EA, and I came off it being turned off by EA aesthetics. But it was where I first heard about the movement, and it was crucial for my involvement in the long term.
One hour (maybe two) fellowship sessions isn’t long enough to get into “late night life-changing conversations” mode, which is important for big changes.
This to me is the main downside.
I got introduced to EA over a 3 week in-person summer program, and my experience is that 2~4 week in person intensive programs have a good track record in getting people excited and engaged. Off the top of my head 1 out of 3 participants in the camps ive been involved in became counterfactually engaged, 1 out of three was engaged but would be anyway and 1 out of three bounced off and didn’t stay engaged.
Late night conversations and a great vibe was a big part of why I stayed engaged, and matches my intuitions of what works best to help people grow and connect.
I would be interested on having more data about 6 month after retention for the Intro EA Fellowships, both for the whole group and for the subgroups that are considered “more promising”.
Thank you!
These papers were ones that William MacAskill was considering citing in his forthcoming book. FF hired me to thoroughly check them.
There is definitely many other persistence papers I didn’t cover!
Eg:
Acemoglu et al, Colonial Origins
Acemoglu et al, Reversal of Fortune
Woodberry (2012). The Missionary Roots of Liberal Democracy
All the papers cited in Kelly’s Understanding Persistence
And many others.
EDIT: Faatima Osman questioned whether it was fair to exclude respondents from Benin, Ghana and Nigeria in Nunn and Wantchekon’s paper, given that Nigeria is the most populated country in Africa by far.
And in hindsight I think she is totally right—respondents from these countries are ~25% of the sample! I now believe that its unfair to call these respondents outliers. Correspondingly, my trust in Nunn and Wantchekon’s paper has gone up, since Kelly’s critique was my main concern about it.
EDIT: Faatima Osman questioned whether it was fair to exclude respondents from Benin, Ghana and Nigeria in Nunn and Wantchekon’s paper, given that Nigeria is the most populated country in Africa by far.
And in hindsight I think she is totally right—respondents from these countries are ~25% of the sample! I now believe that its unfair to call these respondents outliers. Correspondingly, my trust in Nunn and Wantchekon’s paper has gone up, since Kelly’s critique was my main concern about it.
Can we influence the values of our descendants?
Thank you for your thoughful reply. I think you raise interesting points, which move my confidence in my conclusions down.
Here are some comments
[...] averaging log odds will always give more extreme pooled probabilities than averaging probabilities does
As in your post, averaging the probs effectively erases the information from extreme individual probabilities, so I think you will agree that averaging log odds is not merely a more extreme version of averaging probs.
I nonetheless think this is a very important issue—the difficulty of separating the extremizing effect of log odds from its actual effect.
I don’t expect optimally-extremized average log odds to outperform optimally-extremized average probabilities
This is an empirical question that we can settle empirically. Using Simon_M’s script I computed the Brier and log scores for binary Metaculus questions of the extremized means and extremized log odds and extremizing factors between 1 and 3 in intervals of 0.05.
In this setting, the top performing metrics are the “optimally” extremized average log odds in term of log loss, surpassing the “optimally” extremized mean of probs.
Note that the Brier scores are identical, which is consistent with the average log odds outperforming the average probs only when extreme forecasts are involved.
Also notice that the optimal extremizing factor for the average of logodds is lower than for the average of probabilities—this relates to your observation that the average log odds are already relatively extremized compared to the mean of probs.
There are reasons to question the validity of this experiment—we are effectively overfitting the extremizing factor to whatever gives the best results. And of course this is just one experiment. But I find it suggestive.
External Bayesianity seems like an actively undesirable property for probability pooling methods that treat experts symmetrically. When new evidence comes in, this should change how credible each expert is if different experts assigned different probabilities to that evidence.
I am not sure I follow your argument here.
I do agree that when new evidence comes in about the experts we should change how we weight them. But when we are pooling the probabilities we aren’t receiving any extra evidence about the experts (?).
I talked about the argument that averaging probabilities ignores extreme predictions in my post, but the way you stated it, you added the extra twist that the expert giving more extreme predictions is known to be more knowledgeable than the expert giving less extreme predictions. If you know one expert is more knowledgeable, then of course you should not treat them symmetrically.
I agree that the way I presented it I framed the extreme expert as more knowledgeable. I did this for illustrative purposes. But I believe the setting works just as well when we take both experts to be equally knowledgeable / calibrated. Throwing away the information from the extreme prediction seems bad.
Probabilities must add to 1.
I like invariance arguments—I think they can be quite illuminating. In fact I am quite puzzled by the fact that neither the average of probabilities nor the average of log odds seem to satisfy the basic invariance property of respecting annualized probabilities.
The A,B,C example you came up with is certainly a strike against average log odds and in favor of average probs.
It reminds me of Toby Ord’s example with the Jack, Queen and King. I think dependency structures between events make the average log odds fail.
My personal takeaway here is that when you are aggregating probabilities derived from mutually exclusive conditions, then the average probability is the right way to go. But otherwise stick with log-odds.
[...] I maintain that, if you want a quick and dirty heuristic, averaging probabilities is a better quick and dirty heuristic than anything as senseless as averaging log odds.
I notice this is very surprising to me, because averaging log odds is anything but senseless.
This is a far lower confidence argument than the other points I raise here, but I think there is an aesthetic argument for averaging log odds—log odds make Bayes rule additive, and I expect means to work well when the underlying objects are additive (more about this from Owen CB here).
There is also the argument that average logodds are what you get when you try to optimize the minimum log loss in a certain situation—see Eric Neyman’s comment here.
Again, these arguments appeal mostly to aesthetic considerations. But I think it is unfair to call them senseless—they arise naturally in some circumstances.
if the worst odds you’d be willing to bet on are bounds on how seriously you take the hypothesis that someone else knows something that should make you update a particular amount, and you want to get an actual probability, then you should average over probabilities you perhaps should end up at, weighted by how likely it is that you should end up at them. This is an arithmetic mean of probabilities, not a geometric mean of odds.
Being honest I do not fully follow the reasoning here.
My gut feeling is this argument relies on an adversarial setting where you might get exploited. And this probably means that you should come up with a probability range for the additional evidence your opponent might have.
So if you think their evidence is uniformly distributed over −1 and 1 bits, you should combine that with your evidence by adding that evidence to your logarithmic odds. This gives you a probability distribution over the possible values. Then use that spread to decide which bet odds are worth the risk of exploitation.
I do not understand how this is about pooling different expert probabilities. But I might be misunderstanding your point.
Thank you again for writing the post and your comments. I think this is an important and fascinating issue, and I’m glad to see more discussion around it!
Persistence—A critical review [ABRIDGED]
On getting research collaborators
(adapted from a private conversation)
The 80⁄20 advice I would give is: be proactive in reaching out to other people and suggesting to them to work for an evening on a small project, like writing a post. Afterwards you both can decide if you are excited enough to work together on something bigger, like a paper.
For more in depth advice, here are some ways I’ve started collaborations in the past:
Deconfusion sessions.
I often invite other researchers for short sessions of 1-2 hours to focus on a topic, with the goal of coding together a barebones prototype or a sketch of a paper.
For example, I engaged in conversation with Pablo Moreno about Quantum Computing and AI Aligment. We found we disagreed, so I invited him to spend one hour discussing the topic more in depth. During the conversation we wrote down the key points of disagrement, and we resolved to expand them into an article.
Advertise through intermediate outputs.
I found it useful for many reasons to split big research projects into post-size bits. One of those reasons is to let other people know what I am working on, and that I am interested in collaborating.
For example, for the project on studying macroscopic trends in Machine Learning, we resolved to first write a short article about parameter counts. I then advertised the post asking for potential collaborators to reach out.
Interview people on their interests.
Asking people what motivates them and what they want to work on can segue into an opportunity to say “actually, I am also interested in X, do you want to work together on it?”. I think this requires some finesse, but it is a skill that can be practiced.
For example, I had an in depth conversation with Laura González about her interests and what kinds of things she wanted to work on. It came up that she was interested in game design, so I prodded her on whether she would be interested in helping me refine a board game prototype I had previously shown her. This started our collaboration.
Join communities of practice.
I found it quite useful to participate in small communities of people working towards similar goals.
For example, my supervisor helped me join a Slack group for people working on AI Explainability. I reached out to the people for one-on-one conversations, and suggested working together to a few. Miruna Clinciu accepted—and now we are buiding a small research project.
(disclaimer: this is my opinion)
In short: Spanish civil protection would not as of today consider making plans to address specific GCRs
There is this weird tension where they believe that resilience is very important, and that planning in advance is nearly useless for non-recurring risks.
The civil protection system is very geared towards response. Foresight, mitigation and prevention seldom happens.This means they are quite keen on improving their general response capacity but they have no patience for hypotheticals. So they would not consider specific GCRs.
Even if they wanted to address GCRs, their hands are relatively tied (at least at the national level) - the risks they do specific preparation for are encoded in the law and modifying the list of priority risks would require passing an amendment.
In their opinion some things like geomagnetic storms which could theoretically unleash a global catastrophe are to be addressed by the generalist response plans. And, at least one high ranked person thinks a specific plan for responding to solar storms and similar risks is could be created, but not without a coordinated technical and policy response at the European level.
On the other hand we have seen some autonomies that have enacted their own special civil protection plans independently, but for minor risks (eg coastal environmental protection).And for example Madrid’s city hall wants to have better maps of which expertise is needed and where to find it for conceivable future emergencies.
Also bear in mind that while civil protection is a very important part of risk management in Spain, it is not the only part. The national security system and other organizations might have different attitudes towards GCRs.