Oh yes, that is weird. The impression I had was that Ilya might even have been behind Sam’s ousting (based on rumours from the internet). I also understood that sacking Sam need 4 out of 6 board members, and since two of the board members were Sam A and Greg B, that meant everyone else had to have voted for him to leave, including Ilya. Most confusing.
Sanjay
Karma: 4,684
Bravo for writing this stuff up, glad to see that.
I actually didn’t realise that this elephant was an elephant? Indeed, I had the impression that paid ads had been used already by other EA orgs (if memory serves correctly, by EAG, 80k, and SoGive) so I thought they were considered to have legitimacy, as far as I was aware.
I believe the financial system is well-positioned for “consistent pressure on companies”. I have more to say on this based on my own work experience, so if anyone is interested feel free to reach out.
If we’re only considering plant-based meat, and only looking out over the near term (say, 1-3 years) then the claims here seem reasonable. So much so, that I’m surprised that the PTC model is so popular.
It may look like your concerns also apply to other alternative proteins (e.g. lab-grown meat). I don’t believe that’s the case.
We have a long way to go before we have lab-grown meat which is price competitive with traditional meat.
I’m willing to accept the argument that price-competitive lab-grown meat may not be sufficient, because of social and psychological factors.
However if lab-grown meat is lots cheaper than normal meat, then it seems reasonable to accept that it will displace most of the normal meat; based on some of your arguments, some people may persist in wanting the “real thing”, but given the existence of such a good and cheap substitute, it should be much easier to regulate to ensure that farmed animals have good lives (in which case I would consider the purchase of meat to be net positive)
A summary based on the quotes which I included in a separate comment:
Larry Madoff, who served as editor of the program from 2002 to 2021, said he was “forced out” by the organization’s CEO, Linda MacKinnon, according to STATnews.
It seems likely that something unfortunate is happening here, but I’m unclear what.
There was a letter written by several ProMED moderators, it appears that they objected to:
A letter going out to all ProMED subscribers proposing a subscription model;
this was signed by “The ProMED team” without the moderators being informed in advance; I don’t know much about what’s going on here, but this seems like a management issue;
they didn’t say it explicitly, but I’m guessing they are opposed to the subscription model, not just to the way the process was managed.
I don’t know enough to know whether the subscription model is a good idea.
They also objected to not receiving pay in a timely manner.
ISID CEO Linda MacKinnon’s response statement said: “this is unfortunately not a unique situation because ProMED has always operated on a shoestring budget. <...> We have addressed this more recently by stabilizing the predictability of payments. However, as we are currently in a funding pinch, we communicated in July to the ProMED team that some payments would be delayed, potentially up to two months.”
I’m unclear on what it could mean for them to have stabilised the predictability of payments (maybe they opted not to explain that for brevity?) Whatever it is, I’m unclear on how it could have been effective given the communications about payments being delayed for up to two months.
The moderators/editors who wrote the letter took the apparently strong action of suspending their work, even though they described it as a labour of love.
One of the details mentioned in Linda MacKinnon’s statement is that they plan to decommission scrapers. I’m confused by this, since I would have guessed that scrapers would be one of the more cost-effective components.
I’m also concerned about the internal strife within ISID/ProMED. I’ve copied and pasted some quotes below.
Here’s an excerpt from the STATnews article that this post links to:
...Larry Madoff, who served as editor of the program from 2002 to 2021. In spring 2021, Madoff said he was “forced out” by the organization’s CEO, Linda MacKinnon, and Alison Holmes, then president of the ISID executive committee. A professor of infectious diseases at the University of Massachusetts, Madoff refers to himself as editor emeritus of ProMED, a title bestowed upon him by the moderators with whom he worked.
Those same moderators — many of whom are signatories to the letter posted Thursday — wrote to the ISID to protest Madoff’s departure in 2021. They were told the matter was private and management would not discuss the rationale with them.
ProMED has not appointed a new editor, though it named a chief content officer, Jarod Hanson, in July 2022.
The letter linked to by Alex D above is also interesting. I’ve copied in some excerpts below, but the full letter is not loads longer and worth a read.
Some of you were unpleasantly surprised by the letter sent to all ProMED subscribers on 14 July 2023, informing you that, because of severe financial straits, ProMED will be limiting search capabilities and implementing a paid subscription model. Although the letter was signed by “The ProMED Team,” we the undersigned want to assure you that we had no prior knowledge of that letter. On the same day, ProMED moderators were informed by ISID that their pay, which is in arrears, would be delayed for at least 2 months.
<...>
ProMED moderators and editors have formed a tight knit collegial community over many years. For most of us, creating ProMED’s content is a labor of love. However, we cannot be expected to continue working on good will alone. With the loss of our Editor-in-Chief, and no voice in key administrative decisions directly affecting ProMED writers and readers, any such good will has evaporated.
It is therefore with great sadness and regret that we, the undersigned, are hereby suspending our work for ProMED.
<...>
Therefore, to restore our allegiance and legitimately call ProMED’s writers and editors a “ProMED Team,” ISID would need to:
1. Partner with one or more outside organizations in hosting ProMED in order to ensure that the fiscal and infrastructure needs of ProMED will be met.
2. Provide us with transparency and voting representation on the ISID executive committee.
3. Ensure the administrative and editorial independence of ProMED’s content and subscription policies. This could largely be accomplished by restoring a true Editor-in-Chief position with independent executive authority.
There’s a statement from the ISID CEO Linda MacKinnon which went up yesterday:
We recognize that members of the ProMED community are concerned about the continuation of the platform and want to reassure them that we hear their concerns. We know how many in our community start their day with ProMED and rely on its reports for their work. We also know that we could have communicated changes more clearly to the community and apologize for any confusion and distress caused.
We want to reassure users that ProMED services continue in a limited capacity, but there will likely be a temporary reduction in the number of posts on the website while we work with the signatories to yesterday’s letter.
Regarding delays in payments to Moderators, this is unfortunately not a unique situation because ProMED has always operated on a shoestring budget. This means that sometimes payments are delayed due to cash flow or funding issues. Over ProMED’s almost 30 years of existence, payment terms to Moderators have thus changed and evolved. We have addressed this more recently by stabilizing the predictability of payments. However, as we are currently in a funding pinch, we communicated in July to the ProMED team that some payments would be delayed, potentially up to two months. Despite this, some stipends have been paid in full but we still have more to cover. Some colleagues asked to be paid sooner because of personal circumstances, which we have accommodated.
When it comes to the long-term future of ProMED, the management team has been working consistently to address the lack of funding to keep ProMED running. Moving toward a predictable sustained revenue stream for the massive amount of work that takes place within ProMED is not something we should be afraid to talk about. It takes resources to provide early warning alerts to the world that so many industries, bodies, and governments depend on. I want to emphasize that the challenge is not ProMED’s alone. We know many peer organizations around the world are facing similar funding issues and have been working alongside other global bodies to address this gap. Underinvestment in public surveillance is a global issue.
For more than a year, we have been discussing ProMED’s financial plight with the many different entities and all key super-users of ProMED data. We have reiterated the need for sustained funding with such users to allow us to update the technology and deliver a modernized working environment for our ProMED Moderators, Correspondents, and Rapporteurs, as well as provide better curation of data for our many thousands of users around the world. However, no funding has materialized. The only viable path forward is to move to a subscription-based model, and in essence, decommission scrapers. This is not a move we wanted to make but in the absence of external funding, is required to keep ProMED operating.
To address other concerns that ProMED has not considered all investment avenues, I can confirm that to date, ISID has not received any proposals for funding partnerships, endowments, or revenue sharing. We are open to funding proposals from any interested party. Many have stated the need for ProMED and talked about the need for change, but no one has followed this up with concrete funding. Despite this, we have continued to provide the world with many key first alerts in the modern age, with subject matter expert data curation unparalleled by any other entity.
ISID has been covering the costs for ProMED since 1999, and piecing together small funding amounts is not feasible any longer. There is a perception that ProMED has sustained funding, but that has never been the case. With the need for staff, technology upgrades, and all the software as a service we pay for to keep emails flowing to our 20,000 subscribers, we need a new business model. We are working tirelessly on all fronts for unrestricted grants, and are exploring and open to strategic partnerships.
We know our community wants to do even more with ProMED. And we want to deliver for them. Our vision for an upgraded, mobile, and sophisticated platform that meets the needs of our subscribers requires unrestricted operational funding and investment. We ask organizations to get in touch if you can fund, want to partner, or have a collaborative proposal to share at support@isid.org.
It seems that a central bottleneck for the fund is that a few key people are decision-makers, and they are very busy, which makes it hard to operate quickly at scale and be transparent.
When SoGive ran its grants programme last year, we tackled these problems by getting more junior people to help.
I.e. the structure was:
most senior/expert people helped at key moments, but were mostly consultative
mid-level people (i.e. fairly experienced but non-expert) actively reviewed applications, including have a call with applicants (we actually did two rounds of calls, which might have been overkill)
junior people attended calls, wrote up notes, helped with putting the content into an accessible format for senior people to review, but also contributed thoughts and opinions
In many ways, this structure seemed to work well, since it allowed for the following:
call with applicants: having a video call as well as the application form was often genuinely additive and helped with decision-making; I’m therefore more positive about the quality of our grant-making
feedback: we were able to provide feedback to all applicants, which many applicants stated that they appreciated
transparency: we were able to exhibit a good level of transparency about our process and the outcomes of the process
effects on talent pool: the junior and even mid-level people involved in the process largely felt like it was a great opportunity to learn, and got a lot out of it (which might not be true of the experienced fund manager at the LTFF)
better aligned with key considerations: we generally found that lots of our decisions could be made based on junior input alone; i.e. common sense + general awareness of EA thinking + fresh perspective was often sufficient to highlight weaknesses in an application. Senior involvement was still needed in case something was missed, but often if a junior person was inclined to reject an application, it was rare for someone senior to suggest that we should resurrect the application. This meant that the way time was used was better aligned to the people involved, and there were significant time savings for senior people.
Here are some potential downsides:
speed: there are more steps involved in the process; this makes it hard to have a quick turnaround time. I suspect this could be surmountable.
costs: there are more people involved, which means there are more costs incurred; however arguably the most important consideration is the opportunity cost of people’s time, and those with high opportunity cost are spending less time, and more junior people feel like it’s a great opportunity for them to learn.
dependent on junior staff: if the junior staff were not performing well, this would be problematic. In our experience, this was not a problem. I think this reflects the fact that there are lots of very bright, motivated people who are keen to get relevant experience. However it does mean that care is needed with the selection process.
management capability: in order for this to work, senior and mid-level people need to be effective at managing more junior people. This did not seem to be much of a problem for us, but SoGive has experience supporting junior people (mostly volunteers) for many years.
I’m not trying to say that EA Funds should definitely do something like this, more that if it’s useful to explore what happened when someone tried something different, I’d be happy to discuss.
[Question] How many people are neartermist and have high P(doom)?
I was worried that this whole post might omit missing hedging and impact investing:
(a) an investor may wish to invest in equities for mission hedging reasons (e.g. scenarios where markets go up may be correlated with scenarios where more AI safety work is needed, or you might invest heavily in certain types of biotech firms, since their success might be correlated with pandemic risk work being needed)
(b) an investor can have impact on the entities they have a stake in through stewardship/engagement (sometimes referred to as investor activism). Roughly speaking, this involves having conversations with the firm’s management to influence their behaviour for the better, and using your voting power is part of this.
Fortunately, I’m pleased to see that you have mentioned both of these points, albeit only briefly towards the end. Thank you for incorporating those points.
Mission hedging seems to suggest you should hold more equities than you claim, by your logic.
Your concerns about mission hedging were that you weren’t sure whether future (e.g.) AI scenarios would lead to markets going up, or whether there would be some sort of “mayhem” prior to there being an existential catastrophe.
However your conclusion from this seems to be that we should be holding either more equities or less, and you’re not sure which, so it’s unclear how to update.
However I don’t think this is the right conclusion.
For those in the community who believe that certain risks (e.g. the risks from AI or GCBRs) are neglected, it would make sense that they are not adequately priced into the market. Even if you don’t know whether you this means more downside volatility or more upside growth, this means you are, in the jargon, “long volatility”, and you should do the following:
hold more equities (so you do well in the scenario where equities go up)
hold put options (so you outperform in the scenario where equities plummet because of “mayhem”, because the put options provide you with protection)
I think your post does a disservice to shareholder engagement
I think your claim that shareholder engagement has a small effect is reasonable if it describes the median of the engagement which actually happens, but is likely harsh on the optimal shareholder engagement strategy.
However in terms of the conclusions for this post, I’m unsure of how your engagement scales with size, and I wouldn’t be surprised if it were markedly sublinear. If that’s the case, then it doesn’t significantly argue against your claim. In other words, if you’re arguing between holding 60% equities and 80% equities, and you believe that the 80% equities holding gives you significantly less than 80/60x as much influence as holding 60%, then even if shareholder engagement can be more valuable than you claim, that’s not a very strong argument for holding 80%.
I’ve wondered about the interaction between far-UVC and immunity:
as well as protecting us against a scary novel pandemic-level pathogen, far-UVC would also kill off germs for various “common or garden” infections
at first glance, this sounds like a pretty great cherry on the cake
but could it exacerbate pandemic risk by reducing immunity, thereby making it easier for a bioweapon engineer to create a scary pathogen?
I agree with Jason that the specific moral hazard of “people might move to flood-prone areas in order to get cash” seems unlikely to be a concern.
The moral hazard that I was thinking of when I read Robi Rahman’s comment was “people who already live in flood-prone areas might be less prone to invest in flood defences/move away/do other things in light of the information that floods may be coming”
Re your question: “I would be especially interested if you have ideas for other historical case studies that could inform the longtermist project.” Here’s a few ideas:
In Scott Alexander’s post Beware Systemic Change, he argued that by funding Marx, Engels brought about “global mass murder without any lasting positive change”. I’d be quite interested in an assessment of whether this is true.
Did Marx’s work really cause the mass murder, or did the countries led by Marxist dictators happen to find themselves in circumstances where despots were prone to taking over anyway, and without Marxism might an even worse ideology have driven the counterfactual dictator?
Did Marx’s thinking cause any lasting positive change? How would we conceptualise left wing politics today without him? Did he advance the concept that people have rights, even if they don’t have capital/wealth, or was that concept going to take hold anyway?
There’s probably quite a few battles which could have had world-changing consequences if the outcome had been different. The Battle of Tours of 732 was quite interesting. The Ummayad Caliphate was in the midst of spreading Islam across the known world. As they spread north from Spain, they came up against the Franks and lost at the Battle of Tours. Had this not happened, not only might the Carolingian Empire found it harder to take off, but most of Western Europe (and the US?) might be Muslim today.
The US Constitution was the first constitution that I know of that was influenced by Enlightenment concepts. It seems to have been a success (at least, the US does seem to be a successful country today, at least under some measures). Did the constitution matter, or was the US bound to succeed anyway? Did the constitution have any long term impact on the values of Americans? The right to bear arms appears, at least superficially, to have an impact on contemporary American values. Did enlightenment concepts matter?
At the start of your post, you said, rather tantalisingly: “I believe that many of the learnings from the creation of climate risk financial regulation in the UK can be applied to AI regulation.” Could you expand on this?
Also, I’m pleased you wrote this post :-)
This comment will focus on the specific approaches you set out, rather than the high level question, although I’m also interested in seeing comments from others on how difficult it is to solve alignment, and why.
The approach you’ve set out resembles Coherent Extrapolated Volition (CEV), which was described earlier by Bostrom. I’m not sure what the consensus is on CEV, but here’s a few thoughts which I have in my head from when I thought about CEV (several years ago now).
How do we choose the correct philosophers and intellectuals—e.g. would we want Nietsche or Wagner to be on the list of intellectuals, given the (arguable) links to the Nazis?
How do we extrapolate? (i.e. how do you determine whether the list of intellectuals would want the action to happen?)
For example, Plato was arguably in favour of dictatorships and preferred them over democracies, but recent history seems to suggest that democracies have fared better than dictatorships—should we extrapolate that Plato would prefer democracies if he lived today? How do we know?
Another example, perhaps a bit closer to home: some philosophers might argue that under some forms of utilitarianism, the ends justify the means, and it is appropriate to steal resources in order to fund activities which are in the best long-term interests of humanity. Even if those philosophers say they don’t believe that, they might just be pandering to expectations from society, and the AI might extrapolate that they would say that if unfettered.
In other words, I don’t think this does clearly guarantee us against power-seeking behaviour.
Can you expand on why the ideal unit is “the settlement, village, community, or neighborhood”?
I can also confirm that an early employee of W3W told me that supporting development work was one the main original aims of W3W.
If I’m reading claim 3 correctly, are you saying that being a 10% GWWC pledger should be sufficient to get a spot at EAG, and this is true regardless of absolute donation amount?
At the outset, I had the same concern, however thus far it doesn’t appear to have been a problem. It’s possible that this may change in time, in which case we’ll cross that bridge when we get there.
I think it would be easy for someone to confuse the two, but (as Matt_Sharp rightly indicated) the SoGive 18 months and the GiveWell 3 years are referring to different things.
The SoGive 18 month threshold refers to funds where there are no plans to use the money.
GiveWell is referring to money which is planned to be spent.
Great that you did this, really appreciate it.
I’m no expert on the biology, but my intuition would in any case have been that the effect size would be tiny/negligible for 6 weeks of supplementation, and that for non-trivial effects, you would need sustained supplementation over a longer time period.
Is there any reason to doubt my intuition on this?