David T

Karma: 1,525

David T 16 Jun 2026 19:44 UTC
0 points
0 ∶ 1
in reply to: simon’s comment on: Are top existential risk estimates 50,000 times too high? An optimizer’s curse model and analysis
I’m not sure naive total utility maximization [in a static framework] is the best framework to be thinking about dealing with existential risk over time.^[1]
Assuming the number of risks and error bars are not trivially small, the universal outcome of concentrating all your risk mitigations on one is that most risks continue to be a high as they could possibly be. The modal outcome is that the risks ignored includes at least one risk greater than the one all efforts are concentrated on mitigating. Some reasonable assumptions in the article above show this can hold even where the actual biggest risk is orders of magnitude greater than the one targeted. In the diversified approach, less money are devoted to reducing the perceived biggest risk, but the rest is apportioned to reducing other risks. This seems more robust to conventional assumptions like uncertainty and some risks being easier to mitigate than others.
1. ^
  And tbh I’m not even seeing an average utility boost from concentrating on the single largest risk as opposed to mitigating lots of risks without ancillary assumptions like increasing returns to risk reduction expenditure or the actual value of many risks under consideration being 0.

David T 16 Jun 2026 19:05 UTC
1 point
0 ∶ 0
on: What 21,000 WhatsApp messages reveal about AI utility in extreme poverty contexts
It would be interested to see a more detailed and systematic report on the activity and findings so far.
In some respects, it seems like a strange thing for GiveDirectly to be piloting. On the one hand, GiveDirectly has expertise in systematic studies of behavioural change in LDCs , and the chatbot possibly also performed programmatic functions in a cost effective manner. On the other hand it involves a charity known for its “let local people decide how to use money spent on their behalf, Western aid agencies doing it can be disempowering and often wrong” ethos asking “which parameters should we use to fine tune this [adaptation of a commercial] product we’ve designed to give them the most suitable answers before scaling up its deployment”… which seems like a very different ethos and approach.^[1]

The conclusions highlighted from the research so far—both that if you give poor Rwandans access to ChatGPT they have a similar range of interaction to other humans^[2] and that responses generated by an LLM with no meaningful local training dataset were often inadequate—seem unsurprising. I am sympathetic to arguments that people make better decisions with access to information, but I am also sympathetic to arguments a ChatGPT derivative is not the most valuable information Rwandans could receive (and may have minimal or even negative value)

I’m not actually sure what the costs of acquiring relevant local data and training a chatbot to achieve greater fluency in spoken Kinyarwada dialects and safeguarding against advice that is very bad in a local context are,^[3] but they seem like a pretty relevant benchmark, since they might actually be considerable on a per user basis and the alternative for critical information like “what is the nearest health centre” might be something like signing people up to email lists, or a small number of human agents in Kigali costing surprisingly little.^[4] I guess there’s also the “who’s paying?” question, especially when the current implementation appears to involve providing training data for one of the world’s most valuable companies (and obscure languages may or may not add value to their model).
I feel one relevant benchmark for GiveDirectly specifically might be “what is the estimated cost per per person reached to improve it: would locals rather have a better chatbot or the cash?”. It’s possible the insights they’re getting are extremely valuable particularly in the context of limited/no of web access, but it’s possible they’re not…
1. ^
  the relevant comparator might be the One Laptop Per Child project. Well intentioned, theory of change centred on the idea that people in LEDCs can be empowered by interacting with modern technology and better information too, but perhaps actual educational benefits didn’t really stack up with the costs and the participants would have chosen to have something other than a computer
2. ^
  I must admit, I am curious about the extent to which Rwandans engaged in “witty banter” or attempts to manipulate the chatbot into saying something silly...
3. ^
  I don’t know how bad the speaking and dataset is, and whether an adequate “solution” looks like a finetuning prompt with some info or developing a corpus of services data and synthetic idiosyncratic Kinyarwada to fix the model, but the latter option could be very expensive compared with the people it would actually reach...
4. ^
  I suspect you get many person years of Rwandan human call centre time for a month or two of a mid-level AI engineer’s time...

David T 16 Jun 2026 15:06 UTC
2 points
0 ∶ 0
in reply to: Jackson Wagner’s comment on: How bad would it be if GPS satellites were shot down?
yeah, agree there are some military subsystems that would go down (and alternative PNT options have their own drawbacks). But it’s not a particularly decisive advantage: to take the topical example if Russia took out global GNSS it wouldn’t seriously harm the defensive posture of Ukranian front line troops that have been experiencing local GNSS jamming and finding alternative ways to get drones to work for years now, and would disrupt a bunch of Russian based logistics and civilian systems further from the front line nearly as much as it did to Kyiv’s. So it wouldn’t even significantly help them advance in Ukraine, even before the likely response from NATO and China was considered...

David T 16 Jun 2026 14:52 UTC
2 points
0 ∶ 0
in reply to: simon’s comment on: Are top existential risk estimates 50,000 times too high? An optimizer’s curse model and analysis
I think the conclusion that diversification is a good strategy follows trivially from the optimizers’ curse: if you focus all your efforts on the apparent biggest threat, you’ve probably just focused on the cause with the largest risk assessment error and entirely neglected the actual biggest threat. A more diverse allocation is more likely to address the actual biggest threat. If there are diminishing returns to resources allocated to mitigate particular risk areas that makes diversification look better (complex nonlinear returns complicate it). As does the possibility that larger errors in risk assessment for a particular type of risk are inversely correlated with ability to invest in the best mitigation strategy for that type of risk.^[1]
But your point about adverse selection is a good one too. Metrics are gameable, and there are stronger incentives to do so when funding is “winner takes all” rather than “we disburse funds to a wide selection of causes and value rigour and disclosure of uncertainties”
1. ^
  I think there are probably exceptions to this, but I think it’s generally true. Good understanding of celestial mechanics and early warning systems, for example, are absolutely essential to potentially preventing hypothetical large space rocks colliding with earth, but also mean that we are less likely to overestimate the imminence of destruction by a rogue asteroid than we are for more unpredictable phenomenon.

David T 15 Jun 2026 21:22 UTC
10 points
0 ∶ 0
on: How bad would it be if GPS satellites were shot down?
A few comments, some of which you may be intending to cover in updates
- First of all we actually have a pretty decent idea of what happens in GNSS-denied environments because localised GNSS jamming is a thing,^[1] It’s especially a thing in combat zones, which means that people and infrastructure affected typically have other problems^[2]
- Because GNSS denial is a thing, militaries have alternative PNT systems to aid them in combat. So actively disabling GNSS satellites is a pretty extreme measure that mostly hurts civilians, including in about 250 countries not currently at war with you. And if it involves use of anti-satellite weapons or EMPs, probably takes out a whole bunch of other space infrastructure too^[3]
- As it’s a pretty extreme measure that annoys everyone worldwide without even offering you a decisive advantage in a local conflict, it’s most likely to happen during escalation of a great power conflict. Great power conflicts mean that sectors like maritime would be experiencing COVID level downturns already. The “solar storm” is more interesting because it might be largely unexpected (and would also likely impair a lot of non-GNSS comms stuff)
- But costs of nuisance level GNSS jamming in borderlands between states not actually at war (say, the Baltic...) has a scaled down version of this impact which I guess might be underestimated...
  
  Interested to see the followup
1. ^
  Similarly, people working with navigation systems prone to spoofed location results (maritime navigation) have to find workarounds
2. ^
  though electronic warfare can affect neutral neighbours and overflying aircraft too...
3. ^
  current generation GNSS satellites are in fairly empty medium earth orbits and can be disabled without kinetic weapons, but this doesn’t rule out collateral damage and likely won’t be the case for future GNSS (in part because they want more redundancy even though the system(s) just work.

David T 15 Jun 2026 19:46 UTC
4 points
1 ∶ 1
in reply to: MaxReith’s comment on: Are top existential risk estimates 50,000 times too high? An optimizer’s curse model and analysis
Above all it implies don’t focus the vast majority of efforts on one cause.
That might not be practical for career choices,^[1] but it’s certainly possible for a funder or movement
1. ^
  though a corollary of it is “don’t assume that just because you’ve picked direct work that your career choice is maximally good and stuff like donations and helping others is just a distraction”. This is arguably true for speculative career choices even if the optimal cause is the correct one (i.e. even if AI x-risk really does dominate everything, lots of the promising approaches to resolving it that people might choose will have no impact)

David T 15 Jun 2026 19:29 UTC
18 points
6 ∶ 1
on: A frontier AI company should shut down
Other than OpenAI or Anthropic, I don’t see an AI company shutting down being taken remotely seriously by anyone outside a very small number of people that understand how that lab was performing, most of whom already have their own very strong views on AI safety. To most people, it just says “loss making entities coming up with elaborate excuses for AI bubble starting to burst” (or “I told you Elon was full of shit ” or even “look, a European/Chinese lab cannot compete with the US AI innovation because their governments have forced them to think about alignment too much, let’s not make the mistake of regulating...”)
OpenAI and Anthropic doing it would at least cut through to the average person/policymaker. But even if you believe they’re sincerely mission driven and owe their stockholders nothing, OpenAI and Anthropic are not IPOing this year because they believe that shutting down is the correct course of action

David T 15 Jun 2026 19:17 UTC
2 points
0 ∶ 0
in reply to: david_reinstein’s comment on: Craig Green ’s Quick takes
Yeah, I think that’s true on a lot of politics. Just think that many millions have been spent to make “actually international aid is really important both for saving lives and US soft power” an unfashionable argument in Republican circles, and Big Aid already has very good lobbyists working on the Dems too.

An interesting question is whether there’s much more scope in a European context, where questions about foreign aid are driven more by budget constraints and less by partisanship and xenophobia and “might the money be spent better on filling gaps in PEPFAR etc than current policy” is an argument policymakers might not be hearing so much from existing aid lobbyists.

David T 15 Jun 2026 19:07 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
Seems that’s because there’s significantly more scope to diagnose (rightly or wrongly) communication errors between people, who have motivations, expectations and emotional reactions, belong to cultures and hierarchies and communicate with subtext...

David T 12 Jun 2026 19:35 UTC
2 points
0 ∶ 0
in reply to: Craig Green 🔸’s comment on: [unknown]
Musk spent $290m in political giving to help convince Trump to give him the role of dismantling USAID, a decision which largely aligns with MAGA and Republican orthodoxy. He incidentally became the world’s first billionaire today and can run anti-USAID rhetoric on his social media platform at zero marginal cost.
Doesn’t seem like funds to reverse that in the current political climate are going to go very far. More traction is likely achievable longer term with the Democrat party (who aren’t exactly guaranteed to reinstate USAID, but are at least receptive to the standard arguments for it and unreceptive to what Elon thinks) but there are a lot of organizations already motivated to lobby for it because USAID was a major funding source, and some of them know their way around DC...

David T 12 Jun 2026 16:48 UTC
2 points
0 ∶ 0
in reply to: Guy Raveh’s comment on: Technoliberal’s Quick takes
This task, of trying to align them, is something that shouldn’t just be left to researchers in AI companies
In principle I agree.
But would you say that people’s suitability to align AI safely (or more specifically ensuring that Fable does not write nasty software exploits) is defined less by their expertise and alignment with Anthropic’s stated mission and more by how much money they can spend on credits?
Because that’s what Anthropic and the impending IPO marketing is asking you to believe
(tbh I’m not concerned by Fable manipulating its way into world domination. But if I was, I’d be extremely concerned that our most dedicated defenders against manipulative AI agents might be the sort of people who still take statements put out by AI companies at face value)

David T 11 Jun 2026 22:12 UTC
7 points
1 ∶ 0
on: Good-tasting alt meat is a scientific problem we can solve: $10M RFP for Alternative Protein R&D
I’d say driving prices down was as big a priority and potentially more easily achieved.
Taste might not be there for connoisseurs of the finest steaks, but plenty of bland meat products sell in volume. Alternative proteins can match stuff like nuggets effortlessly, but tend to cost at least twice as much, (as well as being in a different corner of the store alongside products that intentionally taste mostly of peas and beans, which is a potential hurdle no matter how authentic your meat alternative tastes)

David T 4 Jun 2026 16:27 UTC
3 points
0 ∶ 0
on: Fundraising is a tax on high-impact work
Where the theory of change is weakest is that whether funders’ reasons for preferring to get their dealflow from personal connections are good or bad, they’re unlikely to want to replace it by receiving unsolicited pitches via an AI platform (still less one they pay for). After all, they can publish a request for projects/startups to fund and an email address and get inbound submissions for free.
Only way you get over that hill is if the pre-sorting process is exceptionally efficient (and unfortunately for a lot of the most isolated founders, that probably means spending a lot of their time with a system trying to coach them into pitching a fundable project only to still be told they’re not ready yet...)
I also think it’s unclear how much value AI systems add in improving/sorting stuff which is actually good (as opposed to above very low thresholds of relevance/quality)

David T 2 Jun 2026 23:17 UTC
7 points
0 ∶ 0
on: Will we really put data centers in space?
The full report is definitely one of the better treatments I’ve seen of the subject (and I’ve read quite a few recently as they’re loosely related to my work). A few comments though...
- Feels like an unduly favourable assumption to ODCs to amortise hardware over exactly the same schedule as terrestrial ones. Any situation in which inference compute hardware doesn’t become obsolete at exactly the point the ODC reaches end of design life favours terrestrial datacentres.^[1] Also feels generous to assume bleed rates in space are no worse than terrestrial bleed^[2]
- I’m not sure what model you’ve determined your satellite hardware costs on and I know you’ve itemised solar and radiator panels else where, but taking the most feasible approach of thousands of smallsats for your 1GW datacentre, the $4b is buying a lot of propulsion systems, GNC subsystems, intersatellite links, space rated batteries and MAIT work as well as aluminium frames^[3]. The article doesn’t really discuss how much more complex satellite buses are than modular buildings...
- I think the regulatory arguments actually mostly stack in favour of terrestrial datacentres, because you get to jurisdiction shop. Whilst space isn’t effectively regulated, spectrum and launch licensing is and you basically only have one choice.^[4] There’s growing concern over orbital congestion and even the effects of deorbiting on the mesosphere to deal with, and plenty of parts of the world where NIMBYs either don’t live or are less relevant to local regulators than your money.
- The cooling estimate for earth may or may not be representative of what people are paying for climate control for racks of servers in a building, but feels a lot easier to improve that than max out the specific power of satellite radiators, especially since for ODCs we’ve already relaxed design principles like “chips should be less than 1km away from each other!”.
So I agree with the overall point it’s going to depend a lot on launch costs but I think there’s still some way to go. And even if the unit economics do work, your “fast takeoff” approach with Starship launches scaling to 10k by 2028 feels very fast assuming no obstacles at all to development, scaling capacity or even launches scrubbed by weather. c.f. industry expectations that Blue Origin losing their only launchpad recently will set them back a full year. SpaceX’s launch cadence being exceptional feels like reason to doubt they’ll 50x it in the next 2 years with an experimental platform...
A couple of further points slightly more favourable to ODCs in the longer run
- costs look different if one assumes a architecture where much of the mass doesn’t get deorbited every five years (especially stuff which suffers minimal degradation like radiators) ^[5] which probably looks a bit more like Starcloud plus orbital refuelling and servicer vehicles to upgrade chips. That sort of modular architecture is still a work in progress and obviously has its own tradeoffs with more expensive components.
- the other use case for orbital datacentres is edge compute for military uses, which isn’t fully defined yet and isn’t nearly as price sensitive
1. ^
  Your terrestrial datacentre can keep on running if its 5 year old chips are still useful, or conversely be replaced much earlier if the economics favours keeping the building/microgrid but replacing the chips with much better ones. Obsolete chips potentially have some resale market too. Your ODC needs deorbiting after 5 years regardless; propellant refuelling and in space chip replacement are theoretically possible but orders of magnitude more expensive than keeping a building powered on.
2. ^
  we have some evidence of COTS inference chips surviving 5yr LEO radiation doses without issue, but might be a different matter for the marginal chip whilst running hot over a 5 year duration in that environment. If needed, radiation hardened variants could be delivered… for a price
3. ^
  if I understand correctly that’s supposed to be just $4 per watt bus/subsystem/MAIT costs excluding solar panels, radiators and inference chips which would be two orders of magnitude cheaper than competitive providers in today’s market. Even with the benefits of vertical integration and automation that seems ambitious for the near term...
4. ^
  I suppose you could pin your hopes on Europe or India improving launch cadence and offering less restrictive licensing regimes but that sounds unlikely...
5. ^
  one reason why some people manage to be both sceptical of ODCs and bullish on SBSP (which has similar pros and cons) is that the latter is supposed to amortise its build costs over decades rather than be replaced every 5 years, which offsets transmission efficiency loss. Though yeah, that does mean more expensive solar panels etc...
6. ^
  I don’t know how much very slight latency between different nodes or sporadic disruption affects inference compute, but feels like something that could be more of a problem than precision pointing and tracking (these are solved problems, albeit problems solved with more expensive hardware and mission ops and regulators might not like dense formations)

David T 16 May 2026 10:38 UTC
4 points
0 ∶ 0
in reply to: Marcus Abramovitch 🔸’s comment on: AI safety is extremely bottlenecked on grantmakers
If accurate, that ratio of grantmakers to employed specialists looks rather low compared with what I understand it to be in many other fields, and I’m thinking of fields like space technology which have 75 page grant applications requiring specialist knowledge to evaluate and monitor, and government subsidy programmes whose application volume is sufficiently high to have <5% funding rates and which have painful audit requirements.
Also wonder how much EA organizations use part time external reviewers to evaluate grants, which is the standard way of broadening evaluations and removing bottlenecks? (although I can see getting AI specialists who both work in industry/research and are truly independent might be more challenging)

David T 13 May 2026 17:27 UTC
2 points
0 ∶ 0
in reply to: Larks’s comment on: We don’t know why Malawi is poor — and what that means for AI-and-growth forecasts
tbf it looks like the categories that drag Malawi’s economy down are the trade restrictions which were discussed and lack of “sound money”: inflation is often a symptom of being poor^[1] rather than anything particularly unusual about its institutions.
Even more so if we compare institutions with Botswana and its relative wealth from being able to export diamonds rather than basic food.
1. ^
  productivity growth locally not keeping up with the cost of necessary imports, basically. Looks like Malawi de-pegged their economy from the dollar a while back because of balance of payment problems rather than because of some weird governmental quirk.

David T 12 May 2026 22:05 UTC
4 points
0 ∶ 0
on: We don’t know why Malawi is poor — and what that means for AI-and-growth forecasts
I think you make a good point well (although I think the answer to why Malawi is poor is “all of the above and more”, and the question “why is Rwanda growing fast” is a more interesting one^[1])
When it comes to the inability of A[G]I to magically transform Malawi’s economy I actually think it’s even broader than just specific things like the existing political settlement restricting exports: AI telling people how to increase crop yields simply isn’t going to make maize smallholding anything other than a poverty living. Demand for maize is finite so making it marginally more efficiently even if one assumes every Malawian smallholder perfectly optimizes their production using the latest frontier yield optimization models (which one definitely shouldn’t assume) makes little difference. AI isn’t going to make the world any more likely to manufacture in a poorly educated, poorly connected backwater; if anything the opposite. Malawians aren’t exactly inundated with funds to buy compute time to start exciting new international businesses either.
1. ^
  Bangladesh and Vietnam I don’t see as really comparable. They have some classic historic problems but they have factors pulling them into a global economy Malawi doesn’t and couldn’t have.

David T 11 May 2026 20:56 UTC
4 points
1 ∶ 0
in reply to: Arno’s comment on: Exporters Without Borders: Why You Should Start a Company Instead of Working in Aid
On paper the absolute perfect people to do this are the diaspora: people who are both sufficiently familiar with local languages and cultural norms and challenges and sufficiently Westernized to be good at dealing with Western consumers and importers and funders.

Obviously some are doing this, just wondering if there’s much [Western] support infrastructure to help people trying to get into it?

David T 8 May 2026 8:45 UTC
9 points
1 ∶ 0
in reply to: Berke’s comment on: Effective Altruism focused on bednets while a malaria vaccine was stuck for 35 years. The case for Abundance.
The vaccine acceleration has some nuance as well. It’s definitely not as simple as “people could have produced enough vaccines to end malaria if the silly people slowing things down didn’t get in the way”. RTS’S, whilst moderately effective at reducing deaths in infants until the effects wear off never had the potential to get remotely close to ending malaria, and has been estimated to be less cost-effective at saving lives than plain old bednets.^[1] Ideally, of course, kids in the most affected areas would have both and probably seasonal chemoprevention too, but we haven’t even funded comprehensive enough bednet distribution yet.

Also these particular decisions don’t get taken by people who are sceptical of the potential of science or who lack skill in number crunching—quite the opposite. The Phase III trials had a further followup very large trial rather than instant mass distribution as soon as it was declared efficacious and probably safe because spending large fractions of developing countries’ healthcare budget on a promising intervention that appeared to work well in some groups but not others is a difficult thing to do.^[2] It’s not like adopting the opposite approach doesn’t have its drawbacks: the COVID vaccines were about as effective and safe as could be possibly have been hoped for in the accelerated timescale they were released on, but still ended up with the indirect result of US medicine being coopted by radical antivaxxers^[3] which seems bad, as does more parents withdrawing their kids from once routine vaccination schedules. Could have been a lot worse if those rare myocarditis side effects people spotted late on weren’t rarer than myocarditis side effects from the COVID everyone was getting anyway. It’s easier to say more, bigger, faster without considering second order effects.
1. ^
  see for example https://www.1daysooner.org/how-cost-effective-is-the-new-r21-vaccine-compared-to-existing-malaria-interventions/ and note that these have wide error bars due to the differences in costs of vaccination programs between countries and uncertainty about how consistently bednets are used, and that R21 which got faster regulatory approval is more cost effective because it appears more potent and is cheaper to manufacture and distribute.
2. ^
  it wasn’t just the overlapping but probably completely coincidental meningitis, there were quirks in the Phase III trial data probably related to the relatively small sample size like the rare but most-likely-to-be-fatal cerebral malaria actually showing up more in vaccinated older kids which brought recommendations for a very large trial rather than universal incorporation into everyone’s vaccine schedule.
3. ^
  Trump moves in mysterious ways, but his decision to appoint RFK Jr to attack vaccine science surely isn’t unlinked to how the fact the one Trump achievement he can get booed by his own supporters for mentioning is vaccine rollout, after the narrative everyone was being injected before the scientists even knew what the side effects were took hold.

David T 7 May 2026 22:13 UTC
4 points
2 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
It’s an interesting idea but as expressed feels a little tendentious, particularly if one looks at what is actually formally considered a war crime these days (much of which would not have been recognised as war crimes in the past, including by actors who believed themselves to be unusually chivalrous). Hard to believe it will be impossible to avoid total war if a few civilians are murdered or chemical weapons are used, never mind if a pilot gets shot after ejecting or if a spy is not afforded a fair trial, and hostage taking was once considered a good way to avoid total war. We see peace agreements between prolific war criminals quite often too. Avoiding total war might be a motivation, but it can’t be the only one.
On the other hand game theory favours opposing sides agreeing to not shoot ejecting pilots or torture each others’ prisoners even if they don’t agree on anything else, whereas it is impossible to win a war if you are not permitted to fire at the other side’s soldiers, and at least in 1949 artillery and aerial bombardments were also too critical to winning for the Geneva Convention to agree to ban them. It is possible for opposing sides to agree not to shoot at people that don’t wear uniform, but only if both sides treat sneaking up on the other side and shooting them whilst not wearing uniform as also a crime.
Also, many people genuinely believe in the idea that people shooting other people in a uniform which indicates they intend to fire back represents some sort of fair play (even if the targets happen to be sleeping conscripts who haven’t had a chance to surrender yet) and the sort of people that believe in that sort of thing are disproportionately likely to be military officers. They tend to believe in just wars too...
I do agree that opposing sides are considerably more likely to respect conventions on war crimes (and even reach other bargains like prisoner swaps) whilst the infrastructure that may allow the sides to mutually end the war still exists. But there’s plenty of evidence of war crimes committed with impunity in conflicts that never came close to total war, and for that matter of individual military units choosing to abide by conventions despite there being no realistic prospect of a near-term peace agreement and plenty of war crimes being committed by others on their side