Luke, you’ve been so strong at the helm of GWWC for so long that I’m often guilty of thinking about you and GWWC as synonymous (that’s a compliment, I swear!). Well done on the amazing work you’ve done, and enjoy a well deserved break. I can’t wait to see what you do next.
MichaelDello
Karma: 524
As someone who is not an AI safety researcher, I’ve always had trouble knowing where to donate if I wanted to reduce x-risk specifically from AI. I think I would have donated quite a larger share of my donations to AI safety over the past 10 years if something like an AI Safety Metacharity existed. Nuclear Threat Initiative tends to be my go to for x-risk donations, but I’m more worried about AI specifically lately. I’m open to being pitched on where to give for AI safety.
Regarding the model, I think it’s good to flesh things out like this, so thank you for undertaking the exercise. I had a bit of a play with the model, and one thing that stood out to me is that the impact of an AI safety professional at different percentiles doesn’t seem to depend on the ideal size, which doesn’t seem right (I may be missing something). Shouldn’t the marginal impact of one AI safety professional be lower if it turned out the ideal size of the AI safety workforce were 10 million rather than 100,000?
No problem! I think my main concern is just that you make sure the water properties at 0.5-1m depth match the water properties at the surface, or at least, you can work out how they vary to apply corrections to the satellite data. But overall I’m positive about this venture.
Applying remote sensing to fish welfare is a neat idea! I’ve got a few thoughts.
I’m surprised that temperature had no/low correlation with the remote sensing data. My understanding is that using infrared radiation to measure water surface temperature was quite robust. The skin depth of these techniques are quite small, e.g., measuring the temperature in the top 10 μm. Do you have a sense of the temperature profile with respect to depth for these ponds? Perhaps you were measuring the temperature below the surface, and the surface temperature as predicted by the satellite was different. Then again, you might expect some systematic error here giving you some kind of correlation anyway.
The methodology used by Captain Fresh is a black box as you say, but maybe you could ask for more detail. When I was working for an exploration company, specialist contractors who gave us data were usually eager to give us presentations on the minutia of the data and methodology and answer our questions because they wanted our future business.
Do you know what water depth your on-site measurements were taken at? Ensuring that this was consistent seems important, and it’s important to remember the depth of penetration of the remote sensing data. If you could ask Captain Fresh for this, that would be ideal, but it’s typically quite small/shallow. I’m less familiar with best practice for data collection, e.g., how important is it to collect on-site data from as close as possible to the surface, but these might be important considerations. Did Captain Fresh or ProDigital give any guidance for this? (I didn’t see anything from a brief skim of the user manual)
You might also want to consider doing more detailed on-site measurements at a few sites to see how well each water property at depth x correlates to depth y. If the remote sensing data gives you good predictions of the properties at the surface but the properties vary greatly at depth, it’s probably not a very useful prediction, unless they vary in a systematic or predictable way.
This study was able to predict pH levels in lakes using Landsat data with an R2 of 0.81, but the lakes were quite large, on the scale of several km wide. I intuitively but weakly suspect that this method would be less effective for small farmed fish ponds.
I’m surprised to see salinity missing from this list. Predicting water salinity with remote sensing also seems to be quite robust, and it seems to be quite important for monitoring fish welfare. Was this omitted just due to limitations of the Captain Fresh data? Your ProDigtal seems to be capable of measuring water salinity on-site.
Happy to chat about this some more if any of this was helpful. It’s been quite a while since I actually did any remote sensing myself, but I’ve relied on remote sensing data for other work from time to time.
Point 4, Be cautious and intentional about mission creep, makes me think of environmental- and animal-focused political parties such as the Greens and Animal Justice Party in Australia, and the Dutch Party for the Animals in the Netherlands. The first formed as as an environmental party, and the latter two formed as animal protection parties.
All three of these have experienced a lot of mission creep since then (Animal Justice Party to a lesser extent than the other two). The prevailing wisdom from many is that this is a good thing. A serious political party should have a position on every issue, some will say. But the sense I get from your post is that this may not be the case, because just like with a movement, a political party can become partisan by taking a position on every issue and adopting a political leaning of some kind.
Thanks for writing this! I had one thought regarding how relevant saying no to some of the technologies you listed is to AGI.
In the case of nuclear weapons programs, the use of fossil fuels, CFCs, and GMOs, we actively used these technologies before we said no (FFs and GMOs we still use despite ‘no’, and nuclear weapons we have and could use at a moments notice). With AGI, once we start using them it might be too late. Geo-engineering experiments is the most applicable out of these, as we actually did say no before any (much?) testing was undertaken.
I supplement iron and vitamin C, as my iron is currently on the lower end of normal (after a few years of being vegan it was too high, go figure).
I tried creatine for a few months but didn’t notice much difference in the gym and while rockclimbing.
I drink a lot of B12 fortified soy milk which seems to cover that.
I have about 30g of protein powder a day with a good range of different amino acids to help hit 140g a day.
I have a multivitamin every few days.
I have iodine fortified salt that I cook with sometimes.
I’ve thought about supplementing omega 3 or eating more omega 3 rich foods but never got around to it.
8 years vegan for reference.
Very fair response, thanks!
I strongly agree that current LLM’s don’t seem to pose a risk of a global catastrophe, but I’m worried about what might happen when LLM’s are combined with things like digital virtual assistants who have outputs other than generating text. Even if it can only make bookings, send emails, etc., I feel like things could get concerning very fast.
Is there an argument for having AI fail spectacularly in a small way which raises enough global concern to slow progress/increase safety work? I’m envisioning something like a LLM virtual assistant which leads to a lot of lost productivity and some security breaches but nothing too catastrophic, which makes people take AI safety seriously, slowing progress on more advanced AI, perhaps.
A complete spitball.
This is cool! I came across EA in early 2015, and I’ve sometimes been curious about what happened in the years before then. Books like The Most Good You Can Do sometimes incidentally give anecdotes, but I haven’t seen a complete picture in one public place. Not to toot our own horn too much, but I wonder if there will one day be a documentary about the movement itself.
Thanks for the great question. I’d like to see more attempts to get legislation passed to lock in small victories. The Sioux Falls slaughterhouse ban almost passing gives me optimism for this. Although it seemed to be more for NIMBY reasons than for animal rights reasons, in some ways that doesn’t matter.
I’m also interested in efforts to maintain the lower levels of speciesism we see in children into their adult lives, and to understand what exactly drives that so we can incorporate it into outreach attempts targeted at adults. Our recent interview with Matti Wilks touches on this a little if you’re interested.
Thank you for the feedback! I just wanted to let you know that while I haven’t had time to write a proper response, I’ve read your feedback and will try to take it on board in my future work.
People more involved with X-risk modelling (and better at math) than I could better say whether this is better than existing tools for X-risk modelling, but I like it! I hadn’t heard of the absorbing state terminology, that was interesting. When reading that, my mind goes to option value, or lack thereof, but that might not be a perfect analogy.
Regarding x-risks requiring a memory component, can you design Markov chains to have the memory incorporated?
Some possible cases where memory might be useful (without thinking about it too much) might be:
How well past social justice movements went may have implications for the success of future movements that relate to X-risk
The way in which problems end may have implications for future problems
Maybe this information can just be captured without memory anyway?
Thanks for sharing, I’m looking forward to this! I’m particularly excited about the sections on measuring suffering and artificial suffering.
Thanks for sharing! I love seeing concrete roadmaps/plans for things like this, and think we should do it more.
Fair enough! I probably wasn’t clear—what I had in mind was one country detecting an asteroid first, then deflecting it into Earth before any other country/‘the global community’ detects it. Just recently we detected a 1.5 km near Earth object that has an orbit which intersects with Earth. The scenario I had in mind was that one country detects this (but probably a smaller one ~50 m) first, then deflects it.
We detect ~50 m asteroids as they make their final approach to Earth all the time, so detecting one first by chance could be a strategic advantage.
I take your other points, though.
“(b) Secondly, while the great powers may see military use for smaller scale orbital bombardment weapons (i.e. ones capable of causing sub-global or Tunguska-like asteroid events), these are only as destructive as nuclear weapons and similarly cannot be used without risking nuclear retaliation.”
I don’t think this is necessarily right. First, an asteroid impact is easier to seem like a natural event, therefore being less likely to result in mutually assured destruction. Also, just because we can’t think of a reason for a nation to use an asteroid strike, doesn’t mean there isn’t one. Accidental deflection of a small planetary body into an Earth-intersecting orbit is possible. And I don’t necessarily think that only great powers will have access to asteroid deflection technology in the future. As the technology develops, it will be possible for smaller and smaller groups to access it.
Given the relatively low probability of asteroid impacts on short-term timescales, it wouldn’t take that much of an increased risk from the deflection dilemma to make developing deflection technology not worth it.
I think improving detection technology might be a safer option to leave our options open while having a similarly positive effect.
If anyone is still reading this today and is curious where I ended up, I just took a job with Sentience Institute as a Strategy Lead & Researcher.
Cost is one factor, but nuclear also has other advantages such as land use, amount of raw material required (to make the renewables and lithium etc. for battery storage), and benefits for power grid.
It’s nice that renewables is getting cheaper, and I’d definitely like to see more renewables in the mix, but my ideal long term scenario is a mix of nuclear, renewables and battery. I’m weakly open to a small amount of gas being used for power generation in the long term in some cases.
Thanks for writing about this! I’ve thought about this as well, but there are a couple of reasons I haven’t done this yet. Primarily, I’ve been thinking more lately about making sure my time is appropriately valued. I’m still fairly early-mid career, and as much as it shouldn’t matter, taking a salary reduction now probably means reduced earnings potential in the future. This obviously matters less if you don’t plan on working for a non-highly impactful non-profit in the near future or if you’re later in your career, but I think this is worth thinking about even if you’re financially secure.
I think the sort of people who frequent this forum tend to be a bit too keen to work for not much because they think the work is so important, kind of like an ‘impact discount’, and I think historically effective non-profits have been a bit too keen to offer impact discount salaries. This seems to be less of an issue in the last 3 years in my experience, maybe because EA-aligned orgs are getting better funded.
I think taking impact discounts probably harm our impact in the longer term. It’s not just about the money per se, but about the perception of value. Unfortunately, people tend to see your hourly rate as a reflection of how much value you provide. Young people should probably care about this a bit more, but granted it’s a tough job market and it’s probably a much lower priority than just getting a job.
It also, of course, depends on whether you think your current employer is the most effective place to be sending your counterfactual dollars. It’s plausible that one might work at a highly impactful non-profit, but they think that another non-profit they could donate to is twice as effective (or whatever the ratio is based on their marginal tax rate, but it’s probably not more than double).