Mid-career climate science researcher in academia
Previously used display name “Pagw”
ClimateDoc
Karma: 566
I don’t really follow why one set of entities getting AGI and not sharing it should necessarily lead to widespread destitution.
Suppose A, B and C are currently working and trading between each other. A develops AGI and leaves B and C to themselves. Would B and C now just starve? Why would that necessarily happen? If they are still able to work as before, they can do that and trade with each other. They would become a bit poorer due to needing to replace the goods that A had a comparative advantage in producing I guess.
For B and C to be made destitute directly, it would seem to require that they are prevented at working at anything like their previous productivity eg if A were providing something essential and irreplaceable for B and C (maybe software products if A is techy?) or if A’s AGI went and pushed B and C off a large fraction of natural resources. It doesn’t seem very likely to me that B and C couldn’t mostly replace what A provided (eg with current open-source software). For A to push B and C off a large enough amount of resources, when the AGI has presumably already made A very rich, would require A to be more selfish and cruel than I hope is likely—but it’s unfortunately not unthinkable.
Of course there would probably still be hugely more inequality—but that doesn’t imply B and C are destitute.
I could imagine there being indirect large harms on B and C if their drop in productivity were large enough to create a depression, with financial system feedbacks amplifying the effects.
In any case, the picture you paint seems to require an additional reason that B and C cannot produce the things they need for themselves.
Are there roles in your current organisation that you think would be more enjoyable and could move into, say more at the level of making direct contributions?
Also, have you very thoroughly thought through the risks of retiring on $700k? I’ve seen in various discussions that it’s common for people to think that a 4% withdrawal rate is likely sustainable to enable early retirement with low risk, but there are various reasons why that’s probably optimistic, so just thought I’d flag it in case that’s what this is based on. Maybe it’s not...
My understanding of these “reasoning” approaches is that they seem to work very well on problems where there is a well-defined correct answer, and where that can be automatically verified. And it seems reasonable to expect much progress in that area.
What is the thinking of how much of human reasoning work is to do with problems like these?
As a counter-example, in my own particular work on climate prediction, we do not get rapid feedback about what works well, and it is contested what methods and frameworks we should even use i.e. it’s not possible presently to say “getting a good answer just requires solving [list of well-defined problems]” (except making computers so fast that we can do pretty much exact simulations of physics). So it doesn’t seem clear to me that these reasoning models will get a lot better at that kind of thing. But this is perhaps towards the far end of the spectrum of complex problems.
I can see these reasoning models becoming very good at things like writing code where requirements to be met can be precisely specified and automatically verified, and improving performance of devices (such as computer chips) according to well-specified benchmarks. How much difference would it make to make fast progress on problems similar to these?
There doesn’t look to me to be a reason to think that systems trained this way will yield impressive performance at solving messier problems without clear right answers, like predicting complex systems (that can’t be observed experimentally or simulated very well), selecting amongst decision options with different strengths on multiple criteria, dealing with organisational politics etc. Does that seem fair?
These are genuine questions—I don’t feel I have a good grasp of what kinds of work most of our economy is engaged in...
It’s not clear to me why the aim ought to be to sample randomly amongst all people—it seems like a different population could reasonably be chosen!
Sounds interesting. I had a go at the tool, but was a bit perplexed that the “lottery story” it showed me was for a Romanian earning $2,500/month, which doesn’t seem like the kind of life that people’s attention needs to be most drawn to or represents people that would be helped by effective development charities (it even says this person is at the 86th percentile of global income). And then below that it talked about ending hunger, eradicating disease etc., which didn’t relate to the story. I’d focus it on stories about the kinds of people that effective charities would actually help. I tried to get it to generate another story to see what else comes up, but it wouldn’t.
I guess it’s hard to know without being in Mill’s head. Though from what I’ve read it doesn’t sound like he ever really wavered from favouring Britain having India as a colony.
I think this is an interesting analysis, but as others have indicated it could be better to frame this in terms of something like how these potential harms from saving human lives could be offset by donations to animal welfare charities, say.
Well, everyone will have their own emotional journey—not everyone with motivations to do good will have an experience like Mill’s! But the point to not make improving social welfare the sole target and to have alternative sources of satisfaction seems to me quite common in discussions around EA and mental health, at least for those who do have difficulties.
I came across this extract from John Stuart Mill’s autobiography on his experience of a period when he became depressed and lost motivation in his goal of improving society. It sounded similar to what I hear from time to time of EAs finding it difficult to maintain motivation and happiness alongside altruism, and thought some choice quotes would be interesting to share. Mill’s solution was finding pleasure in other pursuits, particularly poetry.
Mill writes that his episode started in 1826, when he was 20 years old—but he had already been a keen utilitarian for 5 years and had been working for 3 years by this time, so was perhaps at a development point that not many would reach before they were into their early careers in the modern day.
From the winter of 1821, when I first read Bentham...I had what might truly be called an object in life; to be a reformer of the world. My conception of my own happiness was entirely identified with this object...This did very well for several years
But the time came when I awakened from this as from a dream. It was in the autumn of 1826. I was in a dull state of nerves...unsusceptible to enjoyment or pleasurable excitement...In this frame of mind it occurred to me to put the question directly to myself: “Suppose that all your objects in life were realized; that all the changes in institutions and opinions which you are looking forward to, could be completely effected at this very instant: would this be a great joy and happiness to you?” And an irrepressible self-consciousness distinctly answered, “No!” At this my heart sank within me: the whole foundation on which my life was constructed fell down...I seemed to have nothing left to live for.
For some months the cloud seemed to grow thicker and thicker...I became persuaded, that my love of mankind, and of excellence for its own sake, had worn itself out.
I frequently asked myself, if I could, or if I was bound to go on living, when life must be passed in this manner. I generally answered to myself that I did not think I could possibly bear it beyond a year. When, however, not more than half that duration of time had elapsed, a small ray of light broke in upon my gloom. I was reading, accidentally, Marmontel’s Mémoires, and came to the passage which relates his father’s death...A vivid conception of the scene and its feelings came over me, and I was moved to tears. From this moment my burden grew lighter.
I gradually found that the ordinary incidents of life could again give me some pleasure...and that there was, once more, excitement, though of a moderate, kind, in exerting myself for my opinions, and for the public good. Thus the cloud gradually drew off, and I again enjoyed life; and though I had several relapses, some of which lasted many months, I never again was as miserable as I had been.
The experiences of this period...led me to adopt a theory of life, very unlike that on which I had before I acted...Those only are happy (I thought) who have their minds fixed on some object other than their own happiness...followed not as a means, but as itself an ideal end.
This state of my thoughts and feelings made the fact of my reading Wordsworth for the first time (in the autumn of 1828), an important event of my life...[his poems] proved to be the precise thing for my mental wants at that particular juncture.
The result was that I gradually, but completely, emerged from my habitual depression, and was never again subject to it
Whilst I salute the effort and progress here, this post does seem rather full of spin, given that from what I can tell the court ruling was against the animal advocates. I’d rather see posts that present the facts more clearly.
They seem to say so in their intro video on this page: https://extinctionrebellion.uk/the-truth/the-emergency/. OK they say due to climate and ecological destruction, but it doesn’t really matter for this. The point is just that disagreeing with experts doesn’t generally seem to prevent an organisation from becoming “successful”. (Plenty of examples outside climate too.)
It seems to be a big part in the UK cf Extinction Rebellion, Just Stop Oil.
One of the advantages of the climate protest movements is that they have a wealth of scientific work to point to for credibility.
Scientific work doesn’t give particular support for the idea that climate change will create a substantial extinction risk though, and that doesn’t stop the activists there. I’m not saying you’re wrong or the OP’s approach is justified, but public perceptions of activist groups’ reasonableness seems only loosely linked to expert views (I’ve not seen much evidence of the “then they can go on to check what experts think” bit happening much).
The Humane League is what comes to mind—searching for them in the forum may bring up recent estimates of their cost effectiveness—I don’t know offhand.
But I thought I’d also say sorry you weren’t offered a meal respecting your ethical choice—it seems like an extraordinary thing to happen today (depending on where you are in the world).
I thought I’d follow up on how I wrote a will leaving money to EA charities, following my previous question about it here. I ended up drafting a will myself and haven’t yet had it checked by a solicitor. I’ve gone down this route as I’m still youngish and so having some probability of the will failing does not seem like something worth spending hundreds of pounds to avoid at present—if I were 20 years older, I may have considered that worth it. For context I’m resident in England, and these steps are not necessarily good to follow in other countries—I can’t say.
My process was to firstly get a free draft will from www.freewills.co.uk . I copied the text to a document I could edit and set out the clauses specifying the distribution of the estate, using wording from https://www.givewell.org/legacy-giving and https://www.givingwhatwecan.org/get-involved/bequests, and including some extra wording about cause areas to allocate the bequests to, which I checked with GiveWell and GWWC beforehand. I also added sub-clauses about what to do if these charities no longer exist (just general instructions to say my Trustees should use their judgement and about how they could identify charities in line with my wishes, with a couple of examples—if GiveWell or GWWC did cease existing, I intend to update my will, so these clauses should only matter for a fairly narrow window of time and so super high-quality wording doesn’t seem needed).
Following this previous Forum post, I added the following clause after the clauses setting out the bequests as an extra failsafe:But if the trusts hereinbefore declared shall fail or determine then and in that event my Trustees shall stand possessed of the said residue of my estate UPON TRUST to transfer pay or apply the same to or for such exclusively charitable institution or purpose or exclusively charitable institutions or purposes and if more than one in such proportions as my Trustees may in their absolute discretion select.
Then I got it signed and witnessed. See https://www.gov.uk/make-will/make-sure-your-will-is-legal .
Keywords to aide searching, as searching for “will” brings up lots of other things!: testament, writing will, leave money to charity
Are there any good research articles that do a decent job of isolating the role of reducing mortality rates? Review articles would be particularly useful.
Here’s a link to the GiveWell-commissioned research that I have: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3635855 .
There was some Givewell-commissioned research that did find that saving lives likely leads to future population increases. I imagine there’s a fair amount of uncertainty, but it seemed to be the best information available at the time I was looking into this a few years ago. I could dig it up if it’s of interest and difficult to find.
Even if the current population isn’t consuming much factory-farmed meat, if it’s children’s lives being saved, the amount they consume over the next half century or so may be substantial as the countries develop and adopt more industrialised food production. Also, saving lives today seems likely to increase population in future (I recall a GiveWell-commissioned study on this), so potentially leading to greater factory-farmed meat consumption.
I came across this account of working as an IPCC author and drafting the SPM by a philosopher who was involved in the 5th IPCC report, which provides some insight: link to pdf—see from p.7. @jackva
I had a look, it seems to presume the AI-owners will control all the resources, but this doesn’t seem like a given (though it may pan out that way).
I realise you said you didn’t want to debate these assumptions, but just wanted to point out that the picture painted doesn’t seem inevitable.