You can send me a message anonymously here: https://www.admonymous.co/will
WilliamKiely
Karma: 2,574
So I think the suggestion that there was some way to get to 65-90% certainty which apparently nobody was willing to either make with substantial evidence or cash in on to any significant extent is a pretty extraordinary claim...
I’m skeptical that nobody was rationally (i.e. not overconfidentally) at >65% belief Trump would win before election day. Presumably a lot of people holding Yes and buying Yes when Polymarket was at ~60% Trump believed Trump was >65% likey to win, right? And presumably a lot of them cashed in for a lot of money. What makes you think nobody was at >65% without being overconfident?
I’ll grant the French whale was overconfident, since it seems very plausible that he was overconfident, though I don’t know that for sure, but that doesn’t mean everyone >65% was overconfident.
I’ll also note that just because the market was at ~60% (or whatever precisely) does not mean that there could not have been people participating in the market who were significantly more confident yhat Trump would win and rationally-so.
The information that we gained between then and 1 week before the election was that the election remained close
I’m curious if by “remained close” you meant “remained close to 50/50″?
(The two are distinct, and I was guilty of pattern-matching “~50/50” to “close” even though ~50/50 could have meant that either Trump or Harris was likely to win by a lot (e.g. swing all 7 swing states) and we just had no idea which was more likely.)
Could you say more about “practically possible”?
Yeah. I said some about that in the ACX thread in an exchange with a Jeffrey Soreff here. Initially I was talking about a “maximally informed” forecaster/trader, but then when Jeffrey pointed out that that term was ill-defined, I realized that I had a lower-bar level of informed in mind that was more practically possible than some notions of “maximally informed.”
What steps do you think one could have taken to have reached, say, a 70% credence?
Basically just steps to become more informed and steps to have better judgment. (Saying specifically what knowledge would be sufficient to be able to form a forecast of 70% seems borderline impossible or at least extremely difficult.)
Before the election I was skeptical that people like Nate Silver and his team and The Economist’s election modeling team were actually doing as good a job as they could have been[1] forecasting who’d win the election and now post-election I still remain skeptical that their forecasts were close to being the best they could have been.
[1] “doing as good a job as they could have been” meaning I think they would have made substantially better forecasts in expectation (lower Brier scores in expectation) if figuring out who was going to win was really important to them (significantly more important than it actually was), and if they didn’t care about the blowback for being “wrong” if they made a confident wrong-side-of-maybe forecast, and if they were given a big budget to use to do research and acquire information (e.g. $10M), and if they were highly skilled forecasters with great judgment (like the best in the world but not superhuman (maybe Nate Silver is close to this—IDK; I read his book The Signal and the Noise, but it seems plausible that there could still be substantial room for him to improve his forecasting skill)).
Note that I also made five Manifold Markets questions to also help evaluate my PA election model (Harris and Trump means and SDs) and the claim that PA is ~35% likely to be decisive.
Will Pennsylvania be decisive in the 2024 Presidential Election?
How many votes will Donald Trump receive in Pennsylvania? (Set)
How many votes will Donald Trump receive in Pennsylvania? (Multiple Choice)
How many votes will Kamala Harris receive in Pennsylvania? (Set)
How many votes will Kamala Harris receive in Pennsylvania? (Multiple Choice)
(Note: I accidentally resolved my Harris questions (#4 & #5) to the range of 3,300,000-3,399,999 rather than 3,400,000-3,499,999. Hopefully the mods will unresolve and correct this for me per my comments on the questions.)
This exercise wasn’t too useful as there weren’t enough other people participating in the markets to significantly move the prices from my initial beliefs. But I suppose that’s evidence that they didn’t think I was significantly wrong.
Before the election I made a poll asking “How much would you pay (of your money, in USD) to increase the probability that Kamala Harris wins the 2024 Presidential Election by 0.0001% (i.e. 1⁄1,000,000 or 1-in-a-million)?”
You can see 12 answers from rationalist/EA people after submitting your answer to the poll or jumping straight to the results.
I think elections tend to have low aleatoric uncertainty, and that our uncertain forecasts are usually almost entirely due to high epistemic uncertainty. (The 2000 Presidential election may be an exception where aleatoric uncertainty is significant. Very close elections can have high aleatoric uncertainty.)
I think Trump was actually very likely to win the 2024 election as of a few days before the election, and we just didn’t know that.
Contra Scott Alexander, I think betting markets were priced too low, rather than too high. (See my (unfortunately verbose) comments on Scott’s post Congrats To Polymarket, But I Still Think They Were Mispriced.)
I think some people may have reduced their epistemic certainty significantly and had justified beliefs (not overconfident beliefs) that Trump was ~65-90% likely to win.
I totally am willing to believe that the French whale was not one of those people and actually just got lucky.
But I do think that becoming informed enough to rationally obtain a credence of >65% Trump was practically possible.
Thanks for writing up this post, @Eric Neyman . I’m just finding it now, but want to share some of my thoughts while they’re still fresh in my mind before next election season.
This means that one extra vote for Harris in Pennsylvania is worth 0.3 μH. Or put otherwise, the probability that she wins the election increases by 1 in 3.4 million
My independent estimate from the week before the election was that Harris getting one extra vote in PA would increase her chance of winning the presidential election by about 1 in 874,000.
My methodology was to forecast the number of votes that Harris and Trump would each receive in PA, calculate the probability of a tie in PA given my probability distributions for the number of votes they would each get, then multiply the probability of the PA tie by the probability that PA is decisive (conditional on a tie).
I used normal distributions to model the expected number of votes Harris and Trump would each get for simplicity so I could easily model the outcomes in Google Sheets (even though my credence/PDF did not perfectly match a normal distribution). These were my parameters:
Harris Mean Harris SD Trump Mean Trump SD 3450000 80000 3480000 90000 Simulating 10,000 elections in Google Sheets with these normal distributions found that about 654 elections per 10,000 were within 20,000 votes, which translates to a 1 in ~306,000 chance of PA being tied. I then multiplied this by a ~35%[1] chance that PA would be decisive (conditional on it being tied), to get a 1 in ~874,000 chance of an extra vote for Harris in PA changing who won overall.
99% of the votes in PA are in right now, with the totals currently at: 3,400,854 for Harris and 3,530,234 for Trump.
This means that the vote totals for Harris and Trump are both within 1 standard deviation of my mean expectation. Harris is about half an SD low and Trump was about half an SD high.From this, it’s not clear that my SDs of 80,000 votes for Harris and 90,000 votes for Trump were too narrow, as your (or Nate’s) model expected.
So I think my 1 in ~874,000 of an extra vote for Harris determining the president might have been more reasonable than your 1 in ~3.4 million.[1] Note: I mistakenly privately thought models and prediction markets might be wrong about the chance of PA being decisive, and thought that maybe it was closer to ~50% rather than ~25-35%, but the reason I thought this was bad and I didn’t realize until after the election: I made a simple pattern matching mistake by assuming “the election is a toss-up” meant “it will be a close election”. I failed to consider other possibilities like “the election will not be close, but we just don’t know which side will win by a lot.” (In retrospect, this was a very silly mistake for me to make, especially since I had seen that as of some late-October date The Economist said that the two most likely outcomes of the 128 swing-state combinations was 20% that Trump swings all seven and 7% that Harris swings all 7.)
(copying my comment from Jeff’s Facebook post of this)
I agree with this and didn’t add it (the orange diamond or 10%) anywhere when I first saw the suggestions/asks by GWWC to do so for largely the same reasons as you.I then added added it to my Manifold Markets *profile* (not name field) after seeing another user had done so. I didn’t know who the user was and didn’t know that they had any affiliation with effective giving or EA, and appreciated learning that, hence why I decided to do the same. I’m not committed to this at all and may remove it in the future. https://manifold.markets/WilliamKiely
I have not added it to my EA Forum name or profile. Everyone on the EA Forum knows that a lot of people there engage in effective giving, with a substantial percentage giving 10% or more. And unlike the Manifold Markets case where it was a pleasant surprise to learn that a person gives 10+% (where presumably a much lower percentage of people do than the EA Forum), I don’t particularly care to know whether a person on he EA Forum does effective giving or not. I also kind of want to avoid actively proselytizing for it on the EA Forum, since for a lot of people trying to save money and focus on finding direct work instead may be a more valuable path than giving a mere 10% of a typical income at the expense of saving faster.
I have not added it anywhere else besides my Manifold Markets profile as far as I recall.
These new systems are not (inherently) agents. So the classical threat scenario of Yudkowsky & Bostrom (the one I focused on in The Precipice) doesn’t directly apply. That’s a big deal.
It does look like people will be able to make powerful agents out of language models. But they don’t have to be in agent form, so it may be possible for first labs to make aligned non-agent AGI to help with safety measures for AI agents or for national or international governance to outlaw advanced AI agents, while still benefiting from advanced non-agent systems.
Unfortunately, the industry is already aiming to develop agent systems. Anthropic’s Claude noe has “computer use” capabilities, and I recall that Demis Hassabis has also stated that agentsbstr the next area to make big capability gains. I expect more bad news in this direction in the next five years.
Today, Miles Brundage published the following, referencing this post:
A few years ago, I argued to effective altruists (who are quite interested in how long it may take for certain AI capabilities to exist) that forecasting isn’t necessarily the best use of their time. Many of the policy actions that should be taken are fairly independent of the exact timeline. However, I have since changed my mind and think that most policymakers won’t act unless they perceive the situation as urgent, and insofar as that is actually the case or could be in the future, it needs to be credibly explained why that’s the case. See: COVID and the many warnings about pandemic preparedness before that.
That sounds great too. Perhaps both axes labels should be possible and it should just be specified for each question asked.
I like the idea of operationalizing the Agree/Disagree as probability that the statement is true. So “Agree” is 100%, neutral is 50%, disagree is 0%. In this case, 20% vs 40% means something concrete.
How does marginal spending on animal welfare and global health influence the long-term future?
I’d guess that most of the expected impact in both cases comes from the futures in which Earth-originating intelligent life (E-OIL) avoids near-term existential catastrophe and goes on to create a vast amount of value in the universe by creating a much larger economy and colonizing other galaxies and solar systems, and transforming the matter there into stuff that matters a lot more morally than lifeless matter (“big futures”).
For animal welfare spending, then, perhaps most of the expected impact come from the spending reducing the amount of animal suffering and suffering of other non-human sentient beings (e.g. future AIs) in the universe compared to the big futures without the late 2020s animal welfare spending. Perhaps the causal pathway for this is affecting what people think about the moral value of animal suffering and that positively affecting what E-OIL does with the reachable universe in big futures (less animal suffering and lower probability of neglecting the importance of sentient AI moral patients).
For global health spending, perhaps most of the expected impact comes from increasing the probability that E-OIL goes on to have a big future. Assuming the big futures are net positive (as I think is likely) this would be a good thing.
I think some global health spending probably has much more of an impact on this than others. For example, $100M would only put a dent in annual malaria deaths (~20,000 fewer deaths, a <5% reduction in annual deaths for 1 year) and it seems like that would have quite a small effect on existential risk. Whereas it seems plausible to me that if the money was spent on reducing the probability of a severe global pandemic in the 2030s (spending which seems like it could qualify as “global health” spending) plausibly could have a much more significant effect. I don’t know how much $100M could reduce the odds of a global pandemic in the 2030s, but intuitively I’d guess that it could make enough of a difference to be much more impactful on reducing 21st century existential risk than reducing malaria deaths.
How would the best “global health” spending compare to the “animal welfare” spending? Could it reduce existential risk by enough to do more good than better values achieved via animal welfare spending could do?
I think it plausibly could (i.e. the global health spending plausibly could do much more good), especially in the best futures in which it turns out that AI does our moral philosophy really well such that our current values don’t get locked in, but rather we figure out fantastic moral values after e.g. a long reflection and terraform the reachable universe based on those values.
But I think that in expectation, $100M of the global health spending would only reduce existential risk by a small amount, increasing the EV of the future by a small amount (something like <0.001%), and intuitively $100M extra spent on animal welfare (given the relatively small size of current spending on animal welfare), could do a lot more good (to increase the value of the big future by a larger small amount (than the small amount of increased probability of a big future from the global health scenario).
Initially I was answering about halfway toward Agree from Neutral, but after thinking this out, I’m moving further toward Agree.
This is horrifying! A friend of the author just shared this along with a Business Insider post that was just published that links to this post:
I’m curious if you or the other past participants you know had a good experience with AISC are in a position to help fill the funding gap AISC currently has. Even if you (collectively) can’t fully fund the gap, I’d see that as a pretty strong signal that AISC is worth funding. Or, if you do donate but you prefer other giving opportunities instead (whether in AIS or other cause areas) I’d find that valuable to know too.
But on the other hand, I’ve regularly meet alumni who tell me how useful AISC was for them, which convinces me AISC is clearly very net positive.
Naive question, but does AISC have enough of such past alumni that you could meet your current funding need by asking them for support? It seems like they’d be in the best position to evaluate the program and know that it’s worth funding.
Nevertheless, AISC is probably about ~50x cheaper than MATS
~50x is a big difference, and I notice the post says:
We commissioned Arb Research to do an impact assessment.
One preliminary result is that AISC creates one new AI safety researcher per around $12k-$30k USD of funding.Multiplying that number (which I’m agnostic about) by 50 gives $600k-$1.5M USD. Does your ~50x still seem accurate in light of this?
I’m a big fan of OpenPhil/GiveWell popularizing longtermist-relevant facts via sponsoring popular YouTube channels like Kurzgesagt (21M subscribers). That said, I just watched two of their videos and found a mistake in one[1] and took issue with the script-writing in the other one (not sure how best to give feedback—do I need to become a Patreon supporter or something?):
Why Aliens Might Already Be On Their Way To UsMy comment:
9:40 “If we really are early, we have an incredible opportunity to mold *thousands* or *even millions* of planets according to our visions and dreams.”—Why understate this? Kurzgesagt already made a video imagining humanity colonizing the Milky Way Galaxy to create a future of “a tredecillion potential lives” (10^42 people), so why not say ‘hundreds of billions of planets’ (the number of planets in the Milky Way), ‘or even more if we colonize other galaxies before other loud/grabby aliens reach them first’? This also seems inaccurate because the chance that we colonize between 1,000-9,999,999 planets (or even 1,000-9,999,999 planets) is less than the probability that we colonize >10 million (or even >1 billion) planets.
As an aside, the reason I watched these two videos just now was because I was inspired to look them up after watching the depressing new Veritasium video Do People Understand the Scale of the Universe? in which he shows a bunch of college students from a university with 66th percentile average SAT scores who do not know basic facts about the universe.
[1] The mistake I found was in the most recent video You Are The Center of The Universe (Literally) was that it said (9:10) that the diameter of the observable universe is 465,000 Milky Way galaxies side-by-side, but that’s actually the radius of the observable universe, not the diameter.
I also had a similar experience making my first substantial donation before learning about non-employer counterfactual donation matches that existed.
It was the only donation I regretted since by delaying making it 6 months I could have doubled the amount of money I directed to the charity for no extra cost to me.
I wasn’t a particularly informed forecaster, so me not telling you what information would have been sufficient to justify a rational 65+% confidence in Trump winning shouldn’t be much evidence to you about the practicality of a very informed person reaching 65+% credence rationally. Identifying what information would have been sufficient is a very time-intensive, costly project, and given I hadn’t done it already I wasn’t about to go spend months researching the data that people in principle had access to that might have led to a >65% forecast just to answer your question.
Prior to the election, I had an inside view credence of 65% that Trump would win, but considered myself relatively uninformed and so I meta-updated on election models and betting market prices to be more uncertain, making my all-things-considered view closer to 50⁄50. As I wrote on November 4th:
So I held this suspicion before the election, and I hold it still. I think it’s likely that such forecasters with rational credences of 65+% Trump victory did exist, and even if they didn’t, I think it’s possible that they could have existed if more people cared more about finding out the truth of who would win.