You can send me a message anonymously here: https://www.admonymous.co/will
WilliamKiely
Karma: 2,591
I endorse this for non-EA vegans who aren’t willing to donate the money to wherever it will do the most good in general, but as my other comments have pointed out if a person (vegan or non-vegan) is willing to donate the money to wherever it will so the most good then they should just do that rather than donate it for the purpose of offsetting.
Per my top-level comment citing Claire Zabel’s post Ethical offsetting is antithetical to EA, offsetting past consumption seems worse than just donating that money to wherever it will do the most good in general.
I see you’ve taken the 10% Pledge, so I gather you’re willing to donate effectively.
While you might feel better if you both donate X% to wherever you believe it will do the most good and $Y to the best animal charities to offset your past animal consumption, I think you instead ought to just donate X%+$Y to wherever it will do the most good.
NB: Maybe you happen to think the best giving opportunity to help animals is the best giving opportunity in general, but if not then my claim is that your offsetting behavior is a mistake.
This seems like a useful fundraising tool to target people who are unwilling to give their money to wherever it will do the most good, but I think it should be flagged that if a person is willing to donate their money to wherever it will do the most good then they should do that rather than donate to the best animal giving opportunities for the purpose of ethical offsetting. See Ethical offsetting is antithetical to EA.
FYI you can edit your original comment to add this in.
I’m now over 20 minutes in and haven’t quite figured out what you’re looking for. Just to dump my thoughts—not necessarily looking for a response:
On the one hand it says “Our goal is to discover creative ways to use AI for Fermi estimation” but on the other hand it says “AI tools to generate said estimates aren’t required, but we expect them to help.”
From the Evaluation Rubric, “model quality” is only 20%, so it seems like the primary goal is neither to create a good “model” (which I understand to mean a particular method for making a Fermi estimate on a particular question) nor to see if AI tools can be used to create such models.
The largest score (40%) is whether the *result* of the model that is created (i.e. the actual estimate that the model spits out with the numbers put into it) is surprising or not, with more surprising being better. But it’s unclear to me if the estimate actually needs to be believed or not for it to be surprising. Extreme numbers could just mean that the output is bad or wrong and not that the output should be evidence of anything.
I don’t see any particular reason to believe the means to obtain that knowledge existed and was used when you can’t tell me what that might look like, never mind how a small number of apparently resource-poor people obtained it…
I wasn’t a particularly informed forecaster, so me not telling you what information would have been sufficient to justify a rational 65+% confidence in Trump winning shouldn’t be much evidence to you about the practicality of a very informed person reaching 65+% credence rationally. Identifying what information would have been sufficient is a very time-intensive, costly project, and given I hadn’t done it already I wasn’t about to go spend months researching the data that people in principle had access to that might have led to a >65% forecast just to answer your question.
Prior to the election, I had an inside view credence of 65% that Trump would win, but considered myself relatively uninformed and so I meta-updated on election models and betting market prices to be more uncertain, making my all-things-considered view closer to 50⁄50. As I wrote on November 4th:My 2⁄10 low information inside view judgment is that Trump is about 65% likely to win PA and the election. My all-things-considered view is basically 50%.
However, notably, after about 10 hours of thinking about who will win in the last week, I don’t know if I actually trust Nate and prediction markets to be doing a good job. I suspect that there may be well-informed people in the world who *know* that the markets are wrong and have justified “true” beliefs that one candidate is >65% likely to win. Such people presumably have a lot of money on the line, but not enough to more [sic] the market prices far from 50%.So I held this suspicion before the election, and I hold it still. I think it’s likely that such forecasters with rational credences of 65+% Trump victory did exist, and even if they didn’t, I think it’s possible that they could have existed if more people cared more about finding out the truth of who would win.
So I think the suggestion that there was some way to get to 65-90% certainty which apparently nobody was willing to either make with substantial evidence or cash in on to any significant extent is a pretty extraordinary claim...
I’m skeptical that nobody was rationally (i.e. not overconfidentally) at >65% belief Trump would win before election day. Presumably a lot of people holding Yes and buying Yes when Polymarket was at ~60% Trump believed Trump was >65% likey to win, right? And presumably a lot of them cashed in for a lot of money. What makes you think nobody was at >65% without being overconfident?
I’ll grant the French whale was overconfident, since it seems very plausible that he was overconfident, though I don’t know that for sure, but that doesn’t mean everyone >65% was overconfident.
I’ll also note that just because the market was at ~60% (or whatever precisely) does not mean that there could not have been people participating in the market who were significantly more confident yhat Trump would win and rationally-so.
The information that we gained between then and 1 week before the election was that the election remained close
I’m curious if by “remained close” you meant “remained close to 50/50″?
(The two are distinct, and I was guilty of pattern-matching “~50/50” to “close” even though ~50/50 could have meant that either Trump or Harris was likely to win by a lot (e.g. swing all 7 swing states) and we just had no idea which was more likely.)
Could you say more about “practically possible”?
Yeah. I said some about that in the ACX thread in an exchange with a Jeffrey Soreff here. Initially I was talking about a “maximally informed” forecaster/trader, but then when Jeffrey pointed out that that term was ill-defined, I realized that I had a lower-bar level of informed in mind that was more practically possible than some notions of “maximally informed.”
What steps do you think one could have taken to have reached, say, a 70% credence?
Basically just steps to become more informed and steps to have better judgment. (Saying specifically what knowledge would be sufficient to be able to form a forecast of 70% seems borderline impossible or at least extremely difficult.)
Before the election I was skeptical that people like Nate Silver and his team and The Economist’s election modeling team were actually doing as good a job as they could have been[1] forecasting who’d win the election and now post-election I still remain skeptical that their forecasts were close to being the best they could have been.
[1] “doing as good a job as they could have been” meaning I think they would have made substantially better forecasts in expectation (lower Brier scores in expectation) if figuring out who was going to win was really important to them (significantly more important than it actually was), and if they didn’t care about the blowback for being “wrong” if they made a confident wrong-side-of-maybe forecast, and if they were given a big budget to use to do research and acquire information (e.g. $10M), and if they were highly skilled forecasters with great judgment (like the best in the world but not superhuman (maybe Nate Silver is close to this—IDK; I read his book The Signal and the Noise, but it seems plausible that there could still be substantial room for him to improve his forecasting skill)).
Note that I also made five Manifold Markets questions to also help evaluate my PA election model (Harris and Trump means and SDs) and the claim that PA is ~35% likely to be decisive.
Will Pennsylvania be decisive in the 2024 Presidential Election?
How many votes will Donald Trump receive in Pennsylvania? (Set)
How many votes will Donald Trump receive in Pennsylvania? (Multiple Choice)
How many votes will Kamala Harris receive in Pennsylvania? (Set)
How many votes will Kamala Harris receive in Pennsylvania? (Multiple Choice)
(Note: I accidentally resolved my Harris questions (#4 & #5) to the range of 3,300,000-3,399,999 rather than 3,400,000-3,499,999. Hopefully the mods will unresolve and correct this for me per my comments on the questions.)
This exercise wasn’t too useful as there weren’t enough other people participating in the markets to significantly move the prices from my initial beliefs. But I suppose that’s evidence that they didn’t think I was significantly wrong.
Before the election I made a poll asking “How much would you pay (of your money, in USD) to increase the probability that Kamala Harris wins the 2024 Presidential Election by 0.0001% (i.e. 1⁄1,000,000 or 1-in-a-million)?”
You can see 12 answers from rationalist/EA people after submitting your answer to the poll or jumping straight to the results.
I think elections tend to have low aleatoric uncertainty, and that our uncertain forecasts are usually almost entirely due to high epistemic uncertainty. (The 2000 Presidential election may be an exception where aleatoric uncertainty is significant. Very close elections can have high aleatoric uncertainty.)
I think Trump was actually very likely to win the 2024 election as of a few days before the election, and we just didn’t know that.
Contra Scott Alexander, I think betting markets were priced too low, rather than too high. (See my (unfortunately verbose) comments on Scott’s post Congrats To Polymarket, But I Still Think They Were Mispriced.)
I think some people may have reduced their epistemic certainty significantly and had justified beliefs (not overconfident beliefs) that Trump was ~65-90% likely to win.
I totally am willing to believe that the French whale was not one of those people and actually just got lucky.
But I do think that becoming informed enough to rationally obtain a credence of >65% Trump was practically possible.
Thanks for writing up this post, @Eric Neyman . I’m just finding it now, but want to share some of my thoughts while they’re still fresh in my mind before next election season.
This means that one extra vote for Harris in Pennsylvania is worth 0.3 μH. Or put otherwise, the probability that she wins the election increases by 1 in 3.4 million
My independent estimate from the week before the election was that Harris getting one extra vote in PA would increase her chance of winning the presidential election by about 1 in 874,000.
My methodology was to forecast the number of votes that Harris and Trump would each receive in PA, calculate the probability of a tie in PA given my probability distributions for the number of votes they would each get, then multiply the probability of the PA tie by the probability that PA is decisive (conditional on a tie).
I used normal distributions to model the expected number of votes Harris and Trump would each get for simplicity so I could easily model the outcomes in Google Sheets (even though my credence/PDF did not perfectly match a normal distribution). These were my parameters:
Harris Mean Harris SD Trump Mean Trump SD 3450000 80000 3480000 90000 Simulating 10,000 elections in Google Sheets with these normal distributions found that about 654 elections per 10,000 were within 20,000 votes, which translates to a 1 in ~306,000 chance of PA being tied. I then multiplied this by a ~35%[1] chance that PA would be decisive (conditional on it being tied), to get a 1 in ~874,000 chance of an extra vote for Harris in PA changing who won overall.
99% of the votes in PA are in right now, with the totals currently at: 3,400,854 for Harris and 3,530,234 for Trump.
This means that the vote totals for Harris and Trump are both within 1 standard deviation of my mean expectation. Harris is about half an SD low and Trump was about half an SD high.From this, it’s not clear that my SDs of 80,000 votes for Harris and 90,000 votes for Trump were too narrow, as your (or Nate’s) model expected.
So I think my 1 in ~874,000 of an extra vote for Harris determining the president might have been more reasonable than your 1 in ~3.4 million.[1] Note: I mistakenly privately thought models and prediction markets might be wrong about the chance of PA being decisive, and thought that maybe it was closer to ~50% rather than ~25-35%, but the reason I thought this was bad and I didn’t realize until after the election: I made a simple pattern matching mistake by assuming “the election is a toss-up” meant “it will be a close election”. I failed to consider other possibilities like “the election will not be close, but we just don’t know which side will win by a lot.” (In retrospect, this was a very silly mistake for me to make, especially since I had seen that as of some late-October date The Economist said that the two most likely outcomes of the 128 swing-state combinations was 20% that Trump swings all seven and 7% that Harris swings all 7.)
(copying my comment from Jeff’s Facebook post of this)
I agree with this and didn’t add it (the orange diamond or 10%) anywhere when I first saw the suggestions/asks by GWWC to do so for largely the same reasons as you.I then added added it to my Manifold Markets *profile* (not name field) after seeing another user had done so. I didn’t know who the user was and didn’t know that they had any affiliation with effective giving or EA, and appreciated learning that, hence why I decided to do the same. I’m not committed to this at all and may remove it in the future. https://manifold.markets/WilliamKiely
I have not added it to my EA Forum name or profile. Everyone on the EA Forum knows that a lot of people there engage in effective giving, with a substantial percentage giving 10% or more. And unlike the Manifold Markets case where it was a pleasant surprise to learn that a person gives 10+% (where presumably a much lower percentage of people do than the EA Forum), I don’t particularly care to know whether a person on he EA Forum does effective giving or not. I also kind of want to avoid actively proselytizing for it on the EA Forum, since for a lot of people trying to save money and focus on finding direct work instead may be a more valuable path than giving a mere 10% of a typical income at the expense of saving faster.
I have not added it anywhere else besides my Manifold Markets profile as far as I recall.
These new systems are not (inherently) agents. So the classical threat scenario of Yudkowsky & Bostrom (the one I focused on in The Precipice) doesn’t directly apply. That’s a big deal.
It does look like people will be able to make powerful agents out of language models. But they don’t have to be in agent form, so it may be possible for first labs to make aligned non-agent AGI to help with safety measures for AI agents or for national or international governance to outlaw advanced AI agents, while still benefiting from advanced non-agent systems.
Unfortunately, the industry is already aiming to develop agent systems. Anthropic’s Claude noe has “computer use” capabilities, and I recall that Demis Hassabis has also stated that agentsbstr the next area to make big capability gains. I expect more bad news in this direction in the next five years.
Today, Miles Brundage published the following, referencing this post:
A few years ago, I argued to effective altruists (who are quite interested in how long it may take for certain AI capabilities to exist) that forecasting isn’t necessarily the best use of their time. Many of the policy actions that should be taken are fairly independent of the exact timeline. However, I have since changed my mind and think that most policymakers won’t act unless they perceive the situation as urgent, and insofar as that is actually the case or could be in the future, it needs to be credibly explained why that’s the case. See: COVID and the many warnings about pandemic preparedness before that.
That sounds great too. Perhaps both axes labels should be possible and it should just be specified for each question asked.
I like the idea of operationalizing the Agree/Disagree as probability that the statement is true. So “Agree” is 100%, neutral is 50%, disagree is 0%. In this case, 20% vs 40% means something concrete.
How does marginal spending on animal welfare and global health influence the long-term future?
I’d guess that most of the expected impact in both cases comes from the futures in which Earth-originating intelligent life (E-OIL) avoids near-term existential catastrophe and goes on to create a vast amount of value in the universe by creating a much larger economy and colonizing other galaxies and solar systems, and transforming the matter there into stuff that matters a lot more morally than lifeless matter (“big futures”).
For animal welfare spending, then, perhaps most of the expected impact come from the spending reducing the amount of animal suffering and suffering of other non-human sentient beings (e.g. future AIs) in the universe compared to the big futures without the late 2020s animal welfare spending. Perhaps the causal pathway for this is affecting what people think about the moral value of animal suffering and that positively affecting what E-OIL does with the reachable universe in big futures (less animal suffering and lower probability of neglecting the importance of sentient AI moral patients).
For global health spending, perhaps most of the expected impact comes from increasing the probability that E-OIL goes on to have a big future. Assuming the big futures are net positive (as I think is likely) this would be a good thing.
I think some global health spending probably has much more of an impact on this than others. For example, $100M would only put a dent in annual malaria deaths (~20,000 fewer deaths, a <5% reduction in annual deaths for 1 year) and it seems like that would have quite a small effect on existential risk. Whereas it seems plausible to me that if the money was spent on reducing the probability of a severe global pandemic in the 2030s (spending which seems like it could qualify as “global health” spending) plausibly could have a much more significant effect. I don’t know how much $100M could reduce the odds of a global pandemic in the 2030s, but intuitively I’d guess that it could make enough of a difference to be much more impactful on reducing 21st century existential risk than reducing malaria deaths.
How would the best “global health” spending compare to the “animal welfare” spending? Could it reduce existential risk by enough to do more good than better values achieved via animal welfare spending could do?
I think it plausibly could (i.e. the global health spending plausibly could do much more good), especially in the best futures in which it turns out that AI does our moral philosophy really well such that our current values don’t get locked in, but rather we figure out fantastic moral values after e.g. a long reflection and terraform the reachable universe based on those values.
But I think that in expectation, $100M of the global health spending would only reduce existential risk by a small amount, increasing the EV of the future by a small amount (something like <0.001%), and intuitively $100M extra spent on animal welfare (given the relatively small size of current spending on animal welfare), could do a lot more good (to increase the value of the big future by a larger small amount (than the small amount of increased probability of a big future from the global health scenario).
Initially I was answering about halfway toward Agree from Neutral, but after thinking this out, I’m moving further toward Agree.
Footnote 2 completely changes the meaning of the statement from common sense interpretations of the statement. It makes it so that e.g. a future scenario in which AI takes over and causes existential catastrophe and the extinction of biological humans this century does not count as extinction, so long as the AI continues to exist. As such, I chose to ignore it with my “fairly strongly agree” answer.