Dan_Keys

Karma: 551

Dan_Keys 30 Jan 2015 18:20 UTC
3 points
0 ∶ 0
on: TLYCS Pamphleting Pilot Plan
Have you looked at the history of your 4 metrics (Visitors, Subscribers, Donors, Pledgers) to see how much noise there is in the baseline rates? The noisier they are, the more uncertainty you’ll have in the effect size of your intervention.

Could you have the pamphlets only give a url that no one else goes to, and then directly track how many new subscribers/donors/pledgers have been to that url?

Dan_Keys 19 Feb 2016 2:09 UTC
4 points
0 ∶ 0
on: The most persuasive writing neutrally surveys both sides of an argument
A quick search into the academic research on this topic roughly matches the claims in this post.

Meta-analyses by Allen (1991) (pdf, blog post summary) and O’Keefe (1999) (pdf, blog post summary) defined “refutational two-sided arguments” as arguments that include 1) arguments in favor of the preferred conclusion, 2) arguments against the preferred conclusion, and 3) arguments which attempt to refute the arguments against the preferred conclusion. Both meta-analyses found that refutational two-sided arguments were more persuasive than one-sided arguments (which include only the first of those 3 types of arguments), which in turn were more persuasive than nonrefutational two-sided arguments (which include the first 2 of those 3 types of arguments).

So: surveying both sides of the argument, and making the case for why one side holds more weight than the other, does seem to lead to more convincing writing.

These results are at a fairly broad level of generality. I don’t know if any research has looked at questions like whether it matters if you include the strongest arguments against the preferred conclusion (vs. only including straw man arguments) or if it matters if you act as if the arguments against the preferred conclusion have been completely refuted (vs. somewhat outweighed by the arguments in favor of the preferred conclusion).

A quick skim through the list of articles citing Allen and O’Keefe’s papers turned up some studies which look for additional sources of variability which might moderate this effect, but I didn’t notice any that challenge the general pattern or which get into really good detail on whether normatively good arguments (e.g., non-straw-man, measured conclusions) are more convincing.
What links here?
- The most persuasive writing neutrally surveys both sides of an argument by Robert_Wiblin (18 Feb 2016 8:42 UTC; 14 points)

Dan_Keys 20 Feb 2016 6:50 UTC
2 points
0 ∶ 0
in reply to: Jeff Kaufman’s comment on: Independent re-analysis of MFA veg ads RCT data
I can’t tell what’s being done in that calculation.

I’m getting a p-value of 0.108 from a Pearson chi-square test (with cell values 55, 809; 78, 856). A chi-square test and a two-tailed t-test should give very similar results with these data, so I agree with Michael that it looks like your p=0.053 comes from a one-tailed test.

Dan_Keys 26 Apr 2016 8:05 UTC
7 points
0 ∶ 0
on: How Should a Large Donor Prioritize Cause Areas?
My guess (which, like Michael’s, is based on speculation and not on actual information from relevant decision-makers) is that the founders of Open Phil thought about institutional philosophy before they looked in-depth at particular cause areas. They asked themselves questions like:

How can we create a Cause Agnostic Foundation, dedicated to directing money wherever it will do the most good, without having it collapse into a Foundation For Cause X as soon as its investigations conclude that currently the highest EV projects are in cause area x?

Do we want to create a Cause Agnostic Foundation? Would it be a bad thing if a Cause Agnostic Foundation quickly picked the best cause and then transformed into the Foundation For Cause X?

Apparently they concluded that it was worth creating a (stable) Cause Agnostic Foundation, and that this would work better if they directed significant amounts of resources towards several different cause areas. I can think of several arguments for this conclusion:
1. Spreading EA Ideas. It’s easier to spread the ideas behind effective altruism (and to create a world where more resources are devoted to attempts at effective altruism) if there is a prominent foundation which is known for the methodology that it uses to choose causes rather than for its support of particular causes. And that works best if the foundation gives to several different cause areas.
2. Diminishing Returns to Prestige. Donations can provide value by conferring prestige, not just by transferring money, and prestige can have sharply diminishing returns to amount donated. e.g., Giving to your alma mater, whether it’s $10 or $10,000, lets them say that a higher percentage of alumni are donors. One might hope that this prestige benefit (with diminishing returns) would apply to many of the grants from a Cause Agnostic Foundation, and that it will be well-regarded enough to bring other people’s attention to the causes & organizations that it supports.
3. Ability to Pivot. If a foundation focuses on just one or two cause areas (and hires people to work on those cause areas, publicizes its reasons for supporting those cause areas, builds connections with other organizations in those cause areas, etc.) that can make it hard for it to keep an open mind about cause areas and potentially pivot to a different cause area which starts looking more promising a few years later.
4. Learning. We can learn more if we pursue several different cause areas than if we just focus on one or two. This can include things like: getting better at cause prioritization by doing it a lot, getting better at evaluating organizations by dealing with some organizations that are in cause areas where progress is relatively easy to track, and learning how to interact with governments in the context of criminal justice reform and then being better able to pursue projects involving government in other cause areas.
5. Hits. A foundation which practices hits-based-giving can tolerate a lot of risk, but they may need to have at least some visible hits over the years in order to remain institutionally strong. Diversifying across cause areas can help that happen.
My sense is that this is an incomplete list; there are other arguments like these.

It’s worth noting that many of these lines of reasoning are specific to a foundation like Open Phil, and would not apply to a single wealthy donor looking to donate his or her own money.

Dan_Keys 30 Jul 2016 4:00 UTC
6 points
0 ∶ 0
on: The 2015 Survey of Effective Altruists: Results and Analysis
For the analysis of donations, which asked about donations in 2014, I’d like to see the numbers for people who became EAs in 2013 or earlier (including the breakdowns for non-students and for donations as % of income for those with income of $10,000 or more).

37% of respondents first got involved with EA in 2015, so their 2014 donations do not tell us much about the donation behavior of EAs. Another 24% first got involved with EA in 2014, and it’s unclear how much their 2014 donations tell us given that they only began to be involved in EA midyear.

Dan_Keys 30 Jul 2016 23:08 UTC
0 points
0 ∶ 0
in reply to: ChrisCundy’s comment on: The 2015 Survey of Effective Altruists: Results and Analysis
Thanks for looking this up quickly, and good point about the selection effect due to attrition.

I do think that it would be informative to see the numbers when also limited to nonstudents (or to people above a certain income, or to people above a certain age). I wouldn’t expect to see much donated from young low- (or no-) income students.

Dan_Keys 12 Sep 2016 23:17 UTC
1 point
0 ∶ 0
on: Altruistic Organizations Should Consider Counterfactuals When Hiring
If the prospective employee is an EA, then they are presumably already paying lots of attention to the question “How much good would I do in this job, compared with the amount of good I would do if I did something else instead?” And the prospective employee has better information than the employer about what that alternative would be and how much good it would do. So it’s not clear how much is added by having the employer also consider this.

Dan_Keys 5 Dec 2016 1:14 UTC
7 points
0 ∶ 0
in reply to: Ajeya’s comment on: Contra the Giving What We Can pledge
It appears that this analysis did not account for when people became EAs. It looked at donations in 2014, among people who in November 2015 were nonstudent EAs on an earning to give path. But less than half of those people were nonstudent EAs on an earning to give path at the start of 2014.

In fact, less than half of the people who took the Nov 2015 survey were EAs at the start of 2014. I’ve taken a look at the dataset, and among the 1171 EAs who answered the question about 2014 donations:
40% first got involved in EA in 2013 or earlier
21% first got involved in EA in 2014
28% first got involved in EA in 2015
11% did not answer the question about when they got involved in EA

This makes all of the analyses of median 2014 donation extremely misleading, unless they’re limited to pre-2014 EAs (which they generally have not been).

I’m hoping that the next EA survey will do better with this issue. I believe the plan is to wait until January in order to ask about 2016 donations, which is a good start. Hopefully they will also focus on pre-2016 EAs when looking at typical donation size, since the survey will include a bunch of new EAs who we wouldn’t necessarily expect to see donating within their first few months as an EA.

(Also speaking for myself only, not my employer.)
What links here?
- Dan_Keys's comment on EA Survey 2017 Series: Donation Data by Tee (12 Sep 2017 3:20 UTC; 5 points)

Dan_Keys 9 Aug 2017 17:31 UTC
1 point
0 ∶ 0
in reply to: MichaelPlant’s comment on: High Time For Drug Policy Reform. Part 1/4: Introduction and Cause Summary
https://delib.zendesk.com/hc/en-us/articles/205061169-Creating-footnotes-HTML-anchors

It might be possible to fix in a not-too-tedious way, by using find-replace in the source code to edit all of the broken links (and anchors?) at once.

Dan_Keys 30 Aug 2017 6:23 UTC
0 points
0 ∶ 0
on: EA Survey 2017 Series: Distribution and Analysis Methodology

This year, a “Donations Only” version of the survey was created for respondents who had filled out the survey in prior years. This version was shorter and could be linked to responses from prior years if the respondent provided the same email address each year.

Are these data from prior surveys included in the raw data file, for people who did the Donations Only version this year? At the bottom of the raw data file I see a bunch of entries which appear not to have any data besides income & donations—my guess is that those are either all the people who took the Donations Only version, or maybe just the ones who didn’t provide an email address that could link their responses.

Dan_Keys 12 Sep 2017 3:20 UTC
5 points
0 ∶ 0
on: EA Survey 2017 Series: Donation Data
It is also worth noting that the survey was asking people who identify as EA in 2017 how much they donated in 2015 and 2016. These people weren’t necessarily EAs in 2015 or 2016.

Looking at the raw data of when respondents said that they first became involved in EA, I’m getting that:

7% became EAs in 2017
28% became EAs in 2016
24% became EAs in 2015
41% became EAs in 2014 or earlier

(assuming that everyone who took the “Donations Only” survey became an EA before 2015, and leaving out everyone else who didn’t answer the question about when they became an EA.)

So if we’re looking at donations made in 2015, 35% of the people weren’t EAs then and another 24% had only just become EAs that year. For 2016, 35% of the people weren’t EAs yet at the start of the year and 7% weren’t EAs at the end of the year.

(There were similar issues with the 2015 survey.)

These not-yet-EAs can have a large influence on the median, and to a lesser extent on the percentiles and the mean. They would also tend to create an upward trend in the longitudinal analysis (e.g., if many of the 184 individuals became EAs in 2015).

Dan_Keys 12 Sep 2017 16:03 UTC
0 points
0 ∶ 0
in reply to: Peter Wildeford’s comment on: EA Survey 2017 Series: Donation Data
I agree that asking about 2016 donations in early 2017 is an improvement for this. If future surveys are just going to ask about one year of donations then that’s pretty much all you can do with the timing of the survey.

In the meantime, it is pretty easy to filter the data accordingly—if you look only at donations made by EAs who stated that they joined on 2014 or before, the median donation is $1280.20 for 2015 and $1500 for 2016.

This seems like a better way to do the analyses. I think that the post would be more informative & easier to interpret if all of the analyses used this kind of filter. (For 2016 donations you could also include people who became involved in EA in 2015.)

For example, someone who hears a number for the median non-student donation in 2016 will by default assume that this refers to people who were non-student EAs throughout 2016. If possible, it’s good to give the number which matches the scenario that they’re imagining rather than needing to give caveats about how 35% of the people weren’t EAs yet at the start of 2016. When people hear a non-intuitive analysis with a caveat then they’re fairly likely to either a) forget about the caveat and mistakenly think that the number refers to the thing that they initially assumed that it meant or b) not know what to make of the caveated analysis and therefore not learn anything.

Dan_Keys 26 Aug 2018 22:18 UTC
2 points
0 ∶ 0
in reply to: RandomEA’s comment on: Public Opinion about Existential Risk
That can be tested on these data, just by looking at the first of the 3 questions that each participant got, since the post says that “Participants were asked about the likelihood of humans going extinct in 50, 100, and 500 years (presented in a random order).”

I expect that there was a fair amount of scope insensitivity. e.g., That people who got the “probability of extinction within 50 years” question first gave larger answers to the other questions than people who got the “probability of extinction within 500 years” question first.

Dan_Keys 26 Aug 2018 22:30 UTC
6 points
0 ∶ 0
on: Public Opinion about Existential Risk
Unless a study is done with participants who are selected heavily for numeracy and fluency in probabilities, I would not interpret stated probabilities literally as a numerical representation of their beliefs, especially near the extremes of the scale. People are giving an answer that vaguely feels like it matches the degree of unlikeliness that they feel, but they don’t have that clear a sense of what (e.g.) a probability of ¹⁄₁₀₀ means. That’s why studies can get such drastically different answers depending on the response format, and why (I predict) effects like scope insensitivity are likely to show up.

I wouldn’t expect the confidence question to pick up on this. e.g., Suppose that experts think that something has a 1 in a million chance and a person basically agrees with the experts’ viewpoint but hasn’t heard/remembered that number. So they indicate “that’s very unlikely” by entering “1%” which feels like it’s basically the bottom of the scale. Then on the confidence question they say that they’re very confident of that answer because they feel sure that it’s very unlikely.

Dan_Keys 4 Jan 2020 4:22 UTC
3 points
0 ∶ 0
in reply to: JacobS’s comment on: [Link] Aiming for Moral Mediocrity | Eric Schwitzgebel
In most of his piece, by “aiming to be mediocre”, Schwitzgebel means that people’s behavior regresses to the actual moral middle of a reference class, even though they believe the moral middle is even lower.
This skirts close to a tautology. People’s average moral behavior equals people’s average moral behavior. The output that people’s moral processes actually produce is the observed distribution of moral behavior.
The “aiming” part of Schwitzgebel’s hypothesis that people aim for moral mediocrity gives it empirical content. It gets harder to pick out the empirical content when interpreting aim in the objective sense.

Dan_Keys 20 Feb 2020 0:49 UTC
6 points
0 ∶ 0
in reply to: Gordon Seidoh Worley’s comment on: The Web of Prevention
The initial post by Eliezer on security mindset explicitly cites Bruce Schneier as the source of the term, and quotes extensively from this piece by Schneier.

Dan_Keys 29 Sep 2020 19:37 UTC
12 points
0 ∶ 0
in reply to: Michael_Wiebe’s comment on: Michael_Wiebe’s Shortform
If humanity goes extinct this century, that drastically reduces the likelihood that there are humans in our solar system 1000 years from now. So at least in some cases, looking at the effects 1000+ years in the future is pretty straightforward (conditional on the effects over the coming decades).
In order to act for the benefit of the far future (1000+ years away), you don’t need to be able to track the far future effects of every possible action. You just need to find at least one course of action whose far future effects are sufficiently predictable to guide you (and good in expectation).

Dan_Keys 16 Oct 2020 2:58 UTC
4 points
0 ∶ 0
in reply to: Peter Wildeford’s comment on: Getting money out of politics and into charity
For example: If there are diminishing returns to campaign spending, then taking equal amounts of money away from both campaigns would help the side which has more money.

Dan_Keys 4 May 2021 1:22 UTC
2 points
0 ∶ 0
in reply to: dan.pandori’s comment on: The State of the World — and Why Monkeys are Smarter than You
I got ¹³⁄₁₃.
q11 (endangered species) was basically a guess. I thought that an extreme answer was more likely given how the quiz was set up to be counterintuitive/surprising. Also relevant: my sense is that we’ve done pretty well at protecting charismatic megafauna; the fact that I’ve heard about a particular species being at risk doesn’t provide much information either way about whether things have gotten worse for it (me hearing about it is related to things being bad for it, and it’s also related to successful efforts to protect it).
On q6 (age distribution of population increase) I figured that most people are age 15-74 and that group would increase roughly proportionally with the overall increase, which gives them the majority of the increase. The increase among the elderly will be disproportionately large, but that’s not enough for it to be the biggest in absolute terms since they’re only like 10% of the population.
On q7 (deaths from natural disaster) I wouldn’t have been surprised if the drop in death rate was balanced out by the increase in population, but I had an inkling that it was faster. And the tenor of the quiz was that the surprisingly good answer was correct, so if population growth had balanced it out then probably it would’ve asked about deaths per capita rather than total deaths.

Dan_Keys 1 Apr 2022 20:31 UTC
9 points
0 ∶ 0
on: Announcing What The Future Owes Us
How much overlap is there between this book & Singer’s forthcoming What We Owe The Past?