It might be helpful to have a short section near the topin which you discuss significant sources of uncertainty. The top-line conclusion makes it clear to me that your analysis indicates that this is not an intervention we should be prioritizing at this time. However, listing factual and other assumptions that could significantly change the results in one place does two helpful things.
First, if the reader thinks some of those assumptions may be incorrect, listing the uncertainties upfront invites the reader to keep reading rather than just accept the top-line conclusion of 0.04x GiveWell efficiency and conclude that it isn’t worth investing 20 minutes in the rest of the report given that 0.04x isn’t close to justifying further review. Second, your report becomes part of the EA body of knowledge, and hopefully someone will be conducting a review of past work every few years to see if something has changed. That person may not be you, and it would be helpful for them to see, at a glance, what inputs and assumptions you think have significant uncertainty and importance. Those are the inputs and assumptions that should be re-examined every few years to decide if this should go to a deeper analysis due to changed circumstances.
Footnotes or cross-references can save the reader time spent on details that they may not find helpful. For instance, you explain that you “discount for the probability of the world being destroyed anyway (i.e. general existential risk discount)” but don’t mention what the discount is until the end of a very long paragraph. I would suggest stating in a short sentence that you are applying a general existential risk discount of 0.07 percent per annum and referring the reader to a footnote (or link to your website) for details.
At the shallow stage of research, a 0.07 percent adjustment is basically noise—understanding the details of why you chose that specific figure is unlikely to help any potential user of your report, and if there are exceptions they can read the footnote or cross-reference. Moreover, anyone who already thinks existential risk is much higher is not going to be convinced by a one-paragraph analysis to the contrary (and the adjustment is already near-irrelevant to anyone who thinks you estimated too high).
Again, the approach could depend on who the target audience is—and in some cases, you might decide it isn’t worth expending more time editing the report. But I thought these comments were worth sharing for future possible reference.
Two more suggestions for future reference:
It might be helpful to have a short section near the topin which you discuss significant sources of uncertainty. The top-line conclusion makes it clear to me that your analysis indicates that this is not an intervention we should be prioritizing at this time. However, listing factual and other assumptions that could significantly change the results in one place does two helpful things.
First, if the reader thinks some of those assumptions may be incorrect, listing the uncertainties upfront invites the reader to keep reading rather than just accept the top-line conclusion of 0.04x GiveWell efficiency and conclude that it isn’t worth investing 20 minutes in the rest of the report given that 0.04x isn’t close to justifying further review. Second, your report becomes part of the EA body of knowledge, and hopefully someone will be conducting a review of past work every few years to see if something has changed. That person may not be you, and it would be helpful for them to see, at a glance, what inputs and assumptions you think have significant uncertainty and importance. Those are the inputs and assumptions that should be re-examined every few years to decide if this should go to a deeper analysis due to changed circumstances.
Footnotes or cross-references can save the reader time spent on details that they may not find helpful. For instance, you explain that you “discount for the probability of the world being destroyed anyway (i.e. general existential risk discount)” but don’t mention what the discount is until the end of a very long paragraph. I would suggest stating in a short sentence that you are applying a general existential risk discount of 0.07 percent per annum and referring the reader to a footnote (or link to your website) for details.
At the shallow stage of research, a 0.07 percent adjustment is basically noise—understanding the details of why you chose that specific figure is unlikely to help any potential user of your report, and if there are exceptions they can read the footnote or cross-reference. Moreover, anyone who already thinks existential risk is much higher is not going to be convinced by a one-paragraph analysis to the contrary (and the adjustment is already near-irrelevant to anyone who thinks you estimated too high).
Again, the approach could depend on who the target audience is—and in some cases, you might decide it isn’t worth expending more time editing the report. But I thought these comments were worth sharing for future possible reference.