david_reinstein

Un­jour­nal is look­ing for feed­back & to mea­sure and en­hance our im­pact (sur­vey w/​ mod­est lot­tery prize)

• david_reinstein 4 Dec 2025 11:45 UTC

  2 points

  0 ∶ 0

  on: “How much do plant-based prod­ucts sub­sti­tute for an­i­mal prod­ucts and im­prove welfare?” – An Un­jour­nal Pivotal Ques­tion (up­date: added polls)

  Posted and curated some Metaculus questions in our community space here.

  This is preliminary: I’m looking for feedback and also hoping to post some of these as ‘Metaculus moderated’ (not ‘community’) questions. Also collaborating with Support Metaculus’ First Animal-Focused Forecasting Tournament.
  
  Questions like:
  
  Predict experts’ beliefs over “If the price of the… hamburger-imitating plant products fell by 10% … how many more/​fewer chickens consumed globally in 2030, as a +/​- percent?

• david_reinstein 4 Dec 2025 11:44 UTC

  2 points

  0 ∶ 0

  on: Is Cul­tured Meat Com­mer­cially Vi­able? Un­jour­nal’s first pro­posed ‘Pivotal Ques­tion’ (& re­quest for feed­back)

  Posted and curated some Metaculus questions in our community space here.

  This is preliminary: I’m looking for feedback and also hoping to post some of these as ‘Metaculus moderated’ (not ‘community’) questions. Also collaborating with Support Metaculus’ First Animal-Focused Forecasting Tournament.
  
  Specific questions include
  
  What will be the average production cost (per edible kg) of cell-cultured chicken meat at the end of the following years (between 2026-2051) across all large-scale plants in the world?
  
  and
  
  Predict the median of experts’ beliefs over “Does $100k to fund plant-based hamburger R&D have a greater animal welfare benefit than $100k to corporate animal welfare campaigns for chickens?”

• david_reinstein 4 Dec 2025 11:40 UTC

  4 points

  0 ∶ 0

  on: Sup­port Me­tac­u­lus’ First An­i­mal-Fo­cused Fore­cast­ing Tournament

  Unjournal.org is collaborating with this initiative for our Pivotal Questions projects:
  
  Is Cultured Meat Commercially Viable? Unjournal’s first proposed ‘Pivotal Question’ (& request for feedback) and
  
  “How much do plant-based products substitute for animal products and improve welfare?” – An Unjournal Pivotal Question (update: added polls)
  
  Aiming to integrate this with some of the questions in our community here
  
  Feedback on these questions and operationalizations is highly appreciated.

• david_reinstein 4 Dec 2025 11:30 UTC

  4 points

  0 ∶ 0

  on: The over­all cost-effec­tive­ness of an in­ter­ven­tion of­ten mat­ters less than the coun­ter­fac­tual use of its funding

  I made a similar argument a few years back, advocating that GiveWell should rank, rate, and measure charities beyond the absolute best/​most measurable.
  
  A common response was that the evidence suggested the returns were so heavy-tailed… So moving money from ~ineffective charities (Make a Wish) to ‘near-top’ charities, or to mainstream charities operating in similar areas (say MSF vs. AMF) would have far less value than moving money from near-top to top charities.
  
  My counter-response was … ~we don’t have solid that charities like MSF are much less effective than AMF, my prior is a less heavy-tail, and we should try to collect more evidence on ‘mainstream multi-intervention charities’.
  
  Not sure if a similar argument applies within the context of animal welfare charities – I’d guess so (within the context of farm animal welfare, say).

• david_reinstein 4 Dec 2025 11:20 UTC

  2 points

  0 ∶ 0

  on: Are there any EA hubs that are both warm and sunny?

  As others note, the East Bay/​Berkeley more or less hits the spot if you don’t care about affordability.

  (Would be nice if there were an affordable alternative though.)

• david_reinstein 4 Dec 2025 11:20 UTC

  2 points

  0 ∶ 0

  on: Are there any EA hubs that are both warm and sunny?

  As others note, the East Bay/​Berkeley more or less hits the spot if you don’t care about affordability. Would be nice if there were an affordable alternative though.

• david_reinstein 29 Nov 2025 18:01 UTC

  6 points

  0 ∶ 0

  in reply to: NickLaing’s comment on: Dona­tion Elec­tion Dis­cus­sion Thread

  It’s hard for me to glean what the consensus is in this thread/​on this issue. But if there seems to be a strong case that some outside scrutiny is needed, this might be something The Unjournal (Unjournal.org) could help with. Bringing “outside the EA bubble” academic expertise to weigh in is one of our key things

  We generally focus on economics and social science but we might be able to stretch to this. (Feel free to dm/​suggest/​ping me).

The Un­jour­nal: Bridg­ing the Ri­gor/​Im­pact Gaps for EA-rele­vant Re­search Questions

• david_reinstein 13 Nov 2025 19:22 UTC

  2 points

  0 ∶ 0

  on: Effec­tive Altru­ists Should Pro­mote Fru­gal­ity More

  I like the post and agree with most of it, but I don’t understand this point. Can you clarify? To me it seems like the opposite of this.

  If EA organizations are seen promoting frugality, their actions could be perceived as an example of the rich promoting their own interests over those of the poor. This would increase the view that EA is an elitist movement.

• david_reinstein 13 Nov 2025 19:20 UTC

  7 points

  0 ∶ 0

  on: How to start a 501(c)3 non­profit as quickly as possible

  A quick ~testimonial. Abraham’s advice was very helpful to us at Unjournal.org. As our fiscal sponsor was ending its operations we needed to transition quickly. We were able to get a 501(c)3 with not a tremendous amount of effort much quicker than anticipated.
  
  in retrospect, there would have been a better decision to form a 501c3 as soon as we had our first grant and had applied for a larger grant. It would have saved us a substantial amount of fees and allowed us to earn interest/​investment income on the larger grant. And it’s also easier to access tech discounts as a 501(c)(3) rather than a fiscally sponsored organization.
  

• david_reinstein 12 Nov 2025 20:27 UTC

  2 points

  0 ∶ 0

  in reply to: Jamie_Harris’s comment on: Jamie_Har­ris’s Shortform

  Enjoyed it, a good start.

  I like the stylized illustrations but I think a bit more realism (or at least detail) could be helpful. Some of the activities and pain suffered by the chickens was hard to see.

  The transition to the factory farm/​caged chickens environment was dramatic and the impact I think you were seeking.

  One fact-based question which I don’t have the answer to—does this really depict the conditions for chickens where the eggs are labeled as “pasture raised?” I hope so, but I vaguely heard that that was not a rigorously enforced label.

• david_reinstein 12 Nov 2025 20:24 UTC

  2 points

  0 ∶ 0

  in reply to: david_reinstein’s comment on: Changes to the 80k podcast

  Here’s some suggestions from 6 minutes of ChatGPT thinking. (Not all are relevant, e.g., I don’t think “Probable Causation” is a good fit here.)

• david_reinstein 12 Nov 2025 20:22 UTC

  4 points

  0 ∶ 0

  in reply to: Matt_Lerner’s comment on: Changes to the 80k podcast

  Do you see other podcasts filling the long-form, serious/​in-depth, EA-adjacent/​aligned niche in areas other than AI? E.g., GiveWell has a podcast, but I’m not sure it’s the same sort of thing. There’s also Hear This Idea, often Clearer Thinking or Dwarkesh Patel cover relevant stuff.
  
  (Aside, was thinking of potentially trying to do a podcast involving researchers and research evaluators linked to The Unjournal; if I thought it could fill a gap and we could do it well, which I’m not sure of.)

• david_reinstein 12 Nov 2025 1:30 UTC

  5 points

  0 ∶ 0

  on: Cause prio cruxes in 2026?

  This seems a bit related to the “Pivotal questions”: an Unjournal trial initiative -- we’ve engaged with a small group of organizations and elicited some of these—see here.

  To highlight some that seem potentially relevant to your ask:

  What are the effects of increasing the availability of animal-free foods on animal product consumption? Are alternatives to animal products actually used to replace animal products, and especially those that involve the most suffering? Which plant-based offerings are being used as substitutes versus complements for animal products and why?

  Wellbeing measures/​how to convert between DALY and WELLBY welfare measurements on assessing charities and interventions.

  Is WELLBY the most appropriate (useful, reliable...) measure [for interventions that may have impacts on mental health]

  What is cell-cultured meat likely to cost, by year, as a function of the level of investments made?

  How often do countries honor their (international) agreements in the event of large catastrophes (and what determines this?)

  How probable is it that cell-cultured meat will gain widespread consumer acceptance, and to what timescale? To what extent will consumers replace conventional meat with cell-cultured meat?

  How important is democracy for resilience against global catastrophic risk?

  How generalizable is evidence on the effectiveness of corporate animal welfare outreach [in the North] to the Global South?

  How much will the US government use subjective forecasting approaches (in the way the DoD does) in the next ~50 years?

• david_reinstein 11 Nov 2025 23:36 UTC

  2 points

  0 ∶ 0

  in reply to: The Unjournal (bot)’s comment on: Why we didn’t get a malaria vac­cine sooner, and what we can do bet­ter next time

  https://​​unjournal.pubpub.org

• david_reinstein 11 Nov 2025 18:00 UTC

  3 points

  0 ∶ 0

  in reply to: geoffrey’s comment on: Can we do use­ful meta-anal­y­sis? Un­jour­nal eval­u­a­tions of “Mean­ingfully re­duc­ing con­sump­tion of meat and an­i­mal prod­ucts is an un­solved prob­lem...”

  Thanks for the thoughts. Note that I’m trying to engage/​report here because we’re working hard to make our evaluations visible and impactful, and this forum seems like one of the most promising interested audiences. But also eager to hear about other opportunities to promote and get engagement with this evaluation work, particularly in non-EA academic and policy circles.
  
  I generally aim to just summarize and synthesize what the evaluators had written and the authors’ response, bringing in what seemed like some specific relevant examples, and using quotes or paraphrases where possible. I generally didn’t give these as my opinions but rather, the author and the evaluators’. Although I did specifically give ‘my take’ in a few parts. If I recall my motivation I was trying to make this a little bit less dry to get a bit more engagement within this forum. But maybe that was a mistake.
  
  And to this I added an opportunity to discuss the potential value of doing and supporting rigorous, ambitious, and ‘living/​updated’ meta-analysis here and in EA-adjacent areas. I think your response was helpful there, as was the authors. I’d like to see others’ takes
  
  

  Some clarifications:

  The i4replication groups does put out replication papers/​reports in each case and submits these to journals, and reports on this outcome on social media . But IIRC they only ‘weigh in’ centrally when they find a strong case suggesting systematic issues/​retractions.

  Note that their replications are not ‘opt-in’: they aimed to replicate every paper coming out in a set of ‘top journals’. (And now, they are moving towards a research focusing on a set of global issues like deforestation, but still not opt-in).

  I’m not sure what works for them would work for us, though. It’s a different exercise. I don’t see an easy route towards our evaluations getting attention through ‘submitting them to journals’ (which naturally, would also be a bit counter to our core mission of moving research output and rewards away from the ‘journal publication as a static output.)
  
  Also: I wouldn’t characterize this post as ‘editor commentary’, and I don’t think I have a lot of clout here. Also note that typical peer review is both anonymous and never made public. We’re making all our evaluations public, but the evaluators have the option to remain anonymous.
  
  But your point about a higher-bar is well taken. I’ll keep this under consideration.
  
  

• david_reinstein 11 Nov 2025 14:11 UTC

  3 points

  0 ∶ 0

  in reply to: Seth Ariel Green 🔸’s comment on: Can we do use­ful meta-anal­y­sis? Un­jour­nal eval­u­a­tions of “Mean­ingfully re­duc­ing con­sump­tion of meat and an­i­mal prod­ucts is an un­solved prob­lem...”

  A final reflective note: David, I want to encourage you to think about the optics/​politics of this exchange from the point of view of prospective Unjornal participants/​authors.

  I appreciate the feedback. I’m definitely aware that we want to make this attractive to authors and others, both to submit their work and to engage with our evaluations. Note that in addition to asking for author submissions, our team nominates and prioritizes high-profile and potential-high-impact work, and contact authors to get their updates, suggestions, and (later) responses. (We generally only require author permission to do these evaluations from early-career authors at a sensitive point in their career.) We are grateful to you for having responded to these evaluations.

  There are no incentives to participate.

  I would disagree with this. We previously had author prizes (financial and reputational) focusing on authors who submitted work for our evaluation. although these prizes are not currently active. I’m keen to revise these prizes when the situation permits (funding and partners).

  But there are a range of other incentives (not directly financial) for authors to submit their work, respond to evaluations and engage in other ways. I provide a detailed author FAQ here. This includes getting constructive feedback, signaling your confidence in your paper and openness to criticism, the potential for highly positive evaluations to help your paper’s reputation, visibility, unlocking impact and grants, and more. (Our goal is that these evaluations will ultimately become the object of value in and of themselves, replacing “publication in a journal” for research credibility and career rewards. But I admit that’s a long path.)

  I did it because I thought it would be fun ad I was wondering if anyone would have ideas or extensions that improved the paper. Instead, I got some rather harsh criticisms implying we should have written a totally different paper.

  I would not characterize the evaluators’ reports in this way. Yes, there was some negative-leaning language, which, as you know, we encourage the evaluators to tone down. But there were a range of suggestions (especially from Jané) which I see as constructive, detailed, and useful, both for this paper and for your future work. And I don’t see this as them suggesting “a totally different paper.” To large extent they agreed with the importance of this project, with the data collected, and with many of your approaches. They praised your transparency. They suggested some different methods for transforming and analyzing the data and interpreting the results.

  Then I got this essay, which was unexpected/​unannounced and used, again, rather harsh language to which I objected. Do you think this exchange looks like an appealing experience to others? I’d say the answer is probably not.

  I think it’s important to communicate the results of our evaluations to wider audiences, and not only on our own platform. As I mentioned, I tried to fairly categorize your paper, the nature of the evaluations, and your response. I’ve adjusted my post above in response to some of your points where there was a case to be made that I was using loaded language, etc.

  Would you recommend that I share any such posts with both the authors and the evaluators before making them? It’s a genuine question (to you and to anyone else reading these comments) - I’m not sure the correct answer.
  
  As to your suggestion at the bottom, I will read and consider it more carefully—it sounds good.

  Aside: I’m still concerned with the connotation of replication, extension, and robustness checking being something that should be relegated to graduate students and not. This seems to diminish the value and prestige of work that I believe to be of the highest order practical value for important decisions in the animal welfare space and beyond.

  In the replication/​robustness checking domain, I think what i4replication.org is doing is excellent. They’re working with both graduate students and everyone from graduate students to senior professors to do this work and treating this as a high-value output meriting direct career rewards. I believe they encourage the replicators to be fair – excessively conciliatory nor harsh, and focus on the methodology. We are in contact with i4replication.org and hoping to work with them more closely, with our evaluations and “evaluation games” offering grounded suggestions for robustness replication checks.
  

• david_reinstein 11 Nov 2025 13:46 UTC

  3 points

  0 ∶ 0

  in reply to: Seth Ariel Green 🔸’s comment on: Can we do use­ful meta-anal­y­sis? Un­jour­nal eval­u­a­tions of “Mean­ingfully re­duc­ing con­sump­tion of meat and an­i­mal prod­ucts is an un­solved prob­lem...”

  I meant “constructive and actionable” In that he explained why the practices used in the paper had potentially important limitations (see here on “assigning an effect size of .01 for n.s. results where effects are incalculable”)...
  
  And suggested a practical response including a specific statistical package which could be applied to the existing data:
  
  ”An option to mitigate this is through multiple imputation, which can be done through the metansue (i.e., meta-analysis of non-significant and unreported effects) package”
  
  In terms of the cost-benefit test it depends on which benefit we are considering here. Addressing these concerns might indeed take months to do and might indeed cost hundreds of hours. Indeed, it’s hard to justify this in terms of the current academic/​career incentives alone, as the paper had already been accepted for publication. If this we’re directly tied to grants there might be a case but as it stands I understand that it could be very difficult for you to take this further.
  
  But I wouldn’t characterize doing this as simply “satisfying two critics”. The critiques themselves might be sound and relevant, and potentially impact the conclusion (at least in differentiating between “we have evidence,” the effects are small and “the evidence is indeterminate”, which I think is an important difference). And the value of the underlying policy question (~‘Should animal welfare advocates be using funding existing approaches to reducing mep consumption?’) seems high to me. So I would suggest that the benefit exceeds the cost here in net even if we might not have a formula for making it worth your while to make these adjustments right now.
  
  I also think there might be value in setting an example standard that, particularly for high-value questions like this, we strive for a high level of robustness, following up on a range of potential concerns and critiques etc. I’d like to see these things as long-run living projects that can be continuously improved and updated (and re-evaluated). The current research reward system doesn’t encourage this, which is a gap we are trying to help fill.

• david_reinstein 11 Nov 2025 2:34 UTC

  2 points

  0 ∶ 0

  in reply to: geoffrey’s comment on: Can we do use­ful meta-anal­y­sis? Un­jour­nal eval­u­a­tions of “Mean­ingfully re­duc­ing con­sump­tion of meat and an­i­mal prod­ucts is an un­solved prob­lem...”

  Thanks for the detailed feedback, this seems mostly reasonable. I’ll take a look again at some of the framings, and try to adjust. (Below and hopefully later in more detail).

  the phrase “this meta-analysis is not rigorous enough”. it seems this meta-analysis is par for the course in terms of quality.

  This was my take on how to succinctly depict the evaluators’ reports (not my own take), in a way the casual reader would be able to digest. Maybe this was rounding down too much, but not by a lot, I think. Some quotes from Janés evaluation that I think are representative:

  Overall, aside from its commendable transparency, the meta-analysis is not of particularly high quality Overall, the transparency is strong, but the underlying analytic quality is limited.

  This doesn’t seem to reflect ‘par for the course’ to me, but it depends on what the course is; i.e., what the comparison group. My own sense/​guess is that this more rigorous and careful than most work in this area of meat consumption interventions (and adjacent) but less rigorous than the meta-analyses the evaluators are used to seeing in their academic contexts and the practices they espouse. But academic meta-analysts will tend to focus on areas where they can find a proliferation of high-quality more homogenous research, not necessarily the highest impact areas.

  Note that the evaluators rated this 40th and 25th percentile for methods and 75th and 39th percentile overall.

  And the central claim of the meta-analysis doesn’t seem like something either evaluator disputed (though one evaluator was hesitant).

  To be honest I’m having trouble pinning down what the central claim of the meta-analysis is. Is it a claim that “the main approaches being used to motivate reduced meat consumption don’t seem to work”, i.e., that we can bound the effects as very small, at best? That’s how I’d interpret the reporting of the pooled effects 95% CI as standardized mean effect of 0.02 and 0.12. I would say that both evaluators are sort of disputing that claim.

  However the authors hedge this in places and sometimes it sounds more like they’re saying that ~”even the best meta-analysis possible leaves a lot of uncertainty” … An absence of evidence more than an evidence of absence, and this is something the evaluators seem to agree with.

  Finally, one piece of context that felt odd not to mention was the fundamental difficulty of finding an expert in both food consumption and meta-analysis.

  That is/​was indeed challenging. Let me try to adjust this post to note that.

  a few editorial choices … make the Forum post seem much more critical than the evaluation reports, which is a bit unusual since the Evaluators are the ones who are actually critiquing the paper.

  My goal for this post was to fairly represent the evaluator’s take, to provide insights to people who might want to use this for decision-making and future research, to raise the question of standards in meta-analysis in EA-related areas. I will keep thinking about whether I missed the mark here. One possible clarification though: we don’t frame the evaluator’s role as (only) looking to criticize or find errors in the paper. We ask them to give a fair assessment of it, evaluating its strengths, weaknesses, credibility, and usefulness. These evaluations can also be useful if they give people more confidence in the paper and its conclusions, and thus reason to update more on this for their own decision-making.