Sure. So I’m thinking that for impact, you’d have sort of causal factors (Scale, importance, relation to other work, etc.) But then you’d also have proxies of impact, things that you intuit correlate well with having an impact even if the relationship isn’t causal. For example, having lots of comments praising some project doesn’t normally cause the project to have more impact. See here for the kind of thing I’m going for.
Here is a more cleaned up — yet still very experimental — version of a rubric I’m using for the value of research:
Expected
Probabilistic
% of producing an output which reaches goals
Past successes in area
Quality of feedback loops
Personal motivation
% of being counterfactually useful
Novelty
Neglectedness
Existential
Robustness: Is this project robust under different models?
Reliability: If this is a research project, how much can we trust the results?
Impact
Overall promisingness (intuition)
Scale: How many people affected
Importance: How important for each person
(Proxies of impact):
Connectedness
Engagement
De-confusion
Direct applicability
Indirect impact
Career capital
Information value
Per Unit of Resources
Personal fit
Time needed
Funding needed
Logistical difficulty
See also: Charity Entrepreneurship’s rubric, geared towards choosing which charity to start.
I like it! I think that something in this vein could potentially be very useful. Can you expand more about the proxies of impact?
Sure. So I’m thinking that for impact, you’d have sort of causal factors (Scale, importance, relation to other work, etc.) But then you’d also have proxies of impact, things that you intuit correlate well with having an impact even if the relationship isn’t causal. For example, having lots of comments praising some project doesn’t normally cause the project to have more impact. See here for the kind of thing I’m going for.