In starting RSP, I had an implicit theory of change in my head
There are quite a few facets of this (mechanisms for value produced, a continuum of hypotheses, etc.)
One important facet (particularly for early-RSP) was a sense of “pretty sure there’s significant value available via something in this vicinity, let’s try it and see if we can hone in”
I explicitly share and communicate parts of this model to the extent that it’s accessible for me to do so
This involved some conversations with people before RSP started, and some presenting thoughts to the research scholars as the programme started, and periodically returning to it
As RSP has developed and other people have become major stakeholders, they’ve developed their own implicit theories of change
We make some space to discuss these / exchange models
As RSP matures, it will make more sense to pin down a theory of change and have it explicit and shared
The facet of “let’s work out what here is good” will naturally diminish, and we’ll work out which other facets are best to lean on
Some general thoughts:
Advantages of having an explicit theory of change:
Makes it easier to sync up about direction/priorities/reasons for doing things
Makes it easier for people to engage critically, or otherwise to notice mistakes and course-correct
Disadvantages of having an explicit theory of change:
Easy to have the case where your best expression of something is dumber than your real internal sense of it
In this case it may be preferable to be guided by the internal sense rather than the explicit version
(this is at least some distant relative of Goodhart’s law)
To the extent that you’re going to be guided by your internal sense rather than an explicit version, sharing something as an explicit theory of change can be misleading
In general I think it’s good to encourage lots of explicit discussion about theories of change
Ideally without committing to reaching an “answer”, but having that as a goal may be helpful for prompting the discussion
I think that I find the disadvantages quite emotionally resonant, which may pull me to err too far in the direction of not being explicit. I have appreciated some cases where people have pushed me towards “let’s have a discussion where we’re pretty explicit about best guesses”.
FHI I think has an explicit theory of change even less than RSP does; my guess it that Nick Bostrom is also averse to incurring the costs of these disadvantages (and maybe more strongly so than me), but that’s speculation.
I’ve quoted the part from “Some general thoughts:” to the second-last paragraph in a new comment on my earlier question post Do research organisations make theory of change diagrams? Should they? (I flagged that you weren’t talking about ToC diagrams in particular.) Hope that’s ok.
For RSP, I think that:
In starting RSP, I had an implicit theory of change in my head
There are quite a few facets of this (mechanisms for value produced, a continuum of hypotheses, etc.)
One important facet (particularly for early-RSP) was a sense of “pretty sure there’s significant value available via something in this vicinity, let’s try it and see if we can hone in”
I explicitly share and communicate parts of this model to the extent that it’s accessible for me to do so
This involved some conversations with people before RSP started, and some presenting thoughts to the research scholars as the programme started, and periodically returning to it
As RSP has developed and other people have become major stakeholders, they’ve developed their own implicit theories of change
We make some space to discuss these / exchange models
As RSP matures, it will make more sense to pin down a theory of change and have it explicit and shared
The facet of “let’s work out what here is good” will naturally diminish, and we’ll work out which other facets are best to lean on
Some general thoughts:
Advantages of having an explicit theory of change:
Makes it easier to sync up about direction/priorities/reasons for doing things
Makes it easier for people to engage critically, or otherwise to notice mistakes and course-correct
Disadvantages of having an explicit theory of change:
Easy to have the case where your best expression of something is dumber than your real internal sense of it
In this case it may be preferable to be guided by the internal sense rather than the explicit version
(this is at least some distant relative of Goodhart’s law)
To the extent that you’re going to be guided by your internal sense rather than an explicit version, sharing something as an explicit theory of change can be misleading
In general I think it’s good to encourage lots of explicit discussion about theories of change
Ideally without committing to reaching an “answer”, but having that as a goal may be helpful for prompting the discussion
I think that I find the disadvantages quite emotionally resonant, which may pull me to err too far in the direction of not being explicit. I have appreciated some cases where people have pushed me towards “let’s have a discussion where we’re pretty explicit about best guesses”.
FHI I think has an explicit theory of change even less than RSP does; my guess it that Nick Bostrom is also averse to incurring the costs of these disadvantages (and maybe more strongly so than me), but that’s speculation.
Thanks for that detailed answer!
I’ve quoted the part from “Some general thoughts:” to the second-last paragraph in a new comment on my earlier question post Do research organisations make theory of change diagrams? Should they? (I flagged that you weren’t talking about ToC diagrams in particular.) Hope that’s ok.