Thanks for this Johannes—nice to see the agility and thoughtfulness of the FP Climate Fund!
I’ve got a couple of questions about the tracking the impact of the EEIST report via media uptake:
Is the theory of this change for this grant along the lines: produce great research → media publicity → greater public pressure for innovation policy → policymakers implement innovation policy or more like great research → media publicity → policymakers read articles and are convinced → better innovation policy?
If doing this analysis in greater detail for the latter ToC, would you make some Fermi estimates on the likelihood of key policymakers to read these articles, % likelihood they would be convinced, etc? Or do you think this would be too speculative to be useful and the value of the media attention is evidently valuable yet challenging to quantify confidently?
What amount of media mentions (articles, etc) did you expect before the grant or did you think would be a worthwhile return on your grant?
Roughly, what was the number of media mentions or people reached per £ or $ spent? Would be a useful benchmark for protests.
What is the analytics tool you use to track total people reached via the media mentions? Asking this out of personal interest again as this would be useful for Animal Rebellion / XR / my research.
I’m curious in general to any other thoughts you have about quantifying impact via media mentions as that is one of the main outputs of activist groups that I’ll be researching. It assume it would be more straightforward in this case as there’s no sentiment issues to deal with (I imagine?) in terms of negative coverage vs positive coverage hence no backfire effects to include. The theory of change / path to impact still seems slightly opaque though so any thoughts on that would be helpful.
Re 1, the ToC is actually different—the report was already produced, but—we believe—would not have been sufficiently amplified absent the grant, so it is more about the latter part of your chain.
Re 2, this is roughly what we would do if the sums justified it—this was a small grant and we operate by the principle of keeping detail of analysis roughly proportional to money moved, so we accepted higher uncertainty here. Something we will be thinking more about going forward if we evaluate similar grants.
Re 3, this is recorded in the article—actually we wrote those sections (“what we expected”) before the grants had effects (“what we achieved”) to allow for this comparison.
Re 4, we spent about 30k so reaching >3m million is about 100 people per USD. There’s more media uptake trickling in still, so it could be significantly more once all is set and done.
Re 5, this is a tool that the PR agency uses, I don’t know which tool this is specifically.
Happy to connect more on those issues, though I probably won’t have time to dig deeper into this before December.
Thanks for this Johannes—nice to see the agility and thoughtfulness of the FP Climate Fund!
I’ve got a couple of questions about the tracking the impact of the EEIST report via media uptake:
Is the theory of this change for this grant along the lines: produce great research → media publicity → greater public pressure for innovation policy → policymakers implement innovation policy or more like great research → media publicity → policymakers read articles and are convinced → better innovation policy?
If doing this analysis in greater detail for the latter ToC, would you make some Fermi estimates on the likelihood of key policymakers to read these articles, % likelihood they would be convinced, etc? Or do you think this would be too speculative to be useful and the value of the media attention is evidently valuable yet challenging to quantify confidently?
What amount of media mentions (articles, etc) did you expect before the grant or did you think would be a worthwhile return on your grant?
Roughly, what was the number of media mentions or people reached per £ or $ spent? Would be a useful benchmark for protests.
What is the analytics tool you use to track total people reached via the media mentions? Asking this out of personal interest again as this would be useful for Animal Rebellion / XR / my research.
I’m curious in general to any other thoughts you have about quantifying impact via media mentions as that is one of the main outputs of activist groups that I’ll be researching. It assume it would be more straightforward in this case as there’s no sentiment issues to deal with (I imagine?) in terms of negative coverage vs positive coverage hence no backfire effects to include. The theory of change / path to impact still seems slightly opaque though so any thoughts on that would be helpful.
Hi James,
thanks for your questions!
Re 1, the ToC is actually different—the report was already produced, but—we believe—would not have been sufficiently amplified absent the grant, so it is more about the latter part of your chain.
Re 2, this is roughly what we would do if the sums justified it—this was a small grant and we operate by the principle of keeping detail of analysis roughly proportional to money moved, so we accepted higher uncertainty here. Something we will be thinking more about going forward if we evaluate similar grants.
Re 3, this is recorded in the article—actually we wrote those sections (“what we expected”) before the grants had effects (“what we achieved”) to allow for this comparison.
Re 4, we spent about 30k so reaching >3m million is about 100 people per USD. There’s more media uptake trickling in still, so it could be significantly more once all is set and done.
Re 5, this is a tool that the PR agency uses, I don’t know which tool this is specifically.
Happy to connect more on those issues, though I probably won’t have time to dig deeper into this before December.