Hello! Iâm Toby. Iâm Content Strategist at CEA. I work with the Online Team to make sure the Forum is a great place to discuss doing the most good we can. Youâll see me posting a lot, authoring the EA Newsletter and curating Forum Digests, making moderator comments and decisions, and more.
Before working at CEA, I studied Philosophy at the University of Warwick, and worked for a couple of years on a range of writing and editing projects within the EA space. Recently I helped run the Amplify Creative Grants program, to encourage more impactful podcasting and YouTube projects. You can find a bit of my own creative output on my blog, and my podcast feed.
Here is the very sketchy first salvo at measuring âImpact-weighted clicksâ (not sure how much I endorse this anymore, but feel free to treat me as if I do endorse it in responses to the commentâthatâll help me figure out what I think).
Was going to just share a doc with @Vasco Grilođž but I figured there is no reason not to make it public. Very happy to get input on this as well, especially if you have experience trying to measure the value of content (outside of just âclicks on jobsâ)
Impact-weighted clicks
Not all EA Newsletter clicks are created equal. The EA Newsletter contains links to impactful jobs, but also interesting podcasts, videos, and articles â where the only further call to action is subscribing. This content isnât of no value, but itâs very unlikely to be the source of most of the newsletterâs value.
The metric Iâm making will give categories of clicks a value from 0-1, and then produce a total number of âimpact-weightedâ clicks, where the total amount of unique clicks each category is multiplied by the value assigned to that category.
What is the value of content clicks?
Firstly â retention. Without content (hypothesis 1), people wouldnât be interested in the EA Newsletter.
Secondly â content pipelines. If someone starts reading an EA friendly outlet, subscribes to the 80k podcast, buys a book, subscribes to rational animations, etc⊠They increase their chance of coming across EA content in the future.
Third â Changing minds. People who are aware of big problems might decide to work on them.
How should we categorise EANL clicks?
What value should we give to each type of click?
An obvious ranking is:
Job ads (this person is already engaged and ready to act)
Fellowship/â funding (likewise ready to act, but funding is short term and fellowship is the step before the impactful job)
Courses (intent to get more involved, learn more, but step before deeper engagement)
Donation (signals strong interest, and likely to sign them up to marketing from an effective charity)
Content (value described above, pretty loosely valuable)
I would stand by the obvious ranking (though maybe more quibling should separate âfellowship/â fundingâ). However, we also need to provide values for each step. In the ideal world, we could base this on the drop off rate between each step in our funnel (perhaps X% of people who do a fellowship donât end up getting a relevant job, therefore fellowship clicks should be ranked as the value of job adsâX). We should come back to this, but for now, Iâll do some crude estimates.
Job ad clicks = 1
Letâs say 50% of fellowship graduates , or funding recipients get a relevant job. Fellowship = 0.5
25% of courses graduates do something relevant afterwards = 0.25
Donation = 0.05
Content = 0
How could the metric be better?
If we knew more about our audience, we could multiply the impact-weighted total clicks by the âvalueâ of our current email-opening audience (i.e. each click is more valuable in expectation if the audience is highly motivated, educated, etcâŠ). Hopefully if we get a marketer involved we could learn more about this.
Hypotheses to test:
H1: âWithout content, people wouldnât be interested in reading the EA Newsletterâ. I.e. without content, overall clicks go down.
This might be wrongâthe content doesnât seem to have led to more subscribers.
And yetâpeople come back to click every month.