Thanks for the response. Out of the four responses to nitpicks, I agree with the first two. I broadly agree about the third, forum quality. I just think that peak post quality is at best a lagging indicator—if you have higher volume and even your best posts are not as good anymore, that would bode very poorly. Ideally, the forum team would consciously trade off between growth and average post quality, and in some cases favouring the latter, e.g. performing interventions that would improve the latter even if they slowed growth. And the fourth, understatement, I don’t think we disagree that much.
As for summarising the year, it’s not quite that I want you say that CEA’s year was bad. In one sense, CEA’s year was fine, because these events don’t necessarily reflect negatively on CEA’s current operations. But in another important sense, it was a terrible year for CEA, because these events have a large bearing on whether CEA’s overarching goals are being reached. And this could bear on on what operating activities should be performed in future. I think an adequate summary would capture both angles. In an ideal world (where you were unconstrained by legal consequences etc.), I think an annual review post would note that when such seismic events happen, the standard metrics become relatively less important, while strategy becomes more important, and the focus of discussion then rests on the actually important stuff. I can accept that in the real world, that discussion will happen (much) later, but it’s important that it happens.
Ideally, the forum team would consciously trade off between growth and average post quality [emphasis added]
I disagree. I think the metric I care about is “quality of the average post that a person reads.”
The Forum will have a long tail of posts that are written by newbies just exploring some area for the first time, or are kinda confused, or are bad takes, etc. Many of these posts are net positive! I’m rather in favor of people having their learning experiences in public. Many times the comments on those posts are good places to recapitulate the best of EA. The good news is that most of those posts don’t get much karma or readership. I’m sure you can think of posts that you don’t like that got lots of karma. There’s a complex conversation to be had there, I hope Lizka will post her draft on that soon. But I’m talking more about the much more common post that sits at 0-20 karma. There are lots of them. But they’re by and large pretty harmless. I don’t want my metric to be reducing them.
I don’t know how many people use RSS feeds, but they interact somewhat poorly with this, because the algorithmic post prioritization does not affect RSS readers.
I don’t mean for me personally—I just mean it will bias the web analytics, because low-karma posts will look like they make up a smaller fraction of impressions than in reality (assuming most RSS feed users do not apply such a cutoff).
Thanks for the response. Out of the four responses to nitpicks, I agree with the first two. I broadly agree about the third, forum quality. I just think that peak post quality is at best a lagging indicator—if you have higher volume and even your best posts are not as good anymore, that would bode very poorly. Ideally, the forum team would consciously trade off between growth and average post quality, and in some cases favouring the latter, e.g. performing interventions that would improve the latter even if they slowed growth. And the fourth, understatement, I don’t think we disagree that much.
As for summarising the year, it’s not quite that I want you say that CEA’s year was bad. In one sense, CEA’s year was fine, because these events don’t necessarily reflect negatively on CEA’s current operations. But in another important sense, it was a terrible year for CEA, because these events have a large bearing on whether CEA’s overarching goals are being reached. And this could bear on on what operating activities should be performed in future. I think an adequate summary would capture both angles. In an ideal world (where you were unconstrained by legal consequences etc.), I think an annual review post would note that when such seismic events happen, the standard metrics become relatively less important, while strategy becomes more important, and the focus of discussion then rests on the actually important stuff. I can accept that in the real world, that discussion will happen (much) later, but it’s important that it happens.
I disagree. I think the metric I care about is “quality of the average post that a person reads.”
The Forum will have a long tail of posts that are written by newbies just exploring some area for the first time, or are kinda confused, or are bad takes, etc. Many of these posts are net positive! I’m rather in favor of people having their learning experiences in public. Many times the comments on those posts are good places to recapitulate the best of EA. The good news is that most of those posts don’t get much karma or readership. I’m sure you can think of posts that you don’t like that got lots of karma. There’s a complex conversation to be had there, I hope Lizka will post her draft on that soon. But I’m talking more about the much more common post that sits at 0-20 karma. There are lots of them. But they’re by and large pretty harmless. I don’t want my metric to be reducing them.
Totally, this is what I had in mind—something like the average over posts based on how often they are served on the frontpage.
I don’t know how many people use RSS feeds, but they interact somewhat poorly with this, because the algorithmic post prioritization does not affect RSS readers.
Not sure if this fully helps but you can use a karma cutoff for the RSS feed
I don’t mean for me personally—I just mean it will bias the web analytics, because low-karma posts will look like they make up a smaller fraction of impressions than in reality (assuming most RSS feed users do not apply such a cutoff).