I did some quick, far from scientific analytics work on this to get a feeling for the additional room for participation.
First, I wanted to identify a series of posts to examine. I wanted a consecutive series so that there would not be any introduced bias. I wanted the series to be recent (to reflect recent activity) but not within a couple weeks, as I wanted it to reflect the lifecycle of a post. I also wanted the series to have what seemed like a representative distribution of upvotes.
I selected the 5 post series from Kuhn’s post about GiveWell’s 2015 recommendations to Ben Todd’s post on WALYs.
I ran some analytics. I do not know if there is a norm that EA Forum traffic numbers are not released, so instead I’ll just report a single statistic that seems most relevant.
In these 5 posts, on average 4.17% of unique visitors to the posts voted (I was sure to include both up and down votes). The range was from 1% to 6% of unique visitors to a post voting.
Of course, this only tells us a bit. If we consider the goal to be to determine what percent of visitors with meaningful opinions voted, we could say this number is inflated, since it only considers those that visited a post, and some people may know with confidence that they want to upvote or downvote based on the title on the front-page alone. This number could be an underestimate, because unique visitors is not a 100% reliable statistic, failing, for example, to adjust for the same visitor reading a post on two different devices.
Besides considering whether it is an over or underestimate, it still lacks context, and I do not believe context is going to be truly available. I did find a reddit post that has a slightly interesting discussion of possible voting behavior on that site: http://bit.ly/1OJq25b
Personally, I am of the opinion that voting behavior, percentage-wise is currently OK compared to other sites, but that this largely accounts for an engaged readership, with the other sites I’m referencing not being highly comparable. I think it is highly likely that our voting should well exceed that of other sites. And I think it is optimal to capture as much feedback as possible, so I would have a leaning toward implementation almost regardless of the current participation data.
I did some quick, far from scientific analytics work on this to get a feeling for the additional room for participation.
First, I wanted to identify a series of posts to examine. I wanted a consecutive series so that there would not be any introduced bias. I wanted the series to be recent (to reflect recent activity) but not within a couple weeks, as I wanted it to reflect the lifecycle of a post. I also wanted the series to have what seemed like a representative distribution of upvotes.
I selected the 5 post series from Kuhn’s post about GiveWell’s 2015 recommendations to Ben Todd’s post on WALYs.
I ran some analytics. I do not know if there is a norm that EA Forum traffic numbers are not released, so instead I’ll just report a single statistic that seems most relevant.
In these 5 posts, on average 4.17% of unique visitors to the posts voted (I was sure to include both up and down votes). The range was from 1% to 6% of unique visitors to a post voting.
Of course, this only tells us a bit. If we consider the goal to be to determine what percent of visitors with meaningful opinions voted, we could say this number is inflated, since it only considers those that visited a post, and some people may know with confidence that they want to upvote or downvote based on the title on the front-page alone. This number could be an underestimate, because unique visitors is not a 100% reliable statistic, failing, for example, to adjust for the same visitor reading a post on two different devices.
Besides considering whether it is an over or underestimate, it still lacks context, and I do not believe context is going to be truly available. I did find a reddit post that has a slightly interesting discussion of possible voting behavior on that site: http://bit.ly/1OJq25b
Personally, I am of the opinion that voting behavior, percentage-wise is currently OK compared to other sites, but that this largely accounts for an engaged readership, with the other sites I’m referencing not being highly comparable. I think it is highly likely that our voting should well exceed that of other sites. And I think it is optimal to capture as much feedback as possible, so I would have a leaning toward implementation almost regardless of the current participation data.