I think I’m pretty torn up about this. I agree that this was a failure, but going too far in the other direction seems like a loss of opportunity. I think my ideal would be something like a very competent and large CEA, or another competent and large organization spearheading a bunch of new EA initiatives. I think there’s enough potential work to absorb an additional 30-1000 full time people. I’d prefer small groups to do this to a poorly managed big group, but in general don’t trust small groups all too much for this kind of work in the long run. Major strategic action requires a lot of coordination, and this is really difficult with a lot of small groups.
I think my take is that the failures mentioned were mostly failures of expectations, rather than bad decisions in the ideal. If CEA could have done all these things well, that would have been the ideal scenario to me. The projects often seemed quite reasonable, it just seemed like CEA didn’t quite have the necessary abilities at those points to deliver on them.
Referencing above comments, I think, “Let’s make sure that our organization runs well, before thinking too much about expanding dramatically” is a very legitimate strategy. My guess is that given the circumstances around it, it’s a very reasonable one as well. But I also have some part of me inside screaming, “How can we get EA infrastructure to grow much faster?”.
Perhaps more intense growth, or at least bringing in several strong new product managers, could be more of a plan in 1-2 years or so.
Oh man, happy to have come across this. I’m a bit surprised people remember that article. I was one of the main people that set up the system, that was a while back.
I don’t know specifically why it was changed. I left 80k in 2014 or so and haven’t discussed this with them since. I could imagine some reasons why they stopped it though. I recommend reaching out to them if you want a better sense.
This was done when the site was a custom Ruby/Rails setup. This functionality required a fair bit of custom coding functionality to set up. Writing quality was more variable then than it is now; there were several newish authors and it was much earlier in the research process. I also remember that originally the scores disagreed a lot between evaluators, but over time (the first few weeks of use) they converged a fair bit.
After I left they migrated to Wordpress, which I assume would have required a fair effort to set up a similar system in. The blog posts seem like they became less important than they used to be; in favor of the career guide, coaching, the podcast, and other things. Also the quality has become a fair bit more consistent, from what I can tell as an onlooker.
The ongoing costs of such a system are considerable. First, it just takes a fair bit of time from the reviewers. Second, unfortunately, the internet can be a hostile place for transparency. There are trolls and angry people who will actively search through details and then point them out without the proper context. I think this review system was kind of radical, and can imagine it not being very comfortable to maintain, unless it really justified a fair bit of effort.
I’m of course sad it’s not longer in place, but can’t really blame them.