As you stated, there are some advantages to running this project through Wikipedia:
Bigger audience
Greater context/âlonger articles that dig into more topics
Many articles exist in other languages
It has formatting options that the Forum doesnât (yet)
Contributions from people outside the EA community, including viewpoints that wouldnât be as likely to come from Forum readers
However, thereâs no reason we canât leverage a lot of this with the Forumâs wiki. In many or even most cases, Iâd expect that the âfurther resourcesâ section of an article will include a link to the topicâs Wikipedia page (or some other detailed resource).
And Iâd hope that the people who already work on EA-related Wikipedia pages will keep doing so; I agree with you that this seems really valuable for helping to make EA ideas more mainstream.
However, there are some issues with trying to run everything through Wikipedia //â benefits to making a wiki here:
As Max said, there are notability concerns about a lot of potentially good content. And my impression is that over time, Wikipedia has become progressively more strict about what qualifies as ânotableâ (see this Gwern essay, though perhaps things have gotten much better since 2009).
It hamstrings our ability to pay for the public good of useful EA content. Vipul Naik used to sponsor the creation of EA-related Wikipedia content, but his top editor got banned and Wikipediaâs mods froze the whole thing. If CEA wants to sponsor articles about EA topics that we want people to read, we need something like EA Concepts or the EA Forum wiki (at least, thatâs my understanding â can you think of ways around this that arenât just clear violations of Wikipediaâs rules?).
We could ask volunteers to do this, but it seems really good to have access to both options, and to be able to pay people for really good work.
Wikipedia can be an unwelcoming and intimidating place to make edits, especially for newcomers (Iâve heard this from many people). The Forum has many fewer rules/âregulations and (Iâd hope) somewhat friendlier mods, who are accountable to the broader EA community (Wikipediaâs editors arenât accountable in this way).
CEA and LessWrong both have developers available to add features to the Forumâs wiki; we can be more flexible in adapting to things that are useful for our readers than Wikipedia is (on the other hand, Wikipedia has a much bigger team, and may add features we canât replicate at some point, so this feels like a toss-up).
Wikipediaâs articles are often going to be longer/âmore unfocused than would be ideal for someone trying to get up to speed with EA ideas. For example, an article about âRussiaâ on our wiki is likely to end up being more relevant to our community members than any article about Russia I can imagine Wikipedia allowing.
This doesnât mean people should only read about Russia on the Forum â certainly not! â but I think thereâs value in having a âRussia through an EA lensâ page for people to look at alongside the Internetâs many other sources of information about Russia. Same goes for lots of other topics.
If the Forum wiki project goes well and we end up with a lot of good, well-sourced articles, that makes it easier for volunteers to edit relevant Wikipedia pages.
Thanks Aaron! Consolidating my replies to a few different comments here.
I think the notability concerns are real, and greater than Iâd originally thought. Pablo has lots of experience as a Wikipedia editor and I donât, so Iâll defer to him. And it does seem quite telling that Pablo originally tried the Wikipedia approach for a âfew months and felt pretty disappointed with the outcome.â
That said, Iâm still pretty sure thereâs a lot of low-hanging fruit in terms of content thatâs notable, EA relevant, and not on Wikipedia. âLongtermismâ is a good example. I also suspect (though more experienced Wikipedia people should weigh in) that if we made good progress on the low hanging fruit, terms like âhinge of historyâ and âpatient philanthropyâ would be perceived as considerably more notable than they are now. (FWIW, I also think that having tags for things like âhinge of historyâ is a perfectly reasonable Minimum Viable Product alternative to a dedicated Wiki.)
To clarify my position, I do think a dedicated EA Wiki would be extremely valuable. But I think thereâs a significant chance that a dedicated EA Wiki wonât be completed and/âor maintained. Thatâs whatâs happened to multiple previous efforts to build an EA Wiki, so thatâs my baseline assumption unless I see a plan thatâs obviously thought very long and hard about sustainability. I certainly donât get that impression about this plan given Aaronâs comment about what it would take to keep the site running:
I donât have a great estimate for how much volunteer time weâd need to keep things running, but Iâd expect the bare minimum to be less time than Pablo and I are putting in, such that further volunteer contributions are a nice addition rather than an existential necessity. If we were volunteer-only⊠maybe 15-20 hours per month? Thatâs enough time for a few dozen minor edits plus a couple of substantive new articles.
Iâm very confident that estimate is well short of the time required to upkeep a dedicated Wiki. While I donât have experience as a Wikipedia editor, I have quite a bit of experience with a previous employerâs internal Wiki. It was immensely valuable. It was also immensely difficult to develop and maintain. There were countless emails and meetings on the theme âwe need to clean up the Wiki (and this time we really mean it!)â, and in my experience it takes a combination of that and making the Wiki part of peoples job responsibilities/âevaluations to maintain something useful. Itâs amazing how quickly information gets stale, and it gets harder to keep things updated the as you get more entries in your Wiki. If you build an EA Wiki that gets to the level of having a page for âRussiaâ (to use Aaronâs example), youâre going to need a lot more volunteer (and/âor paid staff) time than a few days of someoneâs time a month.
I also get the sense that youâre underestimating the financial and opportunity costs of having CEA and LW developers responsible for maintaining/âadding functionality, based on this comment:
CEA and LessWrong both have developers available to add features to the Forumâs Wiki; we can be more flexible in adapting to things that are useful for our readers than Wikipedia is (on the other hand, Wikipedia has a much bigger team, and may add features we canât replicate at some point, so this feels like a toss-up).
Wikipedia already has more features (including features that would be valuable for an EA Wiki like translation) and a much bigger team (that doesnât have competing priorities and doesnât cost CEA anything), so it seems to me like CEA/âLW will be constantly playing catchup. And any time they spend adding new features or even just maintaining a dedicated Wiki is time they wonât be able to work on other valuable projects.
My overarching concern is that youâre seriously underestimating the ongoing costs of this project, which will basically continue in perpetuity and increase over time. This has been the issue that sank previous attempts at an EA Wiki, and honestly itâs a pretty big red flag that you âdonât have a great estimate for how much volunteer time weâd need to keep things running.â
Iâd urge you to do some more research into what the costs will look like over time (i.e. talk to people involved in previous EA Wiki attempts and people who have lots of experience with dedicated Wikis) and to think about âall inâ costs as much as possible (for example, youâll want to include the ongoing cost of finding, training, and overseeing volunteers and account for volunteer turnover). I would really love to see a dedicated EA Wiki get built and maintained, I just think that if you undertake this project you need to have a realistic picture of the ongoing costs youâll be committing to.
Thank you for your thoughtful and extended feedback. I appreciate the time you have taken to raise a number of valid concerns. I will just respond to a few of your points, since much of what you say is in reply to Aaronâs previous comment, and I donât want to interfere with that conversation.
I think the worry that the Wiki may fail due to insufficient contributions is very real. As you note, none of the previous attempts to build something like what we are trying to accomplish here have succeeded. And it appears that this is a common phenomenon with general efforts to create specialist wikis. Forecasting is one of my hobbies, and Iâm well aware that the base rates arenât in our favor.
This was is fact my primary concern back when I was considering this project for a grant application. The reason I eventually decided to go aheadâbesides feeling that I had a somewhat higher shot at success than my predecessors based on my experience editing Wikipedia and the insight this experience gave me about my capacity to feel motivated long-term by a project of this natureâwas that I thought I could gain more information by just trying things out for a few months. The money costs for EA Grants were relatively modest, as were the time costs for me: at the time I didnât have any other project I felt excited about, and I donât think I would have spent those months very productively otherwise.
Fortunately, the experiment was a success: by the time the grant was over, I had not only produced more content than I had promised, but had discovered that I found writing these articles a more enjoyable experience than I had anticipated. Since then, I have continued to work on the project, and all the direct evidence indicates that lack of motivation will not be a serious impediment. This may still be insufficient to warrant an update from the naive prior to the point that I feel super confident that I will either continue to work on this project full-time for at least the next five years or find a suitable replacement, but if I had to guess, Iâd estimate the chances of this happening at something like 65%.
How sensitive are your worries to scenarios in which the main paid content-writer fails to stay motivated, relative to scenarios in which the project fails because of insufficient volunteer effort? Iâm inclined to believe that as long as there is someone whose full-time job is to write content for the Wiki (whether itâs me or someone else), in combination with all the additional work that Aaron and the technical team are devoting to it, enough progress will probably occur to sustain growth over time and attract volunteer contributors. Iâm modestly confident in this, but Iâm much more confident in that it makes sense to (again) test the hypothesis experimentally, by trying to make the Wiki happen and see how excited people feel about it after a period of a year or so.
Thanks Pablo⊠I appreciate your thoughtful engagement with my comments, and all the hard work youâve put into this project.
How sensitive are your worries to scenarios in which the main paid content-writer fails to stay motivated, relative to scenarios in which the project fails because of insufficient volunteer effort? Iâm inclined to believe that as long as there is someone whose full-time job is to write content for the Wiki (whether itâs me or someone else), in combination with all the additional work that Aaron and the technical team are devoting to it, enough progress will probably occur to sustain growth over time and attract volunteer contributors.
Iâm not at all concerned that âthe main paid content-writer fails to stay motivatedâ since that can easily be solved by finding a suitable replacement. I worry a bit about âinsufficient volunteer effortâ, but mostly see that as a symptom of my main concern: whether organizational commitment can be sustained.
If CEA has a good understanding of what it will cost to create and maintain the necessary content, technical platform, and volunteer structure and commits to (indefinitely) paying those costs, Iâd feel pretty optimistic about the project. Iâve expressed some concerns that CEA is underestimating those costs, but would like to let Aaron respond to those concerns as I may be underestimating the paid staff time CEA is planning or otherwise missing something.
Note on this response: I really appreciate your engagement on this! My goal for this comment is to clarify some things I didnât go into much detail on before, and better represent the way weâre currently thinking about the project (as something CEA cares about a lot and will continue to care about).
That said, Iâm still pretty sure thereâs a lot of low-hanging fruit in terms of content thatâs notable, EA relevant, and not on Wikipedia. âLongtermismâ is a good example. I also suspect (though more experienced Wikipedia people should weigh in) that if we made good progress on the low hanging fruit, terms like âhinge of historyâ and âpatient philanthropyâ would be perceived as considerably more notable than they are now.
I agree with all of this. However, I think CEA would have to tread carefully to support this work without violating Wikipediaâs rules about paid editing. I may think about this more in future months (right now, Iâm juggling a lot of projects). If you have suggestions for what CEA could do in this area, Iâd be happy to hear them.
Meanwhile, Iâll declare this as clearly as I can, using bold text in lieu of cash: I am very happy to see work done on EA-relevant Wikipedia pages, and I think that such work ought to be appreciated by the community at large (and, where appropriate, considered in grant applications, job applications, etc.)
While I donât have experience as a Wikipedia editor, I have quite a bit of experience with a previous employerâs internal Wiki. It was immensely valuable. It was also immensely difficult to develop and maintain. There were countless emails and meetings on the theme âwe need to clean up the Wiki (and this time we really mean it!)â, and in my experience it takes a combination of that and making the Wiki part of peoples job responsibilities/âevaluations to maintain something useful. Itâs amazing how quickly information gets stale, and it gets harder to keep things updated the as you get more entries in your Wiki.
I also have experience with an employerâs (valuable) internal wiki, and I appreciate this point. However, Iâd expect that keeping information extremely up-to-date (e.g. making weekly updates to a large range of entries) is going to be more important for a corporate wiki than a conceptual wiki.
My employerâs internal wiki had lots of articles that were constantly becoming wrong, in ways that would impede our work if the wrongness wasnât corrected (âthis is no longer the password you needâ, âthis menu item has been renamedâ, etc.). On the other hand, articles like âLongtermismâ or âWild Animal Sufferingâ, may be expanded from time to time, but itâs rare that text in such an article will suddenly become wrong.
This doesnât mean that decay isnât a concern â just that itâs less of a crisis than it would be if e.g. a company were to stop making it anyoneâs responsibility to edit their own wiki.
If you build an EA Wiki that gets to the level of having a page for âRussiaâ (to use Aaronâs example), youâre going to need a lot more volunteer (and/âor paid staff) time than a few days of someoneâs time a month.
Iâd expect the bare minimum to be less time than Pablo and I are putting in, such that further volunteer contributions are a nice addition rather than an existential necessity.
To clarify: I think of âbare minimumâ as something like âmany articles are checked on ~once per year, and the volunteer spends a few minutes thinking about whether they want to add anything; a volunteer looks at each edit made by a Forum user and makes small fixes/âreversions as neededâ.
With âmanyâ, Iâm leaving out articles like âdecision theoryâ and âAbhijit Banerjeeâ, and focusing on e.g. articles about core EA orgs and active research areas.
And when I think of âvolunteer timeâ, Iâm thinking of people who see themselves as âwiki volunteersâ, rather than the general population of the Forum â Iâd also expect people on the Forum to put in quite a bit more time, because (unlike those of other EA wikis) the Forumâs wiki articles will be very salient to them.
Our tag pages got over 2000 views in each of January and February, even before most of the content uploading happened. By the time weâve put in months of additional paid work and added new features to improve the wiki and make it more visible, I expect that number to increase by a lot. Only a tiny fraction of those views will turn into edits, whatever UI we employ to encourage them, but thatâs still a lot of additional hours.
Still, I wouldnât think of a user who makes one five-minute edit every six months as a âvolunteerâ. So my 15-20 hour estimate didnât include this kind of activity.
The result Iâd expect from a âbare minimumâ outcome, combining dedicated volunteer work and other user edits: the wiki continues to be a useful resource which people refer to often, especially for its articles on evergreen concepts that donât change often. It does decay somewhat, and few new articles are written, but it remains substantial enough that a few dedicated people could jump in and resurrect it more easily than they could start a new wiki.
Itâs possible that 15-20 hours/âmonth is too low even to expect this level of maintenance; as I note below, I didnât spend much time coming up with the estimate (as I donât think it will be relevant for a long time).
*****
That said, I would be very disappointed in the âbare minimumâ outcome, and would go to considerable lengths to bring in more volunteer support. I mostly set my own priorities at CEA; even if I came to believe that doing a lot of dedicated wiki work wasnât a good use of my time, and we decided to stop paying for work from Pablo or others like him, I canât imagine not wanting to spend some of my time coordinating other people to do this work.
A number Iâd be happy with, where Iâd expect to see the wiki grow and flourish? At 500 articles, with three hours/âarticle/âyear, thatâs 125 hours/âmonth â not a very confident estimate, but one that seems plausible for a good outcome. That would take a lot of coordination (if we want to have a couple dozen people putting in an hour per week*), but Iâd expect to be on the front lines of that effort.
*Iâd expect the actual structure of editing to look a bit different than this, because so many members of the community are going to be invested in specific articles; Iâd guess that e.g. someone from 80K would make substantial edits to 80Kâs article every so often, and that the same would be true for many other editor/âtopic combinations.
My overarching concern is that youâre seriously underestimating the ongoing costs of this project, which will basically continue in perpetuity and increase over time. This has been the issue that sank previous attempts at an EA Wiki, and honestly itâs a pretty big red flag that you âdonât have a great estimate for how much volunteer time weâd need to keep things running.â
The reason I havenât spent much time thinking about the âvolunteer-onlyâ version of the wiki is that Pablo has a grant to work on this project for many months to come, and the project is also one of my highest current priorities at CEA. If it starts to seem like one or both of those things will stop being true in the foreseeable future, I expect to put a lot more time into preparing for the âvolunteer-onlyâ era.
A comparison: If you asked me âAaron, who would take over the EA Newsletter if you got hit by a bus?â, I wouldnât have a good answer on hand. That doesnât mean I think the EA Newsletter isnât important, or that it doesnât take much time to produce; I just donât expect to stop running it anytime soon, or to be hit by a bus.
Wikipedia already has more features (including features that would be valuable for an EA Wiki like translation) and a much bigger team (that doesnât have competing priorities and doesnât cost CEA anything), so it seems to me like CEA/âLW will be constantly playing catchup. And any time they spend adding new features or even just maintaining a dedicated Wiki is time they wonât be able to work on other valuable projects.
As Iâve said, I really want to see people contribute to the real Wikipedia in addition to our dedicated wiki.
But given that LessWrong already has their own wiki, which ours is a copy of, I expect them to keep adding new features to theirs which ours will adopt (this is already how the EA Forum gets most of its new features). This is how theyâd be spending their time with or without us.
Iâll retract my âtoss-upâ comment; I donât really know what I meant by it, having written that comment quickly and without editing. I do think Wikipedia will always have a better overall feature set than our Wiki â thanks for making that point clearly.
But I expect us to occasionally implement things that are a bit better than Wikipediaâs version of the thing, and I also think thereâs a lot of value in having something that is almost as good as Wikipedia in most basic respects that we also control (rather than being at the mercy of notoriously hard-headed admins).
I think CEA would have to tread carefully to support this work without violating Wikipediaâs rules about paid editing. I may think about this more in future months (right now, Iâm juggling a lot of projects). If you have suggestions for what CEA could do in this area, Iâd be happy to hear them.
The paid editing restrictions are a bigger issue than Iâd originally realized. But I do think it would be helpful for an experienced Wikipedia editor like Pablo to write up some brief advice on how volunteers can add EA content to Wikipedia while adhering to all their rules. Sounds like Pablo has some other experiences to share as well. That plus a list of EA content that would be good to get on Wikipedia (which I believe already exists) would probably be enough to make some good progress.
But I do think it would be helpful for an experienced Wikipedia editor like Pablo to write up some brief advice on how volunteers can add EA content to Wikipedia while adhering to all their rules.
Darius Meissner and I are in the process of writing exactly such a document.
I feel like another thing that might help with causing more EAs to actually do this (as opposed to helping them do it better) is finding a way to make the impact of editing important Wikipedia articles more legible to other EAs and more likely to benefit oneâs status. Maybe it could be as simple as someone high-status in EA emphasising how valuable this is in a salient way (like a new top-level post, rather than a comment or an old post) and encouraging other EAs to link to their Wikipedia user profile from their EA Forum bio.
Our tag pages got over 2000 views in each of January and February, even before most of the content uploading happened.
Out of interest, do you mean all normal tag pages that can be used for tagging, or all âwiki-only tag pagesâ, or both types of tag pages put together?
(Also, Iâve appreciated everyoneâs contributions to this discussion here.)
Sounds like weâre agreed that Wikipedia editing would be beneficial, and that working on Wikipedia vs. a dedicated wiki isnât necessarily in direct conflict.
I mostly set my own priorities at CEA; even if I came to believe that doing a lot of dedicated wiki work wasnât a good use of my time, and we decided to stop paying for work from Pablo or others like him, I canât imagine not wanting to spend some of my time coordinating other people to do this workâŠ
The reason I havenât spent much time thinking about the âvolunteer-onlyâ version of the wiki is that Pablo has a grant to work on this project for many months to come, and the project is also one of my highest current priorities at CEA. If it starts to seem like one or both of those things will stop being true in the foreseeable future, I expect to put a lot more time into preparing for the âvolunteer-onlyâ era.
As I wrote to Pablo, my biggest concern about this project is that CEA wonât sustain a commitment to it. Pablo has a grant âfor many months to comeâ, but what happens after that? How likely do you think it is that CEA/âEA Funds will pay for Pablo or someone else to work full time on content creation for years to come? If you think thatâs unlikely, then you need a realistic âvolunteer-onlyâ plan that accounts for the necessary staff, incentives, etc. to implement (and if thereâs not a realistic version of the âvolunteer-onlyâ plan, thatâs a good thing to learn ahead of time. ) In the same vein, Iâd suggest giving serious thought as to the likelihood that an EA Wiki will remain âone of your highest prioritiesâ (and/âor a top priority for one of your colleagues) over a timeframe of years not months.
Honestly, a significant part of the reason Iâm concerned is because I feel like accurately estimating the cost of projects (and especially the costs to keep them up and running after an initial push, including the opportunity costs of not being able to pursue new projects) has been a historical weakness of CEAâs and likely the root cause of CEAâs historical âunderlying problemâ of ârunning too many projects.â
These are all reasonable concerns, and I agree that there are cases where CEA hasnât done this well in past years.
As soon as the wiki is up and running, and we have a sense for what âmaintenanceâ looks like for Pablo and I (plus the level of volunteer activity we end up with after the festival), I think weâll be in a much better place to make contingency plans, and I picture us doing much of the research/âplanning you called for in April. (I work in a series of monthly sprints; this monthâs sprint is launching the wiki, and future months will involve more thinking on sustainability.)
As you stated, there are some advantages to running this project through Wikipedia:
Bigger audience
Greater context/âlonger articles that dig into more topics
Many articles exist in other languages
It has formatting options that the Forum doesnât (yet)
Contributions from people outside the EA community, including viewpoints that wouldnât be as likely to come from Forum readers
However, thereâs no reason we canât leverage a lot of this with the Forumâs wiki. In many or even most cases, Iâd expect that the âfurther resourcesâ section of an article will include a link to the topicâs Wikipedia page (or some other detailed resource).
And Iâd hope that the people who already work on EA-related Wikipedia pages will keep doing so; I agree with you that this seems really valuable for helping to make EA ideas more mainstream.
However, there are some issues with trying to run everything through Wikipedia //â benefits to making a wiki here:
As Max said, there are notability concerns about a lot of potentially good content. And my impression is that over time, Wikipedia has become progressively more strict about what qualifies as ânotableâ (see this Gwern essay, though perhaps things have gotten much better since 2009).
It hamstrings our ability to pay for the public good of useful EA content. Vipul Naik used to sponsor the creation of EA-related Wikipedia content, but his top editor got banned and Wikipediaâs mods froze the whole thing. If CEA wants to sponsor articles about EA topics that we want people to read, we need something like EA Concepts or the EA Forum wiki (at least, thatâs my understanding â can you think of ways around this that arenât just clear violations of Wikipediaâs rules?).
We could ask volunteers to do this, but it seems really good to have access to both options, and to be able to pay people for really good work.
Wikipedia can be an unwelcoming and intimidating place to make edits, especially for newcomers (Iâve heard this from many people). The Forum has many fewer rules/âregulations and (Iâd hope) somewhat friendlier mods, who are accountable to the broader EA community (Wikipediaâs editors arenât accountable in this way).
CEA and LessWrong both have developers available to add features to the Forumâs wiki; we can be more flexible in adapting to things that are useful for our readers than Wikipedia is (on the other hand, Wikipedia has a much bigger team, and may add features we canât replicate at some point, so this feels like a toss-up).
Wikipediaâs articles are often going to be longer/âmore unfocused than would be ideal for someone trying to get up to speed with EA ideas. For example, an article about âRussiaâ on our wiki is likely to end up being more relevant to our community members than any article about Russia I can imagine Wikipedia allowing.
This doesnât mean people should only read about Russia on the Forum â certainly not! â but I think thereâs value in having a âRussia through an EA lensâ page for people to look at alongside the Internetâs many other sources of information about Russia. Same goes for lots of other topics.
If the Forum wiki project goes well and we end up with a lot of good, well-sourced articles, that makes it easier for volunteers to edit relevant Wikipedia pages.
Thanks Aaron! Consolidating my replies to a few different comments here.
I think the notability concerns are real, and greater than Iâd originally thought. Pablo has lots of experience as a Wikipedia editor and I donât, so Iâll defer to him. And it does seem quite telling that Pablo originally tried the Wikipedia approach for a âfew months and felt pretty disappointed with the outcome.â
That said, Iâm still pretty sure thereâs a lot of low-hanging fruit in terms of content thatâs notable, EA relevant, and not on Wikipedia. âLongtermismâ is a good example. I also suspect (though more experienced Wikipedia people should weigh in) that if we made good progress on the low hanging fruit, terms like âhinge of historyâ and âpatient philanthropyâ would be perceived as considerably more notable than they are now. (FWIW, I also think that having tags for things like âhinge of historyâ is a perfectly reasonable Minimum Viable Product alternative to a dedicated Wiki.)
To clarify my position, I do think a dedicated EA Wiki would be extremely valuable. But I think thereâs a significant chance that a dedicated EA Wiki wonât be completed and/âor maintained. Thatâs whatâs happened to multiple previous efforts to build an EA Wiki, so thatâs my baseline assumption unless I see a plan thatâs obviously thought very long and hard about sustainability. I certainly donât get that impression about this plan given Aaronâs comment about what it would take to keep the site running:
Iâm very confident that estimate is well short of the time required to upkeep a dedicated Wiki. While I donât have experience as a Wikipedia editor, I have quite a bit of experience with a previous employerâs internal Wiki. It was immensely valuable. It was also immensely difficult to develop and maintain. There were countless emails and meetings on the theme âwe need to clean up the Wiki (and this time we really mean it!)â, and in my experience it takes a combination of that and making the Wiki part of peoples job responsibilities/âevaluations to maintain something useful. Itâs amazing how quickly information gets stale, and it gets harder to keep things updated the as you get more entries in your Wiki. If you build an EA Wiki that gets to the level of having a page for âRussiaâ (to use Aaronâs example), youâre going to need a lot more volunteer (and/âor paid staff) time than a few days of someoneâs time a month.
I also get the sense that youâre underestimating the financial and opportunity costs of having CEA and LW developers responsible for maintaining/âadding functionality, based on this comment:
Wikipedia already has more features (including features that would be valuable for an EA Wiki like translation) and a much bigger team (that doesnât have competing priorities and doesnât cost CEA anything), so it seems to me like CEA/âLW will be constantly playing catchup. And any time they spend adding new features or even just maintaining a dedicated Wiki is time they wonât be able to work on other valuable projects.
My overarching concern is that youâre seriously underestimating the ongoing costs of this project, which will basically continue in perpetuity and increase over time. This has been the issue that sank previous attempts at an EA Wiki, and honestly itâs a pretty big red flag that you âdonât have a great estimate for how much volunteer time weâd need to keep things running.â
Iâd urge you to do some more research into what the costs will look like over time (i.e. talk to people involved in previous EA Wiki attempts and people who have lots of experience with dedicated Wikis) and to think about âall inâ costs as much as possible (for example, youâll want to include the ongoing cost of finding, training, and overseeing volunteers and account for volunteer turnover). I would really love to see a dedicated EA Wiki get built and maintained, I just think that if you undertake this project you need to have a realistic picture of the ongoing costs youâll be committing to.
Hi Anonymous,
Thank you for your thoughtful and extended feedback. I appreciate the time you have taken to raise a number of valid concerns. I will just respond to a few of your points, since much of what you say is in reply to Aaronâs previous comment, and I donât want to interfere with that conversation.
I think the worry that the Wiki may fail due to insufficient contributions is very real. As you note, none of the previous attempts to build something like what we are trying to accomplish here have succeeded. And it appears that this is a common phenomenon with general efforts to create specialist wikis. Forecasting is one of my hobbies, and Iâm well aware that the base rates arenât in our favor.
This was is fact my primary concern back when I was considering this project for a grant application. The reason I eventually decided to go aheadâbesides feeling that I had a somewhat higher shot at success than my predecessors based on my experience editing Wikipedia and the insight this experience gave me about my capacity to feel motivated long-term by a project of this natureâwas that I thought I could gain more information by just trying things out for a few months. The money costs for EA Grants were relatively modest, as were the time costs for me: at the time I didnât have any other project I felt excited about, and I donât think I would have spent those months very productively otherwise.
Fortunately, the experiment was a success: by the time the grant was over, I had not only produced more content than I had promised, but had discovered that I found writing these articles a more enjoyable experience than I had anticipated. Since then, I have continued to work on the project, and all the direct evidence indicates that lack of motivation will not be a serious impediment. This may still be insufficient to warrant an update from the naive prior to the point that I feel super confident that I will either continue to work on this project full-time for at least the next five years or find a suitable replacement, but if I had to guess, Iâd estimate the chances of this happening at something like 65%.
How sensitive are your worries to scenarios in which the main paid content-writer fails to stay motivated, relative to scenarios in which the project fails because of insufficient volunteer effort? Iâm inclined to believe that as long as there is someone whose full-time job is to write content for the Wiki (whether itâs me or someone else), in combination with all the additional work that Aaron and the technical team are devoting to it, enough progress will probably occur to sustain growth over time and attract volunteer contributors. Iâm modestly confident in this, but Iâm much more confident in that it makes sense to (again) test the hypothesis experimentally, by trying to make the Wiki happen and see how excited people feel about it after a period of a year or so.
Thanks Pablo⊠I appreciate your thoughtful engagement with my comments, and all the hard work youâve put into this project.
Iâm not at all concerned that âthe main paid content-writer fails to stay motivatedâ since that can easily be solved by finding a suitable replacement. I worry a bit about âinsufficient volunteer effortâ, but mostly see that as a symptom of my main concern: whether organizational commitment can be sustained.
If CEA has a good understanding of what it will cost to create and maintain the necessary content, technical platform, and volunteer structure and commits to (indefinitely) paying those costs, Iâd feel pretty optimistic about the project. Iâve expressed some concerns that CEA is underestimating those costs, but would like to let Aaron respond to those concerns as I may be underestimating the paid staff time CEA is planning or otherwise missing something.
Note on this response: I really appreciate your engagement on this! My goal for this comment is to clarify some things I didnât go into much detail on before, and better represent the way weâre currently thinking about the project (as something CEA cares about a lot and will continue to care about).
I agree with all of this. However, I think CEA would have to tread carefully to support this work without violating Wikipediaâs rules about paid editing. I may think about this more in future months (right now, Iâm juggling a lot of projects). If you have suggestions for what CEA could do in this area, Iâd be happy to hear them.
Meanwhile, Iâll declare this as clearly as I can, using bold text in lieu of cash: I am very happy to see work done on EA-relevant Wikipedia pages, and I think that such work ought to be appreciated by the community at large (and, where appropriate, considered in grant applications, job applications, etc.)
I also have experience with an employerâs (valuable) internal wiki, and I appreciate this point. However, Iâd expect that keeping information extremely up-to-date (e.g. making weekly updates to a large range of entries) is going to be more important for a corporate wiki than a conceptual wiki.
My employerâs internal wiki had lots of articles that were constantly becoming wrong, in ways that would impede our work if the wrongness wasnât corrected (âthis is no longer the password you needâ, âthis menu item has been renamedâ, etc.). On the other hand, articles like âLongtermismâ or âWild Animal Sufferingâ, may be expanded from time to time, but itâs rare that text in such an article will suddenly become wrong.
This doesnât mean that decay isnât a concern â just that itâs less of a crisis than it would be if e.g. a company were to stop making it anyoneâs responsibility to edit their own wiki.
I think I wasnât clear enough in what I meant the â15-20 hoursâ to represent, and I may have come off as blasĂ© in a way I didnât intend.
Quoting myself:
To clarify: I think of âbare minimumâ as something like âmany articles are checked on ~once per year, and the volunteer spends a few minutes thinking about whether they want to add anything; a volunteer looks at each edit made by a Forum user and makes small fixes/âreversions as neededâ.
With âmanyâ, Iâm leaving out articles like âdecision theoryâ and âAbhijit Banerjeeâ, and focusing on e.g. articles about core EA orgs and active research areas.
And when I think of âvolunteer timeâ, Iâm thinking of people who see themselves as âwiki volunteersâ, rather than the general population of the Forum â Iâd also expect people on the Forum to put in quite a bit more time, because (unlike those of other EA wikis) the Forumâs wiki articles will be very salient to them.
Our tag pages got over 2000 views in each of January and February, even before most of the content uploading happened. By the time weâve put in months of additional paid work and added new features to improve the wiki and make it more visible, I expect that number to increase by a lot. Only a tiny fraction of those views will turn into edits, whatever UI we employ to encourage them, but thatâs still a lot of additional hours.
Still, I wouldnât think of a user who makes one five-minute edit every six months as a âvolunteerâ. So my 15-20 hour estimate didnât include this kind of activity.
The result Iâd expect from a âbare minimumâ outcome, combining dedicated volunteer work and other user edits: the wiki continues to be a useful resource which people refer to often, especially for its articles on evergreen concepts that donât change often. It does decay somewhat, and few new articles are written, but it remains substantial enough that a few dedicated people could jump in and resurrect it more easily than they could start a new wiki.
Itâs possible that 15-20 hours/âmonth is too low even to expect this level of maintenance; as I note below, I didnât spend much time coming up with the estimate (as I donât think it will be relevant for a long time).
*****
That said, I would be very disappointed in the âbare minimumâ outcome, and would go to considerable lengths to bring in more volunteer support. I mostly set my own priorities at CEA; even if I came to believe that doing a lot of dedicated wiki work wasnât a good use of my time, and we decided to stop paying for work from Pablo or others like him, I canât imagine not wanting to spend some of my time coordinating other people to do this work.
A number Iâd be happy with, where Iâd expect to see the wiki grow and flourish? At 500 articles, with three hours/âarticle/âyear, thatâs 125 hours/âmonth â not a very confident estimate, but one that seems plausible for a good outcome. That would take a lot of coordination (if we want to have a couple dozen people putting in an hour per week*), but Iâd expect to be on the front lines of that effort.
*Iâd expect the actual structure of editing to look a bit different than this, because so many members of the community are going to be invested in specific articles; Iâd guess that e.g. someone from 80K would make substantial edits to 80Kâs article every so often, and that the same would be true for many other editor/âtopic combinations.
The reason I havenât spent much time thinking about the âvolunteer-onlyâ version of the wiki is that Pablo has a grant to work on this project for many months to come, and the project is also one of my highest current priorities at CEA. If it starts to seem like one or both of those things will stop being true in the foreseeable future, I expect to put a lot more time into preparing for the âvolunteer-onlyâ era.
A comparison: If you asked me âAaron, who would take over the EA Newsletter if you got hit by a bus?â, I wouldnât have a good answer on hand. That doesnât mean I think the EA Newsletter isnât important, or that it doesnât take much time to produce; I just donât expect to stop running it anytime soon, or to be hit by a bus.
As Iâve said, I really want to see people contribute to the real Wikipedia in addition to our dedicated wiki.
But given that LessWrong already has their own wiki, which ours is a copy of, I expect them to keep adding new features to theirs which ours will adopt (this is already how the EA Forum gets most of its new features). This is how theyâd be spending their time with or without us.
Iâll retract my âtoss-upâ comment; I donât really know what I meant by it, having written that comment quickly and without editing. I do think Wikipedia will always have a better overall feature set than our Wiki â thanks for making that point clearly.
But I expect us to occasionally implement things that are a bit better than Wikipediaâs version of the thing, and I also think thereâs a lot of value in having something that is almost as good as Wikipedia in most basic respects that we also control (rather than being at the mercy of notoriously hard-headed admins).
The paid editing restrictions are a bigger issue than Iâd originally realized. But I do think it would be helpful for an experienced Wikipedia editor like Pablo to write up some brief advice on how volunteers can add EA content to Wikipedia while adhering to all their rules. Sounds like Pablo has some other experiences to share as well. That plus a list of EA content that would be good to get on Wikipedia (which I believe already exists) would probably be enough to make some good progress.
Darius Meissner and I are in the process of writing exactly such a document.
Great!
I feel like another thing that might help with causing more EAs to actually do this (as opposed to helping them do it better) is finding a way to make the impact of editing important Wikipedia articles more legible to other EAs and more likely to benefit oneâs status. Maybe it could be as simple as someone high-status in EA emphasising how valuable this is in a salient way (like a new top-level post, rather than a comment or an old post) and encouraging other EAs to link to their Wikipedia user profile from their EA Forum bio.
Nice!
In the meantime or in addition, some readers of this thread might be interested in some related things Brian Tomasik wrote:
The Value of Wikipedia Contributions in Social Sciences
Tomasikâs ideas for pages to create or improve
(I think Iâve only skimmed these myself.)
Out of interest, do you mean all normal tag pages that can be used for tagging, or all âwiki-only tag pagesâ, or both types of tag pages put together?
(Also, Iâve appreciated everyoneâs contributions to this discussion here.)
Thank you Aaron for this detailed engagement!
Sounds like weâre agreed that Wikipedia editing would be beneficial, and that working on Wikipedia vs. a dedicated wiki isnât necessarily in direct conflict.
As I wrote to Pablo, my biggest concern about this project is that CEA wonât sustain a commitment to it. Pablo has a grant âfor many months to comeâ, but what happens after that? How likely do you think it is that CEA/âEA Funds will pay for Pablo or someone else to work full time on content creation for years to come? If you think thatâs unlikely, then you need a realistic âvolunteer-onlyâ plan that accounts for the necessary staff, incentives, etc. to implement (and if thereâs not a realistic version of the âvolunteer-onlyâ plan, thatâs a good thing to learn ahead of time. ) In the same vein, Iâd suggest giving serious thought as to the likelihood that an EA Wiki will remain âone of your highest prioritiesâ (and/âor a top priority for one of your colleagues) over a timeframe of years not months.
Honestly, a significant part of the reason Iâm concerned is because I feel like accurately estimating the cost of projects (and especially the costs to keep them up and running after an initial push, including the opportunity costs of not being able to pursue new projects) has been a historical weakness of CEAâs and likely the root cause of CEAâs historical âunderlying problemâ of ârunning too many projects.â
These are all reasonable concerns, and I agree that there are cases where CEA hasnât done this well in past years.
As soon as the wiki is up and running, and we have a sense for what âmaintenanceâ looks like for Pablo and I (plus the level of volunteer activity we end up with after the festival), I think weâll be in a much better place to make contingency plans, and I picture us doing much of the research/âplanning you called for in April. (I work in a series of monthly sprints; this monthâs sprint is launching the wiki, and future months will involve more thinking on sustainability.)