Suggestion: EAs should post more summaries and collections

[MichaelA🔸]

Tl;dr I recommend more EAs post summaries of useful ideas and collections of quotes, sources, definitions, terms, etc. Collections in particular can be fairly easy to make when you’re learning about or researching a topic anyway. And both summaries and collections can make the road easier for those who follow.

The problem

Effective altruism is still pretty new, as are several of its allied communities (e.g., the rationality community), and as are many of the topics and concepts of interest to EAs. It seems to me, and I believe to many others, that this leads to the following situations cropping up surprisingly often:

• an interesting/​useful idea or topic is discussed, but mostly just in person, or in fragmented ways in various places; there’s no good central write-up, summary, or way to find sources on the topic

• an interesting/​useful term or concept is discussed but never clearly defined, or is used/​defined in various somewhat different ways

• there are various different terms introduced or used for seemingly quite similar or at least related things, without the sources noting the other seemingly related terms

Essentially, a lot of things of interest to EAs, and a lot of work by EAs, seems to not yet be neatly consolidated and easily searchable. This makes it harder to learn from and build on the work of others. It may also lead to:

• some avoidable wheel-reinvention

• a confusing proliferation of multiple terms for the same concept

• a confusing proliferation of multiple concepts/​definitions for the same term

So summarise!

One solution to this is for people who are learning about or researching something anyway to:

• Make a link post to a summary of the thing, if a decent summary exists but isn’t already on the EA Forum or LessWrong

• Write a summary of the thing themselves, if no decent summary exists at all, or one exists but doesn’t quite capture how EAs want to use that thing

I think that solution is great (as does the forum’s moderation team). And I’ve been doing some of that summarising myself.

For example, a bunch of great stuff had already been written on differential progress/​intellectual progress/​technological development, information hazards, and moral uncertainty. But a lot of those writings were long and academic, or took a certain prior familiarity with the topic as assumed.

When learning about those topics out of general interest, or in hopes of writing things that built on existing work, I discovered the lack (in my view) of central, accessible summaries on them. So I figured I should try to fill those gaps. I saw this as adding value simply by collecting together, in an accessible way, all the most important points (at least regarding the concepts themselves, not always all of their implications).

And I’ve found similar summary posts by other EAs really helpful myself, and would be keen to see more EAs writing summaries like that. You could set out purposely to do that, or just learn about things and wait until you happen to stumble upon an interesting topic for which no decent summary exists.

(See also “research debt” and “research distillation”.)

So collect!

But writing summaries can take a decent amount of effort and time, and can be challenging. So a quite low effort, easy alternative I’ve started doing is posting “collections” of:

1. Quotes

   • E.g., the long reflection seems an important idea, but discussions of it that are visible on the internet seemed quite rare. So I collected all of the relevant quotes I could find, and a handful of other somewhat relevant sources or concepts.

     • My hope was that this would bring the idea to more people’s attention, give people a better understanding of it, and make it easier for people to build on the idea.

     • Secondarily, I thought it might help highlight just how little work seemed to have been done on the idea (at least based on what was visible on the internet), and thus prompt people to just get cracking on it, because they really may have a decent chance of coming up with some valuable, new insights.

     • (I’m using past tense because Toby Ord’s book may have changed matters somewhat, which is exciting.)

2. Sources on a topic

   • E.g., when researching information hazards, downside risks, the unilateralist’s curse, civilizational collapse, and differential progress etc. for various posts I was writing, I was making notes of all the relevant sources I found, just to help myself out. And then I realised that I’d noted a decent number of relevant sources, and hadn’t found such a list in my travels, so I posted those collections.

   • (I don’t think I’d bother posting collections for topics about less obscure topics, such as existential and global catastrophic risks, because in those cases it might be easier for people to just find a recent paper/​post and go through its sources.)

3. Definitions

   • E.g., I was writing a post that mentioned global catastrophic risks, and realised I wasn’t really sure what the definition for that was. When I looked into that, I found there were several, different definitions floating about. So I collected all the definitions I found.

4. Terms

   • E.g., I was writing some posts about probabilities, and realised there were a variety of different terms/​descriptions that seemed to relate to a relatively similar concept of how “trustworthy” or “grounded” a given probability is. So I collected all such terms/​descriptions which I’d found.

5. Resources in a particular medium (videos, in my case; podcasts, in the case of a prior list)

In each case, making and posting the collection took between almost no time and not much time, because I was already learning about the topic and collecting things for my own sake anyway.

I’ve also seen other EAs usefully collect:

• activities EAs could usefully do (other than working at an EA organisation)

• EA-relevant research topics, questions, or projects that could be productively investigated (e.g., here, here, and here)

I’d be keen to see more EAs posting such collections. It seems to me that, as with summaries, such collections should make it easier for people to learn of, learn about, and build on interesting ideas and topics.

Ideally, I’d hope that other people would comment to add to the collections, so that they can serve better as go-to lists for whatever they’re about. This makes the collections more valuable to future readers, as well as to the person who originally made the collection. To encourage this, I’ve always noted in my collections something like “I intend to add to this list over time. If you know of other relevant work, please mention it in a comment”—and indeed, a few kind souls have usefully done so.

As above, but with extra exclamation marks!

So (I gently suggest, to those who feel this would suit them) go forth and get summarising! Build paths as you go about your travels!

And get collecting! Leave a trail of breadcrumbs where you walked, and signposts to interesting regions you haven’t had time to visit yourself!

And then hopefully those coming after you can travel these paths and visit these regions more easily. And hopefully they can race ahead, or to the side, or lay further paths and breadcrumbs and signposts. And hopefully this can all add up to a relatively low-effort way to help our community do good a little better.

P.S.

It also seems to me that summarising and collecting might be able to serve as one small version of something like a “Task Y”: i.e., something which has “some or all of the following properties:

• Task Y is something that can be performed usefully by people who are not currently able to choose their career path entirely based on EA concerns*.

• Task Y is clearly effective, and doesn’t become much less effective the more people who are doing it.

• The positive effects of Task Y are obvious to the person doing the task.”

[Jamie_Harris]

Relatedly, I’m quite keen on the idea of writing and posting short literature reviews. I suspect that when people look into a topic, they sometimes spend 10-40 hours looking into particular sub-topics, just to inform their own decision-making. It doesn’t take much more time input to write your own notes in the format of a literature review, then spend a little while editing it at the end for clarity.

That’s what I did with my short lit reviews on issues related to developing and training management and leadership expertise. Charity Entrepreneurship also write short reports on each of the main options that they consider in various stages of their recommendation process.

One danger of this approach is that by formatting it as a literature review you might come across as over-confident in your findings (even with lots of caveats), and you leave yourself open to accusations of low rigour.

[MichaelA🔸]

Yeah, I think that’d be another great thing for people to do. Although I’d add that that might be especially valuable when there’s no existing non-EA lit review on the topic (e.g., when the topic is somewhat obscure, or you’re taking a particular angle on a topic that EAs are unusually interested in).

E.g., perhaps there’s already a fairly good lit review on hiring practices in general, and you could add some value by writing a more summarised or updated version. But you might also be able to capture a lot of that value just by making a link post to that review on the forum and noting/​summarising a few key points. Meanwhile, there might be no lit review that’s focused on hiring practices for start-up-style nonprofits in particular, so writing that might be especially worthwhile (if that’s roughly the subtopic/​angle you were interested in anyway).

On the other hand, I think I would’ve guessed that the topic “how to create a highly impactful/​disruptive research team” would be quite far from obscure, and would already have a solid, up-to-date lit review covering what EA people would want to know. But this post suggests there wasn’t an existing lit review with that particular angle, and the post seemed quite interesting and useful to me. So there are probably more “gaps” than I would naively expect, and thus substantial value in lit reviews on certain topics I would naively expect are already “covered”.

[Aaron Gertler 🔸]

Great post! I second this recommendation. Because I’m gathering a lot of material in the course of working on the next edition of the EA Handbook, I’ll look for opportunities to post some of that to the Forum (in cases where doing so won’t overlap too much with the Handbook).

[saulius]

an interesting/​useful term or concept is discussed but never clearly defined, or is used/​defined in various somewhat different ways

I think that EA Concepts did a good job mitigating this problem regarding concepts, although I’m unsure how many people read it. In general, it seems that it would be better if these sort of collections would be in one place (e.g., like they are in EA Concepts), in addition to being scattered throughout the EA Forum. Maybe an EA wiki (which is already in the works) could solve such a problem, provided enough people use it. Or maybe EA concepts could be expanded. Or maybe we should also post collections of summaries/​collections on the Forum.

[MichaelA🔸]

I agree that having some central directory or collection of the summaries/​collections would be ideal. And I think all of those suggestions for achieving that are good.

Also, I think EA Concepts is great. And I think people who haven’t checked it out should, or should keep it in mind when they encounter a concept they’re unfamiliar with. (Conceptually also performs a similar function. It isn’t explicitly EA-focused, but it was made by EAs and covers a lot of concepts EAs like to use.) However:

• EA Concepts doesn’t cover everything

• The entries are quite short, which is of course valuable in some ways, but also means there could also be value in longer summaries that build on, go beyond, and add detail to what’s in those entries

• The entries seem slightly old now, so they may not reflect the latest work, nor contain links to it. A forum post will later suffer the same issue, but it allows comments, so people could comment to add discussion of or links to more recent work. (That said, it seems relatively rare for people to comment on older posts, which I think is a shame.)

This suggests that one possible solution would be for the people behind EA Concepts to crowdsource (and then vet) new entries, and/​or updated versions of existing entries.

[MichaelA🔸]

P.S. #2: There are three questions I have about how one should do the “collections”, which didn’t seem necessary to include in the above post itself, but which I’d be interested in people’s thoughts on.

1. Should the collections be full posts on shortform comments?

So far, I’ve posted all my collections as “shortform” comments, rather than full posts, except one (Quotes about the long reflection). This is because most have been quite short, and pretty much just lists, so they didn’t feel substantial or refined enough to be full posts.

But shortform comments aren’t displayed as prominently upon posting or when using the search function (the search function will show the user’s name but not any snippet of the comment). So there’s a higher chance that people who would’ve been interested in the collection will miss it.

That might make sense, as people have limited attention, and these collections are arguably not especially important. But I’m unsure precisely where to draw the line. And I think it’d be nice if they were easier to find through the search function, at least (e.g., if the first few words of shortform comments were displayed when searching, or something like that).

2. Is my approach to collecting sources actually useful, given the fact that relevant papers will have reference lists?

I think the answer is “Yes”, given that:

• relevant papers’ reference lists will contain many not especially relevant sources

• relevant papers’ reference lists will typically not contain “informal sources” like blog posts or podcasts, despite these often being quite useful, especially for topics neglected thus far by mainstream academia

• I’m collecting sources on relatively neglected topics (e.g., the unilateralist’s curse), or topics where the work so far may be scattered, use different keywords, and/​or be hard to spot among lots of other somewhat related work that isn’t quite what I/​EAs are after (e.g., for civilizational collapse, arguably)

• EAs might not necessarily even think to look into the topic (i.e., the collection might help EAs learn of or come to care about the topic), or might miss the relevant papers in the first place

But I’m not sure about this. Maybe it actually wouldn’t be worth people’s time to make collections of sources in particular. (Although, again, the time investment can be pretty small when you’re learning about a topic anyway.)

EDIT: I’m now more convinced that collecting sources is useful, and upvotes of my collections seem to provide weak evidence in support of that.

3. Would it be better to spend the same amount of time producing a lower number of more sophisticated collections, as proper databases or something (e.g., using Mendeley?)?

Somehow I didn’t learn about these more sophisticated approaches at all during my university studies or when writing a paper, so I don’t know how those approaches work or how valuable they are, and thus I can’t really answer this. But maybe this is the approach I and/​or others should be taking, either when posting these collections or just when doing research in general.

[MichaelA🔸]

Update: The new tags system now captures some of the value “collections” can capture (which is great). Though there are two aspects* of the tag system that mean collections can still add value:

• You can only tag things that are already on the EA Forum. Many relevant sources will be elsewhere (e.g. EA-aligned organisations’ own sites, LessWrong, Wikipedia, books). Indeed, the fact relevant sources are dispersed across the web was always a key reason I saw “collections” as useful.

• There aren’t tags for all topics. (Arguably, this is a feature, not a bug, but it still provides room for collections to add value.)

So I think there can still be value in collecting sources even on topics that now have tags. In such cases, it seems it would be best to ensure that the EA Forum posts with the relevant tag are in the relevant collection, and vice versa.

And more obviously, there can still be value in collecting sources on topics for which there isn’t a tag. And perhaps if that topic later gets a tag, people can easily kick that tag off using that collection.