I’m interested in the recommendation of CFAR (though I appreciate it is not funded by the LTFF). What do you think are the top ideas regarding epistemics that CFAR has come up with that have helped EA/the world?
You mention double cruxing in the other post discussing CFAR. Rather than an innovation, isn’t this merely agreeing on which premise you disagree on? Similarly, isn’t murphyjitsu just the pre-mortem, which was defined by Kahneman more than a decade ago?
I also wonder why CFAR has to charge people for their advice. Why don’t they write down all of their insights and put it online for free?
Hmm, it seems to me like you are modeling the goals and purpose of CFAR quite differently than I do. I model CFAR primarily as an educational institution, with a bit of research, but mostly with the goal of adapting existing knowledge and ideas from cognitive science and other disciplines into more practical applications (hence the name “Center for Applied Rationality”).
In my review of CFAR last round, I listed
“Establishing Epistemic Norms”
“Recruitment” and
“Training”
as the three primary sources of value add of CFAR, which importantly lacks what you seem to be evaluating above, and I would describe as “research” (the development of new core concepts in a given field).
I think in the space of the three axes I outlined, CFAR has been pretty successful, as I tried to explain during my last review, and with some additional evidence that I have heard particularly good things about the Artificial Intelligence Risk for Computer Scientist workshops, in that they seem to be able to facilitate a very unique environment in which people with a strong technical background can start engaging with AI Alignment questions.
I don’t think of CFAR’s value being primarily generated by producing specific insights of the type of Kahneman’s work (though I do think there have been some generated by CFAR that I found useful) but in the teaching and communication of ideas in this space, and the feedback loop that comes from seeing how trying to teach those techniques actually works (often uncovering many underspecified assumptions, or the complete lack of effectiveness of an existing technique).
Murphyjitsu is indeed just pre-mortem, and I think is cited as such in both the handbook and at the workshop. It’s just that the name of premortem didn’t stick with participants, and so people changed it to something that people seemed to actually be able to engage with (and there was also a bunch of other valuable iteration on how you actually teach the relevant skill).
I also wonder why CFAR has to charge people for their advice. Why don’t they write down all of their insights and put it online for free?
This seems to again approach CFAR’s value add from a different perspective. While I would be in favor of CFAR publishing their handbook, it’s clear to me that this would not in any real way compete with the value of existing CFAR workshops. Universities have classes, and very few people are able to learn from textbooks alone, and their and CFAR’s value comes from the facilitation of classes, active exercises, and the fast feedback loop that comes from having an instructor right in the room with you.
I am in favor of CFAR writing more things down, but similar to how very few people can learn calculus or linear algebra from nothing but a book, with no accountability structure or teacher to ask questions off, is it also unlikely that many people can learn the relevant subsets of cognitive science and decision-making from just a written description (I think some can, and our community has a much higher fraction of autodidacts than the general population, but even for those, learning without a teacher is usually still a lot slower).
I do think there are other benefits to writing things down, like having more cross-examination of your ideas by others, giving you more information about them, “just writing down their ideas” would not effectively replace CFAR’s value proposition.
-----
I am however just straightforwardly confused by what you mean by “isn’t double crux merely agreeing on which premise you disagree on?”, since that seems to have relatively little to do with basically any formulation of double crux I’ve seen. The goal of double crux is to formulate which observations would cause both you and the other person to change their mind. This has at most a tenuous connection to “which premise do we disagree on?”, since not all premises are necessary premises for a conclusion, and observations tend to only very rarely directly correspond to falsifying one specific premise. And human cognition usually isn’t structured by making explicit premises and arguing from them, making whatever methodology you are comparing it to not really be something that I have any idea how to apply in conversation (if you ask me “what are my premises for the belief that Nature is the most prestigious science journal?” then I definitely won’t have a nice list of premises I can respond with, but if you ask me “what would change my mind about Nature being the most prestigious science journal?” I might be able to give a reasonably good answer and start having a productive conversation).
If the retreats are valuable, one would expect them to communicate genuinely useful concepts and ideas. Which ideas that CFAR teaches do you think are most useful?
On the payment model, imagine that instead of putting their material on choosing a high impact career online, 80k charged people £3000 to have 4 day coaching and networking retreats in a large mansion, afterwards giving them access to the relevant written material. I think this would shave off ~100% of the value of 80k. The differences between the two organisations don’t seem to me to be large enough to make a relevant difference to this analysis when applied to CFAR. Do you think there is a case for 80k to move towards the CFAR £3k retreat model?
**
On double cruxing, here is how CFAR defines double cruxing
“Let’s say you have a belief, which we can label A (for instance, “middle school students should wear uniforms”), and that you’re in disagreement with someone who believes some form of ¬A. Double cruxing with that person means that you’re both in search of a second statement B, with the following properties:
1. You and your partner both disagree about B as well (you think B, your partner thinks ¬B)
2. The belief B is crucial for your belief in A; it is one of the cruxes of the argument. If it turned out that B was not true, that would be sufficient to make you think A was false, too.
3. The belief ¬B is crucial for your partner’s belief in ¬A, in a similar fashion.”
So, if I were to double crux with you, we would both establish which were the premises we disagree on that cause our disagreement. B is a premise in the argument for A. This is double cruxing, right?
You say:
“if you ask me “what are my premises for the belief that Nature is the most prestigious science journal?” then I definitely won’t have a nice list of premises I can respond with, but if you ask me “what would change my mind about Nature being the most prestigious science journal?” I might be able to give a reasonably good answer and start having a productive conversation”
Your answer could be expressed in the form of premises right? Premises are just propositions that bear on the likelihood of the conclusion
On the payment model, imagine that instead of putting their material on choosing a high impact career online, 80k charged people £3000 to have 4 day coaching and networking retreats in a large mansion, afterwards giving them access to the relevant written material.
CFAR’s model is actually pretty similar to 80k’s here. CFAR generally either heavily discounts or waives the cost of the workshop for people they think are likely to contribute to the long-term-future, or are more broadly promising, and who don’t have the money to pay for the workshop. As such the relevant comparison is more “should 80k offer paid coaching (in addition to their free coaching) at relatively high rates for people who they think are less likely to contribute to improving the world, if the money they earn from that allows them to offer the other free coaching services (or scale them up by 30% or something like that)”, to which my answer would be “yes”.
My sense is that 80k is in a better-funded position, and so this tradeoff doesn’t really come up, but I would be surprised if they never considered it in the past (though career coaching is probably somewhat harder to monetize than the kind of product CFAR is selling).
I also think you are underestimating to what degree the paid workshops were a necessity for CFAR historically having gotten to exist. Since there is a lot of downtime cost in being able to run workshops (you need to have a critical mass of teaching staff, you need to do a lot of curriculum development, have reliable venues, etc.) and the EA community didn’t really exist yet when CFAR got started, it was never really an option for CFAR to fully run off of donations, and CFAR additionally wanted to make sure it actually produced something that people would be willing to pay for, so offering paid workshops was one of the only ways to achieve those two goals. I also generally think it’s a good idea for projects like CFAR to ensure that they are producing a product that people are willing to pay significant amount of money for, which is at least a basic sanity check on whether you are doing anything real.
As an example, I encouraged Lynette to ask people whether they would be willing to pay for her coaching, and ideally ask them for at least some payment even if she can’t break even, to make sure that the people she is offering services to are filtered for the people who get enough value out of it to spend $50 per session, or something in that space (she had also considered that already on her own, though I don’t remember the current state of her asking her clients for payment).
I just remembered that 80k actually did consider monetizing part of it’s coaching in 2014, which would have probably resulted in a pretty similar model to CFAR:
Is there a subsection of the audience who might be willing to pay forcoaching?
We’re interesting in the possibility of making part of the coaching self-funding. Our best guess was that the people who will be most willing to pay for coaching are people from tech and finance backgrounds aged 25-35. We found that about 20% of the requests fell in this category, which was higher than our expectations.
Re retreats:
I think it’s quite plausible that 80k organizing retreats would be quite valuable, in particular in a world where CFAR isn’t filling that current niche. CEA also organized a large number of retreats of a similar type in the last year (I attended one on individual outreach, and I know that they organized multiple retreats for group organizers, and at least one operations retreat) presumably because they think that is indeed a good idea (the one that I attended did seem reasonably valuable, and a lot of the design of it was clearly influenced by CFAR workshops, though I can’t speak on whether that overall initiative was worth it).
afterwards giving them access to the relevant written material
I agree that 80k also has a lot of impact via their written material, but I think that is because they have invested a very large fraction of their resources into producing those materials (80K would likely be unable to run as many workshops as CFAR and also produce the written material). I think if 80k was focusing primarily on coaching, it would be very unlikely to produce good written material that would stand well on its own, though I expect it would still produce a good amount of value (and it might still produce some writings, but likely not ones that make much sense without the context of the coaching, similar to CFAR). As such I am skeptical of your claim that switching to that model would get rid of ~100% of 80k’s value. I expect it would change their value proposition, but likely still have a good chance of being competitive in terms of impact (and fully switching towards a coaching model was something that I’ve heard 80k consider multiple times over the years).
Your answer could be expressed in the form of premises right? Premises are just propositions that bear on the likelihood of the conclusion
I think if you define “premise” more broadly to mean “propositions that bear on the likelihood of the conclusion” then you are closer, but still not fully there. A crux would then be defined “a set of premises that when falsified, would provide enough evidence that you would change your mind on the high-level claim”, which is importantly still different from “identifying differences in our premises”, in particular it emphasizes identifying specific premises that are particularly load-bearing for the argument at hand.
(This wouldn’t be a very standard usage of “premise” and doesn’t seem to align super well with any definitions I can find in any dictionaries, which all tend to either be about logical inference or about subsets of a specific logical argument that is being outlined, but doesn’t seem like a horrible stretch from available definitions. Though I wouldn’t expect people to intuitively know what you mean by that definition of “premise”)
I do still expect people to give quite drastically different answers if you ask them “is ‘not X’ a premise of your belief?” vs. “would observing X change your mind about this belief?”. So I wouldn’t recommend using that definition if you were actually trying to do the thing that double crux is trying to do, even if you define it beforehand. I do think that the norms from (classical) rhetoric and philosophy of trying to identify differences in your premises are good norms and generally make conversations go better. I agree that Double Crux is trying to operationalize and build on that, and isn’t doing some weird completely novel thing, though I do think it extends on it in a bunch of non-trivial ways.
I disagree that 80k should transition towards a £3k retreat + no online content model, but it doesn’t seem worth getting into why here.
On premises, here is the top definition I have found from googling… “a previous statement or proposition from which another is inferred or follows as a conclusion”. This fits with my (and CFAR’s) characterisation of double cruxing. I think we’re agreed that the question is which premises you disagree on cause your disagreement. It is logically impossible that double cruxing extends this characterisation.
I disagree that 80k should transition towards a £3k retreat + no online content model, but it doesn’t seem worth getting into why here.
I never said 80k should transition towards a retreat + no online content model. What I said is that it seems plausible to me it would still produce a lot of value in that case, though I agree that their current model seems likely a better fit for them, and probably overall more valuable. Presumably you also disagree with that, but it seemed important to distinguish.
“a previous statement or proposition from which another is inferred or follows as a conclusion”
Given that in the scenario as outlined, there was no “previous statement” or “previous proposition”, I am still confused how you think this definition fits. In the scenario at hand, nobody first outlined their complete argument for why they think the claim discussed is true, and as such, there is no “previous statement or proposition” that can be referred back to. This definition seems to refer mostly to logical argument, which doesn’t really apply to most human cognition.
I am not super excited about debating definitions, and we both agree that using the word premise is at least somewhat close to the right concept, so I am not very excited about continuing this thread further. If you really care about this, I would be glad to set up an experiment on mechanical turk in which we ask participants to list the necessary premises of a belief they hold, and see how much their responses differ from asking them what observations would change their mind about X. It seems clear to me that their responses would differ significantly.
which premises you disagree on cause your disagreement.
This is still only capturing half of it, even under the definition of premise that you’ve outlined here, which seems to be a reasonable definition of what a crux for a single participant in the conversation is. A double crux would be “a set of premises, that when viewed as a new conjunctive proposition you both assign opposite truth values to, that when flipped would cause both of you to change their mind”. Though that alone obviously doesn’t yet make a procedure, so there is still a bunch more structure, but I would think of the above as an accurate enough description to start working with it.
It is logically impossible that double cruxing extends this characterisation.
I don’t think I really know how to engage with this. Obviously it’s possible for double-crux to extend this characterization. I even outlined a key piece that was missing from it in the above paragraph.
But it’s also a procedure that is meant to be used with real people, where every bit of framing and instruction matters. If you really believe this, let us run a test and just give one group of people the instruction “find the premises on which you disagree on that cause your disagreement” and the other group the full double crux worksheet. Presumably you agree that the behavior of those groups will drastically differ.
You maybe have something more specific in mind when you mean “logically impossible”, but given that we are talking about a high-level procedure proofs of logical impossibility seem highly unlikely to me.
To give an answer to the question of what material that CFAR teaches at their workshops I consider valuable, here is a list of classes that I’ve seen have a big impact on individuals, sometimes including myself and for which I also have separate reasons to think they are valuable.
Units of Exchange
Basically an introduction into consequentialist reasoning, trying to get people to feel comfortable trading off different resources that they previously felt were incomparable. A lot of the core ideas in EA are based off of this, and I think it’s generally a good introduction into that kind of thinking.
Inner Simulator
Basic practical introduction into System 1 and System 2 level processing, and going into detail on how to interface between S1 and S2 processing.
Trigger-Action Planning
Basic introduction into how associative processing works in the brain, where it tends to work, and where it tends to fail, and how to work around those failure modes. In the literature the specific technique is known as “Mental contrasting with implementation intentions” and is probably one of the most robust findings in terms of behavior change in behavioral psychology.
This class is often particularly valuable because I’ve seen it provide people with their first real mechanistic model of the human mind, even if simplified. A lot of people don’t really have any mechanistic baseline of how human cognition works, and so the simplified statement of “humans cognition can be modeled as a large pile of programmed ‘if-then-statements’ can get people initial traction on figuring out how their own mind works”.
Goal Factoring
For most attendees this has a lot of overlap with basic 80k coaching. Practice in trying to ask yourself repeatedly “why is this thing that I am doing important to me, and could I achieve it some better way?”, and this is probably the class that I’ve seen that had the biggest effects in terms of causing career changes in participants, mostly by getting them to think about why their are pursuing the career they are pursuing, and how they might be able to achieve their goals better.
Understanding Shoulds
This covers a lot of material in the Minding Our Way “Replacing Guilt” series, which many EAs and people that I trust have reported to have benefited a lot from, and which core conclusions are quite important for a lot of thinking about how to have a big impact in the world, how morality works, reminding people that they are allowed to care about things, etc.
Focusing
Based on Gendlin’s “Focusing” book and audiobook, it teaches a technique that forms the basis of a significant fraction of modern therapeutic techniques and I consider a core skill for doing emotional processing. I’ve benefited a lot from this, and it also has a pretty significant amount of evidence behind it (both in that it’s pretty widely practiced, and in terms of studies), though only for the standards of behavioral psychology, so I would still take that with a grain of salt.
Systemization
This is basically “Getting Things Done” the book, in a class. I, and a really large number of people I’ve worked with and who seem to be good at their job, consider this book core reading for basically anyone’s personal productivity, and I think teaching this is pretty valuable. This class in particular tends to help people who bounced off of the book, which still recommends a really large fraction of practices that I’ve seen in particular young people bounce off of, like putting everything into binders and getting lots of cabinets to put those binders in, instead of having good digital systems.
Double Crux
We’ve discussed this one above a good amount. In particular I’ve seen this class cause a bunch of people to have productive conversations that have previously had dozens of hours of unproductive or really conflict-heavy conversations, the most easily referenced and notable of which is probably a conversation between Scott Garrabrant and Eric Drexler that I think significantly moves the conversation around AI Alignment forward
All of the above strike me as pretty robustly good concepts to teach, already make up more than 50% of intro workshops, and that are pretty hard to get a good grasp on without reading ~6 books, and having substantial scaffolding to actually put time into practicing the relevant ideas and techniques.
I agree that these are pretty valuable concepts to learn. At the same time, I also believe that these concepts can be learned easily by studying the corresponding written materials. At least, that’s how I learned them, and I don’t think I’m different from the average EA in this respect.
But I also think we shouldn’t be speculating about this issue, given its centrality to CFAR’s approach. Why not give CFAR a few tens of thousands of dollars to (1) create engaging online content that explains the concepts taught at their workshops and (2) run a subsequent RCT to test whether people learn these concepts better by attending a workshop than by exposing themselves to that content?
I would be open to helping run such an RCT, and by default would expect the written material without further assistance to have relatively little impact.
I also think that for many people asking them to read the related online material will have a much lower completion rate than going to a workshop, and figuring out how to deal with that would be a major uncertainty in the design of the RCT. I have many friends that I tried to get to desperately read the material that explains the above core concepts, sadly without success, who finally got interested enough into all of the above after attending a CFAR workshop.
In my last 5 years of working in EA and the rationality community, I have repeatedly been surprised by the degree to which even very established EAs have not read almost any introductions to the material I outlined above, and where the CFAR workshop was their first introduction into the material. This includes large parts of the staff at CEA, as well as many core group organizers I’ve met.
I don’t expect CFAR putting out online material to help much with this, since roughly the same holds true for 80k material, and a lot of the concepts above actually already have good written explanations to them.
You seem to be very optimistic about getting people to read written content, whereas my experience has been that people are very reluctant to read content of any type that is not fiction or is of very high relevance to some particular niche interest of theirs. Inviting people to a workshop seems to work a lot more reliably to me, though obviously with written material you get a much broader reach, which can compensate for the lower conversion rate (and which medium makes sense to optimize I think hinges a lot on whether you care about getting a small specific set of people to learn something, vs. trying to get as many people as possible to learn something).
Thank you. Your comment has caused me to change my mind somewhat. In particular, I am now inclined to believe that getting people to actually read the material is, for a significant fraction of these people, a more serious challenge than I previously assumed. And if CFAR’s goal is to selectively target folks concerned with x-risk, the benefits of insuring that this small, select group learn the material well may justify the workshop format, with its associated costs.
I would still like to see more empirical research conducted on this, so that decisions that involve the allocation of hundreds of thousands of EA dollars per year rest on firmer ground than speculative reasoning. At the current margin, I’d be surprised if a dollar given to CFAR to do object-level work achieves more than a dollar spent in uncovering “organizational crucial considerations”—that is, information with the potential to induce a major shift in the organization’s direction or priority. (Note that I think this is true of some other EA orgs, too. For example, I believe that 80k should be using randomization to test the impact of their coaching sessions.)
Hi Oliver, Is there a sequence out there explaining these terms? A quick Google/LW/CFAR search didn’t throw anything up which covered all the concepts you mention above (there’s a sequence called Hammertime, but it didn’t cover all the concepts you mention). I think one of the benfits of a centralized source of information is that it’s accessible and intuitive to find. In the current state, it seems that you would have to go out of your way to find these kinds of writeups, and possibly not even know they exist.
I don’t think there is a single link, though most of the concepts have a pretty good canonical resource. I do think it usually takes quite a bit of text to convey each of those concepts, so I don’t think creating a single written reference is easily feasible, unless someone wants to produce multiple books worth of content (I’ve historically been impressed with how much content you can convey in a 1.5 hour long class, often 10 blog posts worth, or about half of a book).
I don’t think I have the time to compile a full list of resources for each of these concepts, but I will share the top things that come to mind.
Units of Exchange: I think microeconomics classes do a pretty good job of this, though are usually a bit abstract. A lot of writing of Scott Alexander gets at this, with the best introduction probably being his “Efficient Charity: Do unto others...”
Inner Simulator: Covered pretty well by Thinking: Fast and Slow by Daniel Kahneman
Trigger-Action Planning: Also covered pretty well by Thinking Fast and Slow, though, with some Getting Things Done thrown into it
Goal Factoring: I don’t actually know a good introduction to this, alas.
I don’t think this is very important for my overall view on CFAR’s curriculum, but FWIW I was quite surprised by you describing Focusing as
a technique that forms the basis of a significant fraction of modern therapeutic techniques and I consider a core skill for doing emotional processing. I’ve benefited a lot from this, and it also has a pretty significant amount of evidence behind it (both in that it’s pretty widely practiced, and in terms of studies), though only for the standards of behavioral psychology, so I would still take that with a grain of salt.
Maybe we’re just using “significant fraction” differently, but my 50% CI would have been that focusing is part of 1-3 of the 29 different “types of psychotherapy” I found on this website (namely “humanistic integrative psychotherapy”, and maybe “existential psychotherapy” or “person-centred psychotherapy and counselling”). [Though to be fair on an NHS page I found, humanistic therapy was one of 6 mentioned paradigms.] Weighting by how common the different types of therapy are, I’d expect an even more skewed picture: my impression is that the most common types of therapy (at least in rich, English-speaking countries and Germany, which are the countries I’m most familiar with) are cognitive-behavioral therapy and various kinds of talking therapy (e.g. psychoanalytic, i.e. broadly Freudian), and I’d be surprised if any of those included focusing. My guess is that less than 10% of psychotherapy sessions happening in the above countries include focusing, potentially significantly less than that.
My understanding had been that focusing was developed by Eugene Gendlin, who after training in continental philosophy and publications on Heidegger became a major though not towering (unlike, say, Freud) figure in psychotherapy—maybe among the top decile but not the top percentile in terms of influence among the hundreds of people who founded their own “schools” of psychotherapy.
I’ve spent less than one hour looking into this, and so might well be wrong about any of this—I’d appreciate corrections.
Lastly, I’d appreciate some pointers to studies on focusing. I’m not doubting that they exist—I’m just curious because I’m interested in psychotherapy and mental health, but couldn’t find them quickly (e.g. I searched for “focusing Gendlin” on Google Scholar).
I haven’t looked super much into the literature on this so I might be wrong, my sense was that it was more of a case of “lots of therapeutic techniques share a lot of structure, and Gendlin formalized it into a specific technique, but a lot of them share a lot of structure with what Gendlin is doing”, which makes sense, because that’s how focusing was developed. From the Wikipedia article:
Gendlin developed a way of measuring the extent to which an individual refers to a felt sense; and he found in a series of studies that therapy clients who have positive outcomes do much more of this. He then developed a way to teach people to refer to their felt sense, so clients could do better in therapy. This training is called ‘Focusing’. Further research showed that Focusing can be used outside therapy to address a variety of issues.
The thing that made me more comfortable saying the above was that Gendlin’s goal (judging from the focusing book I read and the audiobook I listened to) seems to have been in significant parts a study into “what makes existing therapeutic techniques work”, instead of “let’s develop a new technique that will revolutionize therapy”, so even if a school of therapy isn’t downstream of Gendlin, you expect a good fraction to still have focusing-like things in them, since Gendlin seemed to be more interested in refining techniques instead of revolutionizing them.
I do agree that I should probably stop using words like “significant fraction”. I intended to mean something like 20%-30% of therapy sessions will likely include something that is pretty similar to focusing, even if it isn’t exactly called that, which still seems roughly right to me and matches with my own experience of therapy with a practitioner who specialized in CBT and some trauma-specific therapies, but our actual sessions weren’t really utilizing either of those schools and were basically just focusing sessions, which to that therapist seemed like the natural thing to do in the absence of following a more specific procedure.
Some of my impression here also comes from two textbooks I read on therapy whose names I currently forgot, both of which were mostly school-independent and seemed to emphasize a lot of focusing-like techniques.
However, I don’t have super strong models here, and a significant fraction of my models are downstream of Gendlin’s own writing (who as I said seems to describe focusing more as “the thing that makes most type of therapy work”), so I am pretty open to being convinced I am wrong about this. I can particularly imagine that Freudian approaches could do less focusing, since I’ve basically not interacted with anything in that space and feel kinda averse to it, so I am kind of blind to a significant fraction of the therapy landscape.
I hadn’t considered the possibility that techniques prior to Gendlin might have included focusing-like techniques, and especially that he’s claiming to have synthesized what was already there. This makes me less confident in my impression. What you say about the textbooks you read definitely also moves my view somewhat.
(By contrast, what you wrote about studies on focusing probably makes me somewhat reduce my guess on the strength of the evidence of focusing, but obviously I’m highly uncertain here as I’m extrapolating from weak cues—studies by Gendlin himself, correlational claim of intuitively dubious causal validity—rather than having looked at the studies themselves.)
This all still doesn’t square well with my own experiences with and models of therapy, but they may well be wrong or idiosyncratic, so I don’t put much weight on them. In particular, 20-30% of sessions still seems higher than what I would guess, but overall this doesn’t seem sufficiently important or action-relevant that I’d be interested to get at the bottom of this.
This makes sense to me as a response to Halstead’s question. However, it actually makes me a bit less confident that (what you describe as) CFAR’s reluctance to increase legibility is a good idea. An educational institution strikes me as something that can be made legible way more easily and with fewer downsides than an institution doing cutting-edge research in an area that is hard to communicate to non-specialists.
In my experience teaching rationality is more tricky than the reference class education, and is an area which is kind of hard to communicate to non-specialists. One of the main reasons seems to be many people have somewhat illusory idea how much they understand the problem.
I don’t think most of the costs that I described that come from legibility differ that much between research and educational institutions? The american public education system, as well as many other public education systems actually strike me as core examples of systems that have suffered greatly due to very strong forces on legibility in all of their actions (like standardized curricula combined with standardized testing). I think standardized testing is pretty good in a lot of situations, but that in this case it resulted in a massive reduction in variance in a system where most of the value comes from the right tail.
I agree that there are also other separate costs to legibility in cutting-edge domains, but the costs on educational institutions still seem quite significant to me. And most of the costs are relatively domain-general.
Thanks, that helps me understand where you’re coming from, though it doesn’t change my views on CFAR. My guess is we disagree about various more general claims around the costs and benefits of legibility, but unfortunately I don’t have time right now to articulate my view on this.
Very roughly, I think I (i) agree with you that excessive optimization for easily measurable metrics has harmed the public education system, and in particular has reduced benefits from the right tail, (ii) disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole, (iii) don’t think this tells us much about CFAR as I both think that CFAR’s environment makes increased legibility less risky (due to things like high goal-alignment with important stakeholders such as funders, a more narrow target audience, …) and also that there are plenty of ways to become more legible that don’t incur risks similar to standardized testing or narrow optimization for quantitative metrics (examples: qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these; maintain a list of lessons learned).
(I upvoted your reply, not sure why it was downvoted by someone else.)
(Reply written after the paragraph was added above)
Thanks for the elaboration! Some quick thoughts:
qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these
I think CFAR has done at least everything on this list of examples. Which you might already be aware of, but wanted to make sure is common knowledge. There are a significantnumberof posts trying to explain CFAR at a high-level, and the example workshop schedule summarizes all the classes at a high-level. CFAR has also published the number of workshops they’ve run and their total attendance in their impact reports and on their homepage (currently listing 1045 alumni). Obviously I don’t think that alone is sufficient, but it seemed plausible that a reader might walk away thinking that CFAR hadn’t done any of the things you list.
disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole
I think there is some truth to this interpretation, but I think it’s overall still wrong enough that I would want to correct it. I think the education system has many goals, and I don’t think I would summarize it’s primary output as “quality-weighted sum of generated research”. I don’t think going into my models of the education system here is going to be super valuable, though happy to do that at some other point if anyone is interested in them. My primary point was that optimizing for legibility clearly has had large effects on educational institutions, in ways that would at least be harmful to CFAR if affected in the same way (another good example here might be top universities and the competition for getting into all the top 10 ranking, though I am less confident of the dynamics of that effects).
(Edit the below was written before Max edited the second paragraph into his comment)
Seems good! I actually think considerations around legibility are quite important and where I expect a good amount of intellectual progress to be made by talking to each other, so I would like to see your perspective written up and engage with it.
I also want to make sure that it’s clear that I do think CFAR should be more legible and transparent (as I said in the writeup above). I have some concerns with organizations trying to be overly legible, but I think we both agree that at the current margin it would be better for CFAR to optimize more for legibility.
(I’ve sadly had every single comment of mine on this thread strong-downvoted by at least one person, and often multiple people. My sense is that CFAR is a pretty polarizing topic, which I think makes it particularly important to have this conversation, but seems to also cause some unfortunate voting patterns that feel somewhat stressful to deal with.)
I’m sorry to see the strong downvotes, especially when you’ve put in more effort on explaining your thinking and genuinely engaging with critiques than perhaps than all other EA Fund granters put together. I want you to know that I found your explanations very helpful and thought provoking, and really like how you’ve engaged with criticisms both in this thread and the last one.
(I’m wondering whether this phenomenon could also be due to people using downvotes for different purposes. For example, I use votes roughly to convey my answer to the question “Would I want to see more posts like this on the Forum?”, and so I frequently upvote comments I disagree with. By contrast, someone might use votes to convey “Do I think the claims made in this comment are true?”.)
Data point: I often feel a pull towards up-voting comments that I feel have stimulated or advanced my thinking or exemplify a valuable norm of transparency and clarity, but then I hold back because I think I might disagree with the claims made or I think I simply don’t know enough to judge those claims. This is based on a sense that I should avoid contributing to information cascade-type situations (even if, in these cases, any contribution would only be very slight).
This has happened multiple times in this particular thread; there’ve been comments of Oliver’s that I’ve very much appreciated the transparency of, but with which I felt like I still might slightly disagree overall, so I avoided voting either way.
(I’m not saying this is the ideal policy, just that it’s the one I’ve taken so far.)
For whatever it’s worth, this seems right to me, and I do want to make sure that people know that I do think CFAR should try to be more legible at the margin
I mentioned this in my writeup above:
I think that CFAR is still likely optimizing too little towards legibility, compared to what I think would be ideal for it. Being legible allows an organization to be more confident that its work is having real effects, because it acquires evidence that holds up to a variety of different viewpoints.
I do think the question of what the correct outcome measurements for an impact evaluation would be is non-trivial, and would be interested in whether people have any good ideas for good outcome measurements.
An aside: I had never heard of ‘Murphyjitsu’ before, but use pre-mortems in my personal and professional life regularly. I’m surprised people found the name ‘Murphyjitsu’ easier to engage with!
It’s a bit of a more playful term, which I think makes sense in the context of a workshop, but I also use the two terms interchangeably and seen CFAR staff do the same, and usually use pre-mortem when I am not in a CFAR context.
I don’t have strong opinions on which term is better.
I’m interested in the recommendation of CFAR (though I appreciate it is not funded by the LTFF). What do you think are the top ideas regarding epistemics that CFAR has come up with that have helped EA/the world?
You mention double cruxing in the other post discussing CFAR. Rather than an innovation, isn’t this merely agreeing on which premise you disagree on? Similarly, isn’t murphyjitsu just the pre-mortem, which was defined by Kahneman more than a decade ago?
I also wonder why CFAR has to charge people for their advice. Why don’t they write down all of their insights and put it online for free?
Hmm, it seems to me like you are modeling the goals and purpose of CFAR quite differently than I do. I model CFAR primarily as an educational institution, with a bit of research, but mostly with the goal of adapting existing knowledge and ideas from cognitive science and other disciplines into more practical applications (hence the name “Center for Applied Rationality”).
In my review of CFAR last round, I listed
“Establishing Epistemic Norms”
“Recruitment” and
“Training”
as the three primary sources of value add of CFAR, which importantly lacks what you seem to be evaluating above, and I would describe as “research” (the development of new core concepts in a given field).
I think in the space of the three axes I outlined, CFAR has been pretty successful, as I tried to explain during my last review, and with some additional evidence that I have heard particularly good things about the Artificial Intelligence Risk for Computer Scientist workshops, in that they seem to be able to facilitate a very unique environment in which people with a strong technical background can start engaging with AI Alignment questions.
I don’t think of CFAR’s value being primarily generated by producing specific insights of the type of Kahneman’s work (though I do think there have been some generated by CFAR that I found useful) but in the teaching and communication of ideas in this space, and the feedback loop that comes from seeing how trying to teach those techniques actually works (often uncovering many underspecified assumptions, or the complete lack of effectiveness of an existing technique).
Murphyjitsu is indeed just pre-mortem, and I think is cited as such in both the handbook and at the workshop. It’s just that the name of premortem didn’t stick with participants, and so people changed it to something that people seemed to actually be able to engage with (and there was also a bunch of other valuable iteration on how you actually teach the relevant skill).
This seems to again approach CFAR’s value add from a different perspective. While I would be in favor of CFAR publishing their handbook, it’s clear to me that this would not in any real way compete with the value of existing CFAR workshops. Universities have classes, and very few people are able to learn from textbooks alone, and their and CFAR’s value comes from the facilitation of classes, active exercises, and the fast feedback loop that comes from having an instructor right in the room with you.
I am in favor of CFAR writing more things down, but similar to how very few people can learn calculus or linear algebra from nothing but a book, with no accountability structure or teacher to ask questions off, is it also unlikely that many people can learn the relevant subsets of cognitive science and decision-making from just a written description (I think some can, and our community has a much higher fraction of autodidacts than the general population, but even for those, learning without a teacher is usually still a lot slower).
I do think there are other benefits to writing things down, like having more cross-examination of your ideas by others, giving you more information about them, “just writing down their ideas” would not effectively replace CFAR’s value proposition.
-----
I am however just straightforwardly confused by what you mean by “isn’t double crux merely agreeing on which premise you disagree on?”, since that seems to have relatively little to do with basically any formulation of double crux I’ve seen. The goal of double crux is to formulate which observations would cause both you and the other person to change their mind. This has at most a tenuous connection to “which premise do we disagree on?”, since not all premises are necessary premises for a conclusion, and observations tend to only very rarely directly correspond to falsifying one specific premise. And human cognition usually isn’t structured by making explicit premises and arguing from them, making whatever methodology you are comparing it to not really be something that I have any idea how to apply in conversation (if you ask me “what are my premises for the belief that Nature is the most prestigious science journal?” then I definitely won’t have a nice list of premises I can respond with, but if you ask me “what would change my mind about Nature being the most prestigious science journal?” I might be able to give a reasonably good answer and start having a productive conversation).
thanks for this.
If the retreats are valuable, one would expect them to communicate genuinely useful concepts and ideas. Which ideas that CFAR teaches do you think are most useful?
On the payment model, imagine that instead of putting their material on choosing a high impact career online, 80k charged people £3000 to have 4 day coaching and networking retreats in a large mansion, afterwards giving them access to the relevant written material. I think this would shave off ~100% of the value of 80k. The differences between the two organisations don’t seem to me to be large enough to make a relevant difference to this analysis when applied to CFAR. Do you think there is a case for 80k to move towards the CFAR £3k retreat model?
**
On double cruxing, here is how CFAR defines double cruxing
“Let’s say you have a belief, which we can label A (for instance, “middle school students should wear uniforms”), and that you’re in disagreement with someone who believes some form of ¬A. Double cruxing with that person means that you’re both in search of a second statement B, with the following properties:
1. You and your partner both disagree about B as well (you think B, your partner thinks ¬B)
2. The belief B is crucial for your belief in A; it is one of the cruxes of the argument. If it turned out that B was not true, that would be sufficient to make you think A was false, too.
3. The belief ¬B is crucial for your partner’s belief in ¬A, in a similar fashion.”
So, if I were to double crux with you, we would both establish which were the premises we disagree on that cause our disagreement. B is a premise in the argument for A. This is double cruxing, right?
You say:
“if you ask me “what are my premises for the belief that Nature is the most prestigious science journal?” then I definitely won’t have a nice list of premises I can respond with, but if you ask me “what would change my mind about Nature being the most prestigious science journal?” I might be able to give a reasonably good answer and start having a productive conversation”
Your answer could be expressed in the form of premises right? Premises are just propositions that bear on the likelihood of the conclusion
CFAR’s model is actually pretty similar to 80k’s here. CFAR generally either heavily discounts or waives the cost of the workshop for people they think are likely to contribute to the long-term-future, or are more broadly promising, and who don’t have the money to pay for the workshop. As such the relevant comparison is more “should 80k offer paid coaching (in addition to their free coaching) at relatively high rates for people who they think are less likely to contribute to improving the world, if the money they earn from that allows them to offer the other free coaching services (or scale them up by 30% or something like that)”, to which my answer would be “yes”.
My sense is that 80k is in a better-funded position, and so this tradeoff doesn’t really come up, but I would be surprised if they never considered it in the past (though career coaching is probably somewhat harder to monetize than the kind of product CFAR is selling).
I also think you are underestimating to what degree the paid workshops were a necessity for CFAR historically having gotten to exist. Since there is a lot of downtime cost in being able to run workshops (you need to have a critical mass of teaching staff, you need to do a lot of curriculum development, have reliable venues, etc.) and the EA community didn’t really exist yet when CFAR got started, it was never really an option for CFAR to fully run off of donations, and CFAR additionally wanted to make sure it actually produced something that people would be willing to pay for, so offering paid workshops was one of the only ways to achieve those two goals. I also generally think it’s a good idea for projects like CFAR to ensure that they are producing a product that people are willing to pay significant amount of money for, which is at least a basic sanity check on whether you are doing anything real.
As an example, I encouraged Lynette to ask people whether they would be willing to pay for her coaching, and ideally ask them for at least some payment even if she can’t break even, to make sure that the people she is offering services to are filtered for the people who get enough value out of it to spend $50 per session, or something in that space (she had also considered that already on her own, though I don’t remember the current state of her asking her clients for payment).
I just remembered that 80k actually did consider monetizing part of it’s coaching in 2014, which would have probably resulted in a pretty similar model to CFAR:
Re retreats:
I think it’s quite plausible that 80k organizing retreats would be quite valuable, in particular in a world where CFAR isn’t filling that current niche. CEA also organized a large number of retreats of a similar type in the last year (I attended one on individual outreach, and I know that they organized multiple retreats for group organizers, and at least one operations retreat) presumably because they think that is indeed a good idea (the one that I attended did seem reasonably valuable, and a lot of the design of it was clearly influenced by CFAR workshops, though I can’t speak on whether that overall initiative was worth it).
I agree that 80k also has a lot of impact via their written material, but I think that is because they have invested a very large fraction of their resources into producing those materials (80K would likely be unable to run as many workshops as CFAR and also produce the written material). I think if 80k was focusing primarily on coaching, it would be very unlikely to produce good written material that would stand well on its own, though I expect it would still produce a good amount of value (and it might still produce some writings, but likely not ones that make much sense without the context of the coaching, similar to CFAR). As such I am skeptical of your claim that switching to that model would get rid of ~100% of 80k’s value. I expect it would change their value proposition, but likely still have a good chance of being competitive in terms of impact (and fully switching towards a coaching model was something that I’ve heard 80k consider multiple times over the years).
I think if you define “premise” more broadly to mean “propositions that bear on the likelihood of the conclusion” then you are closer, but still not fully there. A crux would then be defined “a set of premises that when falsified, would provide enough evidence that you would change your mind on the high-level claim”, which is importantly still different from “identifying differences in our premises”, in particular it emphasizes identifying specific premises that are particularly load-bearing for the argument at hand.
(This wouldn’t be a very standard usage of “premise” and doesn’t seem to align super well with any definitions I can find in any dictionaries, which all tend to either be about logical inference or about subsets of a specific logical argument that is being outlined, but doesn’t seem like a horrible stretch from available definitions. Though I wouldn’t expect people to intuitively know what you mean by that definition of “premise”)
I do still expect people to give quite drastically different answers if you ask them “is ‘not X’ a premise of your belief?” vs. “would observing X change your mind about this belief?”. So I wouldn’t recommend using that definition if you were actually trying to do the thing that double crux is trying to do, even if you define it beforehand. I do think that the norms from (classical) rhetoric and philosophy of trying to identify differences in your premises are good norms and generally make conversations go better. I agree that Double Crux is trying to operationalize and build on that, and isn’t doing some weird completely novel thing, though I do think it extends on it in a bunch of non-trivial ways.
I disagree that 80k should transition towards a £3k retreat + no online content model, but it doesn’t seem worth getting into why here.
On premises, here is the top definition I have found from googling… “a previous statement or proposition from which another is inferred or follows as a conclusion”. This fits with my (and CFAR’s) characterisation of double cruxing. I think we’re agreed that the question is which premises you disagree on cause your disagreement. It is logically impossible that double cruxing extends this characterisation.
I never said 80k should transition towards a retreat + no online content model. What I said is that it seems plausible to me it would still produce a lot of value in that case, though I agree that their current model seems likely a better fit for them, and probably overall more valuable. Presumably you also disagree with that, but it seemed important to distinguish.
Given that in the scenario as outlined, there was no “previous statement” or “previous proposition”, I am still confused how you think this definition fits. In the scenario at hand, nobody first outlined their complete argument for why they think the claim discussed is true, and as such, there is no “previous statement or proposition” that can be referred back to. This definition seems to refer mostly to logical argument, which doesn’t really apply to most human cognition.
I am not super excited about debating definitions, and we both agree that using the word premise is at least somewhat close to the right concept, so I am not very excited about continuing this thread further. If you really care about this, I would be glad to set up an experiment on mechanical turk in which we ask participants to list the necessary premises of a belief they hold, and see how much their responses differ from asking them what observations would change their mind about X. It seems clear to me that their responses would differ significantly.
This is still only capturing half of it, even under the definition of premise that you’ve outlined here, which seems to be a reasonable definition of what a crux for a single participant in the conversation is. A double crux would be “a set of premises, that when viewed as a new conjunctive proposition you both assign opposite truth values to, that when flipped would cause both of you to change their mind”. Though that alone obviously doesn’t yet make a procedure, so there is still a bunch more structure, but I would think of the above as an accurate enough description to start working with it.
I don’t think I really know how to engage with this. Obviously it’s possible for double-crux to extend this characterization. I even outlined a key piece that was missing from it in the above paragraph.
But it’s also a procedure that is meant to be used with real people, where every bit of framing and instruction matters. If you really believe this, let us run a test and just give one group of people the instruction “find the premises on which you disagree on that cause your disagreement” and the other group the full double crux worksheet. Presumably you agree that the behavior of those groups will drastically differ.
You maybe have something more specific in mind when you mean “logically impossible”, but given that we are talking about a high-level procedure proofs of logical impossibility seem highly unlikely to me.
To give an answer to the question of what material that CFAR teaches at their workshops I consider valuable, here is a list of classes that I’ve seen have a big impact on individuals, sometimes including myself and for which I also have separate reasons to think they are valuable.
Units of Exchange
Basically an introduction into consequentialist reasoning, trying to get people to feel comfortable trading off different resources that they previously felt were incomparable. A lot of the core ideas in EA are based off of this, and I think it’s generally a good introduction into that kind of thinking.
Inner Simulator
Basic practical introduction into System 1 and System 2 level processing, and going into detail on how to interface between S1 and S2 processing.
Trigger-Action Planning
Basic introduction into how associative processing works in the brain, where it tends to work, and where it tends to fail, and how to work around those failure modes. In the literature the specific technique is known as “Mental contrasting with implementation intentions” and is probably one of the most robust findings in terms of behavior change in behavioral psychology.
This class is often particularly valuable because I’ve seen it provide people with their first real mechanistic model of the human mind, even if simplified. A lot of people don’t really have any mechanistic baseline of how human cognition works, and so the simplified statement of “humans cognition can be modeled as a large pile of programmed ‘if-then-statements’ can get people initial traction on figuring out how their own mind works”.
Goal Factoring
For most attendees this has a lot of overlap with basic 80k coaching. Practice in trying to ask yourself repeatedly “why is this thing that I am doing important to me, and could I achieve it some better way?”, and this is probably the class that I’ve seen that had the biggest effects in terms of causing career changes in participants, mostly by getting them to think about why their are pursuing the career they are pursuing, and how they might be able to achieve their goals better.
Understanding Shoulds
This covers a lot of material in the Minding Our Way “Replacing Guilt” series, which many EAs and people that I trust have reported to have benefited a lot from, and which core conclusions are quite important for a lot of thinking about how to have a big impact in the world, how morality works, reminding people that they are allowed to care about things, etc.
Focusing
Based on Gendlin’s “Focusing” book and audiobook, it teaches a technique that forms the basis of a significant fraction of modern therapeutic techniques and I consider a core skill for doing emotional processing. I’ve benefited a lot from this, and it also has a pretty significant amount of evidence behind it (both in that it’s pretty widely practiced, and in terms of studies), though only for the standards of behavioral psychology, so I would still take that with a grain of salt.
Systemization
This is basically “Getting Things Done” the book, in a class. I, and a really large number of people I’ve worked with and who seem to be good at their job, consider this book core reading for basically anyone’s personal productivity, and I think teaching this is pretty valuable. This class in particular tends to help people who bounced off of the book, which still recommends a really large fraction of practices that I’ve seen in particular young people bounce off of, like putting everything into binders and getting lots of cabinets to put those binders in, instead of having good digital systems.
Double Crux
We’ve discussed this one above a good amount. In particular I’ve seen this class cause a bunch of people to have productive conversations that have previously had dozens of hours of unproductive or really conflict-heavy conversations, the most easily referenced and notable of which is probably a conversation between Scott Garrabrant and Eric Drexler that I think significantly moves the conversation around AI Alignment forward
All of the above strike me as pretty robustly good concepts to teach, already make up more than 50% of intro workshops, and that are pretty hard to get a good grasp on without reading ~6 books, and having substantial scaffolding to actually put time into practicing the relevant ideas and techniques.
I agree that these are pretty valuable concepts to learn. At the same time, I also believe that these concepts can be learned easily by studying the corresponding written materials. At least, that’s how I learned them, and I don’t think I’m different from the average EA in this respect.
But I also think we shouldn’t be speculating about this issue, given its centrality to CFAR’s approach. Why not give CFAR a few tens of thousands of dollars to (1) create engaging online content that explains the concepts taught at their workshops and (2) run a subsequent RCT to test whether people learn these concepts better by attending a workshop than by exposing themselves to that content?
I would be open to helping run such an RCT, and by default would expect the written material without further assistance to have relatively little impact.
I also think that for many people asking them to read the related online material will have a much lower completion rate than going to a workshop, and figuring out how to deal with that would be a major uncertainty in the design of the RCT. I have many friends that I tried to get to desperately read the material that explains the above core concepts, sadly without success, who finally got interested enough into all of the above after attending a CFAR workshop.
In my last 5 years of working in EA and the rationality community, I have repeatedly been surprised by the degree to which even very established EAs have not read almost any introductions to the material I outlined above, and where the CFAR workshop was their first introduction into the material. This includes large parts of the staff at CEA, as well as many core group organizers I’ve met.
I don’t expect CFAR putting out online material to help much with this, since roughly the same holds true for 80k material, and a lot of the concepts above actually already have good written explanations to them.
You seem to be very optimistic about getting people to read written content, whereas my experience has been that people are very reluctant to read content of any type that is not fiction or is of very high relevance to some particular niche interest of theirs. Inviting people to a workshop seems to work a lot more reliably to me, though obviously with written material you get a much broader reach, which can compensate for the lower conversion rate (and which medium makes sense to optimize I think hinges a lot on whether you care about getting a small specific set of people to learn something, vs. trying to get as many people as possible to learn something).
Thank you. Your comment has caused me to change my mind somewhat. In particular, I am now inclined to believe that getting people to actually read the material is, for a significant fraction of these people, a more serious challenge than I previously assumed. And if CFAR’s goal is to selectively target folks concerned with x-risk, the benefits of insuring that this small, select group learn the material well may justify the workshop format, with its associated costs.
I would still like to see more empirical research conducted on this, so that decisions that involve the allocation of hundreds of thousands of EA dollars per year rest on firmer ground than speculative reasoning. At the current margin, I’d be surprised if a dollar given to CFAR to do object-level work achieves more than a dollar spent in uncovering “organizational crucial considerations”—that is, information with the potential to induce a major shift in the organization’s direction or priority. (Note that I think this is true of some other EA orgs, too. For example, I believe that 80k should be using randomization to test the impact of their coaching sessions.)
Hi Oliver, Is there a sequence out there explaining these terms? A quick Google/LW/CFAR search didn’t throw anything up which covered all the concepts you mention above (there’s a sequence called Hammertime, but it didn’t cover all the concepts you mention). I think one of the benfits of a centralized source of information is that it’s accessible and intuitive to find. In the current state, it seems that you would have to go out of your way to find these kinds of writeups, and possibly not even know they exist.
I don’t think there is a single link, though most of the concepts have a pretty good canonical resource. I do think it usually takes quite a bit of text to convey each of those concepts, so I don’t think creating a single written reference is easily feasible, unless someone wants to produce multiple books worth of content (I’ve historically been impressed with how much content you can convey in a 1.5 hour long class, often 10 blog posts worth, or about half of a book).
I don’t think I have the time to compile a full list of resources for each of these concepts, but I will share the top things that come to mind.
Units of Exchange: I think microeconomics classes do a pretty good job of this, though are usually a bit abstract. A lot of writing of Scott Alexander gets at this, with the best introduction probably being his “Efficient Charity: Do unto others...”
Inner Simulator: Covered pretty well by Thinking: Fast and Slow by Daniel Kahneman
Trigger-Action Planning: Also covered pretty well by Thinking Fast and Slow, though, with some Getting Things Done thrown into it
Goal Factoring: I don’t actually know a good introduction to this, alas.
Understanding Shoulds: Mindingourway.com’s “Replacing Guilt” series
Focusing: The best introduction into this is Gendlin’s audiobook, which I highly recommend and is relatively short
Systemization: As mentioned, Getting Things Done is the best introduction into this topic
Double Crux: I think Duncan Sabien’s introduction for this is probably the best one
I don’t think this is very important for my overall view on CFAR’s curriculum, but FWIW I was quite surprised by you describing Focusing as
Maybe we’re just using “significant fraction” differently, but my 50% CI would have been that focusing is part of 1-3 of the 29 different “types of psychotherapy” I found on this website (namely “humanistic integrative psychotherapy”, and maybe “existential psychotherapy” or “person-centred psychotherapy and counselling”). [Though to be fair on an NHS page I found, humanistic therapy was one of 6 mentioned paradigms.] Weighting by how common the different types of therapy are, I’d expect an even more skewed picture: my impression is that the most common types of therapy (at least in rich, English-speaking countries and Germany, which are the countries I’m most familiar with) are cognitive-behavioral therapy and various kinds of talking therapy (e.g. psychoanalytic, i.e. broadly Freudian), and I’d be surprised if any of those included focusing. My guess is that less than 10% of psychotherapy sessions happening in the above countries include focusing, potentially significantly less than that.
My understanding had been that focusing was developed by Eugene Gendlin, who after training in continental philosophy and publications on Heidegger became a major though not towering (unlike, say, Freud) figure in psychotherapy—maybe among the top decile but not the top percentile in terms of influence among the hundreds of people who founded their own “schools” of psychotherapy.
I’ve spent less than one hour looking into this, and so might well be wrong about any of this—I’d appreciate corrections.
Lastly, I’d appreciate some pointers to studies on focusing. I’m not doubting that they exist—I’m just curious because I’m interested in psychotherapy and mental health, but couldn’t find them quickly (e.g. I searched for “focusing Gendlin” on Google Scholar).
I haven’t looked super much into the literature on this so I might be wrong, my sense was that it was more of a case of “lots of therapeutic techniques share a lot of structure, and Gendlin formalized it into a specific technique, but a lot of them share a lot of structure with what Gendlin is doing”, which makes sense, because that’s how focusing was developed. From the Wikipedia article:
The thing that made me more comfortable saying the above was that Gendlin’s goal (judging from the focusing book I read and the audiobook I listened to) seems to have been in significant parts a study into “what makes existing therapeutic techniques work”, instead of “let’s develop a new technique that will revolutionize therapy”, so even if a school of therapy isn’t downstream of Gendlin, you expect a good fraction to still have focusing-like things in them, since Gendlin seemed to be more interested in refining techniques instead of revolutionizing them.
I do agree that I should probably stop using words like “significant fraction”. I intended to mean something like 20%-30% of therapy sessions will likely include something that is pretty similar to focusing, even if it isn’t exactly called that, which still seems roughly right to me and matches with my own experience of therapy with a practitioner who specialized in CBT and some trauma-specific therapies, but our actual sessions weren’t really utilizing either of those schools and were basically just focusing sessions, which to that therapist seemed like the natural thing to do in the absence of following a more specific procedure.
Some of my impression here also comes from two textbooks I read on therapy whose names I currently forgot, both of which were mostly school-independent and seemed to emphasize a lot of focusing-like techniques.
However, I don’t have super strong models here, and a significant fraction of my models are downstream of Gendlin’s own writing (who as I said seems to describe focusing more as “the thing that makes most type of therapy work”), so I am pretty open to being convinced I am wrong about this. I can particularly imagine that Freudian approaches could do less focusing, since I’ve basically not interacted with anything in that space and feel kinda averse to it, so I am kind of blind to a significant fraction of the therapy landscape.
Thanks, this is helpful!
I hadn’t considered the possibility that techniques prior to Gendlin might have included focusing-like techniques, and especially that he’s claiming to have synthesized what was already there. This makes me less confident in my impression. What you say about the textbooks you read definitely also moves my view somewhat.
(By contrast, what you wrote about studies on focusing probably makes me somewhat reduce my guess on the strength of the evidence of focusing, but obviously I’m highly uncertain here as I’m extrapolating from weak cues—studies by Gendlin himself, correlational claim of intuitively dubious causal validity—rather than having looked at the studies themselves.)
This all still doesn’t square well with my own experiences with and models of therapy, but they may well be wrong or idiosyncratic, so I don’t put much weight on them. In particular, 20-30% of sessions still seems higher than what I would guess, but overall this doesn’t seem sufficiently important or action-relevant that I’d be interested to get at the bottom of this.
Just a brief reaction:
This makes sense to me as a response to Halstead’s question. However, it actually makes me a bit less confident that (what you describe as) CFAR’s reluctance to increase legibility is a good idea. An educational institution strikes me as something that can be made legible way more easily and with fewer downsides than an institution doing cutting-edge research in an area that is hard to communicate to non-specialists.
In my experience teaching rationality is more tricky than the reference class education, and is an area which is kind of hard to communicate to non-specialists. One of the main reasons seems to be many people have somewhat illusory idea how much they understand the problem.
I don’t think most of the costs that I described that come from legibility differ that much between research and educational institutions? The american public education system, as well as many other public education systems actually strike me as core examples of systems that have suffered greatly due to very strong forces on legibility in all of their actions (like standardized curricula combined with standardized testing). I think standardized testing is pretty good in a lot of situations, but that in this case it resulted in a massive reduction in variance in a system where most of the value comes from the right tail.
I agree that there are also other separate costs to legibility in cutting-edge domains, but the costs on educational institutions still seem quite significant to me. And most of the costs are relatively domain-general.
Thanks, that helps me understand where you’re coming from, though it doesn’t change my views on CFAR. My guess is we disagree about various more general claims around the costs and benefits of legibility, but unfortunately I don’t have time right now to articulate my view on this.
Very roughly, I think I (i) agree with you that excessive optimization for easily measurable metrics has harmed the public education system, and in particular has reduced benefits from the right tail, (ii) disagree with your implied criterion of using something like “quality-weighted sum of generated research” is an appropriate main criterion for assessing the education system, and thus by extension disagree with the emphasis on right-tail outcomes when evaluating the public education system as a whole, (iii) don’t think this tells us much about CFAR as I both think that CFAR’s environment makes increased legibility less risky (due to things like high goal-alignment with important stakeholders such as funders, a more narrow target audience, …) and also that there are plenty of ways to become more legible that don’t incur risks similar to standardized testing or narrow optimization for quantitative metrics (examples: qualitatively describe what you’re trying to teach, and why you think this is a good idea; monitor and publish data such as number of workshops run, attendance etc., without narrowly optimizing for any of these; maintain a list of lessons learned).
(I upvoted your reply, not sure why it was downvoted by someone else.)
(Reply written after the paragraph was added above)
Thanks for the elaboration! Some quick thoughts:
I think CFAR has done at least everything on this list of examples. Which you might already be aware of, but wanted to make sure is common knowledge. There are a significant number of posts trying to explain CFAR at a high-level, and the example workshop schedule summarizes all the classes at a high-level. CFAR has also published the number of workshops they’ve run and their total attendance in their impact reports and on their homepage (currently listing 1045 alumni). Obviously I don’t think that alone is sufficient, but it seemed plausible that a reader might walk away thinking that CFAR hadn’t done any of the things you list.
I think there is some truth to this interpretation, but I think it’s overall still wrong enough that I would want to correct it. I think the education system has many goals, and I don’t think I would summarize it’s primary output as “quality-weighted sum of generated research”. I don’t think going into my models of the education system here is going to be super valuable, though happy to do that at some other point if anyone is interested in them. My primary point was that optimizing for legibility clearly has had large effects on educational institutions, in ways that would at least be harmful to CFAR if affected in the same way (another good example here might be top universities and the competition for getting into all the top 10 ranking, though I am less confident of the dynamics of that effects).
(Edit the below was written before Max edited the second paragraph into his comment)
Seems good! I actually think considerations around legibility are quite important and where I expect a good amount of intellectual progress to be made by talking to each other, so I would like to see your perspective written up and engage with it.
I also want to make sure that it’s clear that I do think CFAR should be more legible and transparent (as I said in the writeup above). I have some concerns with organizations trying to be overly legible, but I think we both agree that at the current margin it would be better for CFAR to optimize more for legibility.
(I’ve sadly had every single comment of mine on this thread strong-downvoted by at least one person, and often multiple people. My sense is that CFAR is a pretty polarizing topic, which I think makes it particularly important to have this conversation, but seems to also cause some unfortunate voting patterns that feel somewhat stressful to deal with.)
I’m sorry to see the strong downvotes, especially when you’ve put in more effort on explaining your thinking and genuinely engaging with critiques than perhaps than all other EA Fund granters put together. I want you to know that I found your explanations very helpful and thought provoking, and really like how you’ve engaged with criticisms both in this thread and the last one.
Seconded.
(I’m wondering whether this phenomenon could also be due to people using downvotes for different purposes. For example, I use votes roughly to convey my answer to the question “Would I want to see more posts like this on the Forum?”, and so I frequently upvote comments I disagree with. By contrast, someone might use votes to convey “Do I think the claims made in this comment are true?”.)
Data point: I often feel a pull towards up-voting comments that I feel have stimulated or advanced my thinking or exemplify a valuable norm of transparency and clarity, but then I hold back because I think I might disagree with the claims made or I think I simply don’t know enough to judge those claims. This is based on a sense that I should avoid contributing to information cascade-type situations (even if, in these cases, any contribution would only be very slight).
This has happened multiple times in this particular thread; there’ve been comments of Oliver’s that I’ve very much appreciated the transparency of, but with which I felt like I still might slightly disagree overall, so I avoided voting either way.
(I’m not saying this is the ideal policy, just that it’s the one I’ve taken so far.)
Thank you! :)
Yes I don’t fully understand why they’re not legible. A 4 day workshop seems pretty well-placed for a carefully done impact evaluation.
For whatever it’s worth, this seems right to me, and I do want to make sure that people know that I do think CFAR should try to be more legible at the margin
I mentioned this in my writeup above:
I do think the question of what the correct outcome measurements for an impact evaluation would be is non-trivial, and would be interested in whether people have any good ideas for good outcome measurements.
An aside: I had never heard of ‘Murphyjitsu’ before, but use pre-mortems in my personal and professional life regularly. I’m surprised people found the name ‘Murphyjitsu’ easier to engage with!
It’s a bit of a more playful term, which I think makes sense in the context of a workshop, but I also use the two terms interchangeably and seen CFAR staff do the same, and usually use pre-mortem when I am not in a CFAR context.
I don’t have strong opinions on which term is better.