I thought I’d just add some data from my own experience.
For context, I’ve been heavily involved in the EA community, most recently running CEA. After I left CEA, I spent the summer researching what to do next and recently decided to join the Leverage Research team. I’m speaking personally here, not on behalf of Leverage.
I wanted to second Ozzie’s comment. My personal experience at least is that I’ve found the Leverage and Paradigm teams really welcoming.
They do employ people with a wide range of political views with the idea that it helps research progress to have a diversity of viewpoints. Sometimes this means looking at difficult topics and I’ve sometimes found it uncomfortable to try and challenge why I hold different viewpoints but I’ve always found that the focus is on understanding ideas and the attitude to the individual people one of deep respect. I’ve found this refreshing.
I wanted to thank Ozzie for posting this in part because I noticed reticence in myself to saying anything because my experience with conversations about Leverage is that they can get weird and personal quite fast. I know people who’ve posted positive things about Leverage on the EA Forum and then been given grief for it on and offline.
For this reason Greg, I can see why Leverage don’t engage much with the EA Forum. You and I know each other fairly well and I respect your views on a lot of topics (I was keen to seek you out for advice this summer). I notably avoided discussing Leverage though because I expected an unpleasant experience and that I had more information on the topic from investigating them myself. This feels like a real shame. Perhaps I could chat with you (and potentially others) about what you’d like to see written up by Leverage. I’m happy to commit to specific Leverage-related posts if you can help ensure that turns into a genuinely useful discussion. What do you think? :-)
I’d be eager to see anything that speaks to Leverage’s past or present research activity: what have they been trying to find out, what have they achieved, and what are they aiming for at the moment (cf).
As you know from our previous conversations re. Leverage, I’m fairly indifferent to ‘they’re shady!’ complaints (I think if people have evidence of significant wrongdoing, they should come forward rather than briefing adversely off the record), but much less so to the concern that Leverage has an has achieved extraordinarily little for an organisation with multiple full-time staff working for the better part of a decade. Showing something like, “Ah, but see! We’ve done all these things,” or, “Yeah, 2012-6 was a bit of a write-off, but here’s the progress we’ve made since”, would hopefully reassure, but in any case be informative for people who would like to have a view on leverage independent of which rumour mill they happen to end up near.
Other things I’d be interested to hear about is what you are planning to work on at Leverage, and what information you investigated which—I assume—leads to a much more positive impression of Leverage than I take the public evidence to suggest.
Thanks for the message and for engaging at the level of what has Leverage achieved and what is it doing. The tone of your reply made me more comfortable in replying and more interested in sharing things about their work so thank you!
Leverage are currently working on a series of posts that are aimed at covering what has been happening at Leverage from its inception in 2011 up until a recent restructure this year. I expect this series to cover what Leverage and associated organisations were working on and what they achieved. This means that I expect Leverage to answer all of your questions in a lot more depth in the future. However, I understand that people have been waiting a long time for us to be more transparent so below I have written out some more informal answers to your questions from my understanding of Leverage to help in the meantime.
Another good way to get a quick overview of the kinds of things Leverage has been working on beyond my notes below is by checking out this survey that we recently sent to workshop participants. It’s designed for people who’ve engaged directly with our content so it won’t be that relevant for people to fill in necessarily but it gives an overview of the kinds of techniques Leverage developed and areas they researched.
What did Leverage 1.0 work on?
A very brief summary is that the first eight and a half years of Leverage (let’s call this “Leverage 1.0” as a catch-all for those organisations before the restructure) was at first a prioritisation research project looking at what should people work on if they want to improve the world. Leverage 1.0 later came to focus more on understanding and improving people as their psychological frameworks and training tools developed but they still conducted a wide range of research.
This means that in the very early days they were thinking a lot about how to prioritise, how to make good long term plans and just trying a bunch of things. I get the impression that at this stage almost nothing was ruled out in terms of what might be worth exploring if you wanted to improve the world. This meant people investigating all sorts of things like technological intelligence amplification, nootropics, and conducting polyphasic sleep experiments. People might be researching what caused the civilisational collapse that led to the dark ages, the beliefs of a particular Christian sect, or what lead to the development of Newtonian physics. Leverage felt this was important for research progress. They wanted researchers to follow what motivated them. They thought that it was important to investigate a lot of areas before deciding where to focus their efforts because deciding what to prioritise is so important to overall impact. This felt particularly important when investigating moon-shots which had the potential to be extremely valuable even if they seemed unlikely at the outset.
Some of the outputs of these early days of research included training sessions on:
Planning—How to build and error-check plans for achieving your goals
Expert assessment—how to determine if someone is an expert in a given domain when you lack domain knowledge
Learning to learn—how to improve and expand the scope of your learning process
Theorizing—how to build models and improve your model building process over time
Prioritisation and goal setting—how to find your goals, back chain plans from them etc
This is far from everything but gives you a flavour.
Geoff had developed a basic model of psychology called Connection Theory (CT) so this was a thing that was investigated alongside everything else. This involved spending a lot of time testing the various assumptions in CT.
Through experimenting with using CT in this way, Leverage eventually found they were able to use ideas from CT to make some basic predictions about individual and group behaviour, help individuals identify and remove bottlenecks so that they could self improve and perhaps even identify and share specific mental moves people were using to make research progress on particular questions. This made the team more excited about psychology research in particular (amongst the array of things people were researching) as a way to improve the world.
From there they (alongside the newly founded Paradigm Academy) developed some of the research into things like
one on one and group training,
A catalogue of different mental procedures individuals use when conducting research so that they could be taught to others to use to tackle different research and other problems. One example intellectual procedure (IP) just to give a sense of this, is the Proposal IP where you use the fact that you have a lot of implicit content in your mind and your taste response to inelegant proposals to speed up your thinking in an area.
Specific training in strategy, theorizing, and research
A collection of specific introspection and self-improvement techniques such as:
Self-alignment (a tool for increasing introspective access which handled a class of cases where our tools previously weren’t working)
Anti-avoidance techniques (making it so you can think clearly in areas you previously didn’t want to think about or had fuzzy thoughts in)
Charting (a belief change tool that has been modified and built out a lot since it’s initial release)
Mythos (tool for introspection with imagery, helpful for more visual people)
Integration and de-zoning (tools for helping people connect previously separate models)
What is Leverage doing now?
As for what Leverage is currently working on, once we have posted our retrospective we’ll then be updating Leverage’s website to reflect its current staff and focus so again a better update than I can provide is pending.
The teaser here is that from the various research threads being pursued in the early years of Leverage 1.0, Leverage today has narrowed their focus to be primarily on two areas that they found the most promising in their early years:
Scientific methodology research
Psychology
We also continue to be interested in sociology research and expect to bring on research fellows (either full time or part of future fellowship programmes) focusing on sociology in the future. However, since we’re relaunching our website and research programme we want to stay focused so we’re punting building out more of our sociology work to further down the line.
The scientific methodology research involves continuing to look at historical examples of scientific breakthroughs in order to develop better models of how progress is made. This continues some of our early threads of research in theorising, methodology, historical case studies and the history of science. We’re particularly interested in how progress was made in the earlier stages of the development of a theory or technology. Some examples include looking at what led to the transition in chemistry from Phlogiston to Lavoisier’s oxygen theory or the challenges scientists had in verifying findings from the first telescopes. We aim to share lessons from this research with researchers in a variety of fields. In particular, we want to support research that is in its earlier, more explorative stages. This is more of a moon-shot area but this means it can get less attention while being potentially high reward.
Our psychology research aims to continue to build on the progress and various research threads Leverage 1.0 was following. While this is quite a moon shot style bet, if we can improve our understanding of people then we potentially improve the ways in which they work together to solve important problems. At this point, we have developed tools for looking at the mind and mental structures that we think work fairly well on the demographics of people we’ve been working with. I got a ballpark estimate from someone at Leverage that Leverage and Paradigm have worked with around 400 people for shallower training, and about 60 for in-depth work but treat those figures as a guess until we write something up formally. We’ve focused in the last few years on improving these tools so they work in harder cases (e.g. people who have trouble introspecting initially) and using the tools to find common mental structures. Moving forward with this research we want to test the tools in a more rigorous way, in particular by communicating with people in academia to see whether or not they can validate our work.
One thing I personally like about the plans for psychology research is that it also acts as a check on our scientific methodology research. If the insights we gain from looking at the history of scientific progress aren’t useful to us in making progress in psychology then that’s one negative sign on their overall usefulness.
Who works for Leverage and Paradigm?
The team is much smaller and the organisation structure slightly more defined (although there is a way to go here still). There are four researchers (including Geoff who is also the Executive Director) and I’ll be joining as a Program Manager managing the researchers and helping communicate with the public about our work. So four in total at the moment, five once I start.
While Leverage Research in its newer form is getting going it still receives a lot of help from its sister organisation Paradigm Academy. This means that while they are two separate organisations, currently Paradigm staff give a lot of time to helping Leverage, particularly in areas like operations and helping with PR and Communications like the website relaunch. This helps allow the researchers to focus on their research and means the burden of public communication won’t all fall on their newest employee (me). Once a lot of that is done though we expect to make the division between the two organisations clearer. Paradigm currently has nine employees including Geoff.
I expect all of this will generate more questions than it answers at the moment and while my answer is to wait for Leverage’s formal content to be published I can see why this is frustrating. I hope my examples give a small amount of insight into our work while we take the time to write things up. You have every reason to be sceptical about Leverage posting content given various promises made in the past. I think given our track record on public communication that scepticism is valid. All I can perhaps offer in the meantime is that I personally am very keen to see both the retrospective and the new Leverage website published and the get sh*t done spirit that you and others on this forum know me for is part of the reason they’ve offered me a job to help with this in the first place.
Why I chose to work at Leverage
As for my personal reasons for choosing to accept an offer from Leverage, I expect this to be hard to transmit just because of inferential distance. My decision was the result of at least five months of discussions, personal research and resultant updates all of which is built on various assumptions that caused me to already be pursuing the plans I was at CEA.
I’ll attempt a short version here anyway in case it’s helpful. If there’s a lot of interest I’ll consider writing this up but I’m not sure it’ll be sufficiently useful or interesting to be worth the time cost.
Broadly speaking, I created a framework (building off a lot of 80Ks work but adapting it to suit my needs) to use to compare options on:
Impact - comparing potential career plans by looking at the scale and likelihood of success across:
the problem being tackled (e.g. preventing human extinction),
the approach to solving that problem (e.g. develop AGI in a way that’s safe)
the organisation (e.g. DeepMind)
and what I personally could contribute (e.g. say in a role as Project Manager)
Personal happiness (personal fit with the culture, how the job would fit into my life etc)
Future potential (what skills would I build and how useful are they, and what flexible resources such as useful knowledge or network would I gain)
I decided that I was willing to bet at least a few more years of my career on the more moon shot type plans to build a much better future (something like continuing personally to follow CEA’s vision of working towards an optimal world).
This narrowed my focus down to primarily considering paths related to avoiding existential risks and investing in institutions or advances that would improve humanity’s trajectory. In exploring some options around contributing to AI safety in some way I came away both not feeling convinced that I wouldn’t potentially cause harm (through speeding up the development of AGI) and less sure of the arguments for now being a particularly important hinge on this. It, therefore, seemed prudent to learn a lot more before considering this field.
This left me then both wanting to invest more time in learning more while also not wanting to delay working on something I thought was high impact indefinitely. In terms of impact, the remaining areas were advances or institutions that might improve humanity’s ability to tackle global problems.
I’d had plenty of conversations with various people about Leverage (including many Leverage sceptics) in the past and interacted with Leverage and Paradigm directly to some degree, mostly around their introspection techniques which I personally have found extremely useful for self-improvement. I knew that they were interested in psychology initially as a potential way to improve humanity’s trajectory (but didn’t yet understand the scope of their other research) so I reached out to chat about this. I found that many of the people there had already thought a lot about the kinds of things I was considering as options for improving the long-term future and they had some useful models. Those interactions plus my positive view of their introspection techniques led me to think that Leverage had the most plausible plan given my current uncertainty for improving the long-term future and was likely to be by far the best option for me in terms of self-improvement and gaining the knowledge I wanted for making better future plans. Their recent restructure, desire to establish a more structured organisation and plans to publish a lot of content meant they had an opening for my particular skill set and the rest, as they say, is history.
Leverage are currently working on a series of posts that are aimed at covering what has been happening at Leverage from its inception in 2011 up until a recent restructure this year.
This aged well… and it reads like what ChatGPT would blurt, if you asked it to “sound like a convincingly respectful and calm cult with no real output.” Your ‘Anti-Avoidance,’ in particular, is deliciously Orwellian. “You’re just avoiding the truth, you’re just confused...”
I was advocating algal and fish farming, including bubbling air into the water and sopping-up the fish poop with crabs and bivalves—back in 2003. Spent a few years trying to tell any marine biologist I could. Fish farming took-off, years later, and recently they realized you should bubble air and catch the poop! I consider that a greater real-world accomplishment than your ‘training 60+ people on anti-avoidance of our pseudo-research.’ Could you be more specific about Connection Theory, and the experimental design of the research you conducted and pre-registered, to determine that it was correct? I’m sure you’d have to get into some causality-weeds, so those experimental designs are going to be top-notch, right? Or, is it just Geoff writing with the rigor of Freud on a Slack he deleted?
Thanks Larissa—the offer to write up posts from Leverage Research is a generous offer. Might it not be a more efficient use of your time, though, to instead answer questions about Leverage the public domain, many of which are fairly straightforward?
For example, you mention that Leverage is welcoming to new staff. This sounds positive—at the same time, the way Leverage treated incoming staff is one of the main kinds of fact discussed in the top-level post. Is it still true that: (i) staff still discuss recruitees on individual slack channels, (ii) mind-mapping is still used during recruitment of staff, (iii) growth-rates are overestimated, (iv) specific lists of attendees are recruited from EA events, and (v) negative rumours are still spread about other organizations that might compete for similar talent? To the extent that you are not sure about (i-v), it would be interested to know whether you raised those concerns with Geoff in the hiring process, before joining the organization.
For other questions raised by the top-level post: (a) are Leverage’s outputs truly as they appear? (b) Is its consumption of financial resources and talent, as it appears? (c) Has it truly gone to such efforts to conceal its activities as described under the general transparency section? (d) How will Leverage measure any impact from its ninth year of operation?
From another post: (e) How many of the staff at Leverage Research are also affiliated with Paradigm Academy? (f) How much of the leadership of Leverage Research is also playing a leading role at Paradigm Academy?
Since your investigations give you much more information on this topic than is available to an interested outsider, it should be very easy for you to help us out with these questions.
First, to try and answer your questions as best I can.
But then second, start to work out how to make future conversations with you about Leverage more productive
1. ANSWERING YOUR QUESTIONS
I’d recommend first reading my recent reply to Greg because this will give you a lot of relevant context and answers some of your questions.
Questions a, b and d: outputs, resources and future impact
Your Questions:
“(a) are Leverage’s outputs truly as they appear?”
“(b) Is its consumption of financial resources and talent, as it appears?”
“(d) How will Leverage measure any impact from its ninth year of operation?”
In terms of questions a, b and d, I will note the same thing as I said in my reply to Greg which is that we’re currently working both on a retrospective of the last eight and a half years of Leverage and on updating Leverage’s existing website. I think these posts and updates will then allow individuals to assess for themselves
our past work and outputs
whether it was worth the resources invested
our plans for the future
For now, though sections “What did Leverage 1.0 work on?” and “What is Leverage doing now” in my reply to Greg will give you some information about things Leverage did and its plans for the future.
It’s hard to comment on whether any of these things are “as they appear” as (1) there isn’t enough public content to assess and (2) different people seem to have had very different experiences with Leverage and have very different views on their work
This means how Leverage’s work appears depends on an individual’s interactions with them. This has made it hard for any consensus to emerge and so debates continue. Once we’ve published more of our content, I hope it will be easier to sync up and assess them.
Questions e and f: staffing cross over between Leverage and Paradigm
Your questions (referencing one of your other posts about Leverage):
“(e) How many of the staff at Leverage Research are also affiliated with Paradigm Academy?”
“(f) How much of the leadership of Leverage Research is also playing a leading role at Paradigm Academy?”
On questions, e and f, see the section in my recent reply to Greg called “Who works for Leverage and Paradigm?”.
Geoff Anders is the founder and Executive Director of both Paradigm and Leverage and as such is currently the only member of staff working at both organisations. The two are sister organisations with different missions that collaborate. Leverage’s mission was essentially conducting research. Paradigm’s focus was much more on training. This means that there is some natural overlap and historically the two organisations have worked closely together. Paradigm uses Leverage’s research content in their training and in return they provide practical support (operations support, help to update the website) to Leverage.
We have no intention as you and the original poster propose of using the Paradigm or other brands to replace Leverage if there are too many problems with its brand. We created other organisations with different brands to distinguish between the different work they were doing. We’ve intentionally continued our current research under the same name so as to link to our origins with Leverage 1.0. When we update both websites (starting with Leverage) over the coming months you will be able to see the teams at each.
Questions i) and iv) discussing potential recruits in slack and recruitment at events
Your questions:
“(i) staff still discuss recruitees on individual slack channels”
“(iv) specific lists of attendees are recruited from EA events”
On question i), yes Leverage discuss people they are considering hiring on slack both in recruitment slack channels and DMs as part of deciding whether to hire someone.
With regards to workshops in particular, we will mention people attending workshops on slack but this is usually in the form of staff coordinating to arrange training and share training relevant information (e.g. “this person had trouble belief reporting, it would be good to assign a trainer during the 1-1 who could help them belief report”). If someone is a workshop attendee and we are considering hiring them they might be discussed in both contexts.
Leverage and Paradigm have also (in answer to question iv) attended (EA and non-EA) events with lists of people they plan to speak to because they might be good potential hires. In addition, at events, they often do things like having tables so that people interested in working there (or interested in their work in general) can ask questions. Of course, they also attend events for non-recruitment reasons (e.g. learning about interesting topics by attending talks and having discussions with people). Leverage are particularly interested in meeting independent researchers. Paradigm are interested in meeting people interested in self-improvement.
Question ii) “mind-mapping is still used during recruitment of staff”
On question ii), here I assume you’re asking about using the charting procedure which I think might have been called mind mapping at some point. This is a way of taking system one beliefs someone has and trying to map out why a person has those beliefs and what actions they taking as a result of those beliefs that they want to change.
Historically Paradigm and Leverage did do some charting with potential hires to see if they could use some of the basic techniques and found them helpful. Since Paradigm used charting in training and Leverage in research it was important new hires are aware of that, have a basic understanding of the tools and some interest in it. This is no longer needed in the Leverage hiring process as not all researchers will be doing any research relating to CT or charting but continues to be relevant to Paradigm for training so they still do some charting with potential hires. Staff will of course use their models of psychology to aid hiring decisions because they have useful models in this area but this is more akin to the way that recruiters might use their recruitment experience and intuition to aid decisions than I expect people are imagining.
Leverage and Paradigm both have a strict confidentiality policy for all one to one sessions with individuals which means this information is not discussed as part of recruitment discussions without permission.
While I don’t expect my experience to necessarily be indicative, I can speak a bit to my personal recruitment experience with Leverage. I worked on a trial project part-time for a couple of months looking at things like types of events different communities run and why people join communities and I attended a formal interview. I did charting with someone on I think two occasions, both at my own personal request and nothing to do with hiring. My most relevant nearby experience is in running recruitment rounds at CEA. I’d say that the Leverage process felt less organised and structured than the CEA process but was otherwise what I’d expect.
Leverage and Paradigm have both recently been working to make their hiring process more structured, in part based on feedback from potential hires. For Leverage, people can send a resume and would be invited to a research interview to discuss their research and our requirements. From there, successful candidates would take part in a trial period. Paradigm candidates go through several stages: sending a resume, attending an initial screening interview, a follow-up interview that aims to test job-relevant skills, a one week trial primarily checking team fit and finally an extended trial testing fit for the role. The process I went through was more like a less structured version of the Paradigm process. This is because while I’ll be working at Leverage, I won’t be a researcher so the Paradigm recruitment process made more sense in my particular case.
Question iii) “growth-rates are overestimated”
On question iii, am I right in thinking you’re referring to the claim in the original post that
“The leadership of Leverage Research have on multiple occasions overstated their rate of staff growth by more than double, in personal conversation”
If so, it’s hard to comment on what happened here without a lot more context on the conversation. I asked Geoff about this and his guesses were that this could have been confusion about hiring and attrition (was the conversation about hiring rates or about the total number of staff), numbers of volunteers or internal estimates for growth rate which turned out to be harder than we thought.
Question v) “negative rumours are still spread about other organizations that might compete for similar talent?”
Intentionally spreading rumours is certainly not Leverage policy nor endorsed or encouraged among staff. Of course, members of staff will have their own views on how best to improve the world and therefore I would expect them to have views on a number of EA and other organisations, both positive and negative.
I have the same difficulty in giving a useful answer to this question as with question iii, in that the original poster has made a claim about Leverage with a strong connotation but without providing the specifics or reasons they believe this.
I think it’s important for individuals to be able to share and discuss their views. I think it would be more helpful in cases like this forum post and when discussing other organisations people try to qualify what they think, why and where that information is from so that recipients of that information come away with a better understanding. This can be difficult but I think it’s important to try.
Question c) “Has it truly gone to such efforts to conceal its activities as described under the general transparency section?”
Finally on question c, yes our past website was removed from the Wayback machine. With hindsight this was a mistake. At the time people were digging up our old content (for example our old long term plan document) and using this to hype up various Leverage conspiracy theories. While this was entertaining, it was certainly distracting and meant people were even more confused about what we were doing. We thought that if we took the content off the wayback it would reduce the conspiracy style hype but removing it only fuelled the fire of conspiracies.
As you can probably tell, Leverage did not invest very many skill points or chips in public communication in the early days, instead spending them pretty much exclusively on research and experimentation. This means we’ve still got a lot to learn in the public communication space and a lot of mistakes to make up for.
As discussed above, Leverage did not use different organisations as a way to tailor messages to donors, different organisations had different focuses and donors understood this. Similarly we have no plans to replace Leverage as a brand.
2. MAKING OUR INTERACTIONS WITH ONE ANOTHER MORE PRODUCTIVE
I’d like to ensure that any future interactions between you and Leverage are a good use of both of our time, informative to readers, and helpful to you on whatever the real issues are.
Looking at your posting history, it seems that you’ve created this anonymous account only to ask questions about Leverage that come across at least as excuses to make insinuations against them. I don’t currently get the impression that you are following the spirit of the EA Forum guidelines and posting with a scout mindset, trying to give people an accurate view or being clear about what you believe and why.
If you would be willing to spend time trying to help me understand and address your particular concerns about Leverage then I am very happy to spend time on that. If you only plan to continue in the same vein as your previous comments I don’t currently expect that to yield anything that feels useful for either of us. If you would like to chat more to figure out a better way forward that gets your concerns addressed let me know. You can reach me here or at larissa.e.rowe [at] gmail.com.
Hi Buck, Ozzie and Greg,
I thought I’d just add some data from my own experience.
For context, I’ve been heavily involved in the EA community, most recently running CEA. After I left CEA, I spent the summer researching what to do next and recently decided to join the Leverage Research team. I’m speaking personally here, not on behalf of Leverage.
I wanted to second Ozzie’s comment. My personal experience at least is that I’ve found the Leverage and Paradigm teams really welcoming.
They do employ people with a wide range of political views with the idea that it helps research progress to have a diversity of viewpoints. Sometimes this means looking at difficult topics and I’ve sometimes found it uncomfortable to try and challenge why I hold different viewpoints but I’ve always found that the focus is on understanding ideas and the attitude to the individual people one of deep respect. I’ve found this refreshing.
I wanted to thank Ozzie for posting this in part because I noticed reticence in myself to saying anything because my experience with conversations about Leverage is that they can get weird and personal quite fast. I know people who’ve posted positive things about Leverage on the EA Forum and then been given grief for it on and offline.
For this reason Greg, I can see why Leverage don’t engage much with the EA Forum. You and I know each other fairly well and I respect your views on a lot of topics (I was keen to seek you out for advice this summer). I notably avoided discussing Leverage though because I expected an unpleasant experience and that I had more information on the topic from investigating them myself. This feels like a real shame. Perhaps I could chat with you (and potentially others) about what you’d like to see written up by Leverage. I’m happy to commit to specific Leverage-related posts if you can help ensure that turns into a genuinely useful discussion. What do you think? :-)
Hello Larissa,
I’d be eager to see anything that speaks to Leverage’s past or present research activity: what have they been trying to find out, what have they achieved, and what are they aiming for at the moment (cf).
As you know from our previous conversations re. Leverage, I’m fairly indifferent to ‘they’re shady!’ complaints (I think if people have evidence of significant wrongdoing, they should come forward rather than briefing adversely off the record), but much less so to the concern that Leverage has an has achieved extraordinarily little for an organisation with multiple full-time staff working for the better part of a decade. Showing something like, “Ah, but see! We’ve done all these things,” or, “Yeah, 2012-6 was a bit of a write-off, but here’s the progress we’ve made since”, would hopefully reassure, but in any case be informative for people who would like to have a view on leverage independent of which rumour mill they happen to end up near.
Other things I’d be interested to hear about is what you are planning to work on at Leverage, and what information you investigated which—I assume—leads to a much more positive impression of Leverage than I take the public evidence to suggest.
Hi Greg,
Thanks for the message and for engaging at the level of what has Leverage achieved and what is it doing. The tone of your reply made me more comfortable in replying and more interested in sharing things about their work so thank you!
Leverage are currently working on a series of posts that are aimed at covering what has been happening at Leverage from its inception in 2011 up until a recent restructure this year. I expect this series to cover what Leverage and associated organisations were working on and what they achieved. This means that I expect Leverage to answer all of your questions in a lot more depth in the future. However, I understand that people have been waiting a long time for us to be more transparent so below I have written out some more informal answers to your questions from my understanding of Leverage to help in the meantime.
Another good way to get a quick overview of the kinds of things Leverage has been working on beyond my notes below is by checking out this survey that we recently sent to workshop participants. It’s designed for people who’ve engaged directly with our content so it won’t be that relevant for people to fill in necessarily but it gives an overview of the kinds of techniques Leverage developed and areas they researched.
What did Leverage 1.0 work on?
A very brief summary is that the first eight and a half years of Leverage (let’s call this “Leverage 1.0” as a catch-all for those organisations before the restructure) was at first a prioritisation research project looking at what should people work on if they want to improve the world. Leverage 1.0 later came to focus more on understanding and improving people as their psychological frameworks and training tools developed but they still conducted a wide range of research.
This means that in the very early days they were thinking a lot about how to prioritise, how to make good long term plans and just trying a bunch of things. I get the impression that at this stage almost nothing was ruled out in terms of what might be worth exploring if you wanted to improve the world. This meant people investigating all sorts of things like technological intelligence amplification, nootropics, and conducting polyphasic sleep experiments. People might be researching what caused the civilisational collapse that led to the dark ages, the beliefs of a particular Christian sect, or what lead to the development of Newtonian physics. Leverage felt this was important for research progress. They wanted researchers to follow what motivated them. They thought that it was important to investigate a lot of areas before deciding where to focus their efforts because deciding what to prioritise is so important to overall impact. This felt particularly important when investigating moon-shots which had the potential to be extremely valuable even if they seemed unlikely at the outset.
Some of the outputs of these early days of research included training sessions on:
Planning—How to build and error-check plans for achieving your goals
Expert assessment—how to determine if someone is an expert in a given domain when you lack domain knowledge
Learning to learn—how to improve and expand the scope of your learning process
Theorizing—how to build models and improve your model building process over time
Prioritisation and goal setting—how to find your goals, back chain plans from them etc
This is far from everything but gives you a flavour.
Geoff had developed a basic model of psychology called Connection Theory (CT) so this was a thing that was investigated alongside everything else. This involved spending a lot of time testing the various assumptions in CT.
Through experimenting with using CT in this way, Leverage eventually found they were able to use ideas from CT to make some basic predictions about individual and group behaviour, help individuals identify and remove bottlenecks so that they could self improve and perhaps even identify and share specific mental moves people were using to make research progress on particular questions. This made the team more excited about psychology research in particular (amongst the array of things people were researching) as a way to improve the world.
From there they (alongside the newly founded Paradigm Academy) developed some of the research into things like
one on one and group training,
A catalogue of different mental procedures individuals use when conducting research so that they could be taught to others to use to tackle different research and other problems. One example intellectual procedure (IP) just to give a sense of this, is the Proposal IP where you use the fact that you have a lot of implicit content in your mind and your taste response to inelegant proposals to speed up your thinking in an area.
Specific training in strategy, theorizing, and research
A collection of specific introspection and self-improvement techniques such as:
Self-alignment (a tool for increasing introspective access which handled a class of cases where our tools previously weren’t working)
Anti-avoidance techniques (making it so you can think clearly in areas you previously didn’t want to think about or had fuzzy thoughts in)
Charting (a belief change tool that has been modified and built out a lot since it’s initial release)
Mythos (tool for introspection with imagery, helpful for more visual people)
Integration and de-zoning (tools for helping people connect previously separate models)
What is Leverage doing now?
As for what Leverage is currently working on, once we have posted our retrospective we’ll then be updating Leverage’s website to reflect its current staff and focus so again a better update than I can provide is pending.
The teaser here is that from the various research threads being pursued in the early years of Leverage 1.0, Leverage today has narrowed their focus to be primarily on two areas that they found the most promising in their early years:
Scientific methodology research
Psychology
We also continue to be interested in sociology research and expect to bring on research fellows (either full time or part of future fellowship programmes) focusing on sociology in the future. However, since we’re relaunching our website and research programme we want to stay focused so we’re punting building out more of our sociology work to further down the line.
The scientific methodology research involves continuing to look at historical examples of scientific breakthroughs in order to develop better models of how progress is made. This continues some of our early threads of research in theorising, methodology, historical case studies and the history of science. We’re particularly interested in how progress was made in the earlier stages of the development of a theory or technology. Some examples include looking at what led to the transition in chemistry from Phlogiston to Lavoisier’s oxygen theory or the challenges scientists had in verifying findings from the first telescopes. We aim to share lessons from this research with researchers in a variety of fields. In particular, we want to support research that is in its earlier, more explorative stages. This is more of a moon-shot area but this means it can get less attention while being potentially high reward.
Our psychology research aims to continue to build on the progress and various research threads Leverage 1.0 was following. While this is quite a moon shot style bet, if we can improve our understanding of people then we potentially improve the ways in which they work together to solve important problems. At this point, we have developed tools for looking at the mind and mental structures that we think work fairly well on the demographics of people we’ve been working with. I got a ballpark estimate from someone at Leverage that Leverage and Paradigm have worked with around 400 people for shallower training, and about 60 for in-depth work but treat those figures as a guess until we write something up formally. We’ve focused in the last few years on improving these tools so they work in harder cases (e.g. people who have trouble introspecting initially) and using the tools to find common mental structures. Moving forward with this research we want to test the tools in a more rigorous way, in particular by communicating with people in academia to see whether or not they can validate our work.
One thing I personally like about the plans for psychology research is that it also acts as a check on our scientific methodology research. If the insights we gain from looking at the history of scientific progress aren’t useful to us in making progress in psychology then that’s one negative sign on their overall usefulness.
Who works for Leverage and Paradigm?
The team is much smaller and the organisation structure slightly more defined (although there is a way to go here still). There are four researchers (including Geoff who is also the Executive Director) and I’ll be joining as a Program Manager managing the researchers and helping communicate with the public about our work. So four in total at the moment, five once I start.
While Leverage Research in its newer form is getting going it still receives a lot of help from its sister organisation Paradigm Academy. This means that while they are two separate organisations, currently Paradigm staff give a lot of time to helping Leverage, particularly in areas like operations and helping with PR and Communications like the website relaunch. This helps allow the researchers to focus on their research and means the burden of public communication won’t all fall on their newest employee (me). Once a lot of that is done though we expect to make the division between the two organisations clearer. Paradigm currently has nine employees including Geoff.
I expect all of this will generate more questions than it answers at the moment and while my answer is to wait for Leverage’s formal content to be published I can see why this is frustrating. I hope my examples give a small amount of insight into our work while we take the time to write things up. You have every reason to be sceptical about Leverage posting content given various promises made in the past. I think given our track record on public communication that scepticism is valid. All I can perhaps offer in the meantime is that I personally am very keen to see both the retrospective and the new Leverage website published and the get sh*t done spirit that you and others on this forum know me for is part of the reason they’ve offered me a job to help with this in the first place.
Why I chose to work at Leverage
As for my personal reasons for choosing to accept an offer from Leverage, I expect this to be hard to transmit just because of inferential distance. My decision was the result of at least five months of discussions, personal research and resultant updates all of which is built on various assumptions that caused me to already be pursuing the plans I was at CEA.
I’ll attempt a short version here anyway in case it’s helpful. If there’s a lot of interest I’ll consider writing this up but I’m not sure it’ll be sufficiently useful or interesting to be worth the time cost.
Broadly speaking, I created a framework (building off a lot of 80Ks work but adapting it to suit my needs) to use to compare options on:
Impact - comparing potential career plans by looking at the scale and likelihood of success across:
the problem being tackled (e.g. preventing human extinction),
the approach to solving that problem (e.g. develop AGI in a way that’s safe)
the organisation (e.g. DeepMind)
and what I personally could contribute (e.g. say in a role as Project Manager)
Personal happiness (personal fit with the culture, how the job would fit into my life etc)
Future potential (what skills would I build and how useful are they, and what flexible resources such as useful knowledge or network would I gain)
I decided that I was willing to bet at least a few more years of my career on the more moon shot type plans to build a much better future (something like continuing personally to follow CEA’s vision of working towards an optimal world).
This narrowed my focus down to primarily considering paths related to avoiding existential risks and investing in institutions or advances that would improve humanity’s trajectory. In exploring some options around contributing to AI safety in some way I came away both not feeling convinced that I wouldn’t potentially cause harm (through speeding up the development of AGI) and less sure of the arguments for now being a particularly important hinge on this. It, therefore, seemed prudent to learn a lot more before considering this field.
This left me then both wanting to invest more time in learning more while also not wanting to delay working on something I thought was high impact indefinitely. In terms of impact, the remaining areas were advances or institutions that might improve humanity’s ability to tackle global problems.
I’d had plenty of conversations with various people about Leverage (including many Leverage sceptics) in the past and interacted with Leverage and Paradigm directly to some degree, mostly around their introspection techniques which I personally have found extremely useful for self-improvement. I knew that they were interested in psychology initially as a potential way to improve humanity’s trajectory (but didn’t yet understand the scope of their other research) so I reached out to chat about this. I found that many of the people there had already thought a lot about the kinds of things I was considering as options for improving the long-term future and they had some useful models. Those interactions plus my positive view of their introspection techniques led me to think that Leverage had the most plausible plan given my current uncertainty for improving the long-term future and was likely to be by far the best option for me in terms of self-improvement and gaining the knowledge I wanted for making better future plans. Their recent restructure, desire to establish a more structured organisation and plans to publish a lot of content meant they had an opening for my particular skill set and the rest, as they say, is history.
Did this series end up being published?
Just wanted to say I super appreciated this writeup.
Thanks Raemon :-) I’m glad it was helpful.
This aged well… and it reads like what ChatGPT would blurt, if you asked it to “sound like a convincingly respectful and calm cult with no real output.” Your ‘Anti-Avoidance,’ in particular, is deliciously Orwellian. “You’re just avoiding the truth, you’re just confused...”
I was advocating algal and fish farming, including bubbling air into the water and sopping-up the fish poop with crabs and bivalves—back in 2003. Spent a few years trying to tell any marine biologist I could. Fish farming took-off, years later, and recently they realized you should bubble air and catch the poop! I consider that a greater real-world accomplishment than your ‘training 60+ people on anti-avoidance of our pseudo-research.’ Could you be more specific about Connection Theory, and the experimental design of the research you conducted and pre-registered, to determine that it was correct? I’m sure you’d have to get into some causality-weeds, so those experimental designs are going to be top-notch, right? Or, is it just Geoff writing with the rigor of Freud on a Slack he deleted?
Thanks Larissa—the offer to write up posts from Leverage Research is a generous offer. Might it not be a more efficient use of your time, though, to instead answer questions about Leverage the public domain, many of which are fairly straightforward?
For example, you mention that Leverage is welcoming to new staff. This sounds positive—at the same time, the way Leverage treated incoming staff is one of the main kinds of fact discussed in the top-level post. Is it still true that: (i) staff still discuss recruitees on individual slack channels, (ii) mind-mapping is still used during recruitment of staff, (iii) growth-rates are overestimated, (iv) specific lists of attendees are recruited from EA events, and (v) negative rumours are still spread about other organizations that might compete for similar talent? To the extent that you are not sure about (i-v), it would be interested to know whether you raised those concerns with Geoff in the hiring process, before joining the organization.
For other questions raised by the top-level post: (a) are Leverage’s outputs truly as they appear? (b) Is its consumption of financial resources and talent, as it appears? (c) Has it truly gone to such efforts to conceal its activities as described under the general transparency section? (d) How will Leverage measure any impact from its ninth year of operation?
From another post: (e) How many of the staff at Leverage Research are also affiliated with Paradigm Academy? (f) How much of the leadership of Leverage Research is also playing a leading role at Paradigm Academy?
Since your investigations give you much more information on this topic than is available to an interested outsider, it should be very easy for you to help us out with these questions.
Hi Anonymoose,
I’d like to do two things with my reply here.
First, to try and answer your questions as best I can.
But then second, start to work out how to make future conversations with you about Leverage more productive
1. ANSWERING YOUR QUESTIONS
I’d recommend first reading my recent reply to Greg because this will give you a lot of relevant context and answers some of your questions.
Questions a, b and d: outputs, resources and future impact
Your Questions:
In terms of questions a, b and d, I will note the same thing as I said in my reply to Greg which is that we’re currently working both on a retrospective of the last eight and a half years of Leverage and on updating Leverage’s existing website. I think these posts and updates will then allow individuals to assess for themselves
our past work and outputs
whether it was worth the resources invested
our plans for the future
For now, though sections “What did Leverage 1.0 work on?” and “What is Leverage doing now” in my reply to Greg will give you some information about things Leverage did and its plans for the future.
It’s hard to comment on whether any of these things are “as they appear” as (1) there isn’t enough public content to assess and (2) different people seem to have had very different experiences with Leverage and have very different views on their work
This means how Leverage’s work appears depends on an individual’s interactions with them. This has made it hard for any consensus to emerge and so debates continue. Once we’ve published more of our content, I hope it will be easier to sync up and assess them.
Questions e and f: staffing cross over between Leverage and Paradigm
Your questions (referencing one of your other posts about Leverage):
On questions, e and f, see the section in my recent reply to Greg called “Who works for Leverage and Paradigm?”.
Geoff Anders is the founder and Executive Director of both Paradigm and Leverage and as such is currently the only member of staff working at both organisations. The two are sister organisations with different missions that collaborate. Leverage’s mission was essentially conducting research. Paradigm’s focus was much more on training. This means that there is some natural overlap and historically the two organisations have worked closely together. Paradigm uses Leverage’s research content in their training and in return they provide practical support (operations support, help to update the website) to Leverage.
We have no intention as you and the original poster propose of using the Paradigm or other brands to replace Leverage if there are too many problems with its brand. We created other organisations with different brands to distinguish between the different work they were doing. We’ve intentionally continued our current research under the same name so as to link to our origins with Leverage 1.0. When we update both websites (starting with Leverage) over the coming months you will be able to see the teams at each.
Questions i) and iv) discussing potential recruits in slack and recruitment at events
Your questions:
On question i), yes Leverage discuss people they are considering hiring on slack both in recruitment slack channels and DMs as part of deciding whether to hire someone.
With regards to workshops in particular, we will mention people attending workshops on slack but this is usually in the form of staff coordinating to arrange training and share training relevant information (e.g. “this person had trouble belief reporting, it would be good to assign a trainer during the 1-1 who could help them belief report”). If someone is a workshop attendee and we are considering hiring them they might be discussed in both contexts.
Leverage and Paradigm have also (in answer to question iv) attended (EA and non-EA) events with lists of people they plan to speak to because they might be good potential hires. In addition, at events, they often do things like having tables so that people interested in working there (or interested in their work in general) can ask questions. Of course, they also attend events for non-recruitment reasons (e.g. learning about interesting topics by attending talks and having discussions with people). Leverage are particularly interested in meeting independent researchers. Paradigm are interested in meeting people interested in self-improvement.
Question ii) “mind-mapping is still used during recruitment of staff”
On question ii), here I assume you’re asking about using the charting procedure which I think might have been called mind mapping at some point. This is a way of taking system one beliefs someone has and trying to map out why a person has those beliefs and what actions they taking as a result of those beliefs that they want to change.
Historically Paradigm and Leverage did do some charting with potential hires to see if they could use some of the basic techniques and found them helpful. Since Paradigm used charting in training and Leverage in research it was important new hires are aware of that, have a basic understanding of the tools and some interest in it. This is no longer needed in the Leverage hiring process as not all researchers will be doing any research relating to CT or charting but continues to be relevant to Paradigm for training so they still do some charting with potential hires. Staff will of course use their models of psychology to aid hiring decisions because they have useful models in this area but this is more akin to the way that recruiters might use their recruitment experience and intuition to aid decisions than I expect people are imagining.
Leverage and Paradigm both have a strict confidentiality policy for all one to one sessions with individuals which means this information is not discussed as part of recruitment discussions without permission.
While I don’t expect my experience to necessarily be indicative, I can speak a bit to my personal recruitment experience with Leverage. I worked on a trial project part-time for a couple of months looking at things like types of events different communities run and why people join communities and I attended a formal interview. I did charting with someone on I think two occasions, both at my own personal request and nothing to do with hiring. My most relevant nearby experience is in running recruitment rounds at CEA. I’d say that the Leverage process felt less organised and structured than the CEA process but was otherwise what I’d expect.
Leverage and Paradigm have both recently been working to make their hiring process more structured, in part based on feedback from potential hires. For Leverage, people can send a resume and would be invited to a research interview to discuss their research and our requirements. From there, successful candidates would take part in a trial period. Paradigm candidates go through several stages: sending a resume, attending an initial screening interview, a follow-up interview that aims to test job-relevant skills, a one week trial primarily checking team fit and finally an extended trial testing fit for the role. The process I went through was more like a less structured version of the Paradigm process. This is because while I’ll be working at Leverage, I won’t be a researcher so the Paradigm recruitment process made more sense in my particular case.
Question iii) “growth-rates are overestimated”
On question iii, am I right in thinking you’re referring to the claim in the original post that
If so, it’s hard to comment on what happened here without a lot more context on the conversation. I asked Geoff about this and his guesses were that this could have been confusion about hiring and attrition (was the conversation about hiring rates or about the total number of staff), numbers of volunteers or internal estimates for growth rate which turned out to be harder than we thought.
Question v) “negative rumours are still spread about other organizations that might compete for similar talent?”
Intentionally spreading rumours is certainly not Leverage policy nor endorsed or encouraged among staff. Of course, members of staff will have their own views on how best to improve the world and therefore I would expect them to have views on a number of EA and other organisations, both positive and negative.
I have the same difficulty in giving a useful answer to this question as with question iii, in that the original poster has made a claim about Leverage with a strong connotation but without providing the specifics or reasons they believe this.
I think it’s important for individuals to be able to share and discuss their views. I think it would be more helpful in cases like this forum post and when discussing other organisations people try to qualify what they think, why and where that information is from so that recipients of that information come away with a better understanding. This can be difficult but I think it’s important to try.
Question c) “Has it truly gone to such efforts to conceal its activities as described under the general transparency section?”
Finally on question c, yes our past website was removed from the Wayback machine. With hindsight this was a mistake. At the time people were digging up our old content (for example our old long term plan document) and using this to hype up various Leverage conspiracy theories. While this was entertaining, it was certainly distracting and meant people were even more confused about what we were doing. We thought that if we took the content off the wayback it would reduce the conspiracy style hype but removing it only fuelled the fire of conspiracies.
As you can probably tell, Leverage did not invest very many skill points or chips in public communication in the early days, instead spending them pretty much exclusively on research and experimentation. This means we’ve still got a lot to learn in the public communication space and a lot of mistakes to make up for.
As discussed above, Leverage did not use different organisations as a way to tailor messages to donors, different organisations had different focuses and donors understood this. Similarly we have no plans to replace Leverage as a brand.
2. MAKING OUR INTERACTIONS WITH ONE ANOTHER MORE PRODUCTIVE
I’d like to ensure that any future interactions between you and Leverage are a good use of both of our time, informative to readers, and helpful to you on whatever the real issues are.
Looking at your posting history, it seems that you’ve created this anonymous account only to ask questions about Leverage that come across at least as excuses to make insinuations against them. I don’t currently get the impression that you are following the spirit of the EA Forum guidelines and posting with a scout mindset, trying to give people an accurate view or being clear about what you believe and why.
If you would be willing to spend time trying to help me understand and address your particular concerns about Leverage then I am very happy to spend time on that. If you only plan to continue in the same vein as your previous comments I don’t currently expect that to yield anything that feels useful for either of us. If you would like to chat more to figure out a better way forward that gets your concerns addressed let me know. You can reach me here or at larissa.e.rowe [at] gmail.com.