LarissaHeskethRowe

Karma: 201

LarissaHeskethRowe 18 Nov 2023 19:52 UTC
−14 points
2 ∶ 5
in reply to: Larks’s comment on: Sam Altman fired from OpenAI
I do enjoy the secret Leverage spy stories as it makes my life seem more exciting than it is but they don’t ever make me feel very optimistic about EA epistemics.

LarissaHeskethRowe 1 Nov 2022 23:41 UTC
−15 points
0 ∶ 0
in reply to: Jonathan_Michel’s comment on: Ben_West’s Shortform
In case it’s helpful and you’ve not read them, I think that the two main pieces of context that would be helpful are Cathleen’s original post and the Twitter thread of Kerry Vaughan’s that Ben West is referring to.

Cathleen has also now commented above which I think clarifies how her request was meant to be taken. Here’s a quote (as context, she’s formatted her comment to be an idea for how CEA might have responded instead):

(At a cursory glance, it’s difficult to determine the most natural interpretation of the scope of Cathleen’s request, in order to assess the likelihood that the user was knowingly violating her wishes. We initially had a quite narrow interpretation, reading the quote out of context, but I think the situation becomes clearer if you take the time to read her entire post or the section that the quote was pulled from, entitled “We want to move forward with our lives (and not be cancelled)”, which includes a direct reference to LinkedIn/her intent to keep her work history private.)

LarissaHeskethRowe 1 Nov 2022 23:31 UTC
4 points
0 ∶ 0
in reply to: Cathleen’s comment on: Ben_West’s Shortform
I would have so much respect for CEA if they had responded like this.

LarissaHeskethRowe 7 Oct 2022 0:26 UTC
10 points
1 ∶ 0
in reply to: Jeff Kaufman’s comment on: Research Deprioritizing External Communication
I just wanted to say thank you for doing this Jeff. I sympathize with Rockwell Schwartz’s general point, but since Cathleen’s post asks that people not use her full name or name her former colleagues I appreciate you taking this seriously.
(For clarity, I don’t mind people using my full name. It’s my forum username and very easily found e.g. on Leverage’s website. But I currently work at Leverage Research and decided to work there knowing full well how some people in EA react when the topic of Leverage comes up. The same is not true of everyone, and I think individuals who have not chosen to be public figures should be allowed to live in peace should they wish to).

LarissaHeskethRowe 7 Oct 2022 0:16 UTC
4 points
2 ∶ 0
on: Research Deprioritizing External Communication
Larissa from Leverage Research here. I think there might be an interesting discussion to be had about the relationship between feedback loops, external communication (engaging with your main external audiences), and public communication (trying to communicate ideas to the wider public).
For a lot of the history of scientific developments, sharing research, let alone widely distributing it was expensive and rare. Early discoveries in the history of electricity, for example, were nonetheless still made, often by researchers who shared little until they had a complete theory, or a new instrument to display. Often the feedback loops were simply direct engagement with the phenomena itself. Only in more recent history has it become cheap and easy enough to widely share research such that this has become the norm. Similarly, as a couple of people have mentioned in the comments, there are more recent examples of groups that have done great research while having little external engagement: Lockheed Martin and the Manhattan Project being two well-known examples.
This suggests that it is feasible to have feedback loops while doing little external communication of any kind. During Leverage 1.0^[1] people relied more on feedback from their own experiences, interactions with teammates’ experiences and views, workshops and coaching.
That said, we do believe (for reasons independent of research feedback loops) that it was a mistake to not do more external communication in the past, which is why this is something Leverage Research has focused on since 2019. More recently, we have also come to think that it is also important to try to communicate to the wider public (in ways that can be broadly understood) as opposed to just your core audience or peer group. One reason for this is that if projects are only communicated about, and criticisms only accepted in, the language of the particular group that developed them, it’s easy for blindspots to remain until it is too late. (I recommend Glen Weyl’s “Why I’m Not A Technocrat” for a more detailed treatment of this topic.)
For anyone interested in some of our other reflections on public engagement, I recommend reading our 2019-2020 annual report or our Experiences Inquiry Report. The former is Leverage Research’s first annual report since the re-organization in 2019, and one topic we discuss is our new focus on external engagement. The latter shares findings from our inquiry last year into the experiences of former collaborators during Leverage 1.0. To see our engagement efforts today, I recommend checking out our website, subscribing to our newsletter, or following us on Twitter or Medium.
For those interested in the exploratory psychology research Jeff mentions, we recommend reading our write-up from earlier this year covering our 2017 − 2019 Intention Research and keeping an eye on our Exploratory Psychology Research Program page. We are currently working on two pieces: one on risks from introspection (we discuss this a bit on Twitter here), and one on Belief Reporting (an introspective tool developed during Leverage 1.0). We’re also thinking of sharing a few documents written pre-2019 that relate to introspection techniques. These would perhaps be less accessible for a wider audience unfamiliar with our introspective tools but may nonetheless be of interest to those who want to dive deeper on our introspective research. All of this will be added to our website when completed.
Finally, I just wanted to thank Jeff for engaging with us in a discussion of his post. Although we disagreed on some things and it ended up a lengthy discussion, I do feel like I came to understand a bit more of where the disagreement stemmed from, and the post was improved through the process. This seems valuable, so I would like to see that norm encouraged.
1. ^
  As context, “Leverage 1.0” is the somewhat clumsy term I introduced as a shorthand for the decentralized research collaboration between a few organizations from 2011 to 2019 that’s commonly referred to as “Leverage,” so as to distinguish it from Leverage Research the organization since 2019 which looks very different.

LarissaHeskethRowe 29 Nov 2019 17:39 UTC
1 point
0 ∶ 0
in reply to: Chris Leong’s comment on: Updates from Leverage Research: history, mistakes and new focus
Hi Casebash,
Thank you for the question; this is an important topic.
We believe that advances in psychology could make improvements to many people’s lives by helping with depression, increasing happiness, improving relationships, and helping people think more clearly and rationally. As a result, we’re optimistic that the sign can be positive. Our past work was primarily focused on these kinds of upsides, especially self-improvement; developing skills, improving rationality, and helping people solve problems in their lives.
That said, there are potential downsides to advancing knowledge in a lot of areas, which are important to think through in advance. I know the EA community has thought about some of the relevant areas such as flow-through effects and how to think about them (e.g. the impact of AMF on population and the meat-eater problem) and cases where extra effort might be harmful (e.g. possible risks to AI safety from increasing hardware capacities and whether or not working on AI safety might contribute to capabilities).
Leverage 1.0 thought a lot about the impact of psychology research and came to the view that sharing the research would be positive. Evaluating this is an area where it’s hard to build detailed models though so I’d be keen to learn more about EA research on these kinds of questions.

LarissaHeskethRowe 29 Nov 2019 17:19 UTC
23 points
0 ∶ 0
on: EA Updates for November 2019
Every time you post these each month, I end up thinking something like “these are so useful, I’m really grateful David does this”. I thought this month I should actually tell you that, so thank you so much for posting these!

LarissaHeskethRowe 24 Nov 2019 22:04 UTC
0 points
0 ∶ 0
in reply to: Chris Leong’s comment on: Updates from Leverage Research: history, mistakes and new focus
Hi casebash,
We are conducting psychology research based on the following assumptions:
1) psychology is an important area to understand if you want to improve the world
2) it is possible to make progress in understanding the human mind
3) the current field of psychology lags behind its potential
4) part of the reason psychology is lagging behind its potential is that it has not completed the relevant early stage science steps
5) during Leverage 1.0, we developed some useful tools that could be used by academics in the field to make progress in psychology.

Assumptions 2) and 5) are based on our experience in conducting psychology research as part of Leverage 1.0. The next step will be to test these assumptions by seeing if we can train academics on a couple of the introspection tools we developed and have them use them to conduct academic research.

Assumptions 3) and 4) are something we have come to believe from our investigations so far into existing psychology research and early stage science. We are currently very uncertain about this and so further study on our part is warranted.

What we are trying to accomplish is to further the field of psychology, initially by providing tools that others in the field can use to develop and test new theories. The hope is that we might make contributions to the field that would help it advance. Contributing to significant progress in psychology is, of course, a very speculative bet but, given our views on the importance of understanding psychology, one that still seems worth making.
I hope that helps. Let me know if you have further questions.

LarissaHeskethRowe 24 Nov 2019 20:38 UTC
5 points
0 ∶ 0
in reply to: Jeff Kaufman’s comment on: Updates from Leverage Research: history, mistakes and new focus
Hi Jeff,
Thanks for taking the time to check out the paper and for sending us your thoughts.
I really like the examples of building new instruments and figuring out how that works versus creating something that’s a refinement of an existing instrument. I think these seem very illustrative of early stage science.
My guess is that the process you were using to work out how your forked brass works, feels similar to how it might feel to be conducting early stage science. One thing that stood out to me was that someone else trying to replicate the instrument found, if I understood this correctly, they could only do so with much longer tubes. That person then theorised that perhaps the mouth and diaphragm of the person playing the instrument have an effect. This is reminiscent of the problems with Galileo’s telescope and the difference in people’s eyesight.
Another thought this example gave me is how video can play a big part in today’s early stage science, in the same way, that demonstrations did in the past. It’s much easier to demonstrate to a wide audience that you really can make the sounds you claim with the instrument you’re describing if they can watch a video of it. If all people had was a description of what you had built, but they couldn’t create the same sound on a replica instrument, they might have been more sceptical. Being able to replicate the experiment will matter more in areas where the claims made are further outside of people’s current expectations. “I can play these notes with this instrument” is probably less unexpected than “Jupiter has satellites we hadn’t seen before and I can see them with this new contraption”. This is outside of the scope of our research, it’s just a thought prompted by the example.
I’ve asked my colleagues to provide an answer to your questions about how controversial the claim that early stage science works differently is and whether it seems likely that there would still be early stage science today. I believe Mindy will add a comment about that soon. We’ll also amend the typo, thanks for pointing that out!

LarissaHeskethRowe 23 Nov 2019 15:04 UTC
3 points
0 ∶ 0
in reply to: JP Addison’s comment on: Updates from Leverage Research: history, mistakes and new focus
Perfect, thank you. I’ve edited it and added a footnote.

LarissaHeskethRowe 23 Nov 2019 14:58 UTC
3 points
0 ∶ 0
in reply to: JP Addison’s comment on: Updates from Leverage Research: history, mistakes and new focus
Thanks JP and Edoarad! 😄

LarissaHeskethRowe 22 Nov 2019 22:09 UTC
3 points
0 ∶ 0
in reply to: Jeff Kaufman’s comment on: Updates from Leverage Research: history, mistakes and new focus
Thanks Jeff :-) I hope it’s helpful.

LarissaHeskethRowe 22 Nov 2019 22:07 UTC
9 points
0 ∶ 0
in reply to: JP Addison’s comment on: Updates from Leverage Research: history, mistakes and new focus
Yeah this makes sense, thanks for asking for clarification. The communication section is meant to be a mixture of i) and ii). I think in many cases it was the right decision for Leverage not to prioritise publishing a lot of their research where doing so wouldn’t have been particularly useful. However we think it was a mistake to do some public communication and then remove it, and not to figure out how to communicate about more of our work.
I’m not sure what the best post etiquette is here, should I just edit the post to put in your suggestion and note that the post was edited based on comments?

LarissaHeskethRowe 22 Nov 2019 22:03 UTC
5 points
0 ∶ 0
in reply to: LarissaHeskethRowe’s comment on: Updates from Leverage Research: history, mistakes and new focus
(totally unrelated to the actual post but how did you include an emoticon JP?)

LarissaHeskethRowe 22 Nov 2019 22:02 UTC
7 points
0 ∶ 0
in reply to: JP Addison’s comment on: Updates from Leverage Research: history, mistakes and new focus
Hi JP,
(Haha, I did wonder about having so many headings, but it just felt so organised that way, you know 😉)
With regards to removing content we published online, I think we hit the obvious failure mode I expect a lot of new researchers and writers run into, which was that we underestimated how time-consuming, but also stressful, posting publicly and then replying to all the questions can be. To be honest, I kind of suspect early and unexpected negative experiences with public engagement led Leverage to be overly sceptical of it being useful and nudged them away from prioritising communicating their ideas.
From what I understand, some of the key things we ended up removing were:
1) content on Connection Theory (CT)
2) a long-term plan document
3) a version of our website that was very focused on “world-saving.”
With the CT content, I don’t think we made sufficiently clear that we thought of CT as a Kuhnian paradigm worth investigating rather than a fully-fledged, literally true-about-the-world claim.
Speaking to Geoff, it sounds like he assumed people would naturally be thinking in terms of paradigms for this kind of research, often discussed CT under that assumption and then was surprised when people mistook claims about CT to be literal truth claims. To clarify, members of Leverage 1.0 typically didn’t think about CT as being literally true as stated, and the same is true of today’s Leverage 2.0 staff. I can understand why people got this impression from some of their earlier writing though.
This confusion meant people critiqued CT as having insufficient evidence to believe it upfront (which we agree with). While the critiques were understandable, it wasn’t a reason to believe that the research path wasn’t worth following, and we struggled to get people to engage with CT as a potential paradigm. I think the cause of the disagreement wasn’t as clear to us at the time, which made our approach challenging to convey and discuss.
With the long-term planning documents, people misinterpreted the materials in ways that we didn’t expect and hadn’t intended (e.g. as being a claim about what we’d already achieved or as a sign that we were intending something sinister). It seems as though people read the plan as a series of predictions about the future and fixed steps that we were confident we would achieve alone. Instead, we were thinking of it as a way to orient on the scale of the problems we were trying to tackle. We think it’s worth trying to think through very long-term goals you have to see the assumptions that are baked in into your current thinking and world model. We expect solving any problems on a large scale to take a great deal of coordinated effort and plans to change a lot as you learn more.
We also found that a) these kinds of things got a lot more focus than any of our other work which distorted people’s perceptions of what we were doing and b) people would frequently find old versions online and then react badly to them (e.g. becoming upset, confused or concerned) in ways we found difficult to manage.
In the end, I think Leverage concluded the easiest way to solve this was just to remove everything. I think this was a mistake (especially as it only intensified everyone’s curiosity) and it would have been better to post something explaining this problem at the time, but I can see why it might have seemed like just removing the content would solve the problem.

LarissaHeskethRowe 22 Nov 2019 19:18 UTC
10 points
0 ∶ 0
in reply to: Milan_Griffes’s comment on: Updates from Leverage Research: history, mistakes and new focus
Hi Milan, we’re still deciding what, if any, information it seems appropriate to share about our donors publicly. We do expect some Leverage 1.0 donors to continue to support Leverage 2.0. We will also soon start fundraising for Leverage 2.0 and will probably engage with communities that are interested in areas like early stage science and meta science.

Updates from Leverage Research: history, mistakes and new focus

LarissaHeskethRowe22 Nov 2019 16:19 UTC

31 points

32 comments20 min readEA link

LarissaHeskethRowe 11 Oct 2019 12:42 UTC
3 points
0 ∶ 0
in reply to: JP Addison’s comment on: How to use the Forum
That’s great! Thanks for sorting that so quickly and for letting me know.

LarissaHeskethRowe 22 Sep 2019 13:22 UTC
21 points
0 ∶ 0
in reply to: anonymoose’s comment on: Leverage Research: reviewing the basic facts
Hi Anonymoose,
I’d like to do two things with my reply here.
1. First, to try and answer your questions as best I can.
2. But then second, start to work out how to make future conversations with you about Leverage more productive
1. ANSWERING YOUR QUESTIONS
I’d recommend first reading my recent reply to Greg because this will give you a lot of relevant context and answers some of your questions.
Questions a, b and d: outputs, resources and future impact
Your Questions:
“(a) are Leverage’s outputs truly as they appear?”
“(b) Is its consumption of financial resources and talent, as it appears?”
“(d) How will Leverage measure any impact from its ninth year of operation?”

In terms of questions a, b and d, I will note the same thing as I said in my reply to Greg which is that we’re currently working both on a retrospective of the last eight and a half years of Leverage and on updating Leverage’s existing website. I think these posts and updates will then allow individuals to assess for themselves
1. our past work and outputs
2. whether it was worth the resources invested
3. our plans for the future
For now, though sections “What did Leverage 1.0 work on?” and “What is Leverage doing now” in my reply to Greg will give you some information about things Leverage did and its plans for the future.
It’s hard to comment on whether any of these things are “as they appear” as (1) there isn’t enough public content to assess and (2) different people seem to have had very different experiences with Leverage and have very different views on their work
This means how Leverage’s work appears depends on an individual’s interactions with them. This has made it hard for any consensus to emerge and so debates continue. Once we’ve published more of our content, I hope it will be easier to sync up and assess them.

Questions e and f: staffing cross over between Leverage and Paradigm
Your questions (referencing one of your other posts about Leverage):
“(e) How many of the staff at Leverage Research are also affiliated with Paradigm Academy?”
“(f) How much of the leadership of Leverage Research is also playing a leading role at Paradigm Academy?”
On questions, e and f, see the section in my recent reply to Greg called “Who works for Leverage and Paradigm?”.
Geoff Anders is the founder and Executive Director of both Paradigm and Leverage and as such is currently the only member of staff working at both organisations. The two are sister organisations with different missions that collaborate. Leverage’s mission was essentially conducting research. Paradigm’s focus was much more on training. This means that there is some natural overlap and historically the two organisations have worked closely together. Paradigm uses Leverage’s research content in their training and in return they provide practical support (operations support, help to update the website) to Leverage.
We have no intention as you and the original poster propose of using the Paradigm or other brands to replace Leverage if there are too many problems with its brand. We created other organisations with different brands to distinguish between the different work they were doing. We’ve intentionally continued our current research under the same name so as to link to our origins with Leverage 1.0. When we update both websites (starting with Leverage) over the coming months you will be able to see the teams at each.

Questions i) and iv) discussing potential recruits in slack and recruitment at events
Your questions:
“(i) staff still discuss recruitees on individual slack channels”
“(iv) specific lists of attendees are recruited from EA events”
On question i), yes Leverage discuss people they are considering hiring on slack both in recruitment slack channels and DMs as part of deciding whether to hire someone.
With regards to workshops in particular, we will mention people attending workshops on slack but this is usually in the form of staff coordinating to arrange training and share training relevant information (e.g. “this person had trouble belief reporting, it would be good to assign a trainer during the 1-1 who could help them belief report”). If someone is a workshop attendee and we are considering hiring them they might be discussed in both contexts.
Leverage and Paradigm have also (in answer to question iv) attended (EA and non-EA) events with lists of people they plan to speak to because they might be good potential hires. In addition, at events, they often do things like having tables so that people interested in working there (or interested in their work in general) can ask questions. Of course, they also attend events for non-recruitment reasons (e.g. learning about interesting topics by attending talks and having discussions with people). Leverage are particularly interested in meeting independent researchers. Paradigm are interested in meeting people interested in self-improvement.

Question ii) “mind-mapping is still used during recruitment of staff”
On question ii), here I assume you’re asking about using the charting procedure which I think might have been called mind mapping at some point. This is a way of taking system one beliefs someone has and trying to map out why a person has those beliefs and what actions they taking as a result of those beliefs that they want to change.
Historically Paradigm and Leverage did do some charting with potential hires to see if they could use some of the basic techniques and found them helpful. Since Paradigm used charting in training and Leverage in research it was important new hires are aware of that, have a basic understanding of the tools and some interest in it. This is no longer needed in the Leverage hiring process as not all researchers will be doing any research relating to CT or charting but continues to be relevant to Paradigm for training so they still do some charting with potential hires. Staff will of course use their models of psychology to aid hiring decisions because they have useful models in this area but this is more akin to the way that recruiters might use their recruitment experience and intuition to aid decisions than I expect people are imagining.
Leverage and Paradigm both have a strict confidentiality policy for all one to one sessions with individuals which means this information is not discussed as part of recruitment discussions without permission.
While I don’t expect my experience to necessarily be indicative, I can speak a bit to my personal recruitment experience with Leverage. I worked on a trial project part-time for a couple of months looking at things like types of events different communities run and why people join communities and I attended a formal interview. I did charting with someone on I think two occasions, both at my own personal request and nothing to do with hiring. My most relevant nearby experience is in running recruitment rounds at CEA. I’d say that the Leverage process felt less organised and structured than the CEA process but was otherwise what I’d expect.
Leverage and Paradigm have both recently been working to make their hiring process more structured, in part based on feedback from potential hires. For Leverage, people can send a resume and would be invited to a research interview to discuss their research and our requirements. From there, successful candidates would take part in a trial period. Paradigm candidates go through several stages: sending a resume, attending an initial screening interview, a follow-up interview that aims to test job-relevant skills, a one week trial primarily checking team fit and finally an extended trial testing fit for the role. The process I went through was more like a less structured version of the Paradigm process. This is because while I’ll be working at Leverage, I won’t be a researcher so the Paradigm recruitment process made more sense in my particular case.

Question iii) “growth-rates are overestimated”
On question iii, am I right in thinking you’re referring to the claim in the original post that
“The leadership of Leverage Research have on multiple occasions overstated their rate of staff growth by more than double, in personal conversation”
If so, it’s hard to comment on what happened here without a lot more context on the conversation. I asked Geoff about this and his guesses were that this could have been confusion about hiring and attrition (was the conversation about hiring rates or about the total number of staff), numbers of volunteers or internal estimates for growth rate which turned out to be harder than we thought.

Question v) “negative rumours are still spread about other organizations that might compete for similar talent?”
Intentionally spreading rumours is certainly not Leverage policy nor endorsed or encouraged among staff. Of course, members of staff will have their own views on how best to improve the world and therefore I would expect them to have views on a number of EA and other organisations, both positive and negative.
I have the same difficulty in giving a useful answer to this question as with question iii, in that the original poster has made a claim about Leverage with a strong connotation but without providing the specifics or reasons they believe this.
I think it’s important for individuals to be able to share and discuss their views. I think it would be more helpful in cases like this forum post and when discussing other organisations people try to qualify what they think, why and where that information is from so that recipients of that information come away with a better understanding. This can be difficult but I think it’s important to try.

Question c) “Has it truly gone to such efforts to conceal its activities as described under the general transparency section?”
Finally on question c, yes our past website was removed from the Wayback machine. With hindsight this was a mistake. At the time people were digging up our old content (for example our old long term plan document) and using this to hype up various Leverage conspiracy theories. While this was entertaining, it was certainly distracting and meant people were even more confused about what we were doing. We thought that if we took the content off the wayback it would reduce the conspiracy style hype but removing it only fuelled the fire of conspiracies.
As you can probably tell, Leverage did not invest very many skill points or chips in public communication in the early days, instead spending them pretty much exclusively on research and experimentation. This means we’ve still got a lot to learn in the public communication space and a lot of mistakes to make up for.
As discussed above, Leverage did not use different organisations as a way to tailor messages to donors, different organisations had different focuses and donors understood this. Similarly we have no plans to replace Leverage as a brand.

2. MAKING OUR INTERACTIONS WITH ONE ANOTHER MORE PRODUCTIVE
I’d like to ensure that any future interactions between you and Leverage are a good use of both of our time, informative to readers, and helpful to you on whatever the real issues are.
Looking at your posting history, it seems that you’ve created this anonymous account only to ask questions about Leverage that come across at least as excuses to make insinuations against them. I don’t currently get the impression that you are following the spirit of the EA Forum guidelines and posting with a scout mindset, trying to give people an accurate view or being clear about what you believe and why.
If you would be willing to spend time trying to help me understand and address your particular concerns about Leverage then I am very happy to spend time on that. If you only plan to continue in the same vein as your previous comments I don’t currently expect that to yield anything that feels useful for either of us. If you would like to chat more to figure out a better way forward that gets your concerns addressed let me know. You can reach me here or at larissa.e.rowe [at] gmail.com.

LarissaHeskethRowe 20 Sep 2019 5:53 UTC
12 points
0 ∶ 0
in reply to: Raemon’s comment on: Leverage Research: reviewing the basic facts
Thanks Raemon :-) I’m glad it was helpful.

LarissaHeskethRowe

Up­dates from Lev­er­age Re­search: his­tory, mis­takes and new focus

Updates from Leverage Research: history, mistakes and new focus