I don’t have a fully-formed gestalt take yet, other than: thanks for writing this.
I do want to focus on 3.2.2 Communication about our work (it’s a very Larissa thing to do to have 3 layers nesting of headers 🙂). You explain why you didn’t prioritize public communication, but not why you restricted access to existing work. Scrubbing yourself from archive.org seems to be an action taken not from desire to save time communicating, but from a desire to avoid others learning. It seems like that’s a pretty big factor that’s going on here and would be worth mentioning.
Scrubbing yourself from archive.org seems to be an action taken not from desire to save time communicating, but from a desire to avoid others learning. It seems like that’s a pretty big factor that’s going on here and would be worth mentioning.
This is especially jarring alongside the subsequent recommendation (3.2.5) that one should withhold judgement on whether ‘Leverage 1.0’ was a good use of resources given (inter alia) the difficulties of assessing unpublished research.
Happily, given the laudable intention of Leverage to present their work going forward (including, one presumes, developments of efforts under ‘Leverage 1.0’), folks who weren’t around at the start of the decade will be able to do this—and those of us who were should find the ~2012 evidence base superseded.
When I wrote this comment, I also wrote the following.
I’ve noticed a difference between a few types of apologies: i) I regret that I took the action and think that I should not have, ii) I think I made the right call with the information I had at the time, but it’s turned out bad and I regret that, and iii) I think I made the right call, but I regret the necessary negative consequences. It seems to me you’re claiming iii, which makes it weird to be under mistakes.
I now think you maybe did mean it as i or ii? Specifically
While often this was the right trade-off
Implies that sometimes it was the right call and sometimes it wasn’t. This is pretty nit-pick-y but if you agree it’s not type iii, maybe you could change it to
While often this was the right trade-off for Leverage 1.0 where the focus was advancing our ideas, sometimes it wasn’t and in either case this makes the job of communicating our work moving forward challenging.
Yeah this makes sense, thanks for asking for clarification. The communication section is meant to be a mixture of i) and ii). I think in many cases it was the right decision for Leverage not to prioritise publishing a lot of their research where doing so wouldn’t have been particularly useful. However we think it was a mistake to do some public communication and then remove it, and not to figure out how to communicate about more of our work.
I’m not sure what the best post etiquette is here, should I just edit the post to put in your suggestion and note that the post was edited based on comments?
Thanks for the clarification and tolerating the nitpick. I don’t know that anyone has an etiquette book for this, but I’d put a footnote with the update.[1]
[1] In the fullness of time we’ll have built in footnotes in our rich-text editor, but for now you can do hacky footnotes like this.
(Haha, I did wonder about having so many headings, but it just felt so organised that way, you know 😉)
With regards to removing content we published online, I think we hit the obvious failure mode I expect a lot of new researchers and writers run into, which was that we underestimated how time-consuming, but also stressful, posting publicly and then replying to all the questions can be. To be honest, I kind of suspect early and unexpected negative experiences with public engagement led Leverage to be overly sceptical of it being useful and nudged them away from prioritising communicating their ideas.
From what I understand, some of the key things we ended up removing were:
1) content on Connection Theory (CT)
2) a long-term plan document
3) a version of our website that was very focused on “world-saving.”
With the CT content, I don’t think we made sufficiently clear that we thought of CT as a Kuhnian paradigm worth investigating rather than a fully-fledged, literally true-about-the-world claim.
Speaking to Geoff, it sounds like he assumed people would naturally be thinking in terms of paradigms for this kind of research, often discussed CT under that assumption and then was surprised when people mistook claims about CT to be literal truth claims. To clarify, members of Leverage 1.0 typically didn’t think about CT as being literally true as stated, and the same is true of today’s Leverage 2.0 staff. I can understand why people got this impression from some of their earlier writing though.
This confusion meant people critiqued CT as having insufficient evidence to believe it upfront (which we agree with). While the critiques were understandable, it wasn’t a reason to believe that the research path wasn’t worth following, and we struggled to get people to engage with CT as a potential paradigm. I think the cause of the disagreement wasn’t as clear to us at the time, which made our approach challenging to convey and discuss.
With the long-term planning documents, people misinterpreted the materials in ways that we didn’t expect and hadn’t intended (e.g. as being a claim about what we’d already achieved or as a sign that we were intending something sinister). It seems as though people read the plan as a series of predictions about the future and fixed steps that we were confident we would achieve alone. Instead, we were thinking of it as a way to orient on the scale of the problems we were trying to tackle. We think it’s worth trying to think through very long-term goals you have to see the assumptions that are baked in into your current thinking and world model. We expect solving any problems on a large scale to take a great deal of coordinated effort and plans to change a lot as you learn more.
We also found that a) these kinds of things got a lot more focus than any of our other work which distorted people’s perceptions of what we were doing and b) people would frequently find old versions online and then react badly to them (e.g. becoming upset, confused or concerned) in ways we found difficult to manage.
In the end, I think Leverage concluded the easiest way to solve this was just to remove everything. I think this was a mistake (especially as it only intensified everyone’s curiosity) and it would have been better to post something explaining this problem at the time, but I can see why it might have seemed like just removing the content would solve the problem.
I don’t have a fully-formed gestalt take yet, other than: thanks for writing this.
I do want to focus on 3.2.2 Communication about our work (it’s a very Larissa thing to do to have 3 layers nesting of headers 🙂). You explain why you didn’t prioritize public communication, but not why you restricted access to existing work. Scrubbing yourself from archive.org seems to be an action taken not from desire to save time communicating, but from a desire to avoid others learning. It seems like that’s a pretty big factor that’s going on here and would be worth mentioning.
[Speaking for myself, not my employer.]
This is especially jarring alongside the subsequent recommendation (3.2.5) that one should withhold judgement on whether ‘Leverage 1.0’ was a good use of resources given (inter alia) the difficulties of assessing unpublished research.
Happily, given the laudable intention of Leverage to present their work going forward (including, one presumes, developments of efforts under ‘Leverage 1.0’), folks who weren’t around at the start of the decade will be able to do this—and those of us who were should find the ~2012 evidence base superseded.
For reference, a version & commentary of some Leverage 1.0 research:
https://rationalconspiracy.com/2014/04/22/the-problem-with-connection-theory/ (a)
When I wrote this comment, I also wrote the following.
I now think you maybe did mean it as i or ii? Specifically
Implies that sometimes it was the right call and sometimes it wasn’t. This is pretty nit-pick-y but if you agree it’s not type iii, maybe you could change it to
Yeah this makes sense, thanks for asking for clarification. The communication section is meant to be a mixture of i) and ii). I think in many cases it was the right decision for Leverage not to prioritise publishing a lot of their research where doing so wouldn’t have been particularly useful. However we think it was a mistake to do some public communication and then remove it, and not to figure out how to communicate about more of our work.
I’m not sure what the best post etiquette is here, should I just edit the post to put in your suggestion and note that the post was edited based on comments?
Thanks for the clarification and tolerating the nitpick. I don’t know that anyone has an etiquette book for this, but I’d put a footnote with the update.[1]
[1] In the fullness of time we’ll have built in footnotes in our rich-text editor, but for now you can do hacky footnotes like this.
Perfect, thank you. I’ve edited it and added a footnote.
Hi JP,
(Haha, I did wonder about having so many headings, but it just felt so organised that way, you know 😉)
With regards to removing content we published online, I think we hit the obvious failure mode I expect a lot of new researchers and writers run into, which was that we underestimated how time-consuming, but also stressful, posting publicly and then replying to all the questions can be. To be honest, I kind of suspect early and unexpected negative experiences with public engagement led Leverage to be overly sceptical of it being useful and nudged them away from prioritising communicating their ideas.
From what I understand, some of the key things we ended up removing were:
1) content on Connection Theory (CT)
2) a long-term plan document
3) a version of our website that was very focused on “world-saving.”
With the CT content, I don’t think we made sufficiently clear that we thought of CT as a Kuhnian paradigm worth investigating rather than a fully-fledged, literally true-about-the-world claim.
Speaking to Geoff, it sounds like he assumed people would naturally be thinking in terms of paradigms for this kind of research, often discussed CT under that assumption and then was surprised when people mistook claims about CT to be literal truth claims. To clarify, members of Leverage 1.0 typically didn’t think about CT as being literally true as stated, and the same is true of today’s Leverage 2.0 staff. I can understand why people got this impression from some of their earlier writing though.
This confusion meant people critiqued CT as having insufficient evidence to believe it upfront (which we agree with). While the critiques were understandable, it wasn’t a reason to believe that the research path wasn’t worth following, and we struggled to get people to engage with CT as a potential paradigm. I think the cause of the disagreement wasn’t as clear to us at the time, which made our approach challenging to convey and discuss.
With the long-term planning documents, people misinterpreted the materials in ways that we didn’t expect and hadn’t intended (e.g. as being a claim about what we’d already achieved or as a sign that we were intending something sinister). It seems as though people read the plan as a series of predictions about the future and fixed steps that we were confident we would achieve alone. Instead, we were thinking of it as a way to orient on the scale of the problems we were trying to tackle. We think it’s worth trying to think through very long-term goals you have to see the assumptions that are baked in into your current thinking and world model. We expect solving any problems on a large scale to take a great deal of coordinated effort and plans to change a lot as you learn more.
We also found that a) these kinds of things got a lot more focus than any of our other work which distorted people’s perceptions of what we were doing and b) people would frequently find old versions online and then react badly to them (e.g. becoming upset, confused or concerned) in ways we found difficult to manage.
In the end, I think Leverage concluded the easiest way to solve this was just to remove everything. I think this was a mistake (especially as it only intensified everyone’s curiosity) and it would have been better to post something explaining this problem at the time, but I can see why it might have seemed like just removing the content would solve the problem.
(totally unrelated to the actual post but how did you include an emoticon JP?)
⌘-^-Space, gets you emoji and unicode on any text field on a Mac. I assume other operating systems have their own versions.
Thanks JP and Edoarad! 😄
Winky-. on Windows (That’s the windows key + dot) 😊