I… feel pretty conflicted and hesitant about this. Overall, I have a sense that debate wasn’t a healthy sport for me, and is unlikely to be healthy for others. And I guess I don’t really want us to end up adopting much of its culture, or use the techniques used in competitive debating to convince people. So I am not sure whether I think this is a good idea.
As long as we are all just treating debate as a game that has as much relationship to figuring out the truth as playing soccer, I am of course OK with that, but I am pretty concerned that it’s actually pretty hard to maintain that relationship to the sport at an event like this.
I used to participate in debate in high school and think it taught me a lot of really bad epistemic habits that took quite a while to unlearn, and most of the American debate tradition seems even more broken than what I was used to in Germany.
I think it’s hard to understate how far away the common practices of competitive debate are from actually doing anything that helps you better understand how the world works, or has much to do with truth-seeking. And at the same time, my experience of the debate community is that they do think that they are learning valuable skills that help them better navigate the world and communicate to each other.
Overall, my sense is that it’s very hard to organize an event like this and actually have the narrative of “To be clear, please don’t ever actually talk to other people in EA the way you talk to other people on stage here. I would consider that quite rude and probably actively harmful, and also, I really don’t think the sport we are practicing here is helping you understand the world better, please don’t take this seriously”.
Like, I would be really surprised if anything like that was said at the opening talk of this event, but like, I do think that is the actually right attitude to have towards debate if you don’t want it to hurt you. I’ve talked to a bunch of people in the EA community who have a debating background, and all of the ones I’ve talked to thought that overall the habits they learned were probably bad, and the whole process really didn’t have much to do with truth-seeking.
For people who are unfamiliar with the degree to which various competitive debating practices have created to me quite horrifying abominations, here is an example of a very high-level competitive debate:
I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much. I think overall, the sport really doesn’t have much to do with even just real and normal political debate, which is already a bad thing to imitate.
This again, doesn’t make me totally confident that the above is a bad project, but I do sure feel like I would warn people against attending events like this, and am pretty worried about adopting more of the competitive debating culture with things like this, and also don’t super think that given how far the sport has deteriorated, that skill in it is really predictive at all in truth-seeking ability, after you control for g (and my guess is negatively correlated after you control for g, though I am less confident of that).
I think these concerns are all pretty reasonable, but also strongly discordant with my personal experience, so I figured it would help third parties if I explained the key insights/skills I think I learned or were strongly reinforced by my debating experience.
Three notable caveats on that experience:
I spent more time judging debates than I did speaking in them, which is moderately unusual. It’s plausible to me that judging was much more useful.
It was 8-12 years ago, and my independent impression is that the top levels of the sport have degenerated somewhat since (e.g. I watched world-class debaters speak and while they spoke fast, I’ve never seen anything like the link Oli posted).
I approached debating with a mindset of ‘this is an area I am naturally weak in and want to get better at’, so it was always more likely to complement my natural quantitative approach to figuring things out, rather than replacing it.
(Edit: Since some other discussions on this thread are talking about various formats, I should also add that my experience is entirely inside British Parliamentary debate.)
All in all, I think it’s very plausible Oli’s experience was closer to a typical 2021 experience than mine. But mostly I’m just not sure, for one thing I’d bet that the ‘cram as many points in as possible’ strategy is still much less prevalent at lower levels.
With that out of the way, here are things I picked up that I think are important and useful for truth-tracking, as opposed to persuasion.
Actually listening to the arguments that have been made, in a way that means I could repeat them back with at-least-comparable eloquence to the speaker. Put another way, I think debating made me much better at ideological Turing tests.
A healthy skepticism of the power of arguments and inner-sense-of-conviction as a truth-tracking device, particularly whenever you are talking to someone smarter and more charismatic than yourself, or whenever you’ve just done something like give a speech (or write a blog post/comment!) in favour of a particular conclusion, or whenever you are surrounded by a group of people who all think the same way. This is very closely related to Epistemic Learned Helplessness. It seems like Scott realised this by reading pseudohistory books, see below quote, but my parallel ‘oh shit’ moment was being thoroughly out-argued and convinced by much better debaters in favour of A, and then being equally out-argued by debaters in favour of not-A. Unlike Scott’s experience, I think those people could argue circles around me on virtually every topic. Which just makes it even more obvious you need a better approach.
Being able to generate (some) strong arguments against things I strongly believe and being able to do it independently. It’s pretty common for novice debaters who are highly committed socialists to be unable to come up with any arguments for free markets, or vice-versa. I often see similar patterns, including on that exact issue but also on many other issues, within EA groups. I think getting better at this is critical if we want to do more policy work. Closely related: Policy debates should not appear one-sided. I’m also reminded of Haidt’s work on moral foundations and how liberals tend to ignore some of the foundations.
Identifying critical disagreements, areas that if they resolved one way would likely result in a win for one side, and if they resolved the other way would win for the other side. These are very close to, though not quite the same as, CFAR’s concept of a double crux.
To state the hopefully-obvious, I doubt debating is the optimal way to learn any of this. If I was talking to an EA without debating experience who really wanted to pick up the things I picked up, I’d advise them to read and reflect on the above links, and probably a few other related links I didn’t think of, rather than getting involved in competitive debating, partly for reasons Oli gives and partly for time reasons. I did it primarily because it was fun and the fact it happened to be (imo) useful was a bonus, not unlike the reasons I played Chess or strategy games. That and the fact that half those posts didn’t even exist back in 2009.
At the same time, if I want to learn things from a conversation with someone that I disagree with, and all I know is that I have the choice between talking to someone with or without debating experience, I’m going with the first person. Past experience has taught me that the conversation is likely to be more efficient, more focused on cruxes and falsifiable beliefs, and thus less frustrating.
And there are people who can argue circles around me. Maybe not on every topic, but on topics where they are experts and have spent their whole lives honing their arguments. When I was young I used to read pseudohistory books; Immanuel Velikovsky’s Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect, that I could barely bring myself to bother to search out rebuttals.
And then I read the rebuttals, and they were so obviously correct, so devastating, that I couldn’t believe I had ever been so dumb as to believe Velikovsky.
And then I read the rebuttals to the rebuttals, and they were so obviously correct that I felt silly for ever doubting.
And so on for several more iterations, until the labyrinth of doubt seemed inescapable...
I don’t have time to respond in super much depth because of a bunch of competing commitments but I want to say that all of these are good points and I appreciate you making them.
You did give some responses elsewhere, so a few thoughts on your responses:
But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats.
Here’s why I think I know the opposite: the standard in British Parliamentary judging is to judge based on the ‘Ordinary Intelligent Voter’, defined as follows:
In particular, judges are asked to conceive of themselves as if they were a hypothetical ‘ordinary intelligent voter’ (sometimes also termed ‘average reasonable person’ or ‘informed global citizen’). This hypothetical ordinary intelligent voter doesn’t have pre-formed views on the topic of the debate and isn’t convinced by sophistry, deception or logical fallacies. They are well informed about political and social affairs but lack specialist knowledge. They are open-minded and concerned to decide how to vote – they are thus willing to be convinced by the debaters who provide the most compelling case for or against a certain policy. They are intelligent to the point of being able to understand and assess contrasting arguments (including sophisticated arguments), that are presented to them; but they keep themselves constrained to the material presented unless it patently contradicts common knowledge or is otherwise wildly implausible.
This definition is basically designed to be hard to Goodhart. It’s still easy for judging cultures to take effect and either reward or fail to punish unhelplful behaviour, and personally I would list ‘speaking too fast’ under this, but nothing in that definition is likely to lead to people ‘debating the institution of debating’. So unsurprisingly, I saw vanishingly little of this. Scanning down recent WUDC finals, the only one where the speakers appear to come close to doing this is the one where the motion itself is “This house believes that University Debating has done more harm than good”. Correspondingly, I see no cases where they end up ‘not debating the topic at all’.
The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.
I mean, I’m sorry you had terrible judges or a terrible format I guess? I judged more high school debates than virtually anyone during my time at university, and these are not things I would have allowed to fly, because they are not things I consider persuasive to the Ordinary Intelligent Voter; the ‘isn’t convinced by sophistry, deception or logical fallacies’ seems particularly relevant.
On that note, I don’t think it’s a coincidence that a significant fraction of my comments on this forum are about challenging errors of math or logic. My rough impression is that other users often notice something is wrong, but struggle to identify it precisely, and so say nothing. It should be obvious why I’m keen on getting more people who are used to structuring their thoughts in such a way that they can explain the exact perceived error. Such exactness has benefits even when the perception is wrong and the original argument holds, because it’s easier to refute the refutation.
I might be wrong here, but I currently don’t really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.
The Oxbridge debating community at least is pretty far to the right of the EA community, politically speaking. I consider this an important form of cognitive diversity, but YMMV.
***
Overall, I’m left with the distinct impression that you’ve made up your mind on this based on a bad personal experience, and that nothing is likely to change that view. Which does happen sometimes when there isn’t much in the way of empirical data (after all, there’s sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..), and isn’t a bad reasoning process per se, but confidence in such views should necessarily be limited.
To be clear, I think very little of my personal experience played a role in my position on this. Or at least very unlikely in the way you seem to suggest.
A good chunk of my thoughts on this were formed talking to Buck Shlegeris and Evan Hubinger at some point and also a number of other discussions about debating with a bunch of EAs and rationalists. I was actually pretty in favor of debate ~4-5 years ago when I remember first discussing this with people, but changed my mind after a bunch of people gave their perspectives and experiences and I thought more through the broader problem of how to fix it.
I also want to clarify the following
after all, there’s sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..
I didn’t say that. I said “This is also pretty common in other debate formats”. I even explicitly said I am less familiar with BP as a debate format. It seems pretty plausible to me that BP has less of the problem of meta-debate. But I do think evidence of problems like meta-debate in other formats is evidence of BP also having problems, even if I am specifically less familiar with BP.
I even explicitly said I am less familiar with BP as a debate format.
The fact that you are unfamiliar with the format, and yet are making a number of claims about it, is pretty much exactly my issue. Lack of familiarity is an anti-excuse for overconfidence.
The OP is about an event conducted in BP. Any future events will presumably also be conducted in BP. Information about other formats is only relevant to the extent that they provide information about BP.
I can understand not realising how large the differences between formats are initially, and so assuming information from other formats has strong relevance at first, which is why I was sympathetic to your original comment, but a bunchofpeople have pointed this out by now.
I expect substantiated criticisms of BP as a truth-seeking device (of which there are many!) to look more like the stuff that Ben Pace is saying here, and less like the things you are writing. In brief, I think the actual biggest issues are:
15-minute prep makes for a better game but for very evidence-light arguments.
Judges are explicitly not supposed to reward applause lights, but they are human, so sometimes they do.
It’s rarely a good idea to explicitly back down, even on an issue you are clearly losing. Instead you end up making a lot of ‘even if’ statements. I think Scott did a good job of explaining why that’s not ideal in collaborative discussions (search for “I don’t like the “even if” framing.”).
(1) isn’t really a problem on the meta (read: relevant) level, since it’s very obvious; mostly I think this ends up teaching the useful lesson ‘you can prove roughly anything with ungrounded arguments’. (2) and (3) can inculcate actual bad habits, which I would worry about more if EA wasn’t already stuffed full of those habits and if my personal experience didn’t suggest that debaters are pretty good at dropping those habits outside of the debates themselves. Still, I think they are things reasonable people can worry about.
By contrast, criticisms I think mostly don’t make sense:
Goodharting
Anything to the effect of ‘the speakers might end up believing what they are saying’, especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots.
Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I’d suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don’t know what it was you meant to say.
Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I’d suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don’t know what it was you meant to say.
I’ve edited the relevant section. The edit was simply “This is also pretty common in other debate formats (though I don’t know how common in BP in particular)”.
By contrast, criticisms I think mostly don’t make sense:
+ Goodharting + Anything to the effect of ‘the speakers might end up believing what they are saying’, especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots.
Alas, those are indeed my primary concerns. It’s of course totally OK if you are not compelled, but I have no idea how you are so confident to dismiss them. Having talked to multiple people who have participated in high-level debate in multiple formats, those are the criticisms that they level as well, including for formats very similar to BP, and for BP in-particular.
I have now watched multiple videos of BP debate, and I wish I didn’t because my guesses of what it would look like were basically right, and I feel like I wasted two hours of my time watching BP debates because you insisted that for some reason I am not allowed to make claims from very nearby points of evidence, even though as far as I can tell after spending those two hours, most of my concerns are on-point and BP looks just like most other forms of debate I’ve seen.
I knew when I wrote the above comment that BP debates look less immediately goodharted. But after engaging more deeply with it, I would be surprised if it is actually much less goodharted. Of course, 4 debates at 2x speed isn’t really enough to judge the whole category, but given that I’ve watched dozens of other debates in multiple other formats, I feel like I’ve pinpointed the type of debate that BP is pretty well in debate space.
Of course, you can insist that only people intimately familiar with the format participate in this discussion, in which case I of course cannot clear that bar, and neither can almost anyone else on the forum (and this will of course heavily select against anyone who is critical of debate).
Let me take a step back. Look… I feel super frustrated by your comment above. I am trying to contribute a number of points to this discussion that feel important, and you now twice just kind of insinuated that I am unfairly biased or being unreasonable without really backing up your points? Like, your comments have been super stressful, and the associated downvotes have felt pretty bad. I think the arguments I’ve made in my comments are pretty straightforward, I stayed civil, and I don’t think I am being particularly irrational about this topic.
I’ve thought about it for 2-3 dozen hours over the years and at multiple points in the last few years have spent full-time work weeks evaluating whether we should have a debate tradition inside of EA as well, which caused me to think through a lot of the relevant considerations and investigate a substantial number of debate formats. I’ve talked to something like 8 people with extensive debate experience in EA for at least an hour and tried to get a sense of what things worked and what didn’t.
And then you come along and just assert:
Overall, I’m left with the distinct impression that you’ve made up your mind on this based on a bad personal experience, and that nothing is likely to change that view.
And this… just feels really unfair? Indeed, phenomenologically, my debate experiences were great. I didn’t have a random bad experience that somehow soured me towards this whole sport. I was positive on it, and then thought about it for at least a dozen hours in total and overall came to a complicated high-level position that overall was a lot more hesitant. I have separately also thought for at least a literal 1000 by-the-clock-hours about our talent funnels and the epistemic norms I want the community to have and how the two interact.
My position also isn’t categorically opposed to debate at all. Indeed, I am personally likely to cause EA and the Rationality community to have more of a debating institution internally, and continue to feel conflicted about this project. I think it’s quite plausible it’s good, but I would want the organizers to think hard about how to avoid the problems that seem pretty deeply embedded in debate, and how to avoid them damaging the social institutions of EA, or attract people that might otherwise cause harm.
I don’t know. It’s fine for you to think I am being irrational about this topic, or for some reason to categorically dismiss the kind of concern that I am having, but I don’t feel like you’ve really justified either of those assertions, and I perceived them both coming with some kind of social slap-down motion that made participating in this thread much more stressful than necessary. I will disengage for now. I hope the people involved in this project make good choices.
I just don’t think this is very relevant to whether outreach to debaters is good. A better metric would be to look at life outcomes of top debaters in high school. I don’t have hard statistics on this but the two very successful debaters I know personally are both now researchers at the top of their respective fields, and certainly well above average in truth-seeking.
I also think the above arguments are common tropes in the “maths vs fuzzies” culture war, and given EA’s current dispositions I suspect we’re systematically more likely to hear and be receptive to anti-debate than to pro-debate talking points. (I say this as someone who loved to hate on debate in high school, especially as it was one of the main competitors with math team for recruiting smart students. But with hindsight from seeing my classmates’ life outcomes I think most of the arguments I made were overrated.)
Hi, thank you for your thoughtful response, I appreciate you taking the time to write it. You presented a complicated issue, and I think discussions like this are further complicated by the fact that debating can vary wildly depending on how it’s taught and practiced, so different people can have extremely different experiences (like in the different comments on this thread). Many people already responded to parts, but I’d like to stress a few things that I believe are worth highlighting.
1) The effect of the tournament. The question of the quality of debating as a tool to evaluate the truth is somewhat orthogonal to the question of whether it is worthwhile to engage the debate community with EA-related concepts. While there is value in discussing the former (and I’ll address it later in the comment), the main purpose of the project was the latter. That is to say that rather than convincing EAs to adopt debating as a tool, the aim of the championship was to engage people that already debate with EA. We believe it is a worthwhile effort for all of the reasons stated in the post—e.g. there is potentially a big future gain due to the prospective influence members of the community are going to carry, the project is positively positioned to generate insights for EA advocacy that may be beneficial broadly, etc. It is of course very valid to hold an opinion that there are issues with debating itself, but somewhat like issues with the conducts of tech companies should not prevent us from nudging them towards an EA direction, we think the same applies here. The way we’ve structured the tournament is focused on this—we’ve publicized this tournament to debaters (not non-debating EAs), the process included lots of exposure to EA content, and so on. Therefore, when evaluating the project we primarily took the prism of the value of engaging the debating community.
2) The opportunities of the EA championship. There is truth to the claim that when debating is taught poorly it can lead to suboptimal habits. However, we think these issues are not inherent to the format. Specifically, in this project we assembled a team with a specialty in global debate education that had successfully organized hundreds of international events, built the complimentary lecture series that allowed people to explore EA more deeply before/after the tournament, created additional channels so that people can reflect on content (e.g. a FB group and Discord channels that were used during the tournament), and had a dedicated team of people that were tasked with making sure that the atmosphere in the tournament is positive & inclusive. While it is hard to prove that such an effort was successful, the anecdotal evidence of participants indicating that they wish to learn more on EA, the adoption of donation norms in other tournaments, and the very positive feedback participants provided serve as evidence that (hopefully) the message was conveyed well and that participants were responsive to the educational element of the tournament.
3) Norms of learning. The norms of debating have shifted quite heavily in recent years and the BP-community is very different from the policy-community (the one featured in the video). There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue.
4) On debating itself. I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion. However, there are multiple benefits to the activity that, in my opinion, make debating a useful tool. To name a few:
a) To debate well one needs to carefully listen to the arguments the other side is making, be able make rapid yet thoughtful responses, build arguments and understand how they relate to each other, predict critical points of disagreements and update them dynamically as the discussion progresses, understand how to work well in a team, etc. There are of course other ways to pick up these skills, but debate provides a useful way to practice them. Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.
b) Debating incentivizes you to learn more about the world. Even if winning is one’s key motivation, being competitive requires a lot of preparation between tournaments. This usually involves researching ideas, discussing thoughts with peers, reflecting on previous rounds, etc. From my experience, the general culture is one where debaters are nudged towards challenging their current perceptions and to enrich their world-views.
c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.
d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.
The bottom line is that I agree that the tool is not perfect. However, I think that since the competition primarily introduces EA to debaters rather than promotes debating as a tool, that should be the main consideration in prioritizing this project. I also tried to note in the above why I think debating as a whole, and these kinds of competitions in particular, are potentially beneficial.
However, we think these issues are not inherent to the format.
I do basically think the problems are kind of inherent to the format and pretty hard to fix. Like, I don’t think it’s physically impossible to fix these issues, but I am very skeptical of any efforts that try to fix them that stay within the existing debate context.
Overall, I am not sure where the key causes of our disagreement lies. The above didn’t really feel like it addresses my core concerns with the sport, and while the list of benefits is nice, it feels like a list that I can basically construct for an arbitrary sport (like, not this specific list of debates, but something of equal benefit), and below I give some pointers why I think some of them don’t hold, or at least don’t hold with the forcefulness that one might expect based on your descriptions.
I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion.
To be clear, neither the complexity of the topics or the limited time are at all anywhere close to the central reason why I think debate isn’t very truth-seeking. I can totally have a meeting with a bunch of friends of mine about a complicated issue with only an hour of time, and we can easily make good approximations and solid progress on understanding it. I am quite confident we would not if we instead spent that time in any competitive debating context. Indeed, it seems likely to me that we would leave the competitive debating context with worse beliefs than we entered it on the relevant topic.
There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue.
None of these (inclusion, extracurricular education or “exploring complexity is a virtue) have really much to do with my concerns for debate, so at least from my perspective, describing these as trends that are gaining momentum in the debate community does very little to make me less concerned. Some of these seem mildly bad to me.
Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.
I don’t really think debate is really helping you understand how arguments relate to each other, at least not in a truth-tracking way. In most debate formats, it’s usually much less about actually making good arguments, but much more about abusing the way judges are told to score various arguments, in a way that has very little to do with the cognitive patterns I would encourage someone to use if they were trying to figure out whether an argument makes sense or not.
c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.
I think this is actually useful, and learning the skill of generating steelmanned-arguments for positions you don’t believe is quite useful. Though because of the problem I pointed out above with the arguments that you are generating having very little to do with actual truth-trackingness, this benefit does fall quite a bit short from the ideal you describe here.
In my experience it creates a kind of “fallacy-of-grey”-like mindset where you are avoiding having any beliefs on these issues at all, or don’t really think of it being your job to actually decide which side is right, which I think is quite bad. Ultimately the goal of understanding both sides of an argument is to still judge which side is right (or of course to do a more complicated synthesis between the two, though that’s I think pretty actively discouraged in the debating format).
d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.
I don’t really believe this? The debating community is overall really insular and narrow, as far as I can tell, being really heavily selected for being full of all the standard ivy-league people that we already have a ton of. I like cognitive diversity, but I don’t really think the debating community is very exciting from that perspective. Indeed it seems to have very similar selection filters to the way the EA community is already filtered for. I might be wrong here, but I currently don’t really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.
Following up on this, as part of me trying to understand the format of BP more, I was watching this video, which is the most watched WUDC video on Youtube. And… I find it terrifying. I find it in some sense more terrifying than the video where everyone talks super fast:
I encourage other people who are trying to evaluate debate as a method for truth-seeking to watch this themselves.
There is no super-fast-talking here, but all the arguments in the opening speech are terrible rhetorical argument. The speaker leverages the laughs and engagement of the audience to dismiss the position of his opponents, and this overall really felt more terrifying to me than many of the big political speeches I’ve seen this year.
Like, I… think I am more terrified of the effect this would have on epistemics than the effect of the super-fast-talkers in policy debate? Like, at least in policy-debate it’s somewhat obvious you are playing a game. In the above, I wouldn’t be surprised if the participants actually come to believe the position they are trying to defend.
I was surprised, this video was much less goodharted than I expected (after having been primed with the super-fast talking example). I was expecting more insane things.
Though overall it had the level of much broad public debate/discourse I’ve seen. I watched the first three speakers, and didn’t learn anything. In good debates I’ve seen I’ve felt that I’ve learned something from the debaters about their fields and their unique world views, these felt like two opposing sides in a broader political debate with kind of no grounding in reality. They were optimized for short-scale (e.g. <30 seconds) applause lights for the audience, when objected they’d make it a fight saying things like “Don‘t even try to win that example”, their examples seemed false yet rewarded (primarily attributing China’s rise out of poverty in the last 50 years to ‘redistribution’ and getting applause for it, which, correct me if I’m wrong, is not at all the primary reason, they had massive growth in industry in part by copying a lot of the west). I wouldn’t expect to learn anything, it just seemed like nobody understood economics and they were indexed off what was like 0-1 inferential steps from what the audience as a whole understood. I guess that was the worst part, how can you discuss interesting ideas if they have to be obvious to an audience that big and generic within 10-20 seconds?
People shared so many bad experiences with debate…
I had a great time debating (BP style) in Russia a few years ago. I clearly remember some moments which helped me to become better at thinking/speaking and world modeling:
The initial feedback I got during the practice session is basically don’t be a guy from the terrible video you shared :-). Make it easy for a judge to understand your arguments: improve the structure and speak slower. Focus on one core argument during your speech: don’t squeeze multiple half-baked ideas in; deliver one but prove it fully.
At my first tournament for newbies, an experienced debater gave a lecture on playing something-something resolutions and concluded with strongly recommending reading up on game theory (IIRC, The Strategy of Conflict and Governing the Commons).
My second tournament was in Jedi format: I, an inexperienced Padawan, played with a skilled Jedi. I matched with a person because both of us liked LessWrong. I think we even managed to use “belief should pay rent” as part of an argument in a debate on the tyranny of the majority. I think it’s plausible that we referred to Moloch at least once.
Later on, improvement came from managing inferential distances during speeches; and grounding arguments in reality: being specific about harms and benefits, delivering appropriate ~examples to support intermediate claims.
I think the experience was worth it. It helped me to think more in-depth and about much more issues than I would have overwise (kinda like forecasting now). I quit because (a) tournaments are time-consuming; (b) I got bored playing social issues & identity politics.
While competitive debating is not about collaborative truth-seeking, in my experience, debtors are high cognitive decouplers. Arguing with them (outside of the game) felt good, and we were able to touch topics far outside of the default Overtone window (like taking the perspective of ISIS).
The culture was healthy because most people were just passionate about debating/grokking complex issues (like investor-state dispute settlements), and their incentives were not screwed up because the only upside to winning debate tournaments in Russia is internet points.
Upd: I feel that one of your main concerns is Goodharting. I think the BP system as we played it basically encouraged maximizing the expected utility of impacts of arguments you brought to the table i.e. harm/benefit to individual × scale × probability occurring × how well you proved it (which can be seen as the probability that your reasoning is correct). It’s a bit harder to fit the importance of framing the issue and principled arguments into my simplification. But the first can be seen as prioritizing based on relative tractability (e.g. in almost all of the debate arguing that “we will save money by not implementing a policy” is a bad move because there are multiple other ways to save money and the benefits of the policy might be unique). The second is about the importance of metagame, incentive structures, commitments, and so on.
I did competitive college debate for four years (American Parliamentary format, which is similar to the BP format used in the EA Debate Championship but not identical) and I think that the extent to which it does/doesn’t encourage truth-seeking is less important than the way it pushes people to justify their values.
Oversimplifying broadly, debate has two layers: one is the arguments about what the impacts of a certain idea/policy are likely to be, and one is arguments about which impacts are more important (known as “weighing”). In order to win rounds, you have to win arguments at both levels. This means that debate requires people to engage with one of issues that is most central to EA — a relatively consequentialist understanding of which issues matter most. In regular life you can say, “I support government funding for the arts because art is good” and not think very hard about how that trades off with, say, funding for healthcare. But if you do that in a debate round, the other team will point out the tradeoff, estimate the number of people who will die as a result of there being less funding for healthcare, and you will lose the round.
I think this is the main benefit of debate from an EA perspective, and I suspect that it has meaningful impacts on people who are forced to confront, over and over again in countless debate rounds, the actual effects (in lives lost and other very serious harms) of different ways of weighing between issues. Anecdotally, a higher-than-average percentage of the debaters I know are EAers, or at least interested in EA. And even debaters who don’t personally support EA very often use EA weighing arguments in rounds. As a result, for some people (I suspect many), debate is the first place they hear about EA. To me, this makes debate leagues a fertile recruiting ground for EA.
...that video is deeply absurd, and I am caught between revulsion and laughter. They are clearly way past the point where there is even a gesture at real usefulness; this might be the most heavily gamed thing I have ever seen.
However, if the video is an example I am confident that this activity can’t do any real harm to our objectives. Who could do this with even the pretense of thinking they would talk to people in the real world that way? It’s just a verbal luge.
Yeah, it’s kinda hilarious. Speaking so fast that your opponents can’t follow your arguments and therefore lose the round is common practice in some forms of competitive debate. But in other debate categories, using this tactic would immediately lose you the round. In my own personal experience of high school debate, the quality of competitive debate depends very heavily on the particular category of debate.
The video above is Policy Debate, the oldest form of debate which degenerated decades ago into unintelligible speed reading and arguments that every policy would result in worldwide nuclear annihilation. In the 1980s, the National Speech and Debate Association instituted a new form of debate called Lincoln Douglas that attempted to reground debate in commonsense questions about moral values; but LD has also fallen victim to speed reading and even galaxy-brained “kritiks” arguing that the structure of debate itself is racist or sexist and therefore that the round should be abandoned.
Public Forum debate, invented in 2002 as an antidote to Lincoln Douglas, is IMO a very healthy and educational form of debate. Here is the final round of the 2018 national championship (starting at 4:05) on the resolution, “On balance, the benefits of United States Participation in the North American Free Trade Agreement outweigh the consequences.” https://m.youtube.com/watch?v=MUnyLbeu7qU&feature=youtu.be
British Parliamentary debate is another form of debate that, in my experience, is more civil and less “game-able” than other forms of debate (though Harrison D disagrees below, with specifics about its pitfalls). One key difference is that, while Public Forum allows and encourages debaters to spend weeks researching and debating a single specific resolution, Parliamentary debate typically a involves generalized preparation on a subject or theme and only reveals the specific resolution a few minutes before the round begins. Because of this, I think Public Forum is more educational for debaters, but Parliamentary is probably easier to run a one-off tournament because debaters won’t be expected to have done as much preparation.
Extemporaneous Speaking is another category involving less preparation, where participants are asked a question about current affairs or politics and have 30 minutes to prepare a 7 minute off-the-cuff speech. There is no “opponent” in Extemp, perhaps limiting the level of discourse, but it might be possible to easily introduce EA-related topics because participants are expected to be conversant in a wide range of topic areas.
On the whole, I’m very glad to see this EA debate tournament being run, and would be very excited to see further work bringing EA topics into debate. I can understand why many people might find some debate tactics toxic and counterproductive, particularly in categories like Policy and LD, but I do think this is the failure of specific categories and tactics and not an indictment of all adversarial debate. Learning the best arguments for both sides of a resolution certainly teaches a bit of an “arguments as soldiers” approach, but I believe the greater effect is to lead debaters to real truths about which arguments are stronger and improve their personal understanding of the issues. In future EA debate events, I would only suggest that organizers be very conscious of these standards and norms when choosing a specific category of debate.
I appreciate this nuanced comparison between formats. I increase my estimation of the goodness of the EA debate project, because this introduces a new pathway to victory:
The creation of an EA debate format, which can be designed against the backdrop of all the poisoned formats to encourage skills and norms more like those we want to develop.
It does feel like the general period for research before the debate would work well for these topics. Do you know if any format has a mechanism for awarding points on the grounds of agreeing with the opponent’s points, or otherwise acknowledging when arguments or data weigh against you?
I think that the culture of British Parliamentary has made it significantly less game-y and more civil than most if not all other formats of collegiate debate, including both prepared formats (e.g., Policy Debate, Lincoln-Douglas, Public Forum) and limited preparation formats (e.g., other forms of parli such as American Parliamentary).
I think that the limited topic prep nature of parliamentary debate makes those formats significantly less game-y and more civil than most other formats of collegiate debate.
My main issue with BP is really just two individual characteristics in the format that represent stark differences from the format I did in high school (American Parli, in the Stoa league): the 4-teams-of-2 (instead of 2v2) combined with the lack of access to published sources on the internet when in prep. Really, most if not all of the main issues I highlighted in my comment relate to the first thing, which I think is more fundamental.
So ultimately, I’m not trying to compare BP to policy debate, nor am I trying to compare it to the actual (culturally-driven) practice of American Parli in collegiate leagues (which I’m not as familiar with), it’s really just me comparing it to what I think an ideal format would be when given a decent culture that isn’t so acceptive of gamification.
It’s unfortunate to hear that you had such a negative experience from debate. As someone who has judged public-school high school policy and public forum debate, I will say that I am not that surprised. That being said, I do take issue with your characterization of all/BP debate with the video plus the statement “I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much.”
I cannot speak to every BP league/competition in the world, but I have never seen nor heard of such drastic gamification of debate in BP—or anything even come close to it—in the four years that I did BP in college. In fact, I have often seen people hold up BP and collegiate policy debate as polar opposites, with BP being one of the least toxic/gamified formats (at least among the major formats) and policy debate being the most. (BP definitely has some problems with left-leaning judge bias, but it could be a lot worse and that’s not really that unique to BP.)
Ultimately, I don’t want to be rude/abrasive, but I feel that the video really gives a deeply misleading picture of BP, even if it is only a mild-moderate exaggeration of collegiate policy debate. I think it’s unfortunate to think that many people who are unfamiliar with debate (let alone BP specifically) may come away with such a misleading picture of BP/debate in general based on this extreme example of a different format. I’m not sure how to say this in a non-confrontational way, but I personally think that some kind of revision/redaction (e.g., a disclaimer saying that the video is not of BP, acknowledgement that BP is different) may be in order.
I will just add the following video to illustrate that the gamifying (e.g., speed and spread tactics) of debate is not so inherent to the sport or even specific formats themselves, but rather are heavily determined by league culture (e.g., what kinds of judges are used, how debaters are taught to debate): https://www.youtube.com/watch?v=TvhNvumnZ1U&t=23s
(Although, do excuse the pathos-heavy story at the beginning, and remember these are just high school students)
A lot of my models here come from talking to Evan Hubinger about this, who has a lot of thoughts on debate and competed at the national level in Policy debate in college.
My guess is that overall debate is really badly goodharted. All of the different variants. One of the ways policy-debate in particular is really badly goodharted is that everyone talks so fast nobody can comprehend them. But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats (though I don’t know how common in BP in particular). Evan won a bunch of his highest-level debates by telling stories about pirates for 15 minutes, and then telling everyone that the institution of debate is so broken and useless and wouldn’t it just be better if we would use this time to learn cool facts about history like the pirate anecdotes I told you in the first 80% of the debate?
If people were just talking fast, I do think that would be a problem, but not a super big one. I can imagine a kind of hyperoptimized court system that functions fine with everyone talking at 2-3x normal speed. The problem is that it’s evidence that the system at large has very little defenses against goodharting and runaway competition effects.
As far as I can tell, some other debate formats do not have the problem of everyone talking at 2-3x speed, but basically have many of the other problems, and importantly, all share the fundamental attribute that they have very few successful defenses against goodhart’s law. And so somehow, even though it might differ from debate format to debate format, the actual thing the competitors are doing has very little to do with seeking the truth.
The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.
I watched about 20 minutes of the video you linked, and I do think just from that slice it seems much less obviously broken than the video I linked, and that does seem important to recognize.
I do also think that I can only really interface with that whole video in a healthy way by repeatedly forcing myself to take a step back and basically throw away all the information that is thrown at me, because I don’t trust it, and I don’t trust the process that produced it. Like, I don’t think I left that video having a better understanding of seatbelt laws (the topic of debate). I learned some useful relevant facts, but I am still worried that I left that debate with worse beliefs about seatbelt laws, and very likely much worse than if I had just read the Wikipedia article on it. Of course, the point of the debate is not for me to learn about seatbelt law, but I do also think the same is probably true for the participants.
“The problem is that it’s evidence that the system at large has very little defenses against goodharting and runaway competition effects.”
Although I acknowledge that there will always be some level of misalignment between truth-seeking and competition, I would push back on the idea that the system has little defense against drastic goodharting like is seen in both high school and collegiate policy debate: the experience of Stoa (the league in which I debated) and NCFCA are evidence of that. In my view and in the view of some others (see e.g., https://www.ethosdebate.com/community-judges-1-necessity-community-judges/), it seems that one of the important front-line defenses against gamification of debate is the use of community judges who recoil at nonsense and speed. Of course, that introduces tradeoffs that debaters (myself included) sometimes huff about, such as biased decisions, but it still seems worth it. Additionally, I feel fairly confident that there are other important factors that explain the stark cultural differences between Stoa/NCFCA and most public-school/collegiate leagues (e.g., the debaters’ personalities/background, parental involvement, the Christian ethos, the observation of and opportunity for self-differentiation from public-school/collegiate practices).
To address your broader point about the truth-seeking vs. competition drive (goodharting): I and many others in my league have considered this question. (For a brief example article from someone I know, see https://www.ethosdebate.com/art-persuasion-vs-pursuit-truth/) I could be wrong/exaggerating, but I get the sense from you that debate should be really strict about promoting truth-seeking above other things—perhaps even to the extent that debate should almost never sacrifice truth-seeking for other goals. Perhaps that is not what you are saying, but regardless, I would push back and emphasize that debate has a wide variety of purposes, crucially including skills education in general (as opposed to topical education). (I actually recently finished a blog series which I started by outlining some of the major purposes of debate: see https://www.ethosdebate.com/purposes-of-debate-pt-1-the-goals-and-anti-goals-of-debate/ ). In short, I think that the experience of Stoa/NCFCA shows that with reasonable safeguards (e.g., including community judges in the judging pool) debate can be at least neutral if not more positive than negative in promoting truth-seeking, while at the same time is a great way to get youth excited about studying topics, scrutinizing their own views, and learning to persuade others.
That last part applies to that NITOC final (regarding seatbelt policy), which focused on a case that was known for being somewhat pathos-heavy (as opposed to, for example, the case for cutting funding for air marshals, which I and many other debaters would likely have never come to see if it were not subject to the adversarial scrutiny of a competitive season of debate): debate shouldn’t be entirely/solely about truth-seeking; teaching persuasion skills is also really important, because if you have the truth but cannot persuade others, then your ability to act on it is sorely limited.
Also: “people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study”—my experience in Stoa was fairly different: I repeatedly had to defend the methodology of some of the studies I relied on, and was able to challenge the methodology of sources.
I was confused about the situation with debate, so I talked to Evan Hubinger about his experiences. That conversation was completely wild; I’m guessing people in this thread might be interested in hearing it. I still don’t know exactly what to make of what happened there, but I think there are some genuine and non-obvious insights relevant to public discourse and optimization processes (maybe less to the specifics of debate outreach). The whole thing’s also pretty funny.
I recorded the conversation; don’t want to share publicly but feel free to DM me for access.
I want to echo this. I think my own experience of debating has been useful to me in terms of my ability to intelligence-signal in person, but was pretty bad overall for my epistemics. One interesting thing about BP (which was the format I competed in most frequently at the highest level) was the importance in the 4th speaker role identifying the cruxes of the debate (usually referring to them as “clash”), which I think is really useful. Concluding that the side you’ve been told to favour has then “won” all of the cruxes is… less so.
+1 from my own experience in debate (also british style). Truth-seeking / identifying and meaningfully resolving points of disagreement between different positions is a very different skill to trying to win a debate, and the skillsets/mindsets developed in the latter seem like they might actively work against the former unless the people doing it are very careful and self-aware.
I did Policy Debate, the format from that video, in high school and college. Policy Debate has its problems, but I think the fast delivery style is good on balance, since it lets a single debate cover so much ground. Speed reading is comprehensible with practice. Here’s an example from college debate with better audio quality—definitely weird, but clear enough to understand.
Sure, debate may involve a lot of techniques that are counterproductive to truth-seeking, and I wouldn’t want people to write on the EA Forum like it’s a debate, for example. However, I think there are many places where it would help to be able to convey more convincing arguments even if being more convincing doesn’t improve truth-seeking—speaking with non-EAs about EA, for example.
I generally want us to use truth-seeking methods when engaging with outsiders as well. Of course, that isn’t always possible, but I also really don’t want us to have a reputation for using lots of rhetorical tricks to convince others (and generally think that doing so is pretty bad).
I… feel pretty conflicted and hesitant about this. Overall, I have a sense that debate wasn’t a healthy sport for me, and is unlikely to be healthy for others. And I guess I don’t really want us to end up adopting much of its culture, or use the techniques used in competitive debating to convince people. So I am not sure whether I think this is a good idea.
As long as we are all just treating debate as a game that has as much relationship to figuring out the truth as playing soccer, I am of course OK with that, but I am pretty concerned that it’s actually pretty hard to maintain that relationship to the sport at an event like this.
I used to participate in debate in high school and think it taught me a lot of really bad epistemic habits that took quite a while to unlearn, and most of the American debate tradition seems even more broken than what I was used to in Germany.
I think it’s hard to understate how far away the common practices of competitive debate are from actually doing anything that helps you better understand how the world works, or has much to do with truth-seeking. And at the same time, my experience of the debate community is that they do think that they are learning valuable skills that help them better navigate the world and communicate to each other.
Overall, my sense is that it’s very hard to organize an event like this and actually have the narrative of “To be clear, please don’t ever actually talk to other people in EA the way you talk to other people on stage here. I would consider that quite rude and probably actively harmful, and also, I really don’t think the sport we are practicing here is helping you understand the world better, please don’t take this seriously”.
Like, I would be really surprised if anything like that was said at the opening talk of this event, but like, I do think that is the actually right attitude to have towards debate if you don’t want it to hurt you. I’ve talked to a bunch of people in the EA community who have a debating background, and all of the ones I’ve talked to thought that overall the habits they learned were probably bad, and the whole process really didn’t have much to do with truth-seeking.
For people who are unfamiliar with the degree to which various competitive debating practices have created to me quite horrifying abominations, here is an example of a very high-level competitive debate:
I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much. I think overall, the sport really doesn’t have much to do with even just real and normal political debate, which is already a bad thing to imitate.
This again, doesn’t make me totally confident that the above is a bad project, but I do sure feel like I would warn people against attending events like this, and am pretty worried about adopting more of the competitive debating culture with things like this, and also don’t super think that given how far the sport has deteriorated, that skill in it is really predictive at all in truth-seeking ability, after you control for g (and my guess is negatively correlated after you control for g, though I am less confident of that).
I think these concerns are all pretty reasonable, but also strongly discordant with my personal experience, so I figured it would help third parties if I explained the key insights/skills I think I learned or were strongly reinforced by my debating experience.
Three notable caveats on that experience:
I spent more time judging debates than I did speaking in them, which is moderately unusual. It’s plausible to me that judging was much more useful.
It was 8-12 years ago, and my independent impression is that the top levels of the sport have degenerated somewhat since (e.g. I watched world-class debaters speak and while they spoke fast, I’ve never seen anything like the link Oli posted).
I approached debating with a mindset of ‘this is an area I am naturally weak in and want to get better at’, so it was always more likely to complement my natural quantitative approach to figuring things out, rather than replacing it.
(Edit: Since some other discussions on this thread are talking about various formats, I should also add that my experience is entirely inside British Parliamentary debate.)
All in all, I think it’s very plausible Oli’s experience was closer to a typical 2021 experience than mine. But mostly I’m just not sure, for one thing I’d bet that the ‘cram as many points in as possible’ strategy is still much less prevalent at lower levels.
With that out of the way, here are things I picked up that I think are important and useful for truth-tracking, as opposed to persuasion.
Actually listening to the arguments that have been made, in a way that means I could repeat them back with at-least-comparable eloquence to the speaker. Put another way, I think debating made me much better at ideological Turing tests.
A healthy skepticism of the power of arguments and inner-sense-of-conviction as a truth-tracking device, particularly whenever you are talking to someone smarter and more charismatic than yourself, or whenever you’ve just done something like give a speech (or write a blog post/comment!) in favour of a particular conclusion, or whenever you are surrounded by a group of people who all think the same way. This is very closely related to Epistemic Learned Helplessness. It seems like Scott realised this by reading pseudohistory books, see below quote, but my parallel ‘oh shit’ moment was being thoroughly out-argued and convinced by much better debaters in favour of A, and then being equally out-argued by debaters in favour of not-A. Unlike Scott’s experience, I think those people could argue circles around me on virtually every topic. Which just makes it even more obvious you need a better approach.
Being able to generate (some) strong arguments against things I strongly believe and being able to do it independently. It’s pretty common for novice debaters who are highly committed socialists to be unable to come up with any arguments for free markets, or vice-versa. I often see similar patterns, including on that exact issue but also on many other issues, within EA groups. I think getting better at this is critical if we want to do more policy work. Closely related: Policy debates should not appear one-sided. I’m also reminded of Haidt’s work on moral foundations and how liberals tend to ignore some of the foundations.
Identifying critical disagreements, areas that if they resolved one way would likely result in a win for one side, and if they resolved the other way would win for the other side. These are very close to, though not quite the same as, CFAR’s concept of a double crux.
To state the hopefully-obvious, I doubt debating is the optimal way to learn any of this. If I was talking to an EA without debating experience who really wanted to pick up the things I picked up, I’d advise them to read and reflect on the above links, and probably a few other related links I didn’t think of, rather than getting involved in competitive debating, partly for reasons Oli gives and partly for time reasons. I did it primarily because it was fun and the fact it happened to be (imo) useful was a bonus, not unlike the reasons I played Chess or strategy games. That and the fact that half those posts didn’t even exist back in 2009.
At the same time, if I want to learn things from a conversation with someone that I disagree with, and all I know is that I have the choice between talking to someone with or without debating experience, I’m going with the first person. Past experience has taught me that the conversation is likely to be more efficient, more focused on cruxes and falsifiable beliefs, and thus less frustrating.
I don’t have time to respond in super much depth because of a bunch of competing commitments but I want to say that all of these are good points and I appreciate you making them.
You did give some responses elsewhere, so a few thoughts on your responses:
(Emphasis added). This seems like a classic case for ‘what do you think you know, and how do you think you know it?’.
Here’s why I think I know the opposite: the standard in British Parliamentary judging is to judge based on the ‘Ordinary Intelligent Voter’, defined as follows:
This definition is basically designed to be hard to Goodhart. It’s still easy for judging cultures to take effect and either reward or fail to punish unhelplful behaviour, and personally I would list ‘speaking too fast’ under this, but nothing in that definition is likely to lead to people ‘debating the institution of debating’. So unsurprisingly, I saw vanishingly little of this. Scanning down recent WUDC finals, the only one where the speakers appear to come close to doing this is the one where the motion itself is “This house believes that University Debating has done more harm than good”. Correspondingly, I see no cases where they end up ‘not debating the topic at all’.
I mean, I’m sorry you had terrible judges or a terrible format I guess? I judged more high school debates than virtually anyone during my time at university, and these are not things I would have allowed to fly, because they are not things I consider persuasive to the Ordinary Intelligent Voter; the ‘isn’t convinced by sophistry, deception or logical fallacies’ seems particularly relevant.
On that note, I don’t think it’s a coincidence that a significant fraction of my comments on this forum are about challenging errors of math or logic. My rough impression is that other users often notice something is wrong, but struggle to identify it precisely, and so say nothing. It should be obvious why I’m keen on getting more people who are used to structuring their thoughts in such a way that they can explain the exact perceived error. Such exactness has benefits even when the perception is wrong and the original argument holds, because it’s easier to refute the refutation.
The Oxbridge debating community at least is pretty far to the right of the EA community, politically speaking. I consider this an important form of cognitive diversity, but YMMV.
***
Overall, I’m left with the distinct impression that you’ve made up your mind on this based on a bad personal experience, and that nothing is likely to change that view. Which does happen sometimes when there isn’t much in the way of empirical data (after all, there’s sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..), and isn’t a bad reasoning process per se, but confidence in such views should necessarily be limited.
To be clear, I think very little of my personal experience played a role in my position on this. Or at least very unlikely in the way you seem to suggest.
A good chunk of my thoughts on this were formed talking to Buck Shlegeris and Evan Hubinger at some point and also a number of other discussions about debating with a bunch of EAs and rationalists. I was actually pretty in favor of debate ~4-5 years ago when I remember first discussing this with people, but changed my mind after a bunch of people gave their perspectives and experiences and I thought more through the broader problem of how to fix it.
I also want to clarify the following
I didn’t say that. I said “This is also pretty common in other debate formats”. I even explicitly said I am less familiar with BP as a debate format. It seems pretty plausible to me that BP has less of the problem of meta-debate. But I do think evidence of problems like meta-debate in other formats is evidence of BP also having problems, even if I am specifically less familiar with BP.
The fact that you are unfamiliar with the format, and yet are making a number of claims about it, is pretty much exactly my issue. Lack of familiarity is an anti-excuse for overconfidence.
The OP is about an event conducted in BP. Any future events will presumably also be conducted in BP. Information about other formats is only relevant to the extent that they provide information about BP.
I can understand not realising how large the differences between formats are initially, and so assuming information from other formats has strong relevance at first, which is why I was sympathetic to your original comment, but a bunch of people have pointed this out by now.
I expect substantiated criticisms of BP as a truth-seeking device (of which there are many!) to look more like the stuff that Ben Pace is saying here, and less like the things you are writing. In brief, I think the actual biggest issues are:
15-minute prep makes for a better game but for very evidence-light arguments.
Judges are explicitly not supposed to reward applause lights, but they are human, so sometimes they do.
It’s rarely a good idea to explicitly back down, even on an issue you are clearly losing. Instead you end up making a lot of ‘even if’ statements. I think Scott did a good job of explaining why that’s not ideal in collaborative discussions (search for “I don’t like the “even if” framing.”).
(1) isn’t really a problem on the meta (read: relevant) level, since it’s very obvious; mostly I think this ends up teaching the useful lesson ‘you can prove roughly anything with ungrounded arguments’. (2) and (3) can inculcate actual bad habits, which I would worry about more if EA wasn’t already stuffed full of those habits and if my personal experience didn’t suggest that debaters are pretty good at dropping those habits outside of the debates themselves. Still, I think they are things reasonable people can worry about.
By contrast, criticisms I think mostly don’t make sense:
Goodharting
Anything to the effect of ‘the speakers might end up believing what they are saying’, especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots.
Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I’d suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don’t know what it was you meant to say.
I’ve edited the relevant section. The edit was simply “This is also pretty common in other debate formats (though I don’t know how common in BP in particular)”.
Alas, those are indeed my primary concerns. It’s of course totally OK if you are not compelled, but I have no idea how you are so confident to dismiss them. Having talked to multiple people who have participated in high-level debate in multiple formats, those are the criticisms that they level as well, including for formats very similar to BP, and for BP in-particular.
I have now watched multiple videos of BP debate, and I wish I didn’t because my guesses of what it would look like were basically right, and I feel like I wasted two hours of my time watching BP debates because you insisted that for some reason I am not allowed to make claims from very nearby points of evidence, even though as far as I can tell after spending those two hours, most of my concerns are on-point and BP looks just like most other forms of debate I’ve seen.
I knew when I wrote the above comment that BP debates look less immediately goodharted. But after engaging more deeply with it, I would be surprised if it is actually much less goodharted. Of course, 4 debates at 2x speed isn’t really enough to judge the whole category, but given that I’ve watched dozens of other debates in multiple other formats, I feel like I’ve pinpointed the type of debate that BP is pretty well in debate space.
Of course, you can insist that only people intimately familiar with the format participate in this discussion, in which case I of course cannot clear that bar, and neither can almost anyone else on the forum (and this will of course heavily select against anyone who is critical of debate).
Let me take a step back. Look… I feel super frustrated by your comment above. I am trying to contribute a number of points to this discussion that feel important, and you now twice just kind of insinuated that I am unfairly biased or being unreasonable without really backing up your points? Like, your comments have been super stressful, and the associated downvotes have felt pretty bad. I think the arguments I’ve made in my comments are pretty straightforward, I stayed civil, and I don’t think I am being particularly irrational about this topic.
I’ve thought about it for 2-3 dozen hours over the years and at multiple points in the last few years have spent full-time work weeks evaluating whether we should have a debate tradition inside of EA as well, which caused me to think through a lot of the relevant considerations and investigate a substantial number of debate formats. I’ve talked to something like 8 people with extensive debate experience in EA for at least an hour and tried to get a sense of what things worked and what didn’t.
And then you come along and just assert:
And this… just feels really unfair? Indeed, phenomenologically, my debate experiences were great. I didn’t have a random bad experience that somehow soured me towards this whole sport. I was positive on it, and then thought about it for at least a dozen hours in total and overall came to a complicated high-level position that overall was a lot more hesitant. I have separately also thought for at least a literal 1000 by-the-clock-hours about our talent funnels and the epistemic norms I want the community to have and how the two interact.
My position also isn’t categorically opposed to debate at all. Indeed, I am personally likely to cause EA and the Rationality community to have more of a debating institution internally, and continue to feel conflicted about this project. I think it’s quite plausible it’s good, but I would want the organizers to think hard about how to avoid the problems that seem pretty deeply embedded in debate, and how to avoid them damaging the social institutions of EA, or attract people that might otherwise cause harm.
I don’t know. It’s fine for you to think I am being irrational about this topic, or for some reason to categorically dismiss the kind of concern that I am having, but I don’t feel like you’ve really justified either of those assertions, and I perceived them both coming with some kind of social slap-down motion that made participating in this thread much more stressful than necessary. I will disengage for now. I hope the people involved in this project make good choices.
I just don’t think this is very relevant to whether outreach to debaters is good. A better metric would be to look at life outcomes of top debaters in high school. I don’t have hard statistics on this but the two very successful debaters I know personally are both now researchers at the top of their respective fields, and certainly well above average in truth-seeking.
I also think the above arguments are common tropes in the “maths vs fuzzies” culture war, and given EA’s current dispositions I suspect we’re systematically more likely to hear and be receptive to anti-debate than to pro-debate talking points. (I say this as someone who loved to hate on debate in high school, especially as it was one of the main competitors with math team for recruiting smart students. But with hindsight from seeing my classmates’ life outcomes I think most of the arguments I made were overrated.)
Hi, thank you for your thoughtful response, I appreciate you taking the time to write it. You presented a complicated issue, and I think discussions like this are further complicated by the fact that debating can vary wildly depending on how it’s taught and practiced, so different people can have extremely different experiences (like in the different comments on this thread). Many people already responded to parts, but I’d like to stress a few things that I believe are worth highlighting.
1) The effect of the tournament. The question of the quality of debating as a tool to evaluate the truth is somewhat orthogonal to the question of whether it is worthwhile to engage the debate community with EA-related concepts. While there is value in discussing the former (and I’ll address it later in the comment), the main purpose of the project was the latter. That is to say that rather than convincing EAs to adopt debating as a tool, the aim of the championship was to engage people that already debate with EA. We believe it is a worthwhile effort for all of the reasons stated in the post—e.g. there is potentially a big future gain due to the prospective influence members of the community are going to carry, the project is positively positioned to generate insights for EA advocacy that may be beneficial broadly, etc. It is of course very valid to hold an opinion that there are issues with debating itself, but somewhat like issues with the conducts of tech companies should not prevent us from nudging them towards an EA direction, we think the same applies here. The way we’ve structured the tournament is focused on this—we’ve publicized this tournament to debaters (not non-debating EAs), the process included lots of exposure to EA content, and so on. Therefore, when evaluating the project we primarily took the prism of the value of engaging the debating community.
2) The opportunities of the EA championship. There is truth to the claim that when debating is taught poorly it can lead to suboptimal habits. However, we think these issues are not inherent to the format. Specifically, in this project we assembled a team with a specialty in global debate education that had successfully organized hundreds of international events, built the complimentary lecture series that allowed people to explore EA more deeply before/after the tournament, created additional channels so that people can reflect on content (e.g. a FB group and Discord channels that were used during the tournament), and had a dedicated team of people that were tasked with making sure that the atmosphere in the tournament is positive & inclusive. While it is hard to prove that such an effort was successful, the anecdotal evidence of participants indicating that they wish to learn more on EA, the adoption of donation norms in other tournaments, and the very positive feedback participants provided serve as evidence that (hopefully) the message was conveyed well and that participants were responsive to the educational element of the tournament.
3) Norms of learning. The norms of debating have shifted quite heavily in recent years and the BP-community is very different from the policy-community (the one featured in the video). There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue.
4) On debating itself. I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion. However, there are multiple benefits to the activity that, in my opinion, make debating a useful tool. To name a few:
a) To debate well one needs to carefully listen to the arguments the other side is making, be able make rapid yet thoughtful responses, build arguments and understand how they relate to each other, predict critical points of disagreements and update them dynamically as the discussion progresses, understand how to work well in a team, etc. There are of course other ways to pick up these skills, but debate provides a useful way to practice them. Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.
b) Debating incentivizes you to learn more about the world. Even if winning is one’s key motivation, being competitive requires a lot of preparation between tournaments. This usually involves researching ideas, discussing thoughts with peers, reflecting on previous rounds, etc. From my experience, the general culture is one where debaters are nudged towards challenging their current perceptions and to enrich their world-views.
c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.
d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.
The bottom line is that I agree that the tool is not perfect. However, I think that since the competition primarily introduces EA to debaters rather than promotes debating as a tool, that should be the main consideration in prioritizing this project. I also tried to note in the above why I think debating as a whole, and these kinds of competitions in particular, are potentially beneficial.
I appreciate the response!
I do basically think the problems are kind of inherent to the format and pretty hard to fix. Like, I don’t think it’s physically impossible to fix these issues, but I am very skeptical of any efforts that try to fix them that stay within the existing debate context.
Overall, I am not sure where the key causes of our disagreement lies. The above didn’t really feel like it addresses my core concerns with the sport, and while the list of benefits is nice, it feels like a list that I can basically construct for an arbitrary sport (like, not this specific list of debates, but something of equal benefit), and below I give some pointers why I think some of them don’t hold, or at least don’t hold with the forcefulness that one might expect based on your descriptions.
To be clear, neither the complexity of the topics or the limited time are at all anywhere close to the central reason why I think debate isn’t very truth-seeking. I can totally have a meeting with a bunch of friends of mine about a complicated issue with only an hour of time, and we can easily make good approximations and solid progress on understanding it. I am quite confident we would not if we instead spent that time in any competitive debating context. Indeed, it seems likely to me that we would leave the competitive debating context with worse beliefs than we entered it on the relevant topic.
None of these (inclusion, extracurricular education or “exploring complexity is a virtue) have really much to do with my concerns for debate, so at least from my perspective, describing these as trends that are gaining momentum in the debate community does very little to make me less concerned. Some of these seem mildly bad to me.
I don’t really think debate is really helping you understand how arguments relate to each other, at least not in a truth-tracking way. In most debate formats, it’s usually much less about actually making good arguments, but much more about abusing the way judges are told to score various arguments, in a way that has very little to do with the cognitive patterns I would encourage someone to use if they were trying to figure out whether an argument makes sense or not.
I think this is actually useful, and learning the skill of generating steelmanned-arguments for positions you don’t believe is quite useful. Though because of the problem I pointed out above with the arguments that you are generating having very little to do with actual truth-trackingness, this benefit does fall quite a bit short from the ideal you describe here.
In my experience it creates a kind of “fallacy-of-grey”-like mindset where you are avoiding having any beliefs on these issues at all, or don’t really think of it being your job to actually decide which side is right, which I think is quite bad. Ultimately the goal of understanding both sides of an argument is to still judge which side is right (or of course to do a more complicated synthesis between the two, though that’s I think pretty actively discouraged in the debating format).
I don’t really believe this? The debating community is overall really insular and narrow, as far as I can tell, being really heavily selected for being full of all the standard ivy-league people that we already have a ton of. I like cognitive diversity, but I don’t really think the debating community is very exciting from that perspective. Indeed it seems to have very similar selection filters to the way the EA community is already filtered for. I might be wrong here, but I currently don’t really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.
Following up on this, as part of me trying to understand the format of BP more, I was watching this video, which is the most watched WUDC video on Youtube. And… I find it terrifying. I find it in some sense more terrifying than the video where everyone talks super fast:
I encourage other people who are trying to evaluate debate as a method for truth-seeking to watch this themselves.
There is no super-fast-talking here, but all the arguments in the opening speech are terrible rhetorical argument. The speaker leverages the laughs and engagement of the audience to dismiss the position of his opponents, and this overall really felt more terrifying to me than many of the big political speeches I’ve seen this year.
Like, I… think I am more terrified of the effect this would have on epistemics than the effect of the super-fast-talkers in policy debate? Like, at least in policy-debate it’s somewhat obvious you are playing a game. In the above, I wouldn’t be surprised if the participants actually come to believe the position they are trying to defend.
I was surprised, this video was much less goodharted than I expected (after having been primed with the super-fast talking example). I was expecting more insane things.
Though overall it had the level of much broad public debate/discourse I’ve seen. I watched the first three speakers, and didn’t learn anything. In good debates I’ve seen I’ve felt that I’ve learned something from the debaters about their fields and their unique world views, these felt like two opposing sides in a broader political debate with kind of no grounding in reality. They were optimized for short-scale (e.g. <30 seconds) applause lights for the audience, when objected they’d make it a fight saying things like “Don‘t even try to win that example”, their examples seemed false yet rewarded (primarily attributing China’s rise out of poverty in the last 50 years to ‘redistribution’ and getting applause for it, which, correct me if I’m wrong, is not at all the primary reason, they had massive growth in industry in part by copying a lot of the west). I wouldn’t expect to learn anything, it just seemed like nobody understood economics and they were indexed off what was like 0-1 inferential steps from what the audience as a whole understood. I guess that was the worst part, how can you discuss interesting ideas if they have to be obvious to an audience that big and generic within 10-20 seconds?
People shared so many bad experiences with debate…
I had a great time debating (BP style) in Russia a few years ago. I clearly remember some moments which helped me to become better at thinking/speaking and world modeling:
The initial feedback I got during the practice session is basically don’t be a guy from the terrible video you shared :-). Make it easy for a judge to understand your arguments: improve the structure and speak slower. Focus on one core argument during your speech: don’t squeeze multiple half-baked ideas in; deliver one but prove it fully.
At my first tournament for newbies, an experienced debater gave a lecture on playing something-something resolutions and concluded with strongly recommending reading up on game theory (IIRC, The Strategy of Conflict and Governing the Commons).
My second tournament was in Jedi format: I, an inexperienced Padawan, played with a skilled Jedi. I matched with a person because both of us liked LessWrong. I think we even managed to use “belief should pay rent” as part of an argument in a debate on the tyranny of the majority. I think it’s plausible that we referred to Moloch at least once.
Later on, improvement came from managing inferential distances during speeches; and grounding arguments in reality: being specific about harms and benefits, delivering appropriate ~examples to support intermediate claims.
I think the experience was worth it. It helped me to think more in-depth and about much more issues than I would have overwise (kinda like forecasting now). I quit because (a) tournaments are time-consuming; (b) I got bored playing social issues & identity politics.
While competitive debating is not about collaborative truth-seeking, in my experience, debtors are high cognitive decouplers. Arguing with them (outside of the game) felt good, and we were able to touch topics far outside of the default Overtone window (like taking the perspective of ISIS).
The culture was healthy because most people were just passionate about debating/grokking complex issues (like investor-state dispute settlements), and their incentives were not screwed up because the only upside to winning debate tournaments in Russia is internet points.
Upd: I feel that one of your main concerns is Goodharting. I think the BP system as we played it basically encouraged maximizing the expected utility of impacts of arguments you brought to the table i.e. harm/benefit to individual × scale × probability occurring × how well you proved it (which can be seen as the probability that your reasoning is correct). It’s a bit harder to fit the importance of framing the issue and principled arguments into my simplification. But the first can be seen as prioritizing based on relative tractability (e.g. in almost all of the debate arguing that “we will save money by not implementing a policy” is a bad move because there are multiple other ways to save money and the benefits of the policy might be unique). The second is about the importance of metagame, incentive structures, commitments, and so on.
I did competitive college debate for four years (American Parliamentary format, which is similar to the BP format used in the EA Debate Championship but not identical) and I think that the extent to which it does/doesn’t encourage truth-seeking is less important than the way it pushes people to justify their values.
Oversimplifying broadly, debate has two layers: one is the arguments about what the impacts of a certain idea/policy are likely to be, and one is arguments about which impacts are more important (known as “weighing”). In order to win rounds, you have to win arguments at both levels. This means that debate requires people to engage with one of issues that is most central to EA — a relatively consequentialist understanding of which issues matter most. In regular life you can say, “I support government funding for the arts because art is good” and not think very hard about how that trades off with, say, funding for healthcare. But if you do that in a debate round, the other team will point out the tradeoff, estimate the number of people who will die as a result of there being less funding for healthcare, and you will lose the round.
I think this is the main benefit of debate from an EA perspective, and I suspect that it has meaningful impacts on people who are forced to confront, over and over again in countless debate rounds, the actual effects (in lives lost and other very serious harms) of different ways of weighing between issues. Anecdotally, a higher-than-average percentage of the debaters I know are EAers, or at least interested in EA. And even debaters who don’t personally support EA very often use EA weighing arguments in rounds. As a result, for some people (I suspect many), debate is the first place they hear about EA. To me, this makes debate leagues a fertile recruiting ground for EA.
...that video is deeply absurd, and I am caught between revulsion and laughter. They are clearly way past the point where there is even a gesture at real usefulness; this might be the most heavily gamed thing I have ever seen.
However, if the video is an example I am confident that this activity can’t do any real harm to our objectives. Who could do this with even the pretense of thinking they would talk to people in the real world that way? It’s just a verbal luge.
Yeah, it’s kinda hilarious. Speaking so fast that your opponents can’t follow your arguments and therefore lose the round is common practice in some forms of competitive debate. But in other debate categories, using this tactic would immediately lose you the round. In my own personal experience of high school debate, the quality of competitive debate depends very heavily on the particular category of debate.
The video above is Policy Debate, the oldest form of debate which degenerated decades ago into unintelligible speed reading and arguments that every policy would result in worldwide nuclear annihilation. In the 1980s, the National Speech and Debate Association instituted a new form of debate called Lincoln Douglas that attempted to reground debate in commonsense questions about moral values; but LD has also fallen victim to speed reading and even galaxy-brained “kritiks” arguing that the structure of debate itself is racist or sexist and therefore that the round should be abandoned.
Public Forum debate, invented in 2002 as an antidote to Lincoln Douglas, is IMO a very healthy and educational form of debate. Here is the final round of the 2018 national championship (starting at 4:05) on the resolution, “On balance, the benefits of United States Participation in the North American Free Trade Agreement outweigh the consequences.” https://m.youtube.com/watch?v=MUnyLbeu7qU&feature=youtu.be
British Parliamentary debate is another form of debate that, in my experience, is more civil and less “game-able” than other forms of debate (though Harrison D disagrees below, with specifics about its pitfalls). One key difference is that, while Public Forum allows and encourages debaters to spend weeks researching and debating a single specific resolution, Parliamentary debate typically a involves generalized preparation on a subject or theme and only reveals the specific resolution a few minutes before the round begins. Because of this, I think Public Forum is more educational for debaters, but Parliamentary is probably easier to run a one-off tournament because debaters won’t be expected to have done as much preparation.
Extemporaneous Speaking is another category involving less preparation, where participants are asked a question about current affairs or politics and have 30 minutes to prepare a 7 minute off-the-cuff speech. There is no “opponent” in Extemp, perhaps limiting the level of discourse, but it might be possible to easily introduce EA-related topics because participants are expected to be conversant in a wide range of topic areas.
On the whole, I’m very glad to see this EA debate tournament being run, and would be very excited to see further work bringing EA topics into debate. I can understand why many people might find some debate tactics toxic and counterproductive, particularly in categories like Policy and LD, but I do think this is the failure of specific categories and tactics and not an indictment of all adversarial debate. Learning the best arguments for both sides of a resolution certainly teaches a bit of an “arguments as soldiers” approach, but I believe the greater effect is to lead debaters to real truths about which arguments are stronger and improve their personal understanding of the issues. In future EA debate events, I would only suggest that organizers be very conscious of these standards and norms when choosing a specific category of debate.
I appreciate this nuanced comparison between formats. I increase my estimation of the goodness of the EA debate project, because this introduces a new pathway to victory:
The creation of an EA debate format, which can be designed against the backdrop of all the poisoned formats to encourage skills and norms more like those we want to develop.
It does feel like the general period for research before the debate would work well for these topics. Do you know if any format has a mechanism for awarding points on the grounds of agreeing with the opponent’s points, or otherwise acknowledging when arguments or data weigh against you?
Just to clarify my position:
I think that the culture of British Parliamentary has made it significantly less game-y and more civil than most if not all other formats of collegiate debate, including both prepared formats (e.g., Policy Debate, Lincoln-Douglas, Public Forum) and limited preparation formats (e.g., other forms of parli such as American Parliamentary).
I think that the limited topic prep nature of parliamentary debate makes those formats significantly less game-y and more civil than most other formats of collegiate debate.
My main issue with BP is really just two individual characteristics in the format that represent stark differences from the format I did in high school (American Parli, in the Stoa league): the 4-teams-of-2 (instead of 2v2) combined with the lack of access to published sources on the internet when in prep. Really, most if not all of the main issues I highlighted in my comment relate to the first thing, which I think is more fundamental. So ultimately, I’m not trying to compare BP to policy debate, nor am I trying to compare it to the actual (culturally-driven) practice of American Parli in collegiate leagues (which I’m not as familiar with), it’s really just me comparing it to what I think an ideal format would be when given a decent culture that isn’t so acceptive of gamification.
It’s unfortunate to hear that you had such a negative experience from debate. As someone who has judged public-school high school policy and public forum debate, I will say that I am not that surprised. That being said, I do take issue with your characterization of all/BP debate with the video plus the statement “I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much.”
I cannot speak to every BP league/competition in the world, but I have never seen nor heard of such drastic gamification of debate in BP—or anything even come close to it—in the four years that I did BP in college. In fact, I have often seen people hold up BP and collegiate policy debate as polar opposites, with BP being one of the least toxic/gamified formats (at least among the major formats) and policy debate being the most. (BP definitely has some problems with left-leaning judge bias, but it could be a lot worse and that’s not really that unique to BP.) Ultimately, I don’t want to be rude/abrasive, but I feel that the video really gives a deeply misleading picture of BP, even if it is only a mild-moderate exaggeration of collegiate policy debate. I think it’s unfortunate to think that many people who are unfamiliar with debate (let alone BP specifically) may come away with such a misleading picture of BP/debate in general based on this extreme example of a different format. I’m not sure how to say this in a non-confrontational way, but I personally think that some kind of revision/redaction (e.g., a disclaimer saying that the video is not of BP, acknowledgement that BP is different) may be in order.
I will just add the following video to illustrate that the gamifying (e.g., speed and spread tactics) of debate is not so inherent to the sport or even specific formats themselves, but rather are heavily determined by league culture (e.g., what kinds of judges are used, how debaters are taught to debate): https://www.youtube.com/watch?v=TvhNvumnZ1U&t=23s (Although, do excuse the pathos-heavy story at the beginning, and remember these are just high school students)
I just want to say I, Ben Pace, feel attacked every time someone criticizes “BP” in this comment thread.
A lot of my models here come from talking to Evan Hubinger about this, who has a lot of thoughts on debate and competed at the national level in Policy debate in college.
My guess is that overall debate is really badly goodharted. All of the different variants. One of the ways policy-debate in particular is really badly goodharted is that everyone talks so fast nobody can comprehend them. But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats (though I don’t know how common in BP in particular). Evan won a bunch of his highest-level debates by telling stories about pirates for 15 minutes, and then telling everyone that the institution of debate is so broken and useless and wouldn’t it just be better if we would use this time to learn cool facts about history like the pirate anecdotes I told you in the first 80% of the debate?
If people were just talking fast, I do think that would be a problem, but not a super big one. I can imagine a kind of hyperoptimized court system that functions fine with everyone talking at 2-3x normal speed. The problem is that it’s evidence that the system at large has very little defenses against goodharting and runaway competition effects.
As far as I can tell, some other debate formats do not have the problem of everyone talking at 2-3x speed, but basically have many of the other problems, and importantly, all share the fundamental attribute that they have very few successful defenses against goodhart’s law. And so somehow, even though it might differ from debate format to debate format, the actual thing the competitors are doing has very little to do with seeking the truth.
The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.
I watched about 20 minutes of the video you linked, and I do think just from that slice it seems much less obviously broken than the video I linked, and that does seem important to recognize.
I do also think that I can only really interface with that whole video in a healthy way by repeatedly forcing myself to take a step back and basically throw away all the information that is thrown at me, because I don’t trust it, and I don’t trust the process that produced it. Like, I don’t think I left that video having a better understanding of seatbelt laws (the topic of debate). I learned some useful relevant facts, but I am still worried that I left that debate with worse beliefs about seatbelt laws, and very likely much worse than if I had just read the Wikipedia article on it. Of course, the point of the debate is not for me to learn about seatbelt law, but I do also think the same is probably true for the participants.
“The problem is that it’s evidence that the system at large has very little defenses against goodharting and runaway competition effects.” Although I acknowledge that there will always be some level of misalignment between truth-seeking and competition, I would push back on the idea that the system has little defense against drastic goodharting like is seen in both high school and collegiate policy debate: the experience of Stoa (the league in which I debated) and NCFCA are evidence of that. In my view and in the view of some others (see e.g., https://www.ethosdebate.com/community-judges-1-necessity-community-judges/), it seems that one of the important front-line defenses against gamification of debate is the use of community judges who recoil at nonsense and speed. Of course, that introduces tradeoffs that debaters (myself included) sometimes huff about, such as biased decisions, but it still seems worth it. Additionally, I feel fairly confident that there are other important factors that explain the stark cultural differences between Stoa/NCFCA and most public-school/collegiate leagues (e.g., the debaters’ personalities/background, parental involvement, the Christian ethos, the observation of and opportunity for self-differentiation from public-school/collegiate practices).
To address your broader point about the truth-seeking vs. competition drive (goodharting): I and many others in my league have considered this question. (For a brief example article from someone I know, see https://www.ethosdebate.com/art-persuasion-vs-pursuit-truth/) I could be wrong/exaggerating, but I get the sense from you that debate should be really strict about promoting truth-seeking above other things—perhaps even to the extent that debate should almost never sacrifice truth-seeking for other goals. Perhaps that is not what you are saying, but regardless, I would push back and emphasize that debate has a wide variety of purposes, crucially including skills education in general (as opposed to topical education). (I actually recently finished a blog series which I started by outlining some of the major purposes of debate: see https://www.ethosdebate.com/purposes-of-debate-pt-1-the-goals-and-anti-goals-of-debate/ ). In short, I think that the experience of Stoa/NCFCA shows that with reasonable safeguards (e.g., including community judges in the judging pool) debate can be at least neutral if not more positive than negative in promoting truth-seeking, while at the same time is a great way to get youth excited about studying topics, scrutinizing their own views, and learning to persuade others. That last part applies to that NITOC final (regarding seatbelt policy), which focused on a case that was known for being somewhat pathos-heavy (as opposed to, for example, the case for cutting funding for air marshals, which I and many other debaters would likely have never come to see if it were not subject to the adversarial scrutiny of a competitive season of debate): debate shouldn’t be entirely/solely about truth-seeking; teaching persuasion skills is also really important, because if you have the truth but cannot persuade others, then your ability to act on it is sorely limited.
Also: “people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study”—my experience in Stoa was fairly different: I repeatedly had to defend the methodology of some of the studies I relied on, and was able to challenge the methodology of sources.
I was confused about the situation with debate, so I talked to Evan Hubinger about his experiences. That conversation was completely wild; I’m guessing people in this thread might be interested in hearing it. I still don’t know exactly what to make of what happened there, but I think there are some genuine and non-obvious insights relevant to public discourse and optimization processes (maybe less to the specifics of debate outreach). The whole thing’s also pretty funny.
I recorded the conversation; don’t want to share publicly but feel free to DM me for access.
I want to echo this. I think my own experience of debating has been useful to me in terms of my ability to intelligence-signal in person, but was pretty bad overall for my epistemics. One interesting thing about BP (which was the format I competed in most frequently at the highest level) was the importance in the 4th speaker role identifying the cruxes of the debate (usually referring to them as “clash”), which I think is really useful. Concluding that the side you’ve been told to favour has then “won” all of the cruxes is… less so.
Some more discussion on competitive debating and EA https://forum.effectivealtruism.org/posts/DEuYo4zxDyY7h7n74/debate-and-effective-altruism-friends-or-foes
+1 from my own experience in debate (also british style). Truth-seeking / identifying and meaningfully resolving points of disagreement between different positions is a very different skill to trying to win a debate, and the skillsets/mindsets developed in the latter seem like they might actively work against the former unless the people doing it are very careful and self-aware.
I did Policy Debate, the format from that video, in high school and college. Policy Debate has its problems, but I think the fast delivery style is good on balance, since it lets a single debate cover so much ground. Speed reading is comprehensible with practice. Here’s an example from college debate with better audio quality—definitely weird, but clear enough to understand.
Sure, debate may involve a lot of techniques that are counterproductive to truth-seeking, and I wouldn’t want people to write on the EA Forum like it’s a debate, for example. However, I think there are many places where it would help to be able to convey more convincing arguments even if being more convincing doesn’t improve truth-seeking—speaking with non-EAs about EA, for example.
I generally want us to use truth-seeking methods when engaging with outsiders as well. Of course, that isn’t always possible, but I also really don’t want us to have a reputation for using lots of rhetorical tricks to convince others (and generally think that doing so is pretty bad).