Interesting. Does anyone do group brainstorming because they actually expect it to make meaningful progress towards solving a problem? At least when you’re at a large event with people who are not high context on the problem, that seems pretty doomed. I assumed the main reason for doing something like that is to get people engaged and actually thinking about ideas and participating in a way that you can’t in a very extremely large group. If any good ideas happen, that’s a fun bonus
If I wanted to actually generate good ideas, I would do a meeting of people filtered for being high context and having relevant thoughts, which is much more likely to work.
A specific format that has worked well for me eg for running research brainstorms for my team is as follows:
Set a topic and a prompt, eg the topic of planning the team’s next quarter, and the prompt of what our goals should be
Set a 5-minute timer and have each participant brainstorm in a separate doc
This means that everyone needs to actually engage, quiet people also get a voice and separate docs minimise groupthink
After the timer, copy these into a central doc that everyone reads through and leaves comments with questions, disagreements, agreements, further thoughts, etc.
Optional: after it seems like everyone has had time to read everyone else’s, give people a bit of time to verbally discuss any common themes, important disagreements, etc
Repeat on another question. This often takes about 20 minutes a question
This seems to work well on groups of three to nine people. If your group is larger than that, well, firstly, why on earth are you doing a brainstorming meeting of this many people? But if you do need to do this, then splitting up into groups who each think about a different question, and share the most interesting takeaways after each round, might work ok.
Panels are almost always a massive waste of time though, strongly agreed there—I suspect they generally happen as a way to get more fancy people to turn up (by inviting them to be panellists), because there’s a hole in the schedule that needs to be filled, or because of copying what events “should” do
I think your suggested format is a significant upgrade on the (much more common, unfortunately) “group brainstorm” set up that Ollie is criticising, for roughly the reasons he outlines; It does much better on “fidelity per person-minute”.
Individual brainstorming is obviously great for this, for the reasons you said (among others).
Commenting on a doc (rather than discussing in groups of 6-8) again allows many more people to be engaging in a high-quality/active way simultaneously.
It also seems worth saying that choosing questions well, which means they are:
worth answering
difficult or contextual enough that multiple people’s thought is required to get to a good answer
scoped well enough that progress can actually be made by a group in the relevant time
is a) necessary for group discussion to be worthwhile, b) difficult, and c) significantly more difficult for a group of mixed ability and context, whom you don’t know well. c, of course, applies much more strongly in the context Ollie is primarily concerned with (EAGs and similar events), to the one you’re describing (research team meeting).
I think that, almost without exception, if event sessions want to incorporate some discussion, they should start with the ‘individual silent thought’ exercise you mention, and then expand to pairs (with some 3s to stop people needing to do a lot of rearranging odd numbers of people). There are lots of reasons that this works better than larger groupings, but again Ollie’s heuristic of ‘fidelity per person-minute’ is one. A less obvious one is that minimising the distance between speaker and listener allows conversation volume to be much quieter, and if you think about how volume scales with distance, this more than outweighs the effect of having more people talking simultaneously.
Feeding back to the whole group from these discussions can happen (and be great), but is worth parallelising where possible, e.g. by commenting on a central gdoc as you suggest, or with a Slido, where people are encouraged to submit questions that their paired/small group discussions did not resolve, which the facilitator can then answer, or suggest steps to answer, at the end.
*I spent a ~decade as a teacher, and have facilitated many highly-reviewed workshops during and since, so I feel like I do have reasonable grounds to claim authority, but this is a joke, it just seemed like a funny epistemic status.
I assumed the main reason for doing something like that is to get people engaged and actually thinking about ideas
I don’t know what motivations people usually have, but I also feel skeptical of this vague “activation” theory of change. If session leads don’t know what actions they want session participants to take, I’m not optimistic about attendees generating useful actions themselves by discussing the topic for 10 minutes in a casual no-stakes, no-rigour, no-guidance setting. I’m more optimistic if the ask is “open a doc and write things that you could do”.
I would do a meeting of people filtered for being high context and having relevant thoughts, which is much more likely to work.
Yep, the thing you’ve described here sounds promising for the reasons Alex covered :) I realise I was thinking of the conference setting in my critique here (and probably should’ve made that explicit), but I’m much more optimistic about brainstorming in small groups of people with shared context, shared goals and using something like the format you’ve described.
I’ve run very successful group brainstorming sessions with experts just in order to require them to actually think about a topic enough to realize what seems obvious to me. Getting people to talk through what the next decade of AI progress will look like didn’t make them experts, or even get to the basic level I could have presented in a 15 minute talk—but it gives me me a chance to push them beyond their cached thoughts, without them rejecting views they see as extremes, since they are the ones thinking them!
Interesting. Does anyone do group brainstorming because they actually expect it to make meaningful progress towards solving a problem? At least when you’re at a large event with people who are not high context on the problem, that seems pretty doomed. I assumed the main reason for doing something like that is to get people engaged and actually thinking about ideas and participating in a way that you can’t in a very extremely large group. If any good ideas happen, that’s a fun bonus
If I wanted to actually generate good ideas, I would do a meeting of people filtered for being high context and having relevant thoughts, which is much more likely to work.
A specific format that has worked well for me eg for running research brainstorms for my team is as follows:
Set a topic and a prompt, eg the topic of planning the team’s next quarter, and the prompt of what our goals should be
Set a 5-minute timer and have each participant brainstorm in a separate doc
This means that everyone needs to actually engage, quiet people also get a voice and separate docs minimise groupthink
After the timer, copy these into a central doc that everyone reads through and leaves comments with questions, disagreements, agreements, further thoughts, etc.
Optional: after it seems like everyone has had time to read everyone else’s, give people a bit of time to verbally discuss any common themes, important disagreements, etc
Repeat on another question. This often takes about 20 minutes a question
This seems to work well on groups of three to nine people. If your group is larger than that, well, firstly, why on earth are you doing a brainstorming meeting of this many people? But if you do need to do this, then splitting up into groups who each think about a different question, and share the most interesting takeaways after each round, might work ok.
Panels are almost always a massive waste of time though, strongly agreed there—I suspect they generally happen as a way to get more fancy people to turn up (by inviting them to be panellists), because there’s a hole in the schedule that needs to be filled, or because of copying what events “should” do
[Epistemic status: argument from authority*]
I think your suggested format is a significant upgrade on the (much more common, unfortunately) “group brainstorm” set up that Ollie is criticising, for roughly the reasons he outlines; It does much better on “fidelity per person-minute”.
Individual brainstorming is obviously great for this, for the reasons you said (among others).
Commenting on a doc (rather than discussing in groups of 6-8) again allows many more people to be engaging in a high-quality/active way simultaneously.
It also seems worth saying that choosing questions well, which means they are:
worth answering
difficult or contextual enough that multiple people’s thought is required to get to a good answer
scoped well enough that progress can actually be made by a group in the relevant time
is a) necessary for group discussion to be worthwhile, b) difficult, and c) significantly more difficult for a group of mixed ability and context, whom you don’t know well. c, of course, applies much more strongly in the context Ollie is primarily concerned with (EAGs and similar events), to the one you’re describing (research team meeting).
I think that, almost without exception, if event sessions want to incorporate some discussion, they should start with the ‘individual silent thought’ exercise you mention, and then expand to pairs (with some 3s to stop people needing to do a lot of rearranging odd numbers of people). There are lots of reasons that this works better than larger groupings, but again Ollie’s heuristic of ‘fidelity per person-minute’ is one. A less obvious one is that minimising the distance between speaker and listener allows conversation volume to be much quieter, and if you think about how volume scales with distance, this more than outweighs the effect of having more people talking simultaneously.
Feeding back to the whole group from these discussions can happen (and be great), but is worth parallelising where possible, e.g. by commenting on a central gdoc as you suggest, or with a Slido, where people are encouraged to submit questions that their paired/small group discussions did not resolve, which the facilitator can then answer, or suggest steps to answer, at the end.
*I spent a ~decade as a teacher, and have facilitated many highly-reviewed workshops during and since, so I feel like I do have reasonable grounds to claim authority, but this is a joke, it just seemed like a funny epistemic status.
I don’t know what motivations people usually have, but I also feel skeptical of this vague “activation” theory of change. If session leads don’t know what actions they want session participants to take, I’m not optimistic about attendees generating useful actions themselves by discussing the topic for 10 minutes in a casual no-stakes, no-rigour, no-guidance setting. I’m more optimistic if the ask is “open a doc and write things that you could do”.
Yep, the thing you’ve described here sounds promising for the reasons Alex covered :) I realise I was thinking of the conference setting in my critique here (and probably should’ve made that explicit), but I’m much more optimistic about brainstorming in small groups of people with shared context, shared goals and using something like the format you’ve described.
I’ve run very successful group brainstorming sessions with experts just in order to require them to actually think about a topic enough to realize what seems obvious to me. Getting people to talk through what the next decade of AI progress will look like didn’t make them experts, or even get to the basic level I could have presented in a 15 minute talk—but it gives me me a chance to push them beyond their cached thoughts, without them rejecting views they see as extremes, since they are the ones thinking them!