As the post says above, I’d like to share updates the team has made on its policies based on the internal review we did following the Time article and Owen’s statement as a manager on the team and the person who oversaw the internal review. (My initial description of the internal review is here). In general, these changes have been progressing prior to knowing the boards’ determinations, though thinking from Zach and the EV legal team has been an important input throughout.
Changes
Overall we spent dozens of hours over multiple calendar months in discussions and doing writeups, both internally to our team and getting feedback from Interim CEA CEO Ben West and others. Several team members did retrospectives or analyses on the case, and we consulted with external people (two EAs with some experience thinking about these topics as well as seven professionals in HR, law, consulting and ombuds) for advice on our processes generally.
From this we created a list of practices to change and additional steps to add. The casework team also reflected on many past cases to check that these changes were robust and applicable across a wide variety of casework.
Our changes, in rough order of importance:
Defaulting more often to getting input from more people on the team (especially if a case involves someone with power in the EA community) by having multiple caseworkers on a case and/or involving managers for oversight
Specifically, most cases now get seen by more than one person, particularly cases that involve someone with sufficient power in the community (confidentiality constraints may limit how many caseworkers work on a case.)
For the cases we take on, we have a lower bar for proactively reaching out to members to the community in order to get more information. For example, if this same case came up again, we would weigh more highly the importance of getting additional information from the woman Zach mentions above. Our previous approach pointed in this direction as well, but this change will make it less likely we miss an important opportunity.
We are creating a system that automatically flags to us when complaints about the same person come to different caseworkers, making us more likely to catch patterns of poor behavior (this previously was caught in a less systematic way)
We have lowered the bar for what’s considered a personal conflict of interest and improved our systems so it is easier to pass a case on to another caseworker. For instance, if the same case came up again, we would have it owned by someone else.
We’ve added and improved written internal guidelines for considerations to take into account during cases, particularly for cases involving someone with power in the EA community. Previously these were distributed across different documents and tacit understanding; they are now more formalized.
Automatic reminders for follow-ups on old cases to see if any more action is needed
An update towards overcommunication and explicitness when giving feedback to people whose behavior we’d like to change; an update against assuming the feedback has been understood and incorporated
An update towards escalations to boards, legal teams and HR departments where relevant.
Better systems (reminders, templates in meetings, etc) for making sure balls aren’t dropped based on when or how information comes in
The casework side of our team meets monthly to check that we are using our systems effectively, that we have implemented these changes with new and existing cases, and to update our internal guidelines as needed.
Some notes on the changes
I’m glad we have these changes, though I think it’s worth noting my guess that a big part of what will end up driving change here is also in large part due to CH team members’ individual reflections on the original events (Time article + statement from Owen + original case).
The list of changes is not exhaustive, since many of the changes stemming from this internal review (depending on your bar for “policy change”) are quite small e.g. “adding a consideration to our list of factors that may make a case more serious.” I think those “nudge” type changes are important and useful for incorporating data without over-updating or over-bureacratizing, though they are harder to assess externally. In general I’m trying to convey our high level changes without creating a laundry list. Nonetheless I hope the descriptions of our process changes help people make accurate updates and have accurate expectations about our team.
There is some broader context, which is that the casework team’s approach in January of 2023 (before the Time article came out) was already in significant ways different than in 2021. For instance, since Catherine joined in late 2021 (and even more so since Charlotte joined in 2023), the casework team has been able to double-check cases more often and have a lower bar for passing on cases where there was a personal connection. The changes as a result of the internal review add on to these previous shifts.
We’ll be checking in as a team at regular intervals to reflect on the value of these changes at getting better outcomes. This refers both to whether the increased time and effort per case looks like the right call (relative to e.g. handling more cases) and whether the decisions made are overall better (rather than the changes overfitting to this specific instance).
What’s still underway
In addition to the changes in how we handle personal COIs (with some person one team member has a COI with), we’re still working on a general approach to institutional COIs and what constitutes one, e.g. if a case involves someone in EV leadership. So far our planned approach is to ask for a person external to EV to give an independent view when such an institutional COI comes up in a case we take on, but it’s not obvious that there will always be an appropriate person willing. This will need more work to figure out.
As Zach indicates above, we’ll be working on a more formalized version of our current processes for escalation, COIs, and anything else the boards ask for.
Have you considered blinded case work / decision making?
Like one person collects the key information annonomises it and then someone else decides the appropriate responce without knowing the names / orgs of the people involved.
Could be good for avoiding some CoIs. Has worked for me in the past for similar situations.
This is hard and that should be recognized. It seems you all are taking this extremely seriously and that should be commended.
The recent discussion around Nonlinear got me wondering about one aspect of CH reports I hadn’t considered before.
Has the CH team ever spoken negatively about the person who made a report to people outside the CH team?
I’m thinking of a scenario like: Steve makes an accusation against Lina. The CH team interacts with Steve and through this interaction comes to view him as somewhat of an unstable character. Even though no one has reported Steve, the CH team advises against hiring Steve to other institutions within the EA space, based on their interactions they had with Steve when the reported about Lina’s behavior.
Thanks for your question, Tiresias. We appreciate people coming to us with concerns, and we absolutely don’t want to disincentive people from doing so. And we know that people usually aren’t at their best when they’re in the midst of stressful situations.
However, we don’t think it’s a workable policy to promise never to take action against people who come to us. Many concerns we get involve two or more people who each have complaints about the other’s actions. In those cases, we don’t want to unfairly advantage the side of the person who raises the topic with us first. Or there could be a concern separate from the problem the person reported. So we have to balance these two considerations.
Thanks for the honest and thoughtful response. I have a couple follow up questions. Since this does occur, I think it would be useful for the community to know what risks they are taking if they choose to report to the CH team.
I am reading what you stated as “reporting something to us does not mean that we will never take action against the person who reported to us.” I see two possible scenarios here, so want to get clarification on what you mean.
One scenario, James reports something to you. You conclude at some point that James has engaged in misconduct, based on reports from people outside of the CH team. Maybe you collected those reports in the process of investigating James’s claim. Maybe someone just separately came to you on an unrelated matter about James. Regardless, it ends up that you have some credibly allege James of misconduct, so you act on that.
Second scenario, James reports something to you. In talking to James, you find him to be abrasive, unreasonable, and generally quite unpleasant to interact with. You grow to have serious concerns about him based on how he’s interacted with you. He’s not engaged in any misconduct (eg, he’s not sexualy harassing you or anything), just generally displaying an unstable personality. You do not receive any reports of misconduct against James.
Have you taken action against someone like “James” in either scenario? Or just in the first scenario?
My second question is: Would it be possible for you to share the percentage of time you’ve taken action against a reporter over the last, say, 5 years?
Thanks again.
P.S. This whole OCB scenario seems like it would be very hard, Julia, and I really appreciate how forthcoming you have been about it. While you admit to making mistakes, I hope the wider community sees that the real mistakes in the CH’s response came from flawed processes more than from any one individual. We all make mistakes in our jobs, and unfortunately yours are publicized more than most. It’s a tough job and I wish you the best
As the post says above, I’d like to share updates the team has made on its policies based on the internal review we did following the Time article and Owen’s statement as a manager on the team and the person who oversaw the internal review. (My initial description of the internal review is here). In general, these changes have been progressing prior to knowing the boards’ determinations, though thinking from Zach and the EV legal team has been an important input throughout.
Changes
Overall we spent dozens of hours over multiple calendar months in discussions and doing writeups, both internally to our team and getting feedback from Interim CEA CEO Ben West and others. Several team members did retrospectives or analyses on the case, and we consulted with external people (two EAs with some experience thinking about these topics as well as seven professionals in HR, law, consulting and ombuds) for advice on our processes generally.
From this we created a list of practices to change and additional steps to add. The casework team also reflected on many past cases to check that these changes were robust and applicable across a wide variety of casework.
Our changes, in rough order of importance:
Defaulting more often to getting input from more people on the team (especially if a case involves someone with power in the EA community) by having multiple caseworkers on a case and/or involving managers for oversight
Specifically, most cases now get seen by more than one person, particularly cases that involve someone with sufficient power in the community (confidentiality constraints may limit how many caseworkers work on a case.)
For the cases we take on, we have a lower bar for proactively reaching out to members to the community in order to get more information. For example, if this same case came up again, we would weigh more highly the importance of getting additional information from the woman Zach mentions above. Our previous approach pointed in this direction as well, but this change will make it less likely we miss an important opportunity.
We are creating a system that automatically flags to us when complaints about the same person come to different caseworkers, making us more likely to catch patterns of poor behavior (this previously was caught in a less systematic way)
We have lowered the bar for what’s considered a personal conflict of interest and improved our systems so it is easier to pass a case on to another caseworker. For instance, if the same case came up again, we would have it owned by someone else.
We’ve added and improved written internal guidelines for considerations to take into account during cases, particularly for cases involving someone with power in the EA community. Previously these were distributed across different documents and tacit understanding; they are now more formalized.
Automatic reminders for follow-ups on old cases to see if any more action is needed
An update towards overcommunication and explicitness when giving feedback to people whose behavior we’d like to change; an update against assuming the feedback has been understood and incorporated
An update towards escalations to boards, legal teams and HR departments where relevant.
Better systems (reminders, templates in meetings, etc) for making sure balls aren’t dropped based on when or how information comes in
The casework side of our team meets monthly to check that we are using our systems effectively, that we have implemented these changes with new and existing cases, and to update our internal guidelines as needed.
Some notes on the changes
I’m glad we have these changes, though I think it’s worth noting my guess that a big part of what will end up driving change here is also in large part due to CH team members’ individual reflections on the original events (Time article + statement from Owen + original case).
The list of changes is not exhaustive, since many of the changes stemming from this internal review (depending on your bar for “policy change”) are quite small e.g. “adding a consideration to our list of factors that may make a case more serious.” I think those “nudge” type changes are important and useful for incorporating data without over-updating or over-bureacratizing, though they are harder to assess externally. In general I’m trying to convey our high level changes without creating a laundry list. Nonetheless I hope the descriptions of our process changes help people make accurate updates and have accurate expectations about our team.
There is some broader context, which is that the casework team’s approach in January of 2023 (before the Time article came out) was already in significant ways different than in 2021. For instance, since Catherine joined in late 2021 (and even more so since Charlotte joined in 2023), the casework team has been able to double-check cases more often and have a lower bar for passing on cases where there was a personal connection. The changes as a result of the internal review add on to these previous shifts.
We’ll be checking in as a team at regular intervals to reflect on the value of these changes at getting better outcomes. This refers both to whether the increased time and effort per case looks like the right call (relative to e.g. handling more cases) and whether the decisions made are overall better (rather than the changes overfitting to this specific instance).
What’s still underway
In addition to the changes in how we handle personal COIs (with some person one team member has a COI with), we’re still working on a general approach to institutional COIs and what constitutes one, e.g. if a case involves someone in EV leadership. So far our planned approach is to ask for a person external to EV to give an independent view when such an institutional COI comes up in a case we take on, but it’s not obvious that there will always be an appropriate person willing. This will need more work to figure out.
As Zach indicates above, we’ll be working on a more formalized version of our current processes for escalation, COIs, and anything else the boards ask for.
Have you considered blinded case work / decision making? Like one person collects the key information annonomises it and then someone else decides the appropriate responce without knowing the names / orgs of the people involved.
Could be good for avoiding some CoIs. Has worked for me in the past for similar situations.
Thanks — yes, we’ve done this in some cases.
(Fwiw, the Forum moderation team does this for many of our cases.)
This is hard and that should be recognized. It seems you all are taking this extremely seriously and that should be commended.
The recent discussion around Nonlinear got me wondering about one aspect of CH reports I hadn’t considered before.
Has the CH team ever spoken negatively about the person who made a report to people outside the CH team?
I’m thinking of a scenario like: Steve makes an accusation against Lina. The CH team interacts with Steve and through this interaction comes to view him as somewhat of an unstable character. Even though no one has reported Steve, the CH team advises against hiring Steve to other institutions within the EA space, based on their interactions they had with Steve when the reported about Lina’s behavior.
Thanks for your question, Tiresias. We appreciate people coming to us with concerns, and we absolutely don’t want to disincentive people from doing so. And we know that people usually aren’t at their best when they’re in the midst of stressful situations.
However, we don’t think it’s a workable policy to promise never to take action against people who come to us. Many concerns we get involve two or more people who each have complaints about the other’s actions. In those cases, we don’t want to unfairly advantage the side of the person who raises the topic with us first. Or there could be a concern separate from the problem the person reported. So we have to balance these two considerations.
Thanks for the honest and thoughtful response. I have a couple follow up questions. Since this does occur, I think it would be useful for the community to know what risks they are taking if they choose to report to the CH team.
I am reading what you stated as “reporting something to us does not mean that we will never take action against the person who reported to us.” I see two possible scenarios here, so want to get clarification on what you mean.
One scenario, James reports something to you. You conclude at some point that James has engaged in misconduct, based on reports from people outside of the CH team. Maybe you collected those reports in the process of investigating James’s claim. Maybe someone just separately came to you on an unrelated matter about James. Regardless, it ends up that you have some credibly allege James of misconduct, so you act on that.
Second scenario, James reports something to you. In talking to James, you find him to be abrasive, unreasonable, and generally quite unpleasant to interact with. You grow to have serious concerns about him based on how he’s interacted with you. He’s not engaged in any misconduct (eg, he’s not sexualy harassing you or anything), just generally displaying an unstable personality. You do not receive any reports of misconduct against James.
Have you taken action against someone like “James” in either scenario? Or just in the first scenario?
My second question is: Would it be possible for you to share the percentage of time you’ve taken action against a reporter over the last, say, 5 years?
Thanks again.
P.S. This whole OCB scenario seems like it would be very hard, Julia, and I really appreciate how forthcoming you have been about it. While you admit to making mistakes, I hope the wider community sees that the real mistakes in the CH’s response came from flawed processes more than from any one individual. We all make mistakes in our jobs, and unfortunately yours are publicized more than most. It’s a tough job and I wish you the best