Summary: I’m a psychiatrist working in a safety-net hospital setting in Seattle. I want to use federated learning with community data governance to reduce iatrogenic harm from involuntary psychiatric detention. Recent evidence shows marginal detentions increase 6-month mortality by significant margins. The propsal below demonstrates practical Constitutional AI—serving collectively-defined community values rather than centrally-determined principles. Looking for feedback, collaborators, and connections to relevant researchers/funders.
The Problem: Iatrogenic Harm at Scale
Involuntary psychiatric detention is meant to prevent harm. Recent empirical evidence suggests it often causes harm instead. These data from Alleghany County PA, surrounding Pittsburgh, were recently published by the New York Federal Reserve. They used physician random assignment as an instrumental variable to study psychiatric detention outcomes. Key finding: for marginal detention cases, involuntary commitment increases 6-month mortality, criminal justice involvement, and substance use disorder mortality.
The mechanism appears to be disruption of social supports—detention causes people to lose jobs, housing, and community connections that protect against bad outcomes.
Where I work, at Harborview Medical Center in Seattle, we’ve developed ML models that predict violent hospital incidents in the proceeding 3 days with F1 scores of 0.77 (in press, expanding on earlier work). Future studies will likely show greater performance as they include wider data sources with salient variables such as regional ED utilization or other social determinants of health. NYU and the Epic EHR Cosmos tool have independently demonstrated the use of independently trained machine learning models to predict a variety of events, which NYU’s team aptly declared a “general prediction engine”. This demonstrates we can identify high-risk situations with reasonable accuracy. But this creates a moral imperative: if we can predict which detentions, or for which patients, whom we will cause more harm than good, we have an obligation to act on that information. This has not been replicated yet and really demands an intervention study, but this would be challenging to test at scale without some existing infrastructure.
The scale:
- ~1.7 million involuntary psychiatric holds annually in the US
- If even 20% are marginal cases with net negative outcomes, that’s 340,000 harmful detentions per year
- Cost of crisis response: ~$50,000 per person per year (ED, hospitalization, jail, police)
- Cost of prevention: ~$3,000-5,000 (housing assistance, early intervention)
- Neglected: No one else working on this specific intersection
- Important: Reducing mortality and incarceration at scale, primarily affecting marginalized populations
- Cost-effective: 10-15x cost differential between prevention and crisis response
- Evidence-based: RCT is possible, outcomes are measurable
The Solution: Federated Learning with Community Governance
Technical Architecture
Federated Learning:
- ML models trained across multiple sites (hospitals, crisis response programs, community clinics)
- Individual-level data never leaves originating site
- Only model updates are shared
- Breaks down data silos without creating centralized surveillance infrastructure
Zero-Knowledge Proofs:
- Cryptographically prove properties about data without revealing the data itself
- Example: “This community has identified housing instability as top risk factor” without exposing individual cases
- Enables aggregate pattern recognition while preserving privacy
Community Data Trusts:
- Legal entities governed by community members (patients, advocates, local organizations)
- Hold fiduciary obligations to serve community health interests
- Have veto power over data use, algorithm deployment, resource allocation
- Can audit algorithms for bias and shut down if causing harm
Why This Is Constitutional AI in Practice
Anthropic’s Constitutional AI uses a fixed constitution to guide model behavior. Our proposal extends this:
Instead of: Anthropic researchers define the constitution
We propose: Communities define their own constitutions
Instead of: Single constitution for all users
We propose: Federated constitutions—each community defines values, AI coordinates across them
Instead of: Constitutional principles are abstract (be helpful, harmless, honest)
We propose: Concrete, measurable outcomes (reduce mortality, incarceration, involuntary detention)
Instead of: Human feedback during training only
We propose: Continuous community governance and outcome monitoring
This addresses the main critique of Constitutional AI: Who decides the constitution, and do they have democratic legitimacy?
The Pilot Proposition: Harborview + Community Partners
Phase 1: Enhanced Crisis Response (Year 1)
Partners:
- Harborview Medical Center (King County WA) - safety-net hospital where I work
- Crisis Care Centers newly funded in King County
- other communities may have local organizations, though one model CAHOOTS in Eugene OR has recently shuttered most of its street outreach services because of funding challenges
- Crisis intervention teams for mental health de-escalation and diversion by police modeled in other communities with high opt-in by potential clients and police and demonstrated feasibility
- Build community data trusts through a local registry
Implementation:
- Crisis responders get secure access to relevant history (with community governance)
- ML identifies patterns: “This person has had 6 crises in past month, housing instability appears to be driver”
- Intervention: Housing assistance, not detention
- Track outcomes: ED visits, detention, mortality, incarceration at 6 months
Phase 2: Predictive Prevention (Year 2)
Early warning system:
- ML identifies people at high risk of crisis in next 3-7 days
- Community-based outreach (voluntary, not coercive)
- Microgrants for immediate needs (back rent, utilities, medication, etc)
- Prevents crisis before it requires 911 call
Example:
- System flags: Person with crisis history, housing court date tomorrow, out of medication
- Community health worker reaches out: “Need help getting to court? Need medication refill?”
- “Housing assistance more effective than transport to shelter for people with X profile”
- Each site maintains sovereignty, collective learning improves outcomes
Why This Matters for AI Safety
1. Practical Alignment Testing
Most AI alignment research is theoretical: “How do we ensure AGI doesn’t kill everyone?”
This project tests alignment empirically: “Can AI demonstrably serve communities it affects?”
**Measurable outcomes:**
- Does this reduce mortality? (primary outcome)
- Does this reduce incarceration? (secondary outcome)
- Do communities maintain meaningful control? (governance outcome)
- Does the system resist capture by institutions? (alignment outcome)
If we can’t build aligned AI for something as straightforward as “don’t cause iatrogenic harm in healthcare,” we have no hope for harder alignment problems.
2. Democratic AI Governance
Current AI governance debate: Should we regulate AI through:
- Government oversight? (slow, captured by industry)
- Industry self-regulation? (fox guarding henhouse)
- Technical safety measures? (necessary but insufficient)
Our proposal adds: Community governance as enforcement mechanism
- Communities have skin in the game (their health outcomes)
- Can shut down systems causing harm (veto power)
- Direct feedback loop (outcomes are visible and measurable)
- Scales horizontally (federated, not centralized)
3. Preventing Extractive AI
The alternative to community-governed AI is corporate-governed AI.
UnitedHealth Group model:
- Centralized data aggregation and vertical integration of health data warehousing, financial underwriting, prior authorization, clinical and pharmacy services
- Proprietary algorithms predict risk and triage to care management regimes optimized on internal cost containment (and a steep time-discount, with average churn of ~1 yr), without accountability to community externalities or priorities
- Extract value from information asymmetry
- $300B+ market cap, serves shareholders, not patients
Our model:
- Federated data (stays local)
- Open algorithms (auditable)
- Create value from coordination efficiency
- Revenue from cost savings (marginal fee on realized prevention benefits)
- Serves communities, sustainable without extraction
If we don’t build community-governed alternatives, corporate surveillance infrastructure becomes default.
- Open-source technical architecture and governance framework
Medium-term (3-7 years): Policy Change
- Successful demonstration creates demand for legal authorization
- State legislation (Washington, Oregon, Colorado) enabling community data trusts
- SAMHSA reform of 42 CFR Part 2 (allows substance use data in federated learning)
- Regional adoption across Pacific Northwest
Long-term (7-15 years): Institutional Alternative
- Federated crisis response network across US
- Model extends to other domains (housing, employment, education)
- Alternative to corporate health data infrastructure
- Proof that AI can serve democratic coordination
Critiques and Responses
“This is just predictive policing for mental health”
It’s the opposite. Predictive policing identifies people for state intervention. This system identifies when state intervention will cause harm and should be avoided in favor of local response. We have evidence that detention increases mortality—this reduces harmful detentions.
“ML will encode racial bias”
The status quo is already encoded (via human clinical judgment) with racial bias—Black men are over-detained. Algorithmic bias is at least auditable and correctable. Community governance with regular bias audits is our accountability mechanism, but relies on a robust data flow empowered by trust. If the system shows disparate impact, communities can shut it down.
“False positives will stigmatize people”
The current system has a false positive problem—we’re detaining people who would be better off without detention. Our goal is to reduce total harm (false positives + false negatives), not eliminate it. And community governance means affected people help set thresholds.
“This is a distraction from structural problems”
We need more housing, better mental health services, accountable public safety, more job and economic security. Those fights continue. But while we fight for structural solutions, we can stop causing iatrogenic harm today. This is harm reduction, not a replacement for systemic change.
“Technology won’t solve social problems”
Correct. Technology is necessary but not sufficient. This project reduces one specific harm (mortality from marginal detentions) while generating evidence for resource reallocation (from crisis response to prevention). The political fight remains essential—we’re just creating better evidence to win it. More representative data at scale will highlight the externalized costs of gaps in basic human needs for marginalized communities, facilitating calls for political action and resource prioritization. Building the data infrastructure to rigorously test interventions in any community will allow every community to prioritize the most salient problems with their own solutions.
What I’m Looking For
1. Feedback
- Is this EA-aligned (tractable, neglected, important, cost-effective)?
- What are failure modes I’m not seeing?
- How can the governance be more robust?
- What evidence would convince you this is worth pursuing?
2. Connections to Researchers/Funders
- AI safety researchers interested in practical alignment
- AI governance scholars (especially GovAI)
- Healthcare researchers working on algorithmic fairness (Emma Pierson, Ziad Obermeyer)
- Critical scholars who can help avoid replicating harm (Safiya Noble, Ruha Benjamin, Rediet Abebe)
- Foundations funding AI + health + governance (Open Philanthropy, RWJF, Arnold Ventures)
3. Potential Collaborators
- Researchers with federated learning + privacy-preserving ML expertise
- Cryptographers working on zero-knowledge proofs
- Community organizers in cities with crisis response programs
- Policy experts on health data governance
- Anyone who has thought about democratic AI coordination
4. Pilot Funding
- Looking for $150k-200k for 18-month pilot
- Harborview + CAHOOTS + community data trusts
- Primary outcome: 6-month mortality
- Secondary outcomes: Incarceration, ED utilization, community governance viability
- Deliverable: Published results, open-source architecture, replication blueprint
Why Now?
Crisis response reform has political momentum:
- Post-George Floyd, cities are experimenting with alternatives to police
- CAHOOTS, STAR, Portland Street Response proving concept
- Bipartisan support (progressives: decarceration, conservatives: cost savings)
- Federal funding available (SAMHSA, HHS, DOJ grants)
AI governance needs practical demonstrations:
- Too much theory, not enough empirical work
- Constitutional AI needs to prove democratic legitimacy
- Federated learning needs real-world use cases
- Community governance needs working examples
Healthcare is ready for prevention models:
- Value-based care incentives (not just fee-for-service)
- Social determinants of health recognized as critical
- States (Washington, Oregon, California) experimenting with policy (Medicaid innovation e.g.)
- Evidence base exists (NY Fed paper provides empirical foundation)
The window won’t stay open:
- If we don’t build community-governed infrastructure, corporate surveillance becomes default
- UnitedHealth and others are building centralized systems now
- Once entrenched, very hard to displace
- Next 3-5 years are critical
Contact
Taylor Mac Black
Consultation Psychiatrist, Harborview Medical Center
**Acknowledgments:** This work builds on ideas from constitutional AI (Anthropic), mechanism design for social good (Rediet Abebe), algorithmic accountability (Safiya Noble, Ruha Benjamin), and harm reduction philosophy. All mistakes are mine.
**Epistemic status:** High confidence in problem (iatrogenic harm exists, is measurable), medium confidence in solution (technical architecture is sound but governance is untested), low confidence in timeline (political/institutional change is unpredictable).
Extending Constitutional AI to Community Governance: Reducing Mortality from Psychiatric Detention
Summary: I’m a psychiatrist working in a safety-net hospital setting in Seattle. I want to use federated learning with community data governance to reduce iatrogenic harm from involuntary psychiatric detention. Recent evidence shows marginal detentions increase 6-month mortality by significant margins. The propsal below demonstrates practical Constitutional AI—serving collectively-defined community values rather than centrally-determined principles. Looking for feedback, collaborators, and connections to relevant researchers/funders.
The Problem: Iatrogenic Harm at Scale
Involuntary psychiatric detention is meant to prevent harm. Recent empirical evidence suggests it often causes harm instead. These data from Alleghany County PA, surrounding Pittsburgh, were recently published by the New York Federal Reserve. They used physician random assignment as an instrumental variable to study psychiatric detention outcomes. Key finding: for marginal detention cases, involuntary commitment increases 6-month mortality, criminal justice involvement, and substance use disorder mortality.
The mechanism appears to be disruption of social supports—detention causes people to lose jobs, housing, and community connections that protect against bad outcomes.
Where I work, at Harborview Medical Center in Seattle, we’ve developed ML models that predict violent hospital incidents in the proceeding 3 days with F1 scores of 0.77 (in press, expanding on earlier work). Future studies will likely show greater performance as they include wider data sources with salient variables such as regional ED utilization or other social determinants of health. NYU and the Epic EHR Cosmos tool have independently demonstrated the use of independently trained machine learning models to predict a variety of events, which NYU’s team aptly declared a “general prediction engine”. This demonstrates we can identify high-risk situations with reasonable accuracy. But this creates a moral imperative: if we can predict which detentions, or for which patients, whom we will cause more harm than good, we have an obligation to act on that information. This has not been replicated yet and really demands an intervention study, but this would be challenging to test at scale without some existing infrastructure.
The scale:
- ~1.7 million involuntary psychiatric holds annually in the US
- If even 20% are marginal cases with net negative outcomes, that’s 340,000 harmful detentions per year
- Cost of crisis response: ~$50,000 per person per year (ED, hospitalization, jail, police)
- Cost of prevention: ~$3,000-5,000 (housing assistance, early intervention)
The EA relevance:
- Tractable: Technical solution exists (federated learning + cryptography)
- Neglected: No one else working on this specific intersection
- Important: Reducing mortality and incarceration at scale, primarily affecting marginalized populations
- Cost-effective: 10-15x cost differential between prevention and crisis response
- Evidence-based: RCT is possible, outcomes are measurable
The Solution: Federated Learning with Community Governance
Technical Architecture
Federated Learning:
- ML models trained across multiple sites (hospitals, crisis response programs, community clinics)
- Individual-level data never leaves originating site
- Only model updates are shared
- Breaks down data silos without creating centralized surveillance infrastructure
Zero-Knowledge Proofs:
- Cryptographically prove properties about data without revealing the data itself
- Example: “This community has identified housing instability as top risk factor” without exposing individual cases
- Enables aggregate pattern recognition while preserving privacy
Community Data Trusts:
- Legal entities governed by community members (patients, advocates, local organizations)
- Hold fiduciary obligations to serve community health interests
- Have veto power over data use, algorithm deployment, resource allocation
- Can audit algorithms for bias and shut down if causing harm
Why This Is Constitutional AI in Practice
Anthropic’s Constitutional AI uses a fixed constitution to guide model behavior. Our proposal extends this:
Instead of: Anthropic researchers define the constitution
Instead of: Single constitution for all users
Instead of: Constitutional principles are abstract (be helpful, harmless, honest)
Instead of: Human feedback during training only
This addresses the main critique of Constitutional AI: Who decides the constitution, and do they have democratic legitimacy?
The Pilot Proposition: Harborview + Community Partners
Phase 1: Enhanced Crisis Response (Year 1)
Partners:
- Harborview Medical Center (King County WA) - safety-net hospital where I work
- Crisis Care Centers newly funded in King County
- other communities may have local organizations, though one model CAHOOTS in Eugene OR has recently shuttered most of its street outreach services because of funding challenges
- Crisis intervention teams for mental health de-escalation and diversion by police modeled in other communities with high opt-in by potential clients and police and demonstrated feasibility
- Build community data trusts through a local registry
Implementation:
- Crisis responders get secure access to relevant history (with community governance)
- ML identifies patterns: “This person has had 6 crises in past month, housing instability appears to be driver”
- Intervention: Housing assistance, not detention
- Track outcomes: ED visits, detention, mortality, incarceration at 6 months
Phase 2: Predictive Prevention (Year 2)
Early warning system:
- ML identifies people at high risk of crisis in next 3-7 days
- Community-based outreach (voluntary, not coercive)
- Microgrants for immediate needs (back rent, utilities, medication, etc)
- Prevents crisis before it requires 911 call
Example:
- System flags: Person with crisis history, housing court date tomorrow, out of medication
- Community health worker reaches out: “Need help getting to court? Need medication refill?”
- Cost: $300 intervention
- Prevents: Eviction → homelessness → crisis → ED visit → detention (cost: $15,000+)
Phase 3: Regional Federation (Year 3)
Expansion to:
- Denver STAR program
- Portland Street Response
- Other crisis response programs
Federated learning across sites:
- Share patterns without sharing data
- “Housing assistance more effective than transport to shelter for people with X profile”
- Each site maintains sovereignty, collective learning improves outcomes
Why This Matters for AI Safety
1. Practical Alignment Testing
Most AI alignment research is theoretical: “How do we ensure AGI doesn’t kill everyone?”
This project tests alignment empirically: “Can AI demonstrably serve communities it affects?”
**Measurable outcomes:**
- Does this reduce mortality? (primary outcome)
- Does this reduce incarceration? (secondary outcome)
- Do communities maintain meaningful control? (governance outcome)
- Does the system resist capture by institutions? (alignment outcome)
If we can’t build aligned AI for something as straightforward as “don’t cause iatrogenic harm in healthcare,” we have no hope for harder alignment problems.
2. Democratic AI Governance
Current AI governance debate: Should we regulate AI through:
- Government oversight? (slow, captured by industry)
- Industry self-regulation? (fox guarding henhouse)
- Technical safety measures? (necessary but insufficient)
Our proposal adds: Community governance as enforcement mechanism
- Communities have skin in the game (their health outcomes)
- Can shut down systems causing harm (veto power)
- Direct feedback loop (outcomes are visible and measurable)
- Scales horizontally (federated, not centralized)
3. Preventing Extractive AI
The alternative to community-governed AI is corporate-governed AI.
UnitedHealth Group model:
- Centralized data aggregation and vertical integration of health data warehousing, financial underwriting, prior authorization, clinical and pharmacy services
- Proprietary algorithms predict risk and triage to care management regimes optimized on internal cost containment (and a steep time-discount, with average churn of ~1 yr), without accountability to community externalities or priorities
- Extract value from information asymmetry
- $300B+ market cap, serves shareholders, not patients
Our model:
- Federated data (stays local)
- Open algorithms (auditable)
- Create value from coordination efficiency
- Revenue from cost savings (marginal fee on realized prevention benefits)
- Serves communities, sustainable without extraction
If we don’t build community-governed alternatives, corporate surveillance infrastructure becomes default.
Theory of Change
Near-term (2-3 years): Demonstrate Feasibility
- Pilot with CAHOOTS + Harborview
- Publish outcomes showing reduced mortality/incarceration
- Open-source technical architecture and governance framework
Medium-term (3-7 years): Policy Change
- Successful demonstration creates demand for legal authorization
- State legislation (Washington, Oregon, Colorado) enabling community data trusts
- SAMHSA reform of 42 CFR Part 2 (allows substance use data in federated learning)
- Regional adoption across Pacific Northwest
Long-term (7-15 years): Institutional Alternative
- Federated crisis response network across US
- Model extends to other domains (housing, employment, education)
- Alternative to corporate health data infrastructure
- Proof that AI can serve democratic coordination
Critiques and Responses
“This is just predictive policing for mental health”
It’s the opposite. Predictive policing identifies people for state intervention. This system identifies when state intervention will cause harm and should be avoided in favor of local response. We have evidence that detention increases mortality—this reduces harmful detentions.
“ML will encode racial bias”
The status quo is already encoded (via human clinical judgment) with racial bias—Black men are over-detained. Algorithmic bias is at least auditable and correctable. Community governance with regular bias audits is our accountability mechanism, but relies on a robust data flow empowered by trust. If the system shows disparate impact, communities can shut it down.
“False positives will stigmatize people”
The current system has a false positive problem—we’re detaining people who would be better off without detention. Our goal is to reduce total harm (false positives + false negatives), not eliminate it. And community governance means affected people help set thresholds.
“This is a distraction from structural problems”
We need more housing, better mental health services, accountable public safety, more job and economic security. Those fights continue. But while we fight for structural solutions, we can stop causing iatrogenic harm today. This is harm reduction, not a replacement for systemic change.
“Technology won’t solve social problems”
Correct. Technology is necessary but not sufficient. This project reduces one specific harm (mortality from marginal detentions) while generating evidence for resource reallocation (from crisis response to prevention). The political fight remains essential—we’re just creating better evidence to win it. More representative data at scale will highlight the externalized costs of gaps in basic human needs for marginalized communities, facilitating calls for political action and resource prioritization. Building the data infrastructure to rigorously test interventions in any community will allow every community to prioritize the most salient problems with their own solutions.
What I’m Looking For
1. Feedback
- Is this EA-aligned (tractable, neglected, important, cost-effective)?
- What are failure modes I’m not seeing?
- How can the governance be more robust?
- What evidence would convince you this is worth pursuing?
2. Connections to Researchers/Funders
- AI safety researchers interested in practical alignment
- AI governance scholars (especially GovAI)
- Healthcare researchers working on algorithmic fairness (Emma Pierson, Ziad Obermeyer)
- Critical scholars who can help avoid replicating harm (Safiya Noble, Ruha Benjamin, Rediet Abebe)
- Foundations funding AI + health + governance (Open Philanthropy, RWJF, Arnold Ventures)
3. Potential Collaborators
- Researchers with federated learning + privacy-preserving ML expertise
- Cryptographers working on zero-knowledge proofs
- Community organizers in cities with crisis response programs
- Policy experts on health data governance
- Anyone who has thought about democratic AI coordination
4. Pilot Funding
- Looking for $150k-200k for 18-month pilot
- Harborview + CAHOOTS + community data trusts
- Primary outcome: 6-month mortality
- Secondary outcomes: Incarceration, ED utilization, community governance viability
- Deliverable: Published results, open-source architecture, replication blueprint
Why Now?
Crisis response reform has political momentum:
- Post-George Floyd, cities are experimenting with alternatives to police
- CAHOOTS, STAR, Portland Street Response proving concept
- Bipartisan support (progressives: decarceration, conservatives: cost savings)
- Federal funding available (SAMHSA, HHS, DOJ grants)
AI governance needs practical demonstrations:
- Too much theory, not enough empirical work
- Constitutional AI needs to prove democratic legitimacy
- Federated learning needs real-world use cases
- Community governance needs working examples
Healthcare is ready for prevention models:
- Value-based care incentives (not just fee-for-service)
- Social determinants of health recognized as critical
- States (Washington, Oregon, California) experimenting with policy (Medicaid innovation e.g.)
- Evidence base exists (NY Fed paper provides empirical foundation)
The window won’t stay open:
- If we don’t build community-governed infrastructure, corporate surveillance becomes default
- UnitedHealth and others are building centralized systems now
- Once entrenched, very hard to displace
- Next 3-5 years are critical
Contact
Taylor Mac Black
Consultation Psychiatrist, Harborview Medical Center
Affiliate Faculty, University of Washington
macblack@uw.edu
I’m especially interested in connecting with:
- Anyone at Anthropic working on Constitutional AI, governance, or healthcare applications
- Researchers in AI safety, algorithmic fairness, or health informatics
- Community organizers in cities with crisis response programs
- Funders interested in AI + health + governance intersection
Happy to:
- Share technical details (federated learning architecture, governance protocols)
- Discuss the empirical evidence (NY Fed paper, our violence prediction work)
- Connect with potential pilot sites
- Collaborate on grant proposals
-----
**Acknowledgments:** This work builds on ideas from constitutional AI (Anthropic), mechanism design for social good (Rediet Abebe), algorithmic accountability (Safiya Noble, Ruha Benjamin), and harm reduction philosophy. All mistakes are mine.
**Epistemic status:** High confidence in problem (iatrogenic harm exists, is measurable), medium confidence in solution (technical architecture is sound but governance is untested), low confidence in timeline (political/institutional change is unpredictable).