Why Validation Matters
Pretrial risk assessment tools like the Public Safety Assessment (PSA) are critical to ensuring fair, data-driven decisions about pretrial release. Senate Bill 36 (SB 36) mandates that:
- Each pretrial services agency using a risk assessment tool must validate the tool by July 1, 2021, and regularly thereafter. If local data is unavailable, use the most recent data collected by a pretrial services agency in a similar county in California.
- Validation must scientifically measure:
- The accuracy of predicting court appearances
- The accuracy of assessing public safety risks
- Potential disparate effects based on gender, race, or ethnicity
Beyond legal requirements, validation helps us:
- Confirm the tool’s accuracy in predicting court appearances and public safety risks
- Identify potential disparities in how the tool performs across different demographic groups
- Continuously improve our approach to pretrial justice
Sonoma’s Validation: Key Findings
The 2023 validation study of Sonoma County’s PSA provides valuable insights into the tool’s performance:
Predictive Accuracy
- The PSA demonstrated excellent predictive power for:
- Failure to appear in court
- New arrests during the pretrial period
- Good predictive capabilities for new criminal filings and convictions
- Fair predictive accuracy for new violent arrests
Demographic Analysis
The study carefully examined how the PSA performed across different racial and gender groups:
- No significant bias detected in predictive accuracy between:
- White and Hispanic individuals
- Men and women
Real-World Context
- Study period: July 2020 to January 2022
- Included 1,761 pretrial observations
- Reflects unique challenges of the COVID-19 pandemic period
Relevance to San Francisco
Like Sonoma County, San Francisco is committed to:
- Using scientifically validated risk assessment tools
- Promoting fair and transparent pretrial decision-making
- Continuously evaluating and improving our justice practices
