Evaluating a National Suicide and Crisis Hotline

Case Study - Findings Analysis and Insights - Program Evaluation and Measurement - Research and Evaluation - Research Design and Execution - Training and Technical Assistance - Veteran Health

Evaluating a National Suicide and Crisis Hotline

Posted on 05.03.24

Designing and conducting a national crisis services evaluation

SAMHSA needed a partner skilled in mental health evaluation for assessing the 988 Lifeline, focusing on potential outcome inequities. This partner needed to design and conduct a national crisis services evaluation, support 988 Lifeline grantees and evaluators with technical assistance, collect and analyze data, and produce reports for stakeholders, including Congress. The evaluation had to encompass 988 crisis centers, mobile teams, receiving and stabilization services and various SAMHSA grantees like tribal nations and community clinics. It also needed to identify disparities in crisis services and strategies to create equity.


SAMHSA partnered with Aptive, a consulting company with extensive experience conducting national suicide prevention evaluations, including the Clay Hunt Act-mandated evaluation and report to Congress.

Evaluation and Data Collection Plan

The Aptive team created an evaluation and data collection plan that includes gathering feedback from local evaluators through virtual meetings and using an evaluation advisory panel (EAP) to provide oversight, guide evaluation design and ensure evaluation activities are shaped by people with lived experiences and those from historically harmed, excluded and marginalized communities.

The evaluation and data collection plan includes all instruments, measures and data collection protocols, as well as plans for data analysis, dissemination, human-subjects protections and the integration of feedback. The plan also describes grantee training and technical assistance (T/TA) to support evaluation participation.

Training and Technical Assistance

Aptive offers comprehensive T/TA to all grantees of 988 crisis continuum services. Using an evidence-based approach ensures grantees are well-equipped to participate in data collection without feeling

overwhelmed. A team of technical assistance liaisons (TALs) brings expertise in suicide prevention and evaluation methodologies informed by both professional knowledge and personal experiences. An annual briefing keeps TALs updated on the latest developments in the 988 Crisis Services Framework.

The T/TA framework developed by Aptive is grounded in best practices from development, capacity building and implementation science, all underpinned by expertise in suicide prevention. This structure aims to maintain active engagement in the evaluation process, enhance data quality, and promote the use of data by grantees for program improvement, dissemination and advocacy.

Data Repository

The Aptive team developed a scalable, customizable Crisis Services Program Data Center (CSPDC) to facilitate data collection, management and evaluation. This center acts as a hub for all data collection tools and processes, offering flexible options for new data protocols and integrating data from various sources.

Aptive’s expertise extends to maintaining and documenting system changes and ensuring data quality, integrity, security and confidentiality. The team adeptly handles both quantitative and qualitative data, performing tasks that include data validation, identifying anomalies, anonymizing data and ensuring overall data accuracy and consistency.


Reporting findings to Congress and other stakeholders

Aptive writes and edits all documents needed to report on evaluation outcomes, including ad hoc reports, PowerPoint presentations, quarterly reports, the annual evaluation progress report, final summary and evaluation reports, talking points, conference documents, posters and other related material. An example of our monitoring is illustrated in the graph below.


Work Plan

The Aptive Team developed the evaluation project work plan, which serves as the foundation for documenting management processes and strategies to guide steps toward the success of the evaluation and to plan how results are achieved, measured and communicated.

Synthesis of Evaluation Advisory Panel Feedback

In assessing the evaluation, the Evaluation Advisory Panel provided comprehensive feedback across various studies. The panel’s insights stressed the importance of adopting comprehensive, culturally sensitive and cooperative methods to evaluate and enhance the nation’s crisis care systems.

Training and Technical Assistance (T/TA)

The T/TA team developed the Evaluation Needs Assessment, which the team disseminated to all grantees to capture feedback and understand their unique challenges. The team continuously develops materials and resources to prepare for upcoming grantee meetings.

OMB Package Submission

In preparing the Office of Management and Budget (OMB) package, the Aptive team is focused on delivering a comprehensive and accurate report that aligns with SAMHSA requirements. This process involves detailed planning and systematic data analysis of evaluation requirements. Our aim is to present a well-rounded view of the performance and impact of the various studies under review.

Upcoming Events

The Aptive team will attend the annual 988 grantee meeting in Chicago to present the evaluation project to various grantees and stakeholders and solidify understanding of the 988 framework.

Results at a Glance

  • Development of work plan
  • Synthesis of evaluation advisory panel feedback
  • OMB package submission

[1] Centers for Disease Control and Prevention, National Center for Health Statistics. National Vital Statistics System, Mortality 1999-2020 on CDC WONDER Online Database, released in 2021. Data are from the Multiple Cause of Death Files, 1999-2020, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Accessed at http://wonder.cdc.gov/ucd-icd10.html on Apr 4, 2024 10:27:27 AM