Case Study: Precision Analytics
First Place for Youth helps young people who have aged out of the child welfare system build the skills they need to make a successful transition to self-sufficiency and responsible adulthood. Gemma Services is a social service agency that operates a long-term residential psychiatric care program for youth.
Both organizations recognized that while they had extensive administrative data systems, very little actionable information was available for their caseworkers. They needed to translate data from specific participant populations into actionable information, so front-line staff could connect each client with the services that were most likely to ensure success.
Scope of Services
BCT Partners developed a precision analytics approach to give practitioners more contextualized and precise information. Precision Analytics (PA) trains machine learning algorithms to build predictive, prescriptive, and evaluation models that can be technologically implemented to generate actionable evidence on demand. The PA process uses program administration and case assessment data to find matched comparison groups of like cases based on their historical and baseline strengths, challenges, and environmental context. Then, machine learning algorithms are trained to discover all the different ways in which children and families in the same group were treated differently; these are the naturally occurring real-world "experiments" in which the same types of children were unconsciously assigned to different treatments. Algorithms find these counterfactual treatments within a very similar group of cases, determine what pattern of treatments achieved the most significant comparative gains, and report on these "ideal treatments."
BCT's team translated these patterns into useful information for front-line staff, including on-demand predictive and prescriptive insights presented in dashboards, reports, and visualizations, while also providing leaders and funders with up-to-date, precise, rigorous evaluation findings about how many lives are being attributable improved.
First Place for Youth and Gemma Services have implemented their dashboards, and practitioners now use real-time actionable evidence. The dashboards and tools resulting from precision modeling also provide valuable insight to program developers, managers, organizational leaders, and system change agents, including policymakers and funders. In addition to insights, resource guides, and the learning and professional support of peers and managers, practitioners can take direct action to increase the desired outcomes. This has led to a culture shift in how First Place for Youth and Gemma Services practitioners do their work and evaluate performance. It is now possible to view daily updates showing each individual's progress, performance, and outcomes, as well as the performance of the practitioner's entire caseload. Organizations are monitoring and tracking the utilization rates for their tools. Currently, each organization's program leaders, managers, and front-line practitioners are accessing and using their respective tools between 300 and 500 times per month.
Program directors are using cluster evaluation information for case planning.
Tools are motivating program directors and front-line practitioners to gather more data.
Practitioners are engaged to help improve the data and gather additional data points.
Precision modeling findings are being leveraged to advocate for policy changes.
To read the full Case Study, click here.