Student attendance is an important outcome to track since it can be a proxy of other forms of student engagement. Rates of chronic absenteeism between 2012-13 (the year before TFF’s investment began) and 2019-2020 (the onset of the COVID-19 pandemic) have varied from 7% to 17% for K-8 students at NBFA and from 14% to 18% for K-8 students in Bridgeport Public Schools (BPS). While chronic absenteeism rates at New Beginnings Family Academy (NBFA) do not show a clear trend over the investment period, K-8 chronic absenteeism rates for BPS showed a small decrease in the early years of the investment, followed by a small increase in the later years of the investment. Although Horizons at Greens Farms Academy (HGFA) does not report attendance rates for their Saturday academy or summer program, a high percentage of students participated in their summer program for all years reported, with the exception of the virtual summer program just after the start of the pandemic.
Some research has shown that school discipline practices that involve excluding students from the classroom are associated with negative short- and long-term outcomes for students. Examination of NBFA data indicates a substantial drop in rates of exclusionary discipline sanctions (defined as out of school suspensions, in-school suspensions, and expulsions) in 2016-17, when NBFA implemented Emotionally Responsive Practice and shifted to a trauma-informed, relationship-based approach, and NBFA sanction rates remained very low through 2019-20. BPS exclusionary discipline sanction rates decreased steadily between 2012-13 and 2019-20.
HGFA, NBFA, and BPS use different assessments to measure students’ academic progress, and each investee has made some changes to the assessments they use over the course of the investment period. HGFA K-8 students consistently achieve substantial growth over the summer in reading (between 1.4 and 3.5 months of growth) and math (between 1.5 and 6.4 months of growth), but there is no clear trend in annual growth values over the investment period. HGFA reported increasing levels of achievement on the SAT for its high school students between 2018-19 to 2019-20, with a decrease in 2020-21, the first full school year of the pandemic. Smarter Balanced Assessment Consortium (SBAC) data for NBFA indicates that the percentage of NBFA students meeting or achieving the standard on the English Language Arts (ELA) and Math SBAC dropped in 2016-17, when NBFA implemented Emotionally Responsive Practice, but this trend reversed, with rates of proficiency in 2018-19 exceeding 2015-16 levels. SBAC data for BPS indicate that the percentage of grade 3-8 students proficient in math increased steadily since 2014-15, whereas the percentage of grade 3-8 students proficient in English Language Arts generally increased since 2015-16. SBAC data is not available for 2019-20 or 2020-21, due to the COVID-19 pandemic.
Members of the TFF staff completed the OMET in 2013-14 and 2014-15 for HGFA and NBFA to assess organizational capacity, before shifting to the iCAT self-assessment in 2015-16 after the OMET became unavailable. HGFA’s overall OMET rating for outcomes-focused management increased from low in 2013-14 to medium in 2014-15, while the other three domains were rated medium in both years. NBFA’s overall rating for strategic leadership and performance management increased from 2013-14 to 2014-15, while the other two domains remained consistent in both years. HGFA’s iCAT scores generally increased in all four domains between 2016 and 2021, with the exception of a decrease in 2018, and scores were highest in outcomes-focused management and lowest in program fidelity, quality, and effectiveness. Between 2016 and 2021, NBFA’s iCAT scores generally increased in all four domains, with the highest scores in outcomes-focused management and the lowest scores in strategic leadership. Because the OMET and iCAT are designed for individual organizations, not initiatives or partnerships, BPS was not required to use those tools. Instead, BPS conducted an educator survey 2-3 times each year from 2014-15 through 2019-20 to collect information about the implementation of SEL in K-8 schools. Survey data shows that SEL implementation generally strengthened over time, as did educators’ perceptions about the usefulness of SEL.
1 In other words, when the EASEL Lab at Harvard Graduate School of Education analyzed SEL Benchmarks assessment data, they detected a statistically significant increase in composite scores between the beginning and end of the summer program. Since the term statistically significant is often equated erroneously with the word significant in the everyday sense of the word (meaningful), this case study describes changes in scores as statistical differences, rather than statistically significant differences. More information on statistical significance is available in the Quantitative Analysis Technical Details.
Social and emotional skills
SEL Benchmarks assessment data for HGFA indicates that students’ SEL skills increased statistically[1] between the beginning and end of the summer program in 2017, 2018, 2019, and 2021. DESSA-mini assessment data for NBFA indicates that in 2018-19, 2019-20, and 2020-21, the percentage of children scored as “need for instruction” decreased over the course of each school year, the percentage of children scored as “strength” generally increased over the course of each school year, and the percentage of children scored as “typical” remained fairly stable. Self-reported SEL survey data for BPS indicates that about half of BPS students report strong emotion self-regulation skills, with the percentage generally increasing over time. Over three-quarters of BPS students report strong behavior self-regulation skills, with percentages decreasing slightly over time.