While the Foundation made significant progress against its strategic objectives over the last eight years of investing, several notable areas stand out as ongoing challenges:
Promoting Outcomes-focused Terminology
The organizational capacity building outcomes terminology was not adopted by investees in the way TFF had originally envisioned.
TFF’s four organizational outcome domains are: Strategic Leadership and Planning; Outcomes-Focused Management; Performance Management; and Program Quality, Fidelity, and Effectiveness.
These domains, described in detail on the Foundation’s website, also function as TFF’s investment categories, against which progress is tracked. Still, there is little evidence of adoption of this specific terminology by the majority of investees–this is not to say these domains are not being advanced, but rather investees are not using this specific language within their organizations. This may be a result of a decreased focus on the terms over time by the Foundation based on investee feedback, or the shift to investing in three initiatives/partnerships in the second cohort versus three individual organizations and one initiative in the first cohort. The “organizational outcomes” language is less relevant to initiatives given there are multiple participating partners that track progress against their own work plans rather than track outcomes via a measurement tool like iCAT. One investee indicated they did not understand TFF’s funding categories or the outcomes the Foundation is pursuing: “One thing I find difficult is trying to determine what needs would be of the most interest to the Foundation. In our strategic plan, we have several goals and if there were a general guide to the types of projects the Foundation was most interested in supporting–or a guide to projects that would not be supported–it would be helpful.”
Measuring Child Outcomes
“I really liked the planning process but I don’t love the labels. My brain doesn’t work that way. It all feels very BCG or Bridgespan (large management consulting firms). Conceptually, I get it but … I felt like I was checking a box. But at a high level, those are the things we’re focused on. Overall, I could take or leave TFF’s terminology.”
- TFF Investee Partner
The Foundation initially explored having all investees use one SEL performance management tool to track five child outcomes identified in TFF’s Logic Model (see graphic above). The data from the tool was to be used for “performance management” or teacher practice improvement purposes. The tool was developed by Child Trends with Foundation staff input and includes teacher and student report items. However, investees perceived the tool as burdensome to use for large numbers of children and not conducive to their emerging SEL measurement needs. This mismatch may have come from the Foundation’s goal of having investees use the tool simply for performance management (that is, very quick, iterative practice improvement cycles using data trends, not precision measurement) while partnering with a research organization to develop the tool whose staff were understandably concerned with ensuring validity and reliability of such a measure. While some constructs measured by the Child Trends tool are still being monitored via surveys, the vision that all investees would use the same tool was quickly dispatched by the Foundation.
The Foundation also expanded the way it characterized social and emotional learning in response to investee shifts. While direct instruction in social and emotional skills remains a central part of SEL implementation, the Foundation increasingly took an ecological approach to child development and sought to promote a systemic view of SEL. In keeping with the SEL field’s expansion and increasing sophistication, TFF’s thinking and support began to more intentionally address environmental conditions of learning, including climate and culture. TFF became more alert to ensuring investee’s SEL implementation did not enforce student behavioral compliance in culturally biased ways and took into account implications of adversity science to learning and development. Consequently, TFF became less prescriptive around measuring the five specific child outcomes in the logic model and invested in approaches and programming whose goals go beyond promoting those child outcomes including New Beginnings Family Academy’s Emotionally Responsive Practice and Bridgeport Public School (BPS)’s efforts to adopt a trauma-informed lens and implement restorative justice approach to student discipline.
“Keep doing what you’re doing, it really does make a difference in children’s lives. SEL is hard to measure and there will be successes and failures but, really, there is no such thing as failure because there are always children who are benefiting from the SEL work that we do.”
- Bridgeport Community Member
Relatedly, TFF continues to learn how to best support institutionalization of SEL data collection in resource constrained contexts. An ongoing TFF strategic objective is to support investee partners to build and sustain the capacity to, with fidelity, collect and utilize SEL or other proxy data for performance management purposes. While every investee has made significant progress toward that goal, that progress has rarely happened in a straight line, with the global pandemic further complicating matters. Some investees have plateaued in their efforts and others have slightly regressed depending on the availability of outside support and staff capacity.
There are unique reasons behind each investee’s difficulties in building SEL data collection and analysis capacity. For example, BPS’s ability to administer and use the data from school climate and SEL surveys was significantly bolstered by the partnership of The Consultation Center at Yale (TCC) evaluation experts with the District SEL Coordinator. TCC transitioned the role of data analysis and evaluation partner to the Bridgeport Child Advocacy Coalition at RYASAP at the beginning of the second five-year investment period, and the dedicated SEL Coordinator position (which was a Teacher on Special Assignment role) was discontinued in summer 2021 and absorbed into another full time role. Effectively collecting and using SEL data is a challenge for most schools–SEL measurement can be a complex task. That said, TFF investees remain committed to growth in this area.
Knowledge Building and Sharing
Cross-portfolio information sharing and learning among investees proved harder to achieve than TFF anticipated. Bandwidth has been understandably thin at both TFF, to design and host ongoing portfolio-wide collaboration opportunities, and among investees, to engage in knowledge sharing.
Additionally, there are a wide array of perceptions of the Foundation’s cross organization learning strategy. While some investees mentioned that the annual meeting hosted by TFF “fostered a sense of community,” several cohort 2 investees share a desire for additional chances to collaborate: “I would like more opportunities to work with others in the community and know what they’re doing” and “bringing folks together more would be beneficial.” The time constraints brought on by the pandemic limited investee capacity to join events and it is unclear if the Foundation met their own goals–and investee expectations–of offering consistent, structured chances for cross-portfolio learning. “I think the pandemic threw a wrench in the peer learning opportunities” one investee surmised. However, in 2020 and 2021, TFF staff focused on offering virtual knowledge-building opportunities in direct response to investee’s expressed needs, which were largely well received. Conversely, two leaders of investee organizations expressed that while they experienced gatherings as “nice” or “pleasant,” some meetings were not perceived as responsive to their most pressing needs. Some investees shared that the demands of their positions, particularly during the pandemic, made it unlikely they would join more frequent meetings with other investees even if they were offered.