Measuring SRS Impact: Seven Years In, How Have We Evolved?

Over the last seven years, Student-Ready Strategies has designed its work to support institutions and partners as they move from big ideas to bold action, in pursuit of measurable change that drives student outcomes. As a mission-driven organization relentlessly committed to improving students' outcomes and experiences, we are often challenged by the question of how best to measure our impact. Our approach to measuring impact has evolved over time: from working within what was currently possible, to pushing the limits of what was possible, and now, building infrastructure to fundamentally change what is possible altogether.

Defining Impact

Initially, we followed a typical model, focusing on the metrics most accessible to us and measuring organizational activity. This includes things like staff size, number of active projects, and total revenue. While these might be a loose proxy for impact, to us, they quickly proved insufficient. Next, we layered on a proxy for quality, asking and tracking how our work is received by our clients and partners. But we could see that while someone enjoying their work with us is important, it does not measure whether they made progress because of it.

In recent years, we’ve gone beyond those metrics to calculate our reach, measuring impact by the number and types of institutions and students served by our work. Throughout this piece, I’ll highlight what we know about SRS’s reach over the past seven years based on the measurement model we’ve developed. I’ll also discuss the next frontier for us and how new infrastructure will enable us to hold ourselves accountable for the core evaluative question for our mission: did student outcomes improve as a result of our work? 

The Past Seven Years

SRS supports the goal of a student-ready system of higher education in two key ways. First, we provide services directly to institutions, either on their own or in cohorts. Second, we provide system-level support to states, systems, and nonprofit organizations. Let’s talk about direct institutional support first. We measure the reach of our direct service projects by counting the number of institutions that participated in a synchronous activity with us, received an institution-specific deliverable from us, or were granted access to Constellation, our technical assistance platform. 

To date, SRS has provided direct service to 488 institutions. 

Here’s what we know about them.

• They collectively serve 6.5 million students per year. 

• Two-thirds of them are community colleges with open admissions policies and a focus on associate degrees and certificates. 

• They are disproportionately minority-serving institutions.

• Ninety-four percent are public institutions.

• Nearly one in five has engaged with SRS multiple times.

As we mentioned, our second lever for advancing student-ready transformation is through systemic work. This occurs when we work with states, systems, and nonprofit organizations to support strategic planning and policy development, as well as research and analysis that result in concrete recommendations. This work affects institutions within that state, system, or network, but those institutions may not have engaged with SRS directly or even know that we played a role in the ultimate system changes. To date, SRS has supported projects that have systemically impacted 603 institutions, about half of which also received direct service from SRS through separate projects. These projects add an additional unduplicated 3+ million students to the total number of students served by SRS's work. 

SRS has directly served and/or impacted over 800 institutions

which collectively serve nearly 10 million students per year.

And since we are supporting structural changes that will impact students for years to come, we are confident that the number of people affected by our past work will continue to grow over time and be further augmented as we serve additional institutions with additional structural reforms in the future.

The Next Frontier

A few years ago, we started working toward a completely different model for delivering services to institutions. The result was Constellation, a web-based platform that delivers technical assistance to institutions by guiding workflows and producing custom deliverables. But the platform isn’t just about digital delivery; we used this exercise in innovation to tackle the issue of impact measurement as well. For each structural change Constellation supports, there is a student-focused success metric, and institutions are guided through a process (which is not optional) to set goals and track progress for that metric. For example, in the module that supports the institutional scale of Open Educational Resources (OER), every participating institution sets a numeric goal for increasing the percentage of students enrolled in at least one OER course. Not only will they set these goals, but they will be prompted to report progress at key checkpoints. 

This structured tracking benefits participating institutions by helping them define and document success, calculate return on investment, and make the case for additional items on their student success agendas. However, the benefits go much further. With systemic, aggregated tracking of institutional changes and their impacts, , an entirely new research base will have been built, helping the field of higher education better understand what structural reforms can accomplish. 

And, while it may seem like a side benefit of this work, the student-centered goal and progress tracking in Constellation are also how SRS will take the next step in measuring our impact and holding ourselves accountable to our mission. As we scale Constellation, we will have moved beyond project counts, anecdotal service reviews, and even student and institutional reach metrics to definitively say that our work is worth it, because it changed students’ lives. I believe that day is not far off, and I can’t wait.