REPORT

June 27, 2017

Executive Summary

The 2015 passage of the Every Student Succeeds Act (ESSA) ushered in a new era for state accountability systems. ESSA provided states an opportunity to help all students succeed by rethinking both how they identify the schools that need improvement and in what ways. The law requires states to submit a formal plan to the Department of Education for peer review and then begin implementing that plan in the 2017–18 school year. Sixteen states plus the District of Columbia submitted their plans in April, and the remainder will do so in September.

Keeping student success at the heart of these plans is paramount. That’s why Bellwether Education Partners, in partnership with the Collaborative for Student Success, convened an objective, independent peer review process of state accountability plans in order to look beyond mere compliance with the federal process, encourage all states to adopt high-quality plans, and provide a resource for state leaders working to help all students succeed.

Process and Methodology:

After the states submitted their official plans, we brought together a bipartisan, nationally-esteemed group of more than 30 education policy experts, with a strong focus on individuals with state-level experience, to review each plan in small groups. This group included specific content experts to address the unique challenges associated with students with disabilities and English Language Learners.

Reviewers used their discretion and expertise to respond to and score each of nine rubric items, and those scores have been normed across states and peers. We gave states the opportunity to respond to our reviews, and that feedback is reflected in our full report. We are pleased that all 17 states responded to our peer review and nearly all engaged in an exchange that resulted in substantive feedback. States clarified portions of their plans and provided additional information to inform our analysis. Any remaining errors are our own, and the findings in this executive summary reflect what we heard from the peers and states collectively.

Importantly, these reviews represent a snapshot in time; that is, they reflect the quality of state plans as submitted to the federal government. As of this writing, the federal process is still ongoing, and states are likely to make changes to their plans up until they earn approval (and even beyond, as they begin implementing their plans). We hope our reviews can help these specific states continue to improve their plans going forward, and that the 34 states that have yet to submit their plans can learn from their peers.

Bright Spots

Our peer review process uncovered some promising policies and practices. Of note:

States are broadening their accountability systems beyond reading and math. Most states are adding science and a more accurate measure of student attendance, not to mention indicators measuring physical education, art, and school climate.

At the high school level, more states included some indicator of college- and career-readiness that attempts to measure whether students are taking positive steps to succeed beyond K-12.

All 17 states included a measure of year-to-year student growth, which gives schools credit for how much progress their students make over time, rather than static determinations about where they are at a given point in time.

Although questions remain about some of the individual choices states made, progress on these fronts is noteworthy. More specifically, several individual state plan components are worthy of praise and should be considered models or exemplars for other states to follow.

Goals:

Louisiana’s goals are ambitious, attainable, and backed by clear data. The state is proposing to sustain its recent gains and annually increase its proficiency rates for all students, especially for groups that are starting further behind.

Standards & Assessments:

Delaware, New Jersey, New Mexico, and Vermont all received top marks for strong commitments to college- and career-ready standards and high-quality, aligned assessments in math and English Language Arts, as well as assessments in science, social studies, and early grades (K-2) that will help address concerns about curriculum narrowing.

Indicators:

New Mexico proposed a simple, high-quality list of meaningful indicators, including the growth of the lowest-performing students, extended-year graduation rates, chronic absenteeism, and a college-readiness indicator. The state also included a plan to add additional measures over time, including a new science indicator and the inclusion of college enrollment and remediation rates, and use feedback from stakeholder meetings with parents and families to ensure the system is usable and easy to understand.

Academic Progress:

Arizona’s plan places a strong weight on student achievement and growth by combining a clear measure of student achievement with two different measures of student growth, including one that compares students to each other and one that compares them to a common, grade-level benchmark.

Tennessee’s plan showed a strong commitment to balancing growth and proficiency, going so far as to include targets specific to each school in the state for both measures.

New Mexico will pair student proficiency rates along with a school’s overall year-to-year student growth, plus specific attention to the growth of each school’s bottom 25% of students.

Identifying Schools:

Louisiana’s A-F school rating system provides stakeholders with a single, clear, summative rating to understand school performance. Additionally, the state plans to identify close to 17 percent of its schools for comprehensive support and improvement, well above the 5 percent required under federal law.

Supporting Schools:

Tennessee based its school improvement strategy on its own previously successful interventions and a broad research base. The state proposes multiple tiers of support and multiple state authorities to drive improvement in the lowest-performing schools and districts. Tennessee should also be commended for planning to award federal school-improvement funds to those districts committed to interventions with the strongest evidence base.

New Mexico clearly states what action must be taken in schools that fail to improve three years after being initially identified for comprehensive support and improvement. Schools must choose between a concrete list of intervention options, or the state department will choose one for it. New Mexico is committed to providing additional funding to plans that use the strongest base of evidence and to providing “Direct Student Services” to support expanded learning time, Advanced Placement course access, K-3 literacy and mathematics, pre-k services, personalized learning, and student transportation.

Exiting Improvement Status:

Nevada put in place clear, rigorous exit criteria that require schools to demonstrate significant improvements over time, against pre-determined benchmarks tied to reaching grade-level expectations, before exiting comprehensive or targeted improvement status.

Continuous Improvement:

New Mexico outlines a number of ways it has and will continue to engage stakeholders on key aspects of its implementation efforts. For example, it plans to conduct additional stakeholder outreach as it develops a future science assessment and an “Opportunity-to-Learn” survey. New Mexico plan also includes an innovative idea for a “Return Tour” across the state to present an overview of the submitted plan, how it changed in response to initial stakeholder feedback, and how the state will implement the plan going forward.

Opportunities for Improvement

While there were promising elements, our peer reviewers found that most state plans failed to provide significant details about how their systems would work in practice. For example:

How will schools be held accountable for the performance of all subgroups of students?

How will schools be identified for improvement?

What steps will schools identified for improvement need to take, and how will they demonstrate they’ve made sufficient progress to exit improvement status?

On these areas, states generally complied with the bare minimum requirements of federal law, often repeating the exact definitions used in the law. Other than a few notable exceptions (see above and below), states provided little indication of how their proposed rules will play out in their state. Despite 18 months since the passage of ESSA, too many states were unable to specify how many schools and how many subgroups each of their proposals would identify.

This lack of detail is worrying, particularly because of the way ESSA works. Although ESSA provides some “guardrails” that every state must follow, it leaves significant discretion to individual states. As we transition to a world where every state has the opportunity to design its own unique systems, states have a responsibility to parents, educators, and the broader public to clearly articulate how their systems will work.

One area where this is particularly problematic is how states address low-performing subgroups of students. Under No Child Left Behind, low performance of any one subgroup carried up to the school’s overall rating. We’ve moved away from that clear-cut rule, first under a set of large-scale waivers issued by the Obama Administration, and now under ESSA. Today, states are proposing systems that combine multiple measures of performance into one overall rating. Six of the 17 first-round states—Connecticut, the District of Columbia, Massachusetts, New Jersey, New Mexico, and Tennessee—deserve recognition for planning to incorporate subgroup performance directly into each school’s rating system. And all of the 17 states, per ESSA’s requirements, have articulated some rules that would identify schools with low-performing subgroups. But only two states—Tennessee and Louisiana—provided data showing what their definitions would mean in practice. The rest gave promises without specificity.

This batch of ESSA plans mark another radical shift. As a country, we’re shifting away from criterion-referenced accountability systems—where all schools are held to the same, pre-determined criteria—to norm-referenced ones, where schools are compared to each other instead of to some external criteria. This change has given states the freedom to adopt more rigorous, and more honest, state standards and assessments, but it has created a disconnect between the standards for students and the standards for schools. As one peer put it, states are now putting “the engine of a Mercedes in a Ford Taurus body.” That is, states have done the hard work of adopting more rigorous standards and more sophisticated assessments, but they’re only holding schools accountable for their place in relative ranking systems. Those systems ignore information on whether or not students are on track to succeed in college and careers.

There is no magic recipe for accountability systems, but the research base on prior efforts suggests that they work best when school leaders understand the system and can respond in kind. Similarly, the research on specific school improvement actions is thin, but there is evidence that the threat of strong interventions can lead schools to improve. Instead of taking the opportunity to design their own school improvement strategies, the state plans are mostly vague and non-specific on how they will support low-performing schools. With the exceptions of New Mexico and Tennessee, states have not yet adequately addressed how they plan to use federal funds to help increase student achievement, increase options for students, or intervene in chronically low-performing schools.

Finally, although it’s not true of all the states in this round, a select few initially submitted plans that were woefully incomplete. There is a good chance that these plans will be improved over time before they earn federal approval. But, at least as originally submitted, they confirmed the worst fears of those who argue that increased flexibility will give states a license to lower expectations, ignore inequities, and give up on the state’s role in meaningfully improving school quality. These incomplete draft plans did a disservice to the rest of the states that put in a good faith effort.

Where We Go from Here

All states in this first round of submissions have room to learn from their peers and continue to improve their plans. Going forward, states can learn from both the strengths and weaknesses from this round of submissions to build better plans that take seriously their obligation to improve education and increase equity.

We encourage the 17 states to use this peer review analysis in concert with the feedback they receive from their internal stakeholders as well as the federal peer review process to continue to strengthen their plans. Even after their plans are approved, states have the opportunity to continuously improve them over time.

We encourage advocates, state departments of education, and governors to use the lessons learned from this peer review process to guide stakeholder engagement conversations and plan development. That’s true of the 17 states in this first round, and doubly true for the 34 states that will submit their plans in September.

This peer review process and final analysis should be viewed as additional guidance from a bipartisan panel of experts and, within the context of each state, can provide important information on how to put in place strong statewide accountability systems that serve all students.