MASSACHUSETTS

Overview

Strengths

 

  • The Massachusetts plan begins with a compelling and coherent vision of the state’s successes and challenges — resulting in the articulation of five overarching strategies that are meant to drive the state’s plan. The state’s Executive Summary is clear, understandable, and accessible for all audiences.

 

  • Massachusetts is also noteworthy for its strong history of taking actions that have resulted in improvements in the state’s lowest-performing schools and districts. The plan offers useful research, experiences, and insights into what the state’s approaches are, and how and why they’ve worked in the past.

 

  • The state’s plan for Supporting Excellent Educators is also a model worthy of study by other states. It emphasizes how educator supports for early literacy and middle school math, for example, will push the thinking and learning of teachers and principals.

 

Weaknesses

 

  • Massachusetts’ plan does not connect the dots between its long-term vision and how that vision will be embodied in its accountability system. As a result, it misses multiple opportunities to tie these together in meaningful and actionable ways.

 

  • Massachusetts clearly spent a lot of time rethinking its performance index, indicators, and determinations. But due to the lack of data, simulations, and examples, this is still somewhat abstract rather than an actionable plan.

 

  • The state proposes to include stakeholders as it continues refining and implementing its ESSA plan, but the state has not proposed a clear stakeholder engagement plan, and it lacks timelines around many critical decisions.

 

Click through the tabs on the left to see how Massachusetts scored in each category.

GOALS

 

Massachusetts aims to reduce the proficiency gap by one-third over the next six years.

 

Although the state is transitioning to new assessments and the results will not be available until the summer of 2017, the state has provided a sample set of performance targets based on its prior state assessments. Similarly, Massachusetts is waiting for more current data on English language proficiency, and in the meantime has provided placeholder goals that it plans to update in the near future.

 

Massachusetts’ proposed “long-term goal” for graduation rates is set for 2020 — but it would still leave large gaps among different student groups.

 

Encouragingly, the state has set gap-closing graduation rate improvement goals for lower-performing subgroups, and it says the goals are based on historical improvement rates. However, the state does not explain how it derived its 2020 goals, nor provide historical data to demonstrate that they are truly ambitious.

 

Massachusetts does not provide a comprehensive vision or set long-term goals for the future.

 

While Massachusetts does highlight five overarching strategies to support all students, based on clear gaps that the state is currently experiencing, these strategies are not cohesively pulled through the plan as submitted.

STANDARDS AND ASSESSMENTS

 

Massachusetts has had strong standards that align with college readiness, but the state is in the midst of revising those standards.

 

The state has been using the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment, but is in the process of developing and administering a new assessment (derived from PARCC) called the Next-Generation MCAS. Due to the transitions taking place and the lack of transparency around those, it is not certain at this time that Massachusetts’ new standards and assessments will be as strong as they have been historically.

 

Massachusetts does not assess students in any language other than English.

 

This policy is due to a state law passed in 2002 — and excludes the high school mathematics assessment, which is offered in Spanish. Despite this legal obstacle, research indicates that English learners are best served by being permitted to take content-area assessments in their native language.

 

Lastly, Massachusetts should strengthen its plan by ensuring that it has a process in place to meet the 1 percent cap on alternate assessments for students with the most significant cognitive disabilities.

INDICATORS

 

Massachusetts’ plan includes a relatively simple list of high-quality indicators.

 

Massachusetts plans to revisit its indicators when it has its new assessment results in summer 2017, but for now the state’s plan includes: academic achievement, academic progress, English language proficiency, graduation rates, and chronic absenteeism.

 

Within those categories, its school quality indicators appear to be well constructed. The state’s proposed use of “Success in grade 9 courses” is notable, as it will serve as an early warning indicator to help schools and districts identify which students are most likely to drop out, and position them to better respond. To produce this measure, the state proposes to use prior-year student performance data to determine if a student failed one or more courses in high school.

 

Massachusetts also deserves credit for including its science assessment in its accountability system.

 

That decision will help lessen concerns about curriculum narrowing and will force schools to pay attention to student performance beyond reading and math.

 

The plan also includes a measure for the successful completion of “challenging coursework,” but it does not describe how that would be defined. The state may want to consider combining this indicator with access to the courses as an incentive for schools to pay attention to both access and success.

 

The state should place greater weight on its four-year graduation rate.

 

At the high school level, it’s not clear if the state’s proposed inclusion of two separate measures of graduation rates (in addition to an annual dropout rate) are sufficiently discrete to add extra information to the system. The state would have a stronger plan, and more closely align its accountability system with its long-term goals, if it placed greater weight on the four-year graduation rate.

ACADEMIC PROGRESS

 

Massachusetts’ accountability system plans to include both student achievement and growth.

 

It is planning to place a strong weight on both, but neither of its proposed measures is aligned to the state’s grade-level standards for students.

 

The state’s proposed achievement measure is an average scale score.

 

This method combines all student scores into one number, which avoids setting rigid cut points for students, but it can obscure information about how many students are meeting various performance thresholds. Similarly, the state’s growth measure, Student Growth Percentiles, compares students with each other, not with an objective standard. While the state plans to add another growth measure based on whether students are growing at a pace sufficient to meet state standards, the plan provides very little information about how that measure will be used or when it will be phased in.

ALL STUDENTS

 

Massachusetts is clearly wrestling with the question of how to incentivize schools to address the needs of every child with a focus on traditionally disadvantaged students.

 

But with few specifics and little backup data to support its outlined approach, it is difficult to predict its impact.

 

Massachusetts plans to identify any school with a subgroup in the lowest performing 10 percent “of all eligible subgroups” for targeted support.

 

However, it’s not entirely clear from the state’s plan if that applies to each subgroup or if the state is envisioning one composite group based on all subgroups. The former would be a stronger plan and would allow for more tailored interventions than one composite group of “achievement gap” schools, but the state could provide greater clarity about how this identification process will work and how many schools might be identified.

 

Massachusetts plans to include “gap-closing for high-needs students” as part of its accountability system.

 

However, more detail is needed about how it would be weighted in its overall school rating system. The state provides data showing that its “high-needs” subgroup would capture virtually all underserved students in schools where the number of English language learner, economically disadvantaged, and special education students is not high enough to be disaggregated alone. This high-needs subgroup could be a useful addition to subgroup analysis and performance, with the stated impact of adding an additional 150 schools to be held accountable for subgroup performance.

 

The state has not committed to how it will address students with disabilities or English learners.

 

Massachusetts has also not committed either way to including or excluding former students with disabilities within the students-with-disabilities subgroup. Similarly, the state does not commit to how it will approach the inclusion of English learners for either recently arrived English learners or for recently exited English learners for a time frame of two to four years. Nor does it say when it will make a decision. 

IDENTIFYING SCHOOLS

 

Massachusetts has the outline of a plan for how it will rate schools.

 

The state has proposed weights of its indicators in its accountability system, but it plans to revisit those decisions in the coming months. Once the indicators are combined, each school will be assigned to a tier, qualifying it for specific levels of support or intervention.

 

The state has also committed to identifying a number of schools with low-performing subgroups for targeted support.

 

Massachusetts states that it will model out its plan once it has additional data, but for now it’s difficult to know what that will look like.

 

Educators, parents, and communities deserve — and indeed, need — to understand the state’s thinking behind its final weighting system. Further, the state explains that it plans to reset the index ranges each year, but it does not explain how the state would make those changes, why it would do so, or how any changes would be communicated to educators and parents. To drive improvement behaviors, all of this must be transparent.

SUPPORTING SCHOOLS

 

Massachusetts has a strong track record in this area that could serve as a model for other states.

 

Massachusetts has commissioned studies to evaluate the effectiveness of its approaches to school turnaround: a 2014 study called Turnaround Practices in Action and a 2016 quasi-experimental study called Evaluation of Level 4 Turnaround Efforts in Massachusetts. Together, these studies form the evidentiary foundation for the state’s current practices and Massachusetts will continue to embed those principles into competitive grant funds dedicated for school-improvement activities.

 

Massachusetts cites its actions that have resulted in improving the state’s lowest-performing schools and districts.

 

The state has the authority to do full takeovers through receivership, to assign third-party organizations to manage a school on behalf of the department or the district, or to support districts in establishing alternative governance structures for specific schools or clusters of schools such as Empowerment Zones. It has used each of these authorities in different circumstances across the state — all to good effect.

 

For schools and districts that do not require the highest level of intervention, Massachusetts provides resources focused on four evidence-based turnaround practices; it provides direct support to the state’s largest districts through the Commissioner’s Districts programs; it supports small and midsized districts through the District and School Assistance Centers; and three additional offices provide specialized support in such areas as social/emotional learning, partner vetting/selection, and managing/evaluating/monitoring those in receivership.

 

It’s not clear how Massachusetts will extend supports to low-performing schools going forward.

 

Information on this issue wasn’t readily available in the state’s plan. In addition, there is a lack of specificity around exactly how, at what time, or in what stages support will be provided to low-performing schools. There are no examples given of how this impressive array of support will be tailored to specific needs. 

Exiting Improvement Status

 

Massachusetts has not yet finalized its exit its criteria for low-performing schools to exit improvement status.

 

However, it has outlined what those might be once the state has more current data. To exit comprehensive support status, schools must meet achievement, growth, and gap closing targets, and submit “exit assurances” that provide a school’s plan to sustain its efforts going forward.

 

For schools identified for targeted support, they can exit improvement status when they rise above the 20th percentile within the group that led to the school’s identification and if that group meet its annual targets. Although these criteria are still in development, they represent a potentially strong combination of requirements that would demonstrate schools were on track for sustained improvements.

Continuous Improvement

 

Massachusetts emphasizes its commitment to continuous improvement.

 

The state says it will continuously engage stakeholders to develop improvements. It is less clear in the plan where specific stakeholder feedback may have influenced a decision or will influence future decisions.

 

Massachusetts also asserts that it has established intentional processes that allow the state to study the effectiveness of initiatives.

 

The state, however, does not identify the frequency or “the how” behind the ongoing conversations and processes and improvements/adjustments to the overall state plan. It is positive that the plan references that the state agency will focus on implementation, including dedicating some staff to the effort. It is also positive that the state plan will be the basis of the report to the commissioner.

 

What is less clear is who is making decisions and then communicating out broadly on things like annual performance targets or the final decisions on index measures and weighting.

 

Massachusetts could strengthen its plan by articulating a more refined monitoring approach.

 

The state’s theory of improvement is based on the belief that it will set targets for each school and district that requires it to “stretch and continually improve,” but it offers little insight as to how any larger continuous improvement framework will be defined, operated, or resourced.

Learn more about why these categories matter: