NEW YORK

Promise to Practice Overview

Equity

New York’s ESSA plan focuses on equity, indicating that the state is providing equity training for school boards. The state is also taking steps to address socioeconomic integration in districts through a grant system, and their annual accountability measures include achievement gap issues. Currently, New York lacks a consistent reference to equity in their materials, but it is likely that the equity focus will strengthen in the future.

Strengths

New York is building upon successful past practices in school improvement, which will be approached as something the state will do in partnership with schools and districts, rather than imposing something on them. The state plans to focus school improvement at the local level and sets expectations of family and community engagement, particularly in diverse communities. New York also has a strong base for evidence and data based strategies for school improvement plans.

Improvements

New York’s school improvement strategy, while clearly outlined in their ESSA plan, is not clearly stated across its school improvement documents. The state could benefit from highlighting a single theory of action throughout their school improvement materials. Additionally, New York gives great flexibility to districts and schools in their turnaround efforts, however, there may not be enough accountability from the state to track progress in school improvement.

Click through the tabs on the left to see how Arizona scored in each category.

Coherent and Aligned Vision for Improving Outcomes

How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?

In its ESSA plan, New York implies a theory of action that is primarily grounded in collaboration and support between the state and districts, and builds upon previous success in school improvement in the state. It explains that the state aims to drive school improvement by providing resources and exemplary examples to districts and working collaboratively with them. While New York does not appear to have put a theory of action statement front and center, this approach is implicit throughout the school improvement language in its plan.

New York does employ a strong school improvement framework called the Diagnostic Tool for School and District Readiness (DTSDE). This framework includes a set of six tenets, for which the state has a high quality rubric, and district and school planning guides reference and require that these tenets be addressed in improvement planning. The DTSDE has uses equity-oriented language throughout. New York has a dedicated website to the DTSDE which is reasonably easy to navigate, and includes tutorials, handbooks, and webinars.

Strategic Use of Funding and Alignment of Resources

Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?

New York’s ESSA plan includes a well-documented flow of how Title I school improvement funds will support the schools most in need, including providing technical assistance to schools in targeted and comprehensive support and improvement. The state is also planning to incentivize a number of improvement efforts for schools to consider, such as socioeconomic integration and other competitive grants.

By and large, New York is using a formula model to allocate school improvement funds to schools and districts. The state’s school improvement application meets the letter of the ESSA law, however it reads as a compliance document with no alignment to any state-level strategy. State feedback has indicated that New York is in the process of creating a new school improvement application, which is a positive sign as it will likely do more to incorporate the strong DTSDE framework found elsewhere in school improvement guidance.

New York’s SIPP grant is an area where the state is encouraging schools and districts to pursue a specific strategic direction for school improvement. Under the program, districts may apply for planning and then implementation grants by describing how they will allow low income students in low performing schools to attend nearby schools with better outcomes. Based on the documented structure of this program, it seems likely that New York has made a commitment to integration as a driver of improvement. This program is a promising approach and one to watch as New York continues to implement it. It was noted, however, that while the SIPP grant should be a strong equity lever, it is not explicitly tied to a theory of action or New York’s DTSDE framework.

Overview

Strengths

• New York took a thoughtful and comprehensive approach in creating its ESSA plan, although the state’s plan is quite complicated, particularly around its performance index and school identification.

• The state’s plan balances incentives for academic proficiency and annual student growth, and it includes incentives for schools to pay attention to other components of school quality, such as science, social studies, college and career readiness, chronic absenteeism, and student suspensions.

• The state will use a data dashboard to provide additional contextual information for parents.

• The school support section of New York’s plan is its school is particularly strong. For the last few years, the state has been implementing a system of differentiated school supports utilizing a high-quality observation rubric, which was completed by a team of external reviewers trained to examine school quality and outline tangible steps forward.

• The state provides compelling evidence of stakeholder engagement in the plan, including survey data and interviews with principals who have completed its school improvement process in the past. New York should continue to engage English learner and disability groups to solicit feedback on current accessibility efforts and on additional needs for these student populations.

Weaknesses

• The state’s approach to its long-term goals and interim targets could run the risk of tolerating stagnation.

• Although New York has a relatively straightforward list of accountability indicators, the way it translates those data into school accountability determinations is complex. New York proposes combining its chosen indicators through complicated logic tables with several permutations. School leaders, parents, or other stakeholders may struggle to understand and respond to this system.

• While New York has proposed a set of rules that appear to ensure schools pay attention to low-performing subgroups, it does not provide data modeling the effects of those decisions.

• Additionally, New York’s proposed accountability system is primarily based on how schools compare to each other rather than to objective goals, and schools will have no front-end predictability about what they would need to accomplish to demonstrate sufficient progress.

• The proposed waiver to allow schools to administer below-grade-level assessments will likely undermine efforts to ensure students with disabilities have access to grade-level instruction.

Click through the tabs on the left to see how New York scored in each category.

GOALS

New York established end goals for academics (all students are proficient), its graduation rate (95 percent graduation rate), and English language proficiency (95 percent of English learners are making progress).

The time frame for achieving these end goals is not specified. Instead, the state sets the goal of reducing achievement and graduation-rate gaps by 20 percent within five years. This asks for faster progress from lower-performing groups, but the state did not present any data showing whether its goals are ambitious or attainable. As a result, some of its targets for certain subgroups appear quite low. For example, in the year 2022, New York is aiming for just 56 percent of its English learners and 63 percent of students with disabilities to graduate in four years.

The state deserves credit for setting a high expectation for rapid English language acquisition among English learners of three to five years.

New York could provide a clearer connection between the goals articulated by its state Board of Regents and its proposed school accountability system.

For example, the Board of Regents has established goals for students entering school ready to learn and 3^rd-grade proficiency, but these do not appear to be reflected in the proposed accountability system.

STANDARDS AND ASSESSMENTS

New York has adopted strong academic standards and assessments.

The state has also produced extensive technical reports on the validity and reliability of its assessments, including its alternate assessments for students with disabilities and assessments for English language proficiency. New York deserves credit for translating its math assessments for grades 3- 8 and its high school Regents assessments into five languages.

The state’s approach to looking at language presence and concentration across the state is excellent.

The use of data to specifically identify need, the number of language translations, and the use of oral-language accommodations are all examples of practices other states should take note of and look to replicate in their own unique circumstances.

The state’s descriptions of accommodations for students with special needs is also strong—but the state’s waiver request is concerning.

New York indicates it is preparing a waiver request to allow schools to administer below-grade-level assessments to a small group of students with disabilities. While the waiver request will be dealt with separately from the state’s ESSA plan, there’s a concern that the proposed approach runs contrary to the letter, spirit, and intent of ESSA and could allow students with disabilities to be excluded from the general classroom and instruction.

INDICATORS

New York identified a relatively short list of high-quality indicators.

Elementary and middle schools will be accountable for student achievement, student growth, English language proficiency, a progress measure designed to capture subgroup performance, and chronic absenteeism. At the high school level, New York will measure graduation rates and college and career readiness. New York also deserves credit for incorporating science (and, in high school, social studies) into its accountability system, which will help broaden the accountability focus beyond reading and math.

Rather than proposing one clean measure of achievement, New York is proposing to calculate two performance indicators with different denominators.

One of those calculations, what the state calls “PI-2,” uses a lower denominator of how many students take a test rather than one of how many students were enrolled in the school. If certain student populations are systematically underrepresented among test takers, the PI-2 index could give a false impression of a school’s true performance for all students.

New York could improve its plan by further clarifying how it intends to define its “Progress” measure—whether this is intended to capture subgroup performance or would represent a school-wide average. Similarly, the state says it will give different amounts of points to schools that “meet” and “exceed” its goals, but it may be difficult for parents and educators to distinguish between those two levels.

The state also makes a number of unexplained decisions about how it will convert raw data into scores for its achievement indicator.

After converting student achievement scores into an index on a 250-point scale, the state then ranks each school. Schools in the bottom 10 percent are placed at Achievement Level 1, schools between the 10^th and 50^th percentile are placed at Level 2, and so on. This process has a number of drawbacks. For one, it imposes unnecessary cut points that oversimplify the data. Schools at the 9^th and 11^th percentile are likely similar, but this system will categorize them differently. On the other hand, schools at the 11^th and 49^th percentiles are likely different, but this system would put them in the same category. Second, because this relies entirely on relative rankings of achievement—how one school compares to another, as opposed to how the school compares to a predetermined standard—it fails to give schools any front-end transparency about what they need to shoot for in the coming year.

New York’s articulation of its chronic absenteeism measure is strong.

It includes both excused and unexcused absences, and it’s based on a percentage calculation, which can adjust even if a student changes schools. While New York deserves credit for proposing to phase out-of-school suspensions into its accountability system as a way to combat disparities in disciplinary action, it should monitor its data to ensure that the indicator accomplishes its goal and does not lead to schools artificially deflating their suspension rates. The state also has a smart plan to publish a range of “opportunity to learn” data points for diagnostic purposes.

At the high school level, New York is proposing to equally weight the four-, five-, and six-year graduation rates.

While it’s a worthwhile goal to give schools extra incentive to help all students graduate, New York’s proposal may risk not giving a sufficient incentive for schools to help all students graduate on time. New York is also proposing to include a “College, Career, and Civic Readiness” measure that will award schools additional points if students complete more challenging high school diplomas and earn an endorsement in career and technical education, a “Seal of Biliteracy,” a qualifying score higher on an Advanced Placement (AP) or International Baccalaureate (IB) test, or the receipt of an industry-recognized credential.

While it’s a positive for New York to reward students in several different career paths, it will be important for the state to monitor its data to ensure all these pathways are equally rigorous and predictive of future success, and whether certain groups of students are disproportionately placed into less rigorous pathways.

ACADEMIC PROGRESS

New York plans to use a performance index for its achievement measure that gives schools credit for student performance across four levels, including advanced performance.

Within that index, the state continues to place a strong incentive on students meeting the state’s grade-level standards. New York’s plan says that instead of specific weights, the state will apply a series of decision rules. These rules are only explained in instances where a school scores a 1 in either category. The state could strengthen its plan by building out these decision tables to address other combinations.

New York’s growth model, Student Growth Percentiles (SGP), compares the progress students make against their similarly-performing peers and converts those scores into percentiles.

This approach does not ensure students learn the content they need to stay on track to achieve mastery at graduation. New York does deserve credit for pairing this type of growth model with a relatively clean measure of achievement to balance the incentives for students to reach proficiency benchmarks and make annual progress over time. However, both these indicators suffer from the issues mentioned above regarding the state’s plans to convert raw data into school ratings.

ALL STUDENTS

Although New York says that it will include disaggregated data in its Achievement Index, the plan does not explain how subgroup performance is included.  

In addition, some technical features of New York’s plan will limit its reach. It plans to use a minimum group size of 30 students for measuring performance and 40 students for determining participation rates. Both are high and likely to exclude many students from the state’s system. In fact, less than half of schools with African-American students will have to report this subgroup’s performance, although New York’s plan to include two years of data when the group size is less than 30 should help mitigate this concern.

Still, it’s not clear why New York would need a separate, larger group size for participation rate, and the accuracy of the system could be undermined by having such a high number. Moreover, by not reporting participation rates when a subgroup has fewer than 40 students or accountability scores when a group has fewer than 30 students, New York is unnecessarily limiting information that could be shared with the public.

The plan could be further strengthened if the state provided greater detail on how it will ensure schools and each individual subgroup reach 95 percent participation in the state assessment.

Further, the state’s approach to reporting by combining groups below the five-student threshold for reporting with the next largest group, until they collectively exceed the threshold, will mask or confuse the performance of all the consolidated subgroups. The state could rethink this approach and just mask the performance of groups that fall below the threshold.

New York is planning to take up flexibility to include former English learners in the English learner subgroup, but the plan doesn’t specify how long it would do so. The state is also seeking a waiver to test, but not include for accountability purposes, recently arrived English learners in their second year of attending school in New York.

IDENTIFYING SCHOOLS

Although New York has a relatively simple list of accountability indicators, the way it plans to translate those data into school accountability determinations is far more complicated.

For example, the state will take raw student test scores, convert them into a numeric performance index, and then give each school and subgroup a 1 to 4 numeric score. Although these conversions align to the state’s performance goals, they also include undefined “lower” and “higher” measures of interim progress.

After converting its indicators to 1 to 4 ratings, New York plans to combine them using a logic table showing whether a school would be identified for comprehensive support based on its combination of indicator rankings. The state’s use of a logic table instead of weighting indicators could be cumbersome and confusing. Although New York says these rules will result in the identification of at least 5 percent of schools, it does not provide data about how many schools would fall into these categories or which types of schools are omitted from them. For example, a school that did not score at a level 1 on either achievement or growth could potentially score very low when the two measures are combined. This type of school would not be identified under New York’s rules, but more data are needed to understand if this hypothetical situation exists or not.

Similarly, the state plans to identify high schools with graduation rates below 67 percent, but it will use the four-, five-, and six-year graduation rates to make this determination. This could potentially set a low bar and allow some high schools with very low on-time completion rates to avoid identification.

New York will identify schools for targeted support using the same methodology as above, but applied to subgroup data and performance levels over two consecutive years.

New York deserves credit for identifying “Target Districts” that include one or more schools identified for comprehensive or targeted support or in which district-wide performance would have resulted in identification for support if the district were a school. The plan also alludes to future consideration of additional criteria for identifying Target Districts based on additional data, such as class sizes, incidence of violence, and teacher engagement.

New York’s plan mentions additional categories of “Schools in Good Standing” and “Recognition Schools,” but it does not articulate any rules for how those schools would be identified or, for the latter group, how they would be recognized.

Although not required to be included in ESSA plans, this seems like a missed opportunity for the state to explain the effects of its system beyond the lowest-performing schools in the state.

SUPPORTING SCHOOLS

New York’s documentation of its state support system is the strongest part of the state’s plan.

It is detailed, thorough, and has the potential to succeed. New York plans to offer a differentiated system of supports to schools based on their identified needs and includes a well-thought-out plan for use of evidence-based interventions. Although many states make statements to this effect, New York has put in place a comprehensive process to ensure schools and districts are equipped to navigate the school improvement process.

All schools identified for comprehensive and targeted support must complete a Comprehensive Diagnostic Needs Assessment process that includes a two- to three-day on-site visit from a team of external reviewers trained to look for the school’s “evidence of impact.” The reviews are focused on six tenets of school quality: school leader practices and decisions, curriculum development and support, teacher practices and decisions, student social and emotional developmental health, family and community engagement, and district leadership and capacity.

Going forward, the state will prohibit teachers rated below “Effective” from transferring into schools identified for comprehensive support. The state also has interesting plans to implement a “participatory budgeting process” that will allow parents to help determine how additional funds allocated to comprehensive-support schools should be allocated, to conduct “resource allocation reviews” of districts with significant numbers of schools identified for support, and to identify “Target Districts,” where entire districts are in need of additional support.

New York has also articulated a smart strategy to allocate the 7 percent of its federal funds dedicated to school improvement activities.

All Title I targeted and comprehensive support schools will receive additional funds, with comprehensive-support schools receiving larger shares. If schools make progress, they can qualify for additional funding designed to help them sustain and consolidate their gains. For schools that fail to make progress, the state will provide extra support and technical assistance before providing additional funding.

If a school identified as a comprehensive support school fails to exit status within three years, the school will automatically be placed in the state’s Receivership program (save for some special situations). Similarly, schools that had previously been identified as “priority” schools under the state’s prior accountability system and that are identified as comprehensive-support schools in the initial year will also be placed in the Receivership program, which can eventually trigger a school for conversion to a charter school, placed under the control of the State University of New York or the City University of New York, or cause it to be closed down.

Finally, New York state should indicate if and how it intends to provide direct student services using the optional 3 percent set-aside, which provides an additional opportunity for the state to align school improvement activities with its statewide goals.

Exiting Improvement Status

New York’s proposed accountability system is based on a normative ranking system, and its exit criteria for low-performing schools also follow a similar approach.

The state has proposed two sets of exit criteria for comprehensive support schools. If, for two consecutive years, the school scores at a Level 2 on achievement, growth, and (for high schools) graduation rates, it may exit status. Or, if after three years, New York reruns its lists and the school is no longer identified, it also exits status. The first is stronger than the second and would require a school to improve rather than just improve its relative ranking. However, because several of New York’s accountability indicators are normative rankings and not tied to objective goals, schools would have no front-end predictability about what they would need to accomplish to show sufficient progress.

New York’s exit criteria for targeted-support schools are also normative.

Schools may exit status if they are no longer identified for two consecutive years. The state should clarify whether the performance of the school or subgroups must be higher for two consecutive years for the targets in those years, or the year in which the school/subgroup was first identified. The state does not appear to have addressed any type of sustainability planning, either.

Continuous Improvement

New York should be commended for its extensive and well-documented evidence of stakeholder engagement up to this point in the development of its ESSA plan.

However, the state does not specifically call out a plan to learn from its implementation, modify actions, and have continued consultation with key stakeholders. Further, given the complexity of the state’s proposed accountability system, providing for consistent and in-depth reviews of the performance—and clarity—of the system would be a valuable addition to the plan.

The state should also be applauded for establishing a review committee and for planning to add new indicators and review its accountability system and potentially revise goals after five to seven years.

That said, the plan is less clear about what the state plans to do going forward, and it lacks specificity about its continued engagement with key stakeholders once it begins implementing its ESSA plan. The state would improve its proposal by clearly describing in more detail its process for continual engagement with stakeholders and for modifying its system as necessary.