New York’s ESSA plan focuses on equity, indicating that the state is providing equity training for school boards. The state is also taking steps to address socioeconomic integration in districts through a grant system, and their annual accountability measures include achievement gap issues. Currently, New York lacks a consistent reference to equity in their materials, but it is likely that the equity focus will strengthen in the future.
New York is building upon successful past practices in school improvement, which will be approached as something the state will do in partnership with schools and districts, rather than imposing something on them. The state plans to focus school improvement at the local level and sets expectations of family and community engagement, particularly in diverse communities. New York also has a strong base for evidence and data based strategies for school improvement plans.
New York’s school improvement strategy, while clearly outlined in their ESSA plan, is not clearly stated across its school improvement documents. The state could benefit from highlighting a single theory of action throughout their school improvement materials. Additionally, New York gives great flexibility to districts and schools in their turnaround efforts, however, there may not be enough accountability from the state to track progress in school improvement.
How well does the state’s approach to school improvement include focused attention on supporting underserved students and closing the achievement gap? Does the state require LEAs to maintain an equity focus in their school improvement plans, activities and resource allocations?
New York’s ESSA plan focuses on equity and ending segregation inequities, and indicates that the state is providing training for school boards on equity. One of the most powerful elements is the plan to create an Equity Report to shine a light on inequities in the state, particularly with regard to teacher effectiveness. New York also created their Socioeconomic Integration Pilot Program (SIPP) grant to help districts tackle socioeconomic integration and was very transparent about the problem the State has in this area and within particular schools. Annual accountability measures include areas that have demonstrated most achievement gap issues, and the state has done substantial messaging through its Equity in ESSA campaign.
However, because New York is still in the process of creating guidance aligned with its ESSA plan, the state’s focus on equity is not yet present in all of the materials reviewed. There are few references to equity in the state’s existing school improvement application, nor is there a strong link between the data examined and the any strategies selected. That said, the state’s SIPP grant does include a statement of purpose describing how it is intended as an equity tool, and communication from the state has included a strong equity focus. From New York’s ESSA plan and communications, it is likely that the equity focus in the school improvement materials themselves will strengthen in the future. New York could do a better job by compiling all ESSA documents in one location for ease of public consumption.
How is the state thoughtfully leveraging ESSA’s flexibility to put in place the necessary policies and procedures that create an enabling environment for effective and sustained school improvement, and that consider state/local lessons learned from past efforts? What parts of the state’s turnaround strategy or guidance to LEAs were strongest or exemplary?
According to its ESSA plan, New York is building upon successful past practices in school improvement. School improvement will be approached as something the state will do in partnership with schools and districts, rather than imposing something on them. The state has an extensive needs assessment and adds state supports for turnaround. The state also aims to move school improvement to be focused at the local level and articulates the expectation of family and community engagement, in particular culturally and linguistically diverse families, in the school improvement process.
New York asks for extensive data analysis from its districts and schools in building school improvement plans, and requires that districts and school link the strategies they have chosen for school improvement to the evidence that led them to take that course of action. The state’s Diagnostic Tool for School and District Readiness (DTSDE) is quite strong, and most of the state’s school improvement materials are aligned to it.
The state has also invested considerable resources into creating documents to help parents and community members what to expect in the ESSA era.
How can the state improve its turnaround efforts? What parts of the state’s strategy or guidance to LEAs were unclear? What risks and challenges might the state face with its current approach?
The state’s DTSDE school improvement framework is high quality but it is not reflected its current school improvement application. While feedback from the state was helpful in understanding its current status and progress implementing its ESSA plan, there remains room for improvement by highlighting one clear theory of action throughout New York’s school improvement materials.
New York has given districts and schools greater autonomy and flexibility while grounding its turnaround efforts using collaboration, partnership and support, assuming that this will help schools improve and address achievement gaps. Because plans set one year targets only, districts and schools in improvement may perceive a moving target.
How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?
In its ESSA plan, New York implies a theory of action that is primarily grounded in collaboration and support between the state and districts, and builds upon previous success in school improvement in the state. It explains that the state aims to drive school improvement by providing resources and exemplary examples to districts and working collaboratively with them. While New York does not appear to have put a theory of action statement front and center, this approach is implicit throughout the school improvement language in its plan.
New York does employ a strong school improvement framework called the Diagnostic Tool for School and District Readiness (DTSDE). This framework includes a set of six tenets, for which the state has a high quality rubric, and district and school planning guides reference and require that these tenets be addressed in improvement planning. The DTSDE has uses equity-oriented language throughout. New York has a dedicated website to the DTSDE which is reasonably easy to navigate, and includes tutorials, handbooks, and webinars.
Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?
New York’s ESSA plan includes a well-documented flow of how Title I school improvement funds will support the schools most in need, including providing technical assistance to schools in targeted and comprehensive support and improvement. The state is also planning to incentivize a number of improvement efforts for schools to consider, such as socioeconomic integration and other competitive grants.
By and large, New York is using a formula model to allocate school improvement funds to schools and districts. The state’s school improvement application meets the letter of the ESSA law, however it reads as a compliance document with no alignment to any state-level strategy. State feedback has indicated that New York is in the process of creating a new school improvement application, which is a positive sign as it will likely do more to incorporate the strong DTSDE framework found elsewhere in school improvement guidance.
New York’s SIPP grant is an area where the state is encouraging schools and districts to pursue a specific strategic direction for school improvement. Under the program, districts may apply for planning and then implementation grants by describing how they will allow low income students in low performing schools to attend nearby schools with better outcomes. Based on the documented structure of this program, it seems likely that New York has made a commitment to integration as a driver of improvement. This program is a promising approach and one to watch as New York continues to implement it. It was noted, however, that while the SIPP grant should be a strong equity lever, it is not explicitly tied to a theory of action or New York’s DTSDE framework.
Is the state applying rigorous criteria and review processes to ensure resources will be used to support effective school improvement efforts? Is the state prioritizing funding to LEAs who demonstrate the greatest need for school improvement funding (including LEAs with a high percentage of CSI and TSI schools) and the strongest commitment to school improvement?
New York describes a solid identification process for comprehensive and targeted schools, including requiring a comprehensive diagnostic needs assessment for comprehensive schools. All schools identified for improvement will receive funding. As noted before, the state’s DTSDE rubric is also strong. New York is looking to expand its Needs Assessment process beyond the DTSDE rubric and on-site review process, so that it now incorporates a review of resources and data, however, at the time of this review, no guidance about these additional components has been released publicly.
Based on the materials available, it is difficult to see how New York would be able to apply any rigorous review process or its DTSDE rubric to its school improvement applications and allocations. The school improvement application seems compliance oriented and, while it does reference alignment between activities and identified needs once, no it is not aligned to the DTSDE needs assessment. Again, a new application appears to be in development that should help to address this issue. New York’s SIPP grant does include a two-question needs assessment attachment, but it neither a comprehensive tool nor used by all schools and districts in improvement.
Each school articulates their coherent plan for improvement via a School Comprehensive Education Plan – not their application for Title I School Improvement funds under Section 1003. School Improvement applications are designed to focus narrowly on those portions of a school’s improvement plans that will be directly supported with those federal resources. In many cases, LEAs are using additional resources well beyond the improvement funds to support the improvement efforts that are not fully detailed in the application for funds. When completing the review of school improvement applications, reviewers leverage DTSDE reviews, improvement plans, and the district’s Consolidated Application to provide feedback and help focus the use of funds. In addition, schools and districts are prompted to set targets for one year, but not beyond, meaning the state will not be able to determine whether local goals are in alignment with New York’s ESSA plan.
Does the state have a robust, data-driven process to monitor LEAs’ implementation of the school improvement plans within their district? Did the state establish clear milestones to ensure improvement over time, and within four years?
New York’s ESSA plan explains that the state requires a progress review to assess the implementation of school improvement plans, review of state reported data, a resource allocation review, and a review of parent, staff, and teacher surveys. Feedback from the state indicated that all comprehensive schools are expected to receive on-site support during the first year. New York also explained that it plans to withhold funds from schools that fail to make gains until further technical assistance is provided. Schools that still fail to improve can be placed in the state’s receivership program, through which the state provides intense supports and oversight.
New York’s school current improvement application prompts districts and schools to examine a strong selection of data points in order to make a self-judgment on progress and articulate next steps. For example, districts and schools must review disaggregated student outcome data, both student and teacher attendance, and the number of discipline referrals. An attachment to this application asks for even more data, including both “leading indicators,” such as chronic absenteeism and teacher quality, and academic indicators. Setting aside some confusion over school improvement grant continuation versus a potential new comprehensive and targeted school identification under ESSA, it is clear that New York has put serious thought into the kinds of data in which the state expects districts and schools to plan for improvement.
New York’s school improvement strategy is to support local districts to improve their schools. Given the wide variety of implementation efforts, it is important that New York ensure schools are effectively utilizing the data to effectively monitor and evaluate their progress. In addition, districts and schools must only analyze historical data, not set targets for the future. This could be a missed opportunity for districts and schools to not only look backwards at existing trends but also be forward-thinking about how they want these data to show improvement. Again, a new school improvement application is forthcoming.
The state plans to use its DTSDE framework to monitor progress against improvement plans, and feedback indicated that teams of reviewers will visit schools as part of monitoring. The state’s SIPP grant application asks districts to self-report on progress, but no information could be found describing how the state will come to a conclusion on progress and to make decisions about future support and funding.
To what extent is the state mandating LEAs use evidence-based strategies in their improvement efforts? Does the state provide guidance and supports to LEAs to help them identify and implement the most effective strategies based upon their needs?
According to New York’s ESSA plan, the State requires at least one evidence-based intervention and has identified evidence-based school improvement best practices for schools and districts. The state’s plan also identifies technical assistance providers that districts can access for support in evidence based interventions. New York provides a sizeable resource guide to districts which contains potential strategies for district leaders to consider, aligned with the six tenets of the state’s DTSDE school improvement framework. This document appears to be a strong resource, as district leaders can use their DTSDE needs assessment to find strategies that will help target areas for improvement. While the link between identified needs and chosen strategies is therefore potentially strong, the current school improvement application will need to be revised to include this alignment as well.
How well does the state articulate, delineate or set parameters around which interventions and responsibilities belong to the state, LEA and/or school? Does the state provide support or guidance to help LEAs identify and reduce barriers to school improvement? Does the state have a framework or process to support and monitor outside entities who partner with the state, LEAs or schools in school improvement efforts?
In its ESSA plan, New York articulates the different performance categories of schools and the interventions and responsibilities for each type of identified school, giving the schools autonomy and relying on them to be their own change agents. The state describes eight ways in which it will support comprehensive and targeted support and intervention schools, and makes it clear that schools and principals are responsible for improvement, and the state is a critical support. The state requires a review of principal capacity in identified schools as well.
While New York has publicly posted an impressive array of public-facing documents explaining the state’s plan under ESSA to the layperson, surprisingly nothing was found articulating and delineating school improvement roles and responsibilities for the state, districts, and schools. A brief document explaining these responsibilities to a district audience would be helpful.
The state does not appear to take a position on the extent to which local districts and schools should partner with external providers. It would be helpful for districts to know which providers they may consider, how to evaluate them, and the funding streams that can be used to pay for those services. Feedback from the State indicated that entities partnering with the school will use the DTSDE rubric to guide how it works with schools, and while the State ESSA plan proposes offering technical assistance on school improvement spending to schools that do not make progress, it is unclear at the moment how the State will assist all identified schools and districts in determining the effectiveness of outside entities used for school improvement.
Does the state require LEAs to engage with stakeholders such as parents and community members in the development and implementation of their school improvement plans? Does the state provide sufficient guidance and resources to LEAs to effectively do so, helping them foster local buy-in and promote sustainability?
New York’s ESSA plan is clear about requiring annual parent, community and stakeholder engagement, as the use of an annual survey to gauge progress on engagement. It specifically articulates how the engagement of culturally and linguistically diverse communities is required and especially important. Expectations for creating safe and supporting learning environments were identified in order to promote local engagement, including the use of culturally responsive education. The state has certainly encouraged public engagement with school improvement in the ESSA era through its explanation documents designed for parents and community members.
While the state’s ESSA plan is strong in how it incorporates stakeholder engagement, it is yet unknown how this focus will translate to the field. When implemented, there may be concern regarding whether districts would have the capacity to engage in stakeholder engagement to the level described in the plan and, if they do not, what will occur. Feedback from the state indicates that further stakeholder engagement supports are being developed with involvement from stakeholders themselves, and that students will be required to participate in the school improvement planning process.
Does the state have a plan in place to review the school improvement efforts statewide and evaluate the impact and effectiveness? Does the state have a process in place to support LEAs and schools by enhancing their capacity to maintain their improvement efforts upon exiting identification and intervention?
While New York plans to have strong supports while a school and districts are in school improvement and additional technical assistance and oversight for its lowest performing identified schools, this model puts responsibility on the school and district to improve, and it is unclear how the state will evaluate the supports it provides to schools struggling to make gains. The state does describe in its ESSA plan that schools that make gains for two consecutive years will receive a supplemental allocation designed to assist the school in transitioning to improvement efforts that can be sustained, which is a promising approach. It seems the state has a strong school improvement framework in place on which to build an approach to long-term sustainability.
The time frame for achieving these end goals is not specified. Instead, the state sets the goal of reducing achievement and graduation-rate gaps by 20 percent within five years. This asks for faster progress from lower-performing groups, but the state did not present any data showing whether its goals are ambitious or attainable. As a result, some of its targets for certain subgroups appear quite low. For example, in the year 2022, New York is aiming for just 56 percent of its English learners and 63 percent of students with disabilities to graduate in four years.
The state deserves credit for setting a high expectation for rapid English language acquisition among English learners of three to five years.
For example, the Board of Regents has established goals for students entering school ready to learn and 3rd-grade proficiency, but these do not appear to be reflected in the proposed accountability system.
The state has also produced extensive technical reports on the validity and reliability of its assessments, including its alternate assessments for students with disabilities and assessments for English language proficiency. New York deserves credit for translating its math assessments for grades 3- 8 and its high school Regents assessments into five languages.
The use of data to specifically identify need, the number of language translations, and the use of oral-language accommodations are all examples of practices other states should take note of and look to replicate in their own unique circumstances.
New York indicates it is preparing a waiver request to allow schools to administer below-grade-level assessments to a small group of students with disabilities. While the waiver request will be dealt with separately from the state’s ESSA plan, there’s a concern that the proposed approach runs contrary to the letter, spirit, and intent of ESSA and could allow students with disabilities to be excluded from the general classroom and instruction.
Elementary and middle schools will be accountable for student achievement, student growth, English language proficiency, a progress measure designed to capture subgroup performance, and chronic absenteeism. At the high school level, New York will measure graduation rates and college and career readiness. New York also deserves credit for incorporating science (and, in high school, social studies) into its accountability system, which will help broaden the accountability focus beyond reading and math.
One of those calculations, what the state calls “PI-2,” uses a lower denominator of how many students take a test rather than one of how many students were enrolled in the school. If certain student populations are systematically underrepresented among test takers, the PI-2 index could give a false impression of a school’s true performance for all students.
New York could improve its plan by further clarifying how it intends to define its “Progress” measure—whether this is intended to capture subgroup performance or would represent a school-wide average. Similarly, the state says it will give different amounts of points to schools that “meet” and “exceed” its goals, but it may be difficult for parents and educators to distinguish between those two levels.
After converting student achievement scores into an index on a 250-point scale, the state then ranks each school. Schools in the bottom 10 percent are placed at Achievement Level 1, schools between the 10th and 50th percentile are placed at Level 2, and so on. This process has a number of drawbacks. For one, it imposes unnecessary cut points that oversimplify the data. Schools at the 9th and 11th percentile are likely similar, but this system will categorize them differently. On the other hand, schools at the 11th and 49th percentiles are likely different, but this system would put them in the same category. Second, because this relies entirely on relative rankings of achievement—how one school compares to another, as opposed to how the school compares to a predetermined standard—it fails to give schools any front-end transparency about what they need to shoot for in the coming year.
It includes both excused and unexcused absences, and it’s based on a percentage calculation, which can adjust even if a student changes schools. While New York deserves credit for proposing to phase out-of-school suspensions into its accountability system as a way to combat disparities in disciplinary action, it should monitor its data to ensure that the indicator accomplishes its goal and does not lead to schools artificially deflating their suspension rates. The state also has a smart plan to publish a range of “opportunity to learn” data points for diagnostic purposes.
While it’s a worthwhile goal to give schools extra incentive to help all students graduate, New York’s proposal may risk not giving a sufficient incentive for schools to help all students graduate on time. New York is also proposing to include a “College, Career, and Civic Readiness” measure that will award schools additional points if students complete more challenging high school diplomas and earn an endorsement in career and technical education, a “Seal of Biliteracy,” a qualifying score higher on an Advanced Placement (AP) or International Baccalaureate (IB) test, or the receipt of an industry-recognized credential.
While it’s a positive for New York to reward students in several different career paths, it will be important for the state to monitor its data to ensure all these pathways are equally rigorous and predictive of future success, and whether certain groups of students are disproportionately placed into less rigorous pathways.
Within that index, the state continues to place a strong incentive on students meeting the state’s grade-level standards. New York’s plan says that instead of specific weights, the state will apply a series of decision rules. These rules are only explained in instances where a school scores a 1 in either category. The state could strengthen its plan by building out these decision tables to address other combinations.
This approach does not ensure students learn the content they need to stay on track to achieve mastery at graduation. New York does deserve credit for pairing this type of growth model with a relatively clean measure of achievement to balance the incentives for students to reach proficiency benchmarks and make annual progress over time. However, both these indicators suffer from the issues mentioned above regarding the state’s plans to convert raw data into school ratings.
In addition, some technical features of New York’s plan will limit its reach. It plans to use a minimum group size of 30 students for measuring performance and 40 students for determining participation rates. Both are high and likely to exclude many students from the state’s system. In fact, less than half of schools with African-American students will have to report this subgroup’s performance, although New York’s plan to include two years of data when the group size is less than 30 should help mitigate this concern.
Still, it’s not clear why New York would need a separate, larger group size for participation rate, and the accuracy of the system could be undermined by having such a high number. Moreover, by not reporting participation rates when a subgroup has fewer than 40 students or accountability scores when a group has fewer than 30 students, New York is unnecessarily limiting information that could be shared with the public.
Further, the state’s approach to reporting by combining groups below the five-student threshold for reporting with the next largest group, until they collectively exceed the threshold, will mask or confuse the performance of all the consolidated subgroups. The state could rethink this approach and just mask the performance of groups that fall below the threshold.
New York is planning to take up flexibility to include former English learners in the English learner subgroup, but the plan doesn’t specify how long it would do so. The state is also seeking a waiver to test, but not include for accountability purposes, recently arrived English learners in their second year of attending school in New York.
For example, the state will take raw student test scores, convert them into a numeric performance index, and then give each school and subgroup a 1 to 4 numeric score. Although these conversions align to the state’s performance goals, they also include undefined “lower” and “higher” measures of interim progress.
After converting its indicators to 1 to 4 ratings, New York plans to combine them using a logic table showing whether a school would be identified for comprehensive support based on its combination of indicator rankings. The state’s use of a logic table instead of weighting indicators could be cumbersome and confusing. Although New York says these rules will result in the identification of at least 5 percent of schools, it does not provide data about how many schools would fall into these categories or which types of schools are omitted from them. For example, a school that did not score at a level 1 on either achievement or growth could potentially score very low when the two measures are combined. This type of school would not be identified under New York’s rules, but more data are needed to understand if this hypothetical situation exists or not.
Similarly, the state plans to identify high schools with graduation rates below 67 percent, but it will use the four-, five-, and six-year graduation rates to make this determination. This could potentially set a low bar and allow some high schools with very low on-time completion rates to avoid identification.
New York deserves credit for identifying “Target Districts” that include one or more schools identified for comprehensive or targeted support or in which district-wide performance would have resulted in identification for support if the district were a school. The plan also alludes to future consideration of additional criteria for identifying Target Districts based on additional data, such as class sizes, incidence of violence, and teacher engagement.
Although not required to be included in ESSA plans, this seems like a missed opportunity for the state to explain the effects of its system beyond the lowest-performing schools in the state.
It is detailed, thorough, and has the potential to succeed. New York plans to offer a differentiated system of supports to schools based on their identified needs and includes a well-thought-out plan for use of evidence-based interventions. Although many states make statements to this effect, New York has put in place a comprehensive process to ensure schools and districts are equipped to navigate the school improvement process.
All schools identified for comprehensive and targeted support must complete a Comprehensive Diagnostic Needs Assessment process that includes a two- to three-day on-site visit from a team of external reviewers trained to look for the school’s “evidence of impact.” The reviews are focused on six tenets of school quality: school leader practices and decisions, curriculum development and support, teacher practices and decisions, student social and emotional developmental health, family and community engagement, and district leadership and capacity.
Going forward, the state will prohibit teachers rated below “Effective” from transferring into schools identified for comprehensive support. The state also has interesting plans to implement a “participatory budgeting process” that will allow parents to help determine how additional funds allocated to comprehensive-support schools should be allocated, to conduct “resource allocation reviews” of districts with significant numbers of schools identified for support, and to identify “Target Districts,” where entire districts are in need of additional support.
All Title I targeted and comprehensive support schools will receive additional funds, with comprehensive-support schools receiving larger shares. If schools make progress, they can qualify for additional funding designed to help them sustain and consolidate their gains. For schools that fail to make progress, the state will provide extra support and technical assistance before providing additional funding.
If a school identified as a comprehensive support school fails to exit status within three years, the school will automatically be placed in the state’s Receivership program (save for some special situations). Similarly, schools that had previously been identified as “priority” schools under the state’s prior accountability system and that are identified as comprehensive-support schools in the initial year will also be placed in the Receivership program, which can eventually trigger a school for conversion to a charter school, placed under the control of the State University of New York or the City University of New York, or cause it to be closed down.
Finally, New York state should indicate if and how it intends to provide direct student services using the optional 3 percent set-aside, which provides an additional opportunity for the state to align school improvement activities with its statewide goals.
The state has proposed two sets of exit criteria for comprehensive support schools. If, for two consecutive years, the school scores at a Level 2 on achievement, growth, and (for high schools) graduation rates, it may exit status. Or, if after three years, New York reruns its lists and the school is no longer identified, it also exits status. The first is stronger than the second and would require a school to improve rather than just improve its relative ranking. However, because several of New York’s accountability indicators are normative rankings and not tied to objective goals, schools would have no front-end predictability about what they would need to accomplish to show sufficient progress.
Schools may exit status if they are no longer identified for two consecutive years. The state should clarify whether the performance of the school or subgroups must be higher for two consecutive years for the targets in those years, or the year in which the school/subgroup was first identified. The state does not appear to have addressed any type of sustainability planning, either.
However, the state does not specifically call out a plan to learn from its implementation, modify actions, and have continued consultation with key stakeholders. Further, given the complexity of the state’s proposed accountability system, providing for consistent and in-depth reviews of the performance—and clarity—of the system would be a valuable addition to the plan.
That said, the plan is less clear about what the state plans to do going forward, and it lacks specificity about its continued engagement with key stakeholders once it begins implementing its ESSA plan. The state would improve its proposal by clearly describing in more detail its process for continual engagement with stakeholders and for modifying its system as necessary.