It is not clear how North Dakota will be focusing on underserved students or increasing equity in the state’s school improvement efforts. While they expect schools to address subgroup disparities, they lack explanations or evidence that districts will take this seriously.
North Dakota will be providing all schools with continuous improvement support for AdvancED, with targeted support schools getting increased training and professional development.
North Dakota’s current structure for school improvement appears to be mostly based on “boxes checked” versus the quality of the plans proposed. The details asked of schools and districts are mostly surface-level. Additionally, there is little publicly accessible information about how the state is implementing their vision for improvement as outlined in the state’s ESSA plan.
How well does the state’s approach to school improvement include focused attention on supporting underserved students and closing the achievement gap? Does the state require LEAs to maintain an equity focus in their school improvement plans, activities and resource allocations?
North Dakota uses a multi-tiered system of supports as their primary school turnaround framework, and while the mission of the program includes support for all students, it is not clear how North Dakota is focusing their attention on underserved students either through funding or through the state’s school improvement strategy. North Dakota does expect schools with subgroup disparities to discuss their plan to address the gap as part of their school improvement application. However, beyond the initial explanation there is little evidence to indicate that districts will take this seriously, especially given that the district application scoring rubric addresses the extent to which components are present, and not the quality of the proposed applications. This question on the application appears to be the only place where North Dakota brings attention to the achievement gaps in the state and the needs of the underserved populations.
How is the state thoughtfully leveraging ESSA’s flexibility to put in place the necessary policies and procedures that create an enabling environment for effective and sustained school improvement, and that consider state/local lessons learned from past efforts? What parts of the state’s turnaround strategy or guidance to LEAs were strongest or exemplary?
North Dakota is providing each comprehensive and targeted support and intervention school with improvement funding, and all schools receive continuous improvement support from AdvancED. Comprehensive support and improvement schools are assigned coaches who play an active role in leadership training, data coaching, and other professional development and technical assistance designed to improve how the school serves its students. All told, North Dakota seems to provide a suite of third party improvement assistance to its districts and schools. The state also assigns staff liaisons who help to direct resources to districts that need them.
How can the state improve its turnaround efforts? What parts of the state’s strategy or guidance to LEAs were unclear? What risks and challenges might the state face with its current approach?
North Dakota’s ESSA plan presents a compelling vision of a state that drives continuous improvement, and anecdotal accounts from the state corroborate that good work is happening. However, its current structures and documentation for school improvement appear entirely compliance-oriented. For instance, the application and rubric appear to be focused on whether elements of the plan were present, rather than the quality of the plans proposed. The questions and level of detail asked of districts and schools are surface-level and do not prompt deep planning for school improvement.
This is a missed opportunity for districts and schools to use the application as a coherent tool to guide their improvement planning and turnaround efforts. The role of AdvancED and the alignment with the multi-tiered system of supports is unclear, while it could be interwoven throughout the school improvement plan and guidance from the state. The materials available do not fully explain how districts and schools are using the framework and data systems to drive their improvement efforts.
In short, there is too little publicly accessible information about how the state is implementing their vision for improvement as outlined in the state’s ESSA plan.
How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?
There is a significant difference between the vision that can be gleaned from North Dakota’s school improvement application and guidance materials versus the state’s school improvement-related activity in the field. North Dakota adopted the AdvancED Continuous Improvement System (CIS) as their statewide continuous improvement framework. In this model, all schools and districts are expected to consistently, continuously, and holistically evaluate their policies, practices and conditions against research-based standards and factors of school and system quality. This vision was outlined extensively in the state’s ESSA plan.
While this is a promising approach, the materials reviewed did not provide sufficient evidence of this vision being realized in in the actual school improvement process or application. For instance, the state’s application for Title I 1003a funding includes perfunctory mention of some turnaround elements, such as asking districts to describe their strategy and the evidence to support it. However, the application’s questions are surface-level and do not require the districts to differentiate among the issues to which they are responding and how the strategies will support their implementation work.
It is understood that North Dakota is assembling an approach to school improvement new to the state through and partnerships with AdvancED and Ed Direction. Nevertheless, it was difficult to determine how the state was executing on its vision based upon the information available. While good work may be occurring, it remains largely hidden from those who are not a part of it, which is a missed opportunity for the state.
Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?
North Dakota has indicated that it will provide school improvement funds to both comprehensive and targeted schools. If established correctly, North Dakota could set up a process whereby the state’s limited school improvement dollars can be used to support the schools in most need. North Dakota noted that they will be allocating funding through a competitive approach, at least at first, however it is unclear from the available documents what priorities North Dakota is establishing through the competitive process or how they are maximizing the funding in a strategic way. For instance, to access the funding, the district or school simply completes the application and includes a budget by spending category such as staff salaries, travel and supplies.
In the state’s school improvement materials, there is no apparent requirement that districts or schools connect programs they will implement to improve with the proposed budget; the budget sits apart from any plan. The review rubric does ask whether the applicant connected the expenditures to the plan, but because the application asks for a narrative by spending category, it is hard to imagine how this will drive real connections between specific interventions and detailed expenditures.
Is the state applying rigorous criteria and review processes to ensure resources will be used to support effective school improvement efforts? Is the state prioritizing funding to LEAs who demonstrate the greatest need for school improvement funding (including LEAs with a high percentage of CSI and TSI schools) and the strongest commitment to school improvement?
While North Dakota is allocating funding through a competitive approach, the rubric appears quite weak because it asks generic questions and provides generic assessments of the quality of the application. In addition, North Dakota’s rubric weighs all parts of a district’s application equally. As a result, the narrative account of “how the school will develop and implement a plan” is worth 10 points, as is the narrative account of “the process used to recruit, screen, select, and regularly review external providers/contractors to assist the school in making improvements.” The equal weighting seems unlikely to tilt the scales in favor of schools or districts who have given significantly more thought to their plans for school improvement. Additionally, the equal weighting of all factors does not create a system that prioritizes elements that are more important for driving significant and meaningful change.
North Dakota should consider giving more weight on indicator 1, which includes the description of how the plan will be developed and implemented in response to being identified for comprehensive or targeted support, as well as a description of “student performance against the state-determined long-term goals.” Instead, the review process feels like a compliance exercise. It is not clear that the review categories will necessarily lead to a system that prioritizes funding to districts who demonstrate the greatest need.
Again, however, the impression from North Dakota’s school improvement materials is different than that from work occurring on the ground. Feedback from the state indicated that state and district staff connect often during the planning process to discuss feedback and ensure that both district plans match needs and the state knows how it can best support the plan. It seems that most guidance is taking place during one-on-one interactions and is not yet codified and accessible to the field at large.
Does the state have a robust, data-driven process to monitor LEAs’ implementation of the school improvement plans within their district? Did the state establish clear milestones to ensure improvement over time, and within four years?
According to North Dakota’s ESSA plan, there is a clear focus, with support from AdvancED, on continuous improvement. Every school will be supported on its “Improvement Journey” and each will have its own “roadmap” to get there. Feedback from the state indicated support to all districts from AdvancED, not just schools in improvement, though identified schools receive more. In addition, the state contracts with Ed Direction to provide high-touch coaching and technical assistance to all comprehensive support and improvement schools. These partnerships have the making of a strong support system.
It was unclear however, how these supports, improvement journeys and roadmaps are connected to the state’s accountability system or the interim benchmarks established in the ESSA plan. Given that districts and schools were not required to outline their improvement goals in their application, and the rubric does not assess whether or not a district or school established benchmarks for improvement, it is hard to determine how progress will be monitored by AdvancED or the state. On paper, North Dakota has good pieces in place and is requiring local leaders to consider focus on important issues, but it is difficult to see how they are ensuring that the various checklists and rubrics will drive substantive new thinking at the school and district level.
The state described in its feedback that Ed Direction not only provides monthly progress updates to the state, but also engages in quarterly progress monitoring meetings where Ed Direction and state staff discuss and agree on next steps to support districts and schools that are struggling to improve. This has the makings of a strong process for staying updated on progress in the field and directing resources where they are most needed. It will be interesting to learn from North Dakota as this process continues how the state is able to use it to improve the support it provides to the field.
To what extent is the state mandating LEAs use evidence-based strategies in their improvement efforts? Does the state provide guidance and supports to LEAs to help them identify and implement the most effective strategies based upon their needs?
North Dakota’s application asks that districts and schools describe how “proposed interventions and strategies are evidence-based,” and the rubric’s “proficient” criterion says that “the school described how the proposed interventions and strategies are evidence-based. The school demonstrated multiple levels of evidence-based strategies as defined in ESSA.” Asking for the information in this way appears to be a compliance exercise rather than an opportunity to determine whether or not the strategies used are connected to the reason the school was identified.
Based on this documentation, evidence-based interventions seem to play a minimal role in the state’s school improvement work. There is some documentation explaining the different ways in which the state’s education partners can support districts, but nothing that might help a district understand which to choose. Again, without more information about how these resources are used, it is unclear the extent to which the state or AdvancED staff are ensuring districts and schools are helping districts hone in on evidence-based approaches that are connected to their needs.
How well does the state articulate, delineate or set parameters around which interventions and responsibilities belong to the state, LEA and/or school? Does the state provide support or guidance to help LEAs identify and reduce barriers to school improvement? Does the state have a framework or process to support and monitor outside entities who partner with the state, LEAs or schools in school improvement efforts?
While the ESSA-related documentation from North Dakota is currently thin, the state deserves credit for what appears to be the beginnings of a solid set of supports for not only identified schools, but all schools and districts willing to take advantage of them. AdvancED works with all schools, and the state’s regional education agencies also provide assistance.
Identified schools receive dedicated coaching from Ed Direction. North Dakota uses a multi-tiered system of supports (NDMTSS) to support their school improvement efforts. North Dakota notes that this system “is a framework to provide all students with the best opportunities to succeed academically and behaviorally in school. NDMTSS focuses on providing high-quality instruction and interventions matched to student need, monitoring progress frequently to make decisions about changes in instruction or goals.” Through this model, districts and schools are paired with a MTSS coach who works with the identified school to conduct an assessment and develop an improvement plan. The coaches check in with the school leaders three times throughout the year and work with the leaders to evaluate the progress and determine an appropriate path for the following school year. They then report back to the state on progress and also build the capacity of state staff themselves to support school improvement efforts.
The next step will be for the state to codify this support in a way that ensures stakeholders understand what the state is doing to help schools improve and ensure that all students are provided the opportunity to learn. North Dakota should clearly define the role of the coach, the relationship between the coach and district and schools, and the expectations of all parties throughout the coaching process. It is currently unclear, aside from anecdotal description, how the state, district and school work together within the framework provided. In the materials reviewed, there is a lack of intentionality on how the framework is aligned with the school improvement process.
Does the state require LEAs to engage with stakeholders such as parents and community members in the development and implementation of their school improvement plans? Does the state provide sufficient guidance and resources to LEAs to effectively do so, helping them foster local buy-in and promote sustainability?
Community engagement is not a priority in the state’s school improvement plans. The application to receive funding does not ask whether or not stakeholders were involved in the development of the school improvement plan nor is engagement referenced in any of the guidance materials provided to states or as a part of the multi-tiered system of support. While the scoring rubric asks whether “there is significant evidence to demonstrate an assessment of needs at the school, taking into consideration input from all stakeholders,” this appears to be the only reference to stakeholder engagement. In addition, no guidance could be found for districts or schools on how to engage parents and the community in the development of the plans.
Meanwhile North Dakota’s accountability system includes an engagement survey which AdvancED administers, and the organization almost certainly supports districts and schools in improving their survey results if they are underperforming. Again, information available to the public does not describe the activity taking place in the field. Feedback from the state did indicate that it is participating in a parent and family engagement cohort during which it will build a team of stakeholders and community members who will help carry the state’s school improvement message.
Does the state have a plan in place to review the school improvement efforts statewide and evaluate the impact and effectiveness? Does the state have a process in place to support LEAs and schools by enhancing their capacity to maintain their improvement efforts upon exiting identification and intervention?
There was not enough information to evaluate this category.
North Dakota has articulated a vision that “All students will graduate choice ready with the knowledge, skills, and disposition to be successful in whatever they choose to do, whether they pursue a post-secondary degree, enroll in a technical college, enter the workforce, or join the military.”
The state set a goal of reducing the number of nonproficient students by 33 percent in six years. Meeting this goal would still result in a statewide reading proficiency rate of 68 percent, but less than 40 percent among English-language learners.
However, the state also set these goals without analysis of historical data or state trends, so it is difficult to determine whether they are ambitious or attainable.
Its four-year graduation rate goal is also 90 percent for all students and each student group. North Dakota includes GED students in its graduation rate calculation. Including GEDs would put North Dakota out of alignment with other states, out of compliance with federal requirements, and potentially out of alignment with its goal of ensuring that all students graduate “Choice Ready.”
North Dakota has been using Smarter Balanced, which was aligned to the state’s standards. But, as the state revises its standards and chooses a new assessment, many questions remain about process – especially ensuring that the new standards will be rigorous and aligned to college and career readiness, and that the new assessments will be aligned to the standards.
The state could also strengthen its plan by ensuring that it has a process in place to meet the 1 percent cap on alternate assessments for students with the most significant cognitive disabilities.
However, it is not clear what process the state will use to approve assessments proposed by local districts and to ensure districts provide reasonable accommodations to English learners and students with disabilities.
In addition to achievement, growth, and English language proficiency, North Dakota has included two indicators in its accountability system that are worthy of highlighting: student engagement and a “Choice Ready” measure.
This has the potential to capture student perceptions of their school environment, but North Dakota should proceed thoughtfully to ensure the validity and reliability of such an instrument, and ensure that its use and weighting are supported. The state may also want to consider how it will respond and support schools based on what the survey reveals.
This indicator sets forth a common set of expectations for all students, as well as additional requirements specific to each pathway. The state should monitor implementation and ensure that this measure does not lower teacher expectations for certain groups of students, or lower those same students’ expectations of themselves.
The state’s graduation rate indicators have strengths and weaknesses. On the positive side, North Dakota’s decision to include five- and six-year graduation rates will encourage schools to prioritize students who may need additional time in high school. However, it is unclear how the 13 percent weight assigned to graduation rates is distributed across the state’s three proposed measures. North Dakota could further strengthen its plan by weighting the four-year graduation rate more heavily.
It proposes to include a simple measure of student achievement, and a growth model that expects greater progress from lower-performing students. For elementary and middle schools, the state plans to give the same weight to achievement and growth.
North Dakota’s growth model incentivizes both academic growth and academic proficiency. However, the state should consider carefully how it communicates the results to parents to ensure they have the information they need to understand how their child’s school is performing, and how that performance compares with other schools.
At the high school level, the state will weight proficiency at 25 percent, and growth and the “Choice Ready” measure will be worth 22 percent combined. It’s not clear how the 22 percent will be divided, and it’s also not clear how the state will be able to measure academic growth at the high school level if every district can select its own assessment and if the test is administered only once. Given that North Dakota intends to allow high schools to choose their own assessments, it will be critical that these assessments meet validity, reliability, and comparability requirements.
North Dakota has decided to use a growth-to-target model for setting interim student-level goals for English language learners’ annual English acquisition. Factoring in age and grade along with baseline performance data would help determine the most appropriate timeline for language acquisition.
Because there are so many small and rural schools in North Dakota, the state decided to use a small n-size of 10 students. This approach was reinforced through extensive engagement with stakeholders who wanted to make sure every student counted. However, the state could help ensure more subgroups are identified for improvement by changing the proposed confidence interval.
It also proposes to cap the number of schools identified for targeted support at 10 percent of schools. Additionally, North Dakota is proposing to look only at four historically under-performing groups. The state provides compelling data behind its focus on these four groups, but it appears inconsistent with the requirements of ESSA to capture any low-performing group.
North Dakota also proposes to include former students with disabilities in its students-with-disabilities subgroup. Since exiting students tend to have higher performance, the state should monitor its data to ensure it is not masking the performance of students who are still receiving services. On the positive side, the state is measuring English language growth for all K–12 students, which goes beyond ESSA’s requirement to track it only in grades 3-8 and once in high school.
North Dakota will annually identify the lowest-performing 5 percent of Title I schools based on all of the accountability model indicators. These schools will be labeled comprehensive support schools.
North Dakota will identify an additional 10 percent of schools from across the state for targeted support and improvement. Five percent of schools with the lowest subgroup performance will be identified, as well as the 5 percent of schools with the largest gaps in student achievement.
But North Dakota must analyze its data to ensure that it is identifying those schools with low-performing subgroups who are not making progress toward the long-term goal of reducing the percentage of nonproficient students, and to ensure that its 5 percent caps are not protecting schools with low-performing groups that would otherwise be identified.
Additionally, North Dakota proposes to wait until the 2019-2020 school year to identify targeted schools, but ESSA requires the first group to be identified in 2018-19.
Finally, it is worth pointing out that North Dakota identifies only three categories of schools: general, targeted, and comprehensive. This misses the opportunity to identify, highlight, and learn from the state’s highest-performing schools.
While school improvement funds are being made available to identified schools, the constellation of supports is not comprehensive, and many of the state’s planned interventions lack specificity. The state also does not specify any more rigorous interventions for those schools that fail to exit comprehensive support and improvement status.
The model includes a needs assessment, training in developing plans, professional development, data analysis support, monitoring, and access to research based strategies for student support. Unfortunately, it is not clear how these supports will be accessed or implemented.
North Dakota plans to distribute its 7 percent set-aside of funds dedicated for school-improvement strategies through a formula. Since the state has opted not to use competitive funding, it should at least consider using a rigorous process for reviewing and approving these improvement plans.
North Dakota sets reasonable exit criteria that require schools to meet predetermined expectations that directly relate to why schools were in need of corrective action for three consecutive years. Determinations of which schools are identified for comprehensive support are revisited every three years.
Schools exit when they no longer meet the eligibility criteria or meet their interim goals for the identified subgroup for two consecutive years. They appear to be revisited annually, raising the concern that schools could bounce in and out of status.
The state will keep its ESSA Planning Committee intact after its plan is submitted, and the committee will work with the state Department of Education to monitor progress toward statewide goals, and review and revise the state’s plan as needed.
Schools will work with AdvancED to develop their own improvement plans and submit them to the state. The success of this partnership is yet to be determined, but it is crucial to North Dakota’s implementation of ESSA. It is an interesting model and one worth watching and learning from.