MONTANA

OVERVIEW

Strengths

• Montana has a strong system for standards and assessments: The state uses Common Core-aligned standards and Smarter Balanced assessments.

• In addition, the state has chosen a potentially comprehensive and useful set of indicators that includes an assessment of science in K-8 schools and college and career readiness in high school.

• The state also deserves credit for seeking input from stakeholders and selecting a low minimum group size of 10 for student subgroups in order to include more schools and students in the accountability system. Montana’s approach to providing structure, process, and assistance to schools in comprehensive support and improvement also appears promising.

• Finally, the state was thoughtful in describing how it will support and improve school conditions and transitions.

Weaknesses

• Montana’s plan does not provide sufficient detail or description about the proposed accountability system, how it will be integrated with other state initiatives, or—most importantly—how it will substantially improve student achievement and close performance gaps.

• The state does not set a high bar for academic proficiency or growth. The long-term goals for proficiency appear low, especially for the subgroups that are already the furthest behind.

• Montana proposes goals that would narrow gaps, but still leave large gaps intact.

• The state perhaps places too much weight on indicators such as chronic absenteeism, while deemphasizing academic achievement.

• Critically, the state’s system for identifying schools is relative instead of objective, which means schools can be perceived as improving merely because other schools are declining, irrespective of their own progress or lack thereof.

• Generally, the state’s plan lacks detail in a number of critical areas. It does not appear that the state sufficiently analyzed data or provided enough detail to fully understand the implications of the various choices it is making.

Click through the tabs on the left to see how Montana scored in each category.

GOALS

Montana’s vision, “Putting Montana Students First,” is rooted in local control and flexibility.

The plan says the state will focus on serving its diverse student population. However, the state’s goals do not accelerate growth quickly enough for all students in order to meet the state’s vision.

Montana has a six-year long-term proficiency goal, which expects schools to reduce the number of students who are not proficient by 4 percent each year. Montana does not explain its rationale for the 4 percent reduction goal structure. While these goals would still leave large gaps intact, they would, if achieved, effectively double the proficiency of the state’s lowest-performing subgroups in just six years.
 

The state sets a goal for English language arts proficiency for all students that would only raise achievement by 11 percent over six years—less than 2 percent per year.

Groups that are the furthest behind, such as English learners, children with disabilities, and American Indians, who would reach proficiency rates of 29 percent, 37 percent, and 39 percent respectively, are all still very low. Even the highest-achieving subgroup, white students (the largest group by far in the state), would only increase proficiency from 55 percent to 65 percent, meaning one in three of these students would still not reach proficiency. The numbers in math are substantially lower and, at the end of the process, two out of three English learners (the lowest-performing subgroup) would not be proficient and one out of three white students (the highest-achieving subgroup) would also not be proficient.

Montana uses the same 4 percent calculation for its graduation rate goal. This goal would only modestly raise the state’s overall graduation rate over six years and only slightly narrow graduation rate gaps.

The state’s goals for English learners are slightly more aggressive, with expectations set that English language proficiency will increase by 15 percentage points over six years.

STANDARDS AND ASSESSMENTS

Montana has put in place strong college- and career-ready standards based on the Common Core State Standards and is using Smarter Balanced Assessment Consortium as its statewide assessment system in English language arts and math in grades 3-8.

The state is developing a new science assessment aligned to the state-adopted Next Generation Science Standards (NGSS) that will be implemented in 2020, which is commendable.

Montana requires all high school juniors to take the ACT, which allows all students to take a college-ready entrance exam.

Montana is currently creating performance-level descriptors for the ACT, as well as beginning a standards validation process to ensure that the ACT is a valid measure of the state’s content standards. For example, it would be beneficial for the state to provide a timeline to achieve its stated goal of completing the standards validation and performance-level descriptors in time for testing in the 2017-18 school year and school identifications for the 2018-19 school year. It is unclear if the state provides sufficient accommodations on those tests to English learners and students with disabilities.

The plan could be strengthened by addressing how students with the most significant cognitive disabilities will participate in the state’s assessment system using an alternate assessment. Further, the state should provide the steps it will take to ensure that it does not exceed the 1 percent cap on participation in those alternate assessments.

INDICATORS

Montana has chosen a lengthy list of potentially high-quality indicators to include in its accountability system.

They include: academic achievement, academic growth, English-language acquisition, graduation rates, science proficiency, chronic absenteeism, college and career preparedness, and school quality measures.

The state deserves credit for including its science assessment (currently the CRT, changing to the new NGSS test in 2020) in its accountability system, which will help address concerns about curriculum narrowing. This decision signals schools to pay attention to student performance beyond reading and math. Because Montana mentions STEM education as a goal in the opening of its plan, it could go even further to ensure that the indicator measures the “TEM” (technology, engineering, and math) of STEM.

Montana’s weight for its satisfactory attendance indicator seems high. The state counts chronic absenteeism as 20 percent of the summative accountability score for K-8 schools and 15 percent for high schools. It is only five points lower than the academic achievement weight, giving it almost the same importance. This approach diminishes the focus on academic performance and may permit schools to improve their ratings without improving achievement or closing gaps.

Montana took an innovative approach to its school quality indicator.

Rather than selecting a single measure, the state provides a menu of indicators covering a variety of important areas that vary between elementary, middle, and high school, including school climate, student behavior, and student engagement. Montana chose this indicator to show the state’s commitment to creating positive learning environments, using data effectively, and implementing evidence-based practices with fidelity. However, the list of factors contributing to such an indicator is vague and may be difficult to quantify. It will be critical to ensure that the factors included here are ultimately compliant with the law’s requirement for the school quality indicator to be valid, reliable, and comparable across all schools and districts statewide.

Montana’s “college and career preparedness” indicator is noteworthy and should help more students across the state access these opportunities.

The state proposes reporting the percentage of students who are postsecondary-ready, as evidenced by demonstrating college readiness (earning at least one of these: a college-ready benchmark on the ACT, completion of a dual enrollment course, college credit via the Advanced Placement or International Baccalaureate exam, or a concentration in career and technical education). Including participation and a passing grade in an AP or IB course is a positive step that could yield powerful information about student postsecondary preparedness—but including performance on an AP or IB exam would be a more accurate indicator of college and career preparedness.

ACADEMIC PROGRESS

Montana could weight academic achievement and growth as more significant factors in its accountability system.

Achievement is weighted at 25 percent and growth is weighted at 30 percent, which cumulatively account for just over half of the overall score. Proficiency on the science assessment counts for another 10 percent, but that is still only 65 percent of the total school accountability system.

Because Montana’s weighting of the growth indicator for K-8 is greater than proficiency, it provides an incentive to focus on growth. It appears the state may be using relative scoring of growth that assigns points based on a comparative ranking of schools, which does not provide a strong motivation to increase the achievement of underperforming students because the relative scoring leaves schools unclear on what level of improvement will move schools up in the rankings. Moreover, the state fails to provide any goals or targets for subgroup or school improvement for this growth indicator for purposes of accountability.
 

The state is also proposing to give the school quality measure more points than the growth indicator in its accountability system, which could counter-weight a lack of growth.

At the high school level, achievement counts for 30 percent. However, Montana plans to use ACT scale scores to measure achievement because proficiency levels are not developed. The scale score method combines all student scores into one number, which avoids setting rigid cut points for students and can obscure information about how many students are meeting various performance thresholds.

ALL STUDENTS

Montana’s plan includes little information about how it would incorporate subgroup performance into school ratings.

The state’s proposed minimum group size of 10 is encouraging, and the state presents useful data about how many Title I schools would be included (582) and excluded (92) because of their small size. In addition, for any schools that do not meet the 95 percent student participation rate in statewide mathematics and reading/English language arts assessments—either overall for all students, or for any student subgroup consisting of 10 or more students—schools will be assigned the lowest average score on each measure for the missing student values.

The state uses a ranking system that identifies schools based on relative performance instead of an objective standard of excellence.

This lowers the bar for students, especially low-performing groups of students in schools that are not identified for support. This approach could mask subgroup performance depending on how the state aggregates school and subgroup performance.

IDENTIFYING SCHOOLS

Montana uses a relative ranking to identify schools, reviewing school performance relative to other schools, as opposed to an objective standard of performance.

Therefore, schools that score “well” in this system may not actually be producing excellent performance, but rather, they might only be performing better than the worst-performing schools. This system could create a pattern of the same schools cycling in and out of identification with little actual improvement.

Montana does not provide enough detail on its methodology for calculating a school’s summative rating.

This leaves many questions unanswered for how it is identifying comprehensive support and improvement schools. For example, how will a school’s various indicators — individually by student and by subgroup — be aggregated? This question is particularly acute for the growth indicator, the college- and career-readiness indicator, and the school-quality indicator.

Montana also provides no data or detail to gauge the consequences of its proposed approach, including how many schools may be identified for comprehensive and targeted support.

SUPPORTING SCHOOLS

Montana provides a detailed explanation on how it plans to support schools.

The focus on addressing operational and instructional components and conditions is commendable, as is the potential role for external partners, and the state’s timeline for identification and intervention is clear and helpful, although little detail is provided in the plan about how the external partners will be evaluated/selected or how they will be recruited. The most promising strategies include specialized literacy and math support and wraparound services. The state might also consider investing more in growing the instructional skills of teachers and principals, since they are closer to the students than the school board.

The state’s plans for technical assistance to comprehensive support schools and a Continuous Improvement Plan are promising, but lack detail.

Montana’s provision of direct technical assistance to comprehensive support schools is a positive step, but it is not sufficiently detailed or clear as to how assistance will be provided based on the identified needs and/or why these forms of assistance are effective. The reference to the state’s Early Warning System is a nice example of leveraging existing systems and processes, but it should be more fully developed on how it could help identify and intervene in struggling schools.
 

The state’s reference to a Continuous School Improvement Plan based on a comprehensive needs assessment also seems to have merit, but is not sufficiently detailed in the plan.

Montana makes multiple references to evidence-based practices but does not describe what they could be, what evidence is used to identify them, or how they would be aligned with identified needs. The state also references increasing the intensity of interventions for schools that fail to exit in three years, but no detail is provided as to what “increasing intensity” actually means.

Montana’s approach to high-priority districts in which 100 percent of schools are identified for comprehensive support does not appear adequate for the challenges those schools face. Interventions in areas of the state where students have access to uniformly poor districts and schools must be aggressive, and while a seven-year timeframe offers an extended period of support, it also diminishes urgency.

The state does not provide detail on how it plans to use the 7 percent of federal funds intended for school improvement activities and it should indicate if and how it intends to provide direct student services using the optional 3 percent set-aside.

EXITING IMPROVEMENT STATUS

Montana has set up a multistep process for schools to exit comprehensive support.

The first step is that the school is no longer in the lowest performing 5 percent and high schools have improved their graduation rates above 67 percent. Once schools have met the first criterion, they must also meet the academic growth goals cited in the school’s Continuous School Improvement Plan and show continual improvement in all of the components within the comprehensive needs assessment. While these components have promise, the state’s plan is vague as to what they would mean in practice.

Targeted support schools could potentially exit after only one year of improvement.

The state’s requirement that all subgroups perform better than the lowest 5 percent threshold holds promise, but could still result in schools exiting when a subgroup or subgroups have not improved—or have actually declined. Because of the relative nature of the exit criteria, a school’s performance can decline and a subgroup that simply declines less is viewed as sufficient progress for the school to exit improvement status. If implemented without relative performance expectations, this approach to all subgroup improvement could guard against a targeted support school improving performance for the subgroup for which it was identified, while not succeeding with a different subgroup.

CONTINUOUS IMPROVEMENT

Montana’s plan details engagement with various stakeholders throughout the state, including teachers, administrators, parents, tribal leaders, and leaders in business and industry.

In several places, the state refers to consulting with stakeholders around schools in improvement and schools’ continuous improvement plans, as well as in finalizing its accountability indicators. There is more to be done in terms of learning from implementation efforts, and it is especially important to look closely at and consider amending the interventions, and to consider other evidence-based interventions that might yield additional progress. The state’s stakeholder participation and buy-in seems to have been a priority, and building on that investment in engagement has the potential to assist in the revisions that will be essential to strengthening this plan.