UTAH

OVERVIEW

Strengths

• Utah’s plan is based on transparent goals and targets for college and career readiness, places a strong emphasis on academic achievement and growth, and ensures that stakeholders, educators, and students have a clear understanding of how schools are serving students each year.

• Utah’s inclusion of science achievement and growth is a strong element of its plan, as is the state’s inclusion of different types of growth measures to create incentives for all students, including the lowest-performing students, to both achieve large gains and make sufficient progress to reach grade-level standards.

• In addition, the state clearly listened to stakeholders and created an accountability system with strong alignment between its indicators, A-F school grades, and identification criteria for low-performing schools.

Weaknesses

• While Utah’s description of the indicators in its A-F system is straightforward, but the state does not directly incorporate individual subgroup performance into a school’s rating. Without additional data, it is difficult to know whether schools receiving high grades overall could be masking very low-performing subgroups, including those performing poorly enough to require the school’s identification for targeted support.

• The state’s non-inclusion of test participation rates is a red flag, deemphasizing the importance of measuring progress for all students.

• Utah omits some key details, such as how certain indicators (particularly growth) are awarded points within the overall index and, most critically, the grading scale that distinguishes the final grade a school receives. As a result, it is unclear whether Utah is setting rigorous expectations for school performance that are aligned with its long-term goals and appropriately weighting those indicators most associated with student success.

• Given that the state indicates it may not identify 5 percent of Title I schools every year with F grades and has not included the distribution of grades across schools, it is possible the state has set too low a bar for students.

• Utah provides minimal information in its plan for the actions and strategies that will be implemented to improve low-performing schools, and does not provide sufficient detail about its ongoing activities to engage stakeholders and educators.

Click through the tabs on the left to see how Utah scored in each category.

GOALS

Utah has established a clear method for setting goals that expects increased performance and gap closing by 2022.

Its long-term goals for achievement in grades 3-8 and for on-time graduation would cut the gap between current performance and 100 percent attainment by one-third in five years. This method expects lower-performing groups to make faster progress. That said, it is concerning that the goals are below 50 percent for a number of subgroups, including American Indian and African-American students, students with disabilities, and English learners, which may be the result of Utah’s relatively short timeline for setting goals.

The state provides some data demonstrating that its goals are ambitious, but these analyses are based on normative judgments.

The long-term goal for all students in elementary and middle school math is currently achieved by 11 percent of schools, while the English language arts (ELA) goal is achieved by 8 percent of schools. The plan would be improved if the state included historical performance data to demonstrate that its progress expectations are both attainable and ambitious.

Utah has also included a set of long-term goals for high schools on the ACT.

The state uses a similar approach to cut the gap between the current average score and a composite score of 18 by 2022. It is encouraging that Utah is aiming to increase ACT scores; however, Utah should consider partnering with its higher education institutions to determine the appropriate score to indicate preparation for entry-level credit-bearing courses, especially regarding the state’s math score goals.

Utah also needs to establish additional goals to fully align its system. Because ELA and math tests in grades 9-10 are used in its achievement indicator, Utah should expand its long-term goals to include test results in these grades. Similarly, Utah could set goals for science and five-year graduation rates, as these measures are both included in the state’s indicators.

It unclear whether Utah’s approach will result in ambitious goals for progress toward English language proficiency (ELP) among English learners.

The goals for English learners in grades 9-12 are less ambitious a nine-point increase, from 31 to 40 percent, compared to a 16-point increase, from 57 to 73 percent, in the lower grades.

Utah will have more time with English learners at lower grade levels, but the state does not provide a strong rationale for why the high school goal is significantly lower. From the disparities in its data, it is apparent Utah must do more to support older English learners.

In addition, Utah needs to indicate the timeline it is using to set expectations for sufficient growth each year among English learners.

STANDARDS AND ASSESSMENTS

Utah is implementing its version of the Common Core State Standards, which are aligned to college- and career-readiness expectations, and has a long track record of using its state assessment system, called SAGE.

This system includes state-developed ELA, math, and science assessments in grades 3-10, although the state plans to modify its assessments in grades 9-10.

While transition details are still uncertain, the state is committed to continuing with assessments that are predictive of ACT success and can be used to calculate growth. However, Utah does not discuss the level of alignment between its cut scores and college and career readiness, or how alignment will be maintained as it modifies its system.

The state requires high school juniors to take the ACT, which allows all students to take a college entrance exam.

Utah’s plan includes some detail about the accommodations that are available for English learners on the ACT, including processes to ensure they receive some key benefits of the ACT like college-reportable scores, but does not include any similar information for students with disabilities.

Utah should also include more information about its alternate achievement standards and aligned assessments for students with the most severe cognitive disabilities, including the steps it will take to ensure that the state does not exceed the 1 percent cap on participation in the alternate assessment.

Utah’s plan includes a compelling analysis of the need to implement SAGE tests in grades 3-5 in Spanish.

The state should consider concrete action steps to develop a Spanish-language version.

INDICATORS

Utah has a clear list of indicators, aligned to its goals and backed by data, showing differentiation of school performance across its proposed measures, but lacks detail for how some indicators will be calculated to award points in the state’s A-F grading system.

The state’s indicators focus on proficiency and growth in ELA, math, and science, and the achievement indicator includes a simple measure of proficiency on the state’s ELA and math tests in grades 3-10.

Utah will also measure growth on these tests within the achievement indicator for high schools and in a separate indicator in grades 4-8. Utah considers students’ relative growth compared to similarly performing peers, as well as whether the rate of growth is sufficient for a student to stay proficient or catch up within three years.

Utah includes a school-quality indicator measuring growth among the lowest-performing quartile of students.

Utah deserves significant credit for including both science achievement and growth in its indicators. This decision may help ameliorate concerns about curriculum narrowing.

High schools in Utah will also be held accountable for a composite indicator of postsecondary readiness.

This composite indicator has three equally weighted components: graduation, completion of readiness coursework, and ACT performance.

Points for readiness coursework are based on the rate of graduates earning a C grade or better in Advanced Placement, International Baccalaureate, or dual enrollment classes or completing a career and technical education pathway, while points for ACT performance are based on the percentage of students who receive a score of 18 or higher, which is aligned with the state’s goals.

While this is a good start, the state’s plan would be improved by including a more meaningful discussion of how it will measure career readiness in its pathway options, including when pathways lead to an industry-recognized credential.

Although Utah’s list of indicators is clear, the weighting and combination of measures within indicators could be clarified in the plan.

For example, the state should explain how its two types of growth measures come together to determine the number of points a school is awarded. Similarly, Utah should clarify how points will be awarded to schools based on their graduation rates, as it plans to consider both the four- and five-year graduation rate.

While the weighting of the two clearly emphasizes on-time completion, the plan should explain how the calculation will work in practice. Utah also does not provide information about the length of time expected for students to demonstrate ELP or student-level progress targets for English learners.

ACADEMIC PROGRESS

Utah has created strong incentives for schools to care about proficiency and growth.

It weights academic achievement and growth in ELA, math, and science as 50 to 75 percent of overall school grades, depending on grade level, and equally weights proficiency and growth measures.

Utah is a leading state in this area because it will include both growth during high school and for science, though the state’s plan could be improved if it clarified how the ACT is part of these growth calculations.

In addition to a simple percent proficient measure of achievement, Utah has proposed an innovative method of measuring growth in two ways.

The first measure will give students an individual growth target that puts them on track for proficiency within three years or on track to maintain proficiency, which will be paired with a simpler student growth percentile measure that compares the progress students make against their similarly performing peers, and converts those scores into percentiles.

In this manner, the state rewards both the amount of progress made (regardless of where a student falls on the performance spectrum), as well as whether progress is sufficient for students to master grade-level content. Utah should, however, clarify in its plan how the two types of measures come together in the indicator to award schools points.

Utah is also proposing to use student growth percentiles of the lowest-performing quartile of students as a school quality indicator.

This “Equitable Education Opportunity” indicator adds further incentive for schools to focus on growth, demonstrating the state’s commitment to closing gaps and helping low-performing students catch up to their peers.

That said, Utah again needs to clarify which subjects are included within this indicator, how they are weighted, and how student growth percentiles will translate into a number of points earned by schools.

ALL STUDENTS

Utah will continue to use a small minimum number of students (10) for disaggregating data in its accountability and reporting systems.

However, that choice is less impactful than it could have otherwise been, as none of the state’s indicators is disaggregated for any individual subgroup to award school grades. As a result, it is likely that a school could have a low-performing subgroup and still receive a high overall school grade. Utah’s inclusion of growth for the bottom quartile of students in each school may help capture these students, as individual subgroups are typically overrepresented in the lowest-performing quartile. The plan could be strengthened by adding data showing that this is the case in Utah.

Utah’s lack of consequences beyond reporting participation rates in annual testing is also problematic, and could undermine the integrity of school grades and the inclusion of all students.

Utah asserts that it will identify schools with low-performing subgroups for targeted support where subgroups, on their own, would receive an F grade for two years.

This could be a strong rule, but the state does not present any data on how many schools will be identified, nor does it provide the specific performance thresholds that separate an F from a D or C.

Further, so long as a subgroup’s performance is at the D level or above, the extent of gaps between groups of students within a school does not appear to matter in targeted-support identification.

IDENTIFYING SCHOOLS

Utah’s A-F grading system will provide stakeholders with a clear way to understand school performance overall and why schools may be identified for improvement.

The state’s plan specifies how each of its indicators will be weighted overall and emphasizes academic outcomes, but does not include its grading scale to determine final grades.

Given that the plan states Utah may not even identify 5 percent of Title I schools as F schools each year, the state should provide additional data in its plan about its grading scale or other evidence to demonstrate that it sets a high bar for schools and students. Utah may also want to align its timelines for identifying schools to make them as straightforward as its grading system.

Schools will be identified for comprehensive support if they earn F grades for two consecutive years.

Utah is proposing to identify any school that meets this criterion, not only those receiving Title I funds. However, because the state indicates this approach may not identify 5 percent of Utah’s Title I schools each year, the state’s plan would be improved by adding its distribution of school grades and the precise grading scale.

Any high school with a four-year graduation rate below 67 percent for two years will also be identified. Given the state’s overall high graduation rates, Utah could strengthen its plan by using a two-year average or a single year of data, or identifying high schools annually. Targeted support schools that are Title I schools and fail to improve will also be flagged for comprehensive support.

While the criteria for comprehensive support identification are clear, Utah’s different timelines are confusing and may allow schools with large, persistent gaps to delay more comprehensive interventions.

Utah does not provide a rationale for identifying low-performing schools annually (based on two years of data) while identifying high schools with low graduation rates once every two years. Waiting until 2022-23 to identify schools with chronically low-performing groups (based on four years of data) corresponds with the end date of the state’s long-term goals for those subgroups. It may also be simpler for stakeholders if all types of comprehensive support schools are identified annually.

Utah will identify schools for targeted support if any subgroup would have earned an F on its own for two consecutive years.

This is a clear definition of a consistently underperforming group and considers all indicators, but the plan could be strengthened by providing data on how many schools meet this definition, especially as that is the primary way Utah considers subgroup performance in its accountability system.

SUPPORTING SCHOOLS

Utah provides limited information about how interventions will fit together and the role the state will play in supporting both its districts and schools.

Although Utah has provided some description and rationale for the assistance it provides to schools in improvement, the information is limited. The plan could be strengthened by adding further detail in several areas. For example, the state provides no explanation for how it will use its 7 percent of federal funds dedicated for school improvement activities or whether it will use the optional 3 percent set-aside for direct student services. Much of the plan focuses on needs assessments and root cause analyses (examining issues like staffing, professional development, and budgeting) that the state will help facilitate in districts with significant numbers of identified schools, with less emphasis on what actions will be taken as a result.

Utah has instituted a cross-department team to support improvement, including vetting resources on evidence-based practices for an online repository.

This could help build district capacity to select the strongest interventions that will fit identified schools’ and districts’ needs. In addition, Utah is developing a process for districts to evaluate and address inequities in resources and is working to design a school-level expenditures report for districts to review resource allocation in identified schools—a significant and much-needed tool. These efforts have potential, and the state should take care to follow through on them prior to identifying schools in the 2018-19 school year.

Utah has authority to direct significant interventions in its lowest-performing schools that fail to improve over time, but the plan lacks critical details.

These interventions can include: restructuring a school via contract management, charter school conversion, or takeover; terminating a school’s charter or transferring operations and control to another operator; or closing charter schools.

However, the plan lacks important details, such as triggers, timelines, and specific supports, to do so. The plan indicates that the state could merely prescribe “other appropriate actions.” It will be critical for Utah to craft strong policies and regulations to implement this authority so that “other appropriate actions” does not become a loophole that prevents more rigorous interventions from occurring.

EXITING IMPROVEMENT STATUS

Utah’s exit criteria for identified schools are simple and schools may exit improvement whenever they meet them.

To improve the plan, the state could consider shorten the time period after which—if the criteria are not met—more rigorous interventions will be required. In addition, the state omits exit criteria from comprehensive support status for high schools with low graduation rates and for Title I schools with chronically low-performing subgroups that were previously in targeted support. These should be added to the plan.

To exit comprehensive support, a school must earn a D grade or higher and no longer be performing within the lowest 5 percent of schools during that time. By including both criteria, Utah ensures that exit is based on real improvement, and not just relative rankings.

If schools do not exit within four years, they are subject to more rigorous interventions. For targeted support schools to exit, the criteria are simpler: No subgroup of students in the school can earn an F grade for two consecutive years. Schools are expected to meet these criteria within four years, as well. For both exit criteria, the ESSA plan would be improved by including the state’s established cut scores associated with an “F” versus a “D” grade.

CONTINUOUS IMPROVEMENT

Utah’s plan does not describe the steps it will take to continuously improve the state’s efforts to implement ESSA.

The state does mention how it hopes to support district efforts at continuous improvement, especially with regard to districts with low-performing schools, but it is short on details on its own feedback mechanisms at the state level and how it would change its own practices over time based on that feedback. For example, the state could outline the steps it will take to monitor and adjust its long-term goals or grading system, especially the grading scale, if student and school outcomes improve.