PENNSYLVANIA

OVERVIEW

Strengths

• Pennsylvania vision for its K-12 education system is comprehensive and the state’s chosen accountability indicators are clear and easy to understand.

• Pennsylvania conducted robust stakeholder engagement in the development of its plan, and it appears to have learned lessons from previous efforts to improve schools and districts.

• The state’s new career-readiness indicator—and its components—are worth further study and consideration by other states. Notably, the indicator creates a path and a process for all students, and the adults around them, to think about what is needed to be prepared for various careers as early as 5^th grade.

• Pennsylvania should also be applauded for the substantial emphasis it places on student achievement and growth. Balancing proficiency and growth is essential because it incentivizes schools to help all students improve, while remaining focused on the important goal of ensuring students graduate prepared for success after high school.

• Finally, the state should be commended for its use of behavior interventions and supports to create positive school climate and support students’ social and emotional needs.

Weaknesses

• Pennsylvania set significantly lower goals for certain subgroups of students, which is a troubling approach given the amount of time it plans to allow for the attainment of those goals.

• Pennsylvania’s lack of a summative rating for schools will present significant challenges for parents seeking to quickly and easily understand the performance of their child’s school, and compare it with the performance of other schools.

• While it’s positive that the state will be including a measure of student growth in its accountability system; the state’s chosen measure will not necessarily indicate whether students are making sufficient progress to be prepared for college and career.

• Pennsylvania’s plan lacks details such as deadlines and timeliness related to its plans for intervening in low-performing schools that fail to improve. It also fails to provide schools and other stakeholders with meaningful opportunities to understand, and attempt to influence, the nature of the turnaround strategies that will be employed.

• While the career-readiness indicator is a strength in lower grades as a means of engaging in meaningful planning with students and families, as written it has serious weaknesses at the 11^th-grade level; more attention should be paid to a nationally recognized assessment before exit from high school.

Click through the tabs on the left to see how Pennsylvania scored in each category.

GOALS

Pennsylvania’s long-term academic goal is to cut the percentage of non-proficient students on statewide assessments in half by 2030.

Pennsylvania plans to use the same goal structure for its four- and five-year graduation rate goals. Under this approach, lower-performing subgroups will be expected to make greater progress than higher-performing groups. However, it’s unclear if Pennsylvania’s ultimate goal (80 percent proficient on English language arts and 72 percent on math) is sufficient to meet the state’s long-term vision, especially since it results in substantially lower targets for some subgroups (e.g., students of color, and students with disabilities).

The state’s broad goal for English language proficiency is for students to reach a 5.0 scale score on the WIDA ACCESS assessment within six years.

The state’s approach to determining student progress toward that goal is complicated. It seems the state will compare students’ actual progress with the progress necessary to reach proficiency within six years. If the student is on track to achieve proficiency, the student earns 1.10 performance index points. Students whose performance is behind schedule receive partial credit—0.75 if they are one year behind, 0.5 for two years, and 0.25 for three years behind. This approach may unnecessarily create confusion, but it clearly encourages schools to support English learners reaching English proficiency in a timely fashion.

Pennsylvania should be applauded for also aiming to have 60 percent of its residents earn a postsecondary degree or credential by 2025.

STANDARDS AND ASSESSMENTS

Pennsylvania has high academic standards and administers the Pennsylvania System of School Assessment (PSSA) in grades 3-8, and Keystone Exams in high school.

The state’s academic standards in English language arts and math are based on the Common Core State Standards.

On the assessment front, the state administers the Pennsylvania System of School Assessment (PSSA) in grades 3 through 8 for math, English language arts, and science. When compared with the rigor of the National Assessment of Educational Progress (NAEP), Pennsylvania’s math exams appear relatively rigorous, while the expectations on the English language arts exams are substantially beneath that of NAEP.

In high school, the state uses Keystone Exams, which are end-of-course tests in Algebra I, Biology, and English I. It is unclear if these end-of-course tests are sufficient for determining college and career readiness for all high school students, given how early these classes are typically taken in high school.

Pennsylvania defines languages other than English that are present to a significant extent as a language that is spoken by 0.5 percent of the state’s overall student population.

Spanish is the only language that meets this definition. As such, the state provides side-by-side translations in Spanish for many of its state assessments.

Pennsylvania could strengthen its plan by also providing the steps it will take to ensure that it does not exceed the 1 percent cap on participation in the alternate assessments for students with the most significant cognitive disabilities.

Furthermore, the state should provide more information about its alternative achievement standards and aligned assessments for students with the most severe cognitive disabilities.

The state’s plan also would be stronger if it included consequences for schools that miss the 95 percent participation threshold, overall or for particular subgroups.

The state says it will require assessment audits, but a stronger protection would help preserve the integrity of the state’s accountability system and ensure that all students are represented equitably.

INDICATORS

Pennsylvania’s accountability system includes a small and reasonable set of indicators. However, some of the indicators do not appear to be finalized and may have potential weaknesses that the state should monitor going forward.

The state should be commended for including a growth model, the Pennsylvania Value-Added Assessment System (PVAAS), in its accountability system. The plan would be improved by providing more information and data on the state’s chosen growth calculation.

For its graduation rate indicator, Pennsylvania proposes calculating the percentage of students who earn a diploma within four or five years.

While the state will report the rates separately, it seeks a waiver from the U.S. Department of Education to be able to use the higher of the two rates in accountability determinations. The five-year rate will almost always be higher than the four-year rate, so this proposal may undermine the state’s emphasis on on-time completion. However, if Pennsylvania wants to include the five-year rate, the state could include both measures and still place a stronger emphasis on the four-year rate to focus on helping all students graduate on time.

The state’s English language proficiency measure is a strong indicator that will be based on whether English learners meet individual scale score growth targets.

Targets are based on the progress necessary for students to reach proficiency in six years or less.

Pennsylvania selected two school-quality indicators: chronic absenteeism and career readiness in 5^th, 8^th, and 11^th grades.

The first, chronic absenteeism, measures the number of students absent for more than 10 percent of school days. This measure applies to all elementary and middle schools, and would include any absence that results in a loss of instructional time.

The second school quality indicator measures career readiness in 5^th, 8^th, and 11^th grades. At each grade level the measure is slightly different, beginning with the percentage of students demonstrating career awareness in 5^th grade on a state or locally driven career exploration program, and culminating with the percentage of 11^th-grade students who implement an individualized career plan and develop a career portfolio.

While Pennsylvania should be applauded for its focus on career readiness throughout each student’s school career, these indicators do raise several concerns.

At the 5^th– and 8^th-grade levels, Pennsylvania should ensure that all students are encouraged to include postsecondary education options in their planning. In addition, the state should review data to ensure the indicators are differentiating across schools and to confirm that the completion of these career-readiness steps is associated with improved student outcomes. In 11^th grade, the state should include externally validated, college- and career-ready indicators, such as the successful completion of Advanced Placement or International Baccalaureate exams, the SAT, or industry certifications.

ACADEMIC PROGRESS

Pennsylvania’s plan should be commended for placing a strong emphasis on student achievement and growth, proposing to use a simple measure of student achievement (percent proficient) and a value-added growth model, weighing both equally.

Pennsylvania’s growth model, called the Pennsylvania Value-Added Assessment System (PVAAS), is a value-added measure. It compares the progress students make in a given year against historical performance of similar students in the state. While the state has been using this approach since 2006, it is not explicitly a growth-to-standard model. Pennsylvania should monitor its data to ensure students, including underserved students, are progressing sufficiently to access the content they need to stay on track toward mastery at graduation.

ALL STUDENTS

Pennsylvania should be commended for significantly lowering its minimum subgroup size from 40 to 20 students, which will result in the state including more students in its accountability system.

While the state deserves credit for including an analysis of how the various n-sizes will affect the number of schools and students included in the accountability system to demonstrate why 20 makes sense, Pennsylvania may want to consider reducing its n-size further as part of its continuous improvement process.

Unfortunately, Pennsylvania’s annual evaluation of school performance does not take into account—or put any additional weight on—the performance of individual subgroups.

Instead, the performance of individual student subgroups is considered only in the state’s methodology to identify schools for targeted support and improvement.

The state’s decision to include former English learners (EL) in the EL subgroup for accountability purposes for four years is worth monitoring, as it could inflate the performance of the EL subgroup and mask gaps and underperformance.

Pennsylvania is also proposing a long timeline for including newly arrived English learners in school accountability decisions. In their first year after arrival, the state plans to exempt these English learners from participating in the state English language arts and math assessments. In their second year, students would take the assessments, but their scores would not count in accountability determinations. It would not be until their fourth year that student scores would count fully in the state’s accountability system. This lengthy timeline appears inconsistent with federal requirements.

IDENTIFYING SCHOOLS

Pennsylvania’s approach to identifying schools ensures that academic measures count most significantly, as they comprise the whole of the first step in the identification process.

The state will identify schools in need of comprehensive support based on a multistep process, which starts by placing schools in a quadrant with achievement and growth as the two axes. The state will focus on the quadrant where schools are demonstrating both low proficiency and low growth.

At this point the state has not yet identified final cut scores, but it plans to in the fall of 2018 based on data from the 2016-17 and 2017-18 school years.

The state will also identify for comprehensive support all high schools, not already identified, which have graduation rates below 67 percent.

However, this is based on the state’s proposed four- and five-year cohort graduation calculation, which does not place adequate emphasis on on-time completion.

This is a potential model that other states may want to consider watching, as it guarantees that every identified school has academic performance among the lowest in the state. However, this approach could have several weaknesses.

The state’s Future Ready PA dashboard may help address some concerns about transparency, but the lack of an overall rating for schools could make communicating information about overall school performance to parents a significant challenge for the state.

Pennsylvania should consider what steps it will take to ensure that the index is parent-friendly and provides actionable information for all stakeholders.

Pennsylvania is waiting on additional data before it makes its final determination, which means the current plan lacks context about how many schools the state might identify for comprehensive and targeted support. The state could also strengthen its plan by providing a definition of what it means to be a high-performing school.

With respect to identification of comprehensive support schools, the second step of Pennsylvania’s process focuses on schools that also fall in the bottom quartile of performance on at least one of the other indicators.

These indicators include chronic absenteeism, career readiness, high school graduation rate, and English language proficiency. This approach undervalues these important indicators of school performance and may fail to comply with ESSA’s requirement that identification be based on “all indicators.”

Pennsylvania’s identification of targeted support schools also uses the same two-step process and, therefore, suffers from some of the same weaknesses described above.

SUPPORTING SCHOOLS

Pennsylvania should be applauded for its commitment to tailored intervention strategies based on the needs of individual schools and districts. However, the state’s intervention plan is lacking some detail and clarity.

The state should provide more detail on what it considers an evidence-based intervention, as well as how it will create, share, and maintain an inventory of best practices.

The state should also provide greater clarity about the kinds of tools that it will use to turn around struggling schools. For example, Pennsylvania mentions the authorization of a Chief Recovery Office under state law, but fails to describe how that authority will be used by the state and on what timeline.

Pennsylvania could consider including triggers for how and when the more transformative interventions for schools that fail to improve will be utilized. Finally, the state should afford stakeholders greater opportunity to understand and influence school turnaround strategies.

Notable strengths in the state’s support system include the partial use of competitive grants for school improvement funds, the commitment to address both academic and social-emotional barriers utilizing community resources and services, and the existence of roles like the Chief Recovery Officer.

However, the state’s plan also lacks specific details on how it will leverage the kind of transformative reforms that most of its lowest-performing schools will need to improve.

Pennsylvania plans to administer the 7 percent set-aside of its Title I funds for school improvement activities through a hybrid competitive and formula process.

For the first two years, schools identified for comprehensive support will receive funds through a formula. Thereafter, funds will be distributed on a competitive basis. Additionally, the state should indicate if and how it intends to provide direct student services using the optional 3 percent set-aside, which provides an additional opportunity for the state to align school improvement activities with its statewide goals.

EXITING IMPROVEMENT STATUS

Pennsylvania’s exit criteria are not sufficiently rigorous to demonstrate sustained improvement.

The state has not yet finalized its exit criteria for either comprehensive or targeted support schools. Pennsylvania should require schools to demonstrate sustained improvement prior to exiting status. In its current form, the plan simply states that schools must show “measurable” progress in proficiency or growth, which is not detailed enough to determine whether it’s appropriately rigorous.

CONTINUOUS IMPROVEMENT

Pennsylvania appears to be committed to continuous improvement through its articulation of lessons learned and substantial stakeholder engagement.

The plan also demonstrates that the state is committed to continuously evaluating and improving its new indicators and new system of identification.

That said, there are insufficient details about how the state will put into practice the lessons learned from its implementation process, or how it will use evidence and feedback to improve the quality of its support and interventions for low-performing schools.

The state requires all districts to submit their implementation plans to the state via an online system.

This system will be monitored not only for compliance with statutory and regulatory requirements, but also on a four-year cycle for other risk factors.

Risk factors include but are not limited to: staff and management turnover, allocation size, fiscal findings, missed deadlines, and on-site reviews. During site visits, districts are required to provide reviewers with data and evidence in support of progress made toward their goals.

The state could provide greater clarity about whether schools that show signs of being at risk will receive more frequent site visits or other kinds of monitoring.