EVAAS

Misconception: EVAAS cannot measure growth for groups of students who have missing data.

EVAAS can include students even if they have missing test data, and this is a critical advantage to a sophisticated value-added approach.

EVAAS in Theory

Students with missing test scores are more likely to have a history of lower achievement, and it is important to include these students to avoid selection bias, which could provide misleading growth estimates to systems and schools that serve students with a history of lower achievement or highly mobile populations of students.|| While more simplistic value-added or growth estimates might require that students have the same set of predictors or that students have all required predictors, this often has the result of excluding certain kind of students, and this would disproportionately affect educators serving those types of students.

EVAAS does not require that students have the same set of predictors or all required predictors, and this approach includes more students in the growth measures. When estimating students' entering achievement, the modeling considers the quantity and quality of information available to each student, as well as student mobility among schools from year to year.

To accomplish this without imputing student test scores, EVAAS uses a sophisticated modeling approach that provides more reliable estimates of growth.¶

As a simple example, consider the following scenario. Ten students are given a test in two different years. The goal is to measure academic growth (gain) from one year to the next. The right side of the figure below shows the same students, some of whom now have missing scores. Two simple approaches when data are missing are to calculate the mean of the differences, or to calculate the differences of the means. When there are no missing data, these two simple methods provide the same answer (5.8 in the left side of the figure). However, when there are missing data, each method provides a different result (6.9 vs. 4.6 on the right side of the figure).

SCORES WITHOUT MISSING DATA AND SCORES WITH MISSING DATA

The problem of missing data is very common to student testing data and must be taken into consideration. As illustrated above, a more sophisticated model is needed to address this problem. The approach used by EVAAS estimates the means in each of these cells using relationships between students' test scores as if there were no missing test scores. In this way, the model provides more reliable and less biased growth measures without imputing any data. Furthermore, EVAAS uses much more student data to obtain these relationships in the growth estimates for systems and schools.

EVAAS in Practice

For assessments analyzed with the gain model, all students are included, regardless of their testing histories, their number of prior test scores, and which test scores they haveso long as the students meet the business rules for inclusion in the analysis. For assessments analyzed with the predictive model, all students are included so long as they have three prior test scores in any test, grade, and subject and meet the business rules for inclusion in the analysis.

EVAAS reporting is available using Michigan statewide data for the state summative assessments and state-approved benchmark/interim assessments, such as MAP, for districts that opted to submitusing a large proportion of Michigan districts that opted to submit MAP assessments. As a result, students and their testing history can be tracked as they move within a year (for MAP) and within the state.

Furthermore, regularly excluding highly mobile student populations who tend to be at-risk presents possible problems with educational equity since highly mobile student populations may not otherwise receive the same level of attention as non-mobile students.