In this article, we describe current research findings on assessment accommodations and universal design within the context of emerging interactive digital assessment tasks that employ simulations such as in science, technology, engineering, and mathematics (STEM). STEM education in many classrooms now includes digitally based activities such as science simulations and virtual laboratories that have been shown in some cases to promote learning gains. When such technologies are used in STEM assessments, a major challenge is to ensure assessments are accessible so all students can show what they know and can do. Federal laws and regulations including the Individuals with Disabilities Education Act, Elementary & Secondary Education Act, and Americans with Disabilities Act require that students with disabilities (SWD) be provided an opportunity to participate in educational programing and services available to nondisabled peers. In addition to implementing principles of universal design in assessment contexts, reasonable accommodations must be afforded to ensure accessibility. This article focuses on universal design and accommodations where the STEM construct is not adjusted or modified. Here, we employ synthesis of the research literature to document accessibility recommendations and practices around interactive assessment tasks, especially in STEM. We illustrate with an example and highlight directions that future development might take. The intention is to inform educators, school administrators, state and local policy makers, and assessment developers on the availability and use of accommodations in interactive assessment contexts such as simulation, and what is needed to ensure appropriate accessibility for SWD.
Scalise, K., Irvin, P. S., Alresheed, F., Zvoch, K., Yim, H., Park, S., Landis, B., Meng, P., Kleinfelder, B., Halladay, L.,
& Partsafas, A. (2018). Accommodations in digital interactive STEM assessment tasks: Current accommodations
and promising practices for enhancing accessibility for students with disabilities. Journal of Special Education Technology.
Advance online publication. doi.org/10.1177/0162643418759340
A Validity Argument for a Mathematics Curriculum-Based Measure: Implications for Response to Intervention Decision-Making
Within a response to intervention (RTI) framework, many schools initially group students into tiers on the basis
of normative achievement. Using a screener and benchmarks with curriculum-based measurement (CBM), students
are classified as being academically “at-risk” or not. In the current study, we present a validity argument for the
use of a mathematics CBM as a classification tool within RTI and explore the relation between a fall CBM administration
and state accountability test results using a large sample in Oregon (located in the Pacific Northwest
in the United States) through regression analyses. Our analyses indicate a strong relation between easyCBM® and
the state achievement tests, which suggests that accurate screening of students using CBM can help teachers and
administrators make more informed decisions for both instruction and school resource allocation. For more successful
RTI, educators should implement effective and efficient decision-making processes based on assessment data,
and researchers should continue exploring ways to improve the classification accuracy of CBM and incorporate
Park, B. J., Anderson, D., Tindal, G., & Alonzo, J. (2017). A validity argument for a mathematics curriculum-based measure:
Implications for response to intervention decision-making. Journal of Educational Administration and Policy, 2, 33-46.
Estimating School Effects with a State Testing Program Using Transition Matrices
For the past decade, the accountability model associated with No Child Left Behind (NCLB) emphasized proficiency on end of year tests; with Every Student Succeeds Act (ESSA) the emphasis on proficiency within statewide testing programs, though now integrated with other measures of student learning, nevertheless remains a primary metric for reporting achievement. We examine transition over categories across two years for three cohorts of middle school students as one means for evaluating change. We document school effects for seven models by analyzing relations among them and comparing them to the actual change in movement across categories. We report moderate to high correlations across the transition models though they differed vastly in the (a) assumptions behind them, and (b) ranking of schools with actual transitions. The findings, therefore, need to be considered in the context of assumptions embodied in each transition model to adequately understand their worth in reporting school effects.
Tindal, G., Nese, J. F. T., & Stevens, J. J. (2017). Estimating school effects with a state testing program using transition
matrices. Educational Assessments, 22(3), 189-204. Estimating School Efeects w/ State Testing
Exploring the Robustness of a Unidimensional Item Response Theory Model with Empirically Multidimensional Data
Unidimensionality and local independence are two common assumptions of item response theory. The former implies that all items measure a common latent trait, while the latter implies that responses are independent, conditional on respondents’ location on the latent trait. Yet, few tests are truly unidimensional. Unmodeled dimensions may result in test items displaying dependencies, which can lead to misestimated parameters and inflated reliability estimates. In this article, we investigate the dimensionality of interim mathematics tests and evaluate the extent to which modeling minor dimensions in the data change model parameter estimates. We found evidence of minor dimensions, but parameter estimates across models were similar. Our results indicate that minor dimensions outside the primary trait have negligible consequences on parameter estimates. This finding was observed despite the ratio of multidimensional to unidimensional items being above previously recommended thresholds.
Anderson, D., Kahn, J. D., & Tindal, G. (2017). Exploring the robustness of a unidimensional item response theory
model with empirically multidimensional data. Applied Measurement in Education, 30(3), 163-177.
Unidimensional Item Response Theory Model
Assessment of Students with Learning Disabilities: Using Students’ Performance and Progress to Inform Instruction.
The combination of RTI and formative assessment, particularly curriculum-based measurement (CBM), provides an explicit system for better understanding and validating instructional decision-making using student responses.
Tindal, G., Alonzo, J., Sáez, L., & Nese, J. F. T. (2017). Assessment of students with learning disabilities: Using
students’ performance and progress to inform instruction. In K. Ercikan & J. W. Pellegrino (Eds.), Validation of Score
Meaning in the Next Generation of Assessments. Validation of Score Meaning in the Next Generation of Assessments
Modeling Reading Growth in Grades 3 to 5 with an Alternate Assessment.
Modeling growth for students with significant cognitive disabilities (SWSCD) is difficult due to a variety of factors, including,
but not limited to, missing data, test scaling, group heterogeneity, and small sample sizes. These challenges may account
for the paucity of previous research exploring the academic growth of SWSCD. Our study represents a unique context in
which a reading assessment, calibrated to a common scale, was administered statewide to students in consecutive years
across Grades 3 to 5. We used a nonlinear latent growth curve pattern-mixture model to estimate students’ achievement
and growth while accounting for patterns of missing data. While we observed significant intercept differences across
disability subgroups, there were no significant slope differences. Incorporating missing data patterns into our models
improved model fit. Limitations and directions for future research are discussed.
Farley, D., Anderson, D., Irvin, P. S., & Tindal, G. (2016). Modeling reading growth in grades 3 to 5 with an alternate
assessment. Remedial and Special Education. ModelingRdgGrwthGr3-5_Farley_Anderson_ETAL
Achievement Gaps for Students with Disabilities: Stable, Widening, or Narrowing on a State-wide Reading Comprehension Test?
Reading comprehension growth trajectories from 3rd to 7th grade were estimated for 99,919 students on a state reading comprehension assessment. We examined whether differences between students in general education (GE) and groups of students identified as exceptional learners were best characterized as stable, widening, or narrowing. The groups included students with disabilities (SWD) from 8 exceptionality groups and 2 groups of academically gifted students (AG).
Schulte, Ann C., Stevens, Joseph J., Elliott, Stephen N., Tindal, G., & Nese, Joseph F. T. (2016). Achievement gaps for
students with disabilities: Stable, widening, or narrowing on a state-wide reading comprehension test? American
Psychological Association. Achievemnet Gaps for Students with Disabilities_Schulte_Stevens_ETAL
Influence of Opportunity to Learn Indices and Education Status on Students’ Mathematics Achievement Growth.
We examined instructional processes in classrooms where students with and without disabilities received mathematics instruction to understand the relationship among key instructional process variables and achievement as measured by interim and end-of-year summative assessments.
Elliott, S. N., Kurz, A., Tindal, G. & Yel, N. (2016). Influence of opportunity to learn indices and education status
on students’ mathematics achievement growth. Remedial and Special Education. Published online before print
August 16, 2016, doi:10.1177/0741932516663000. OpportunityToLrn_Elliott_Kurz_ETAL
Using Explicit and Systematic Instruction to Support Working Memory.
Differences in working-memory capacity affect how students learn new skills and complete activities. Students with learning disabilities often experience challenges associated with limited working-memory capacity. This article offers strategies both for recognizing and supporting working-memory processing constraints and to facilitate working-memory processing during reading
and math instruction.
Smith, J. L. M., Sáez, L., & Doabler, C. T. (2016). Using explicit and systematic instruction to support working memory.
Teaching Exceptional Children, 48, 6, 275-281. InstrctnSupportWkngMemory_Smith_Saez_Doabler
The Relation Between Smarter Balanced and easyCBM Mathematics and Reading Assessments.
This study investigated the relation between the easyCBM Benchmark Assessments in both mathematics and reading and the Smarter Balanced assessment, widely adopted across the United States
Alonzo, J. (2016). The relation between Smarter Balanced and easyCBM mathematics and reading assessments.
Journal of School Administration Research and Development, 1, 17-35. Access article through http://www.jsard.org/
Individual Differences in Kindergarten Through Grade 2 Fluency Relations.
Despite long-standing interest in reading fluency, little has been documented about the specific factors that developmentally contribute to individual differences. Consequently, precursory relations were longitudinally examined for students grouped at the end of Grade 2 as low, average, or high fluency readers to describe early alphabetic and word fluency contributions to later passage reading fluency outcomes.
Using structural equation modeling, we modeled Kindergarten letter sounds, Grade 1 word reading, and Grade 2 passage reading fluency with 2302 students to identify early fluency differences in the emergence and growth of beginning reading skills for each fluency group. We compared initial level and growth slope estimates, and reported cross-year developmental fluency relation patterns by group. In general, our findings point to the importance of building letter sound fluency in kindergarten, and shed light on the disruptive impact of delayed skill progress on reinforcing relations during reading skills development.
Sáez, L., Nese, J. F. T., Alonzo, J., & Tindal, G. (2016). Individual differences in kindergarten through grade 2 fluency
relations. Learning and Individual Differences, 49, 100-109. FluencyDiffGrK-2_Saez_Nese_ETAL
Technology-based Assessment and Problem Analysis
This chapter discusses features of technology-based assessments particularly relevant for applications in school settings implementing response to intervention (RTI). Using easyCBM®, a popular learning management system developed to support RTI, the chapter illustrates measurement development and delivery that can provide instructionally relevant reports. Three important constructs are addressed that need additional research and development: measurement sufficiency, instructional adequacy, and data-based decision making. Consideration of these areas has received little attention in the empirical literature but is likely to have profound implications for successful RTI implementation.
Tindal, G., & Alonzo, J. (2016). Technology-based assessment and problem analysis (pp. 473-492). In S. R.
Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.) Handbook of Response to Intervention: The Science and
Practice of Multi Tiered Systems of Support (2nd edition). New York: Springer Science Inc.
Technology-based Assessment and Problem Analysis
Documenting Reading Achievement and Growth for Students Taking Alternate Assessments.
Students with disabilities have been included in state accountability systems for more than a decade; however, only in the past few years have alternate assessments of alternate achievement standards (AA-AAS) become stable enough to allow examination of these students’ achievement growth. Using data from Oregon’s AA-AAS in Reading during the period 2008–2009 to 2010–2011, we examined the achievement growth for a sample of 1,061 elementary students using two growth models: a transition matrix and a multilevel linear growth model. The authors found with the transition matrix model that a majority of students remained at the same performance level from one year to the next, whereas with the multilevel linear growth model, students’ scores revealed small, but statistically meaningful, growth year to year. The article concludes by noting advantages and disadvantages of these models to characterize growth and their implications for policy and practice.
Tindal, G., Nese, J. F. T., Farley, D., Saven, J. L., Elliott, S. N. (2016). Documenting reading achievement and growth
for students taking alternate assessments. Exceptional Children. DocumentingRdgAchvmnt_Tindal_Nese_ETAL
Achievement Gaps for Students with Disabilities: Stable, Widening, or Narrowing on a State-wide Reading Comprehension Test.
Reading comprehension growth trajectories from 3rd to 7th grade were estimated for 99,919 students on a state reading comprehension assessment. We examined whether differences between students in general education (GE) and groups of students identified as exceptional learners were best characterized as stable, widening, or narrowing. The groups included students with disabilities (SWD) from 8 exceptionality groups and 2 groups of academically gifted students (AG). Initial reading comprehension achievement differed for all exceptionalities. Controlling for sociodemographic variables, small, but statistically significant differences in growth rate were observed, with SWD groups growing more rapidly and AG groups growing more slowly than GE students. Given that differences in growth for SWD were small relative to the magnitude of the initial achievement gaps, the observed pattern of growth was one of stable differences. There was evidence of some narrowing of the achievement gap for students identified with learning disabilities in reading. The findings were interpreted within the simple view of reading where increases in word recognition skills for SWD in the grade range examined may have accounted for their more rapid growth in reading comprehension relative to GE students. The findings suggest that similar expectations for rate of reading growth for GE students and SWD might be incorporated into growth-based accountability models, but they also suggest that reading comprehension growth sufficient to have an impact on SWD achievement gaps does not routinely occur in typical educational practice.
Schulte, A. C., Stevens, J. J., Elliott, S. N., Tindal, G., & Nese, J. F. T. (2016). Achievement gaps for students with
disabilities: Stable, widening, or narrowing on a state-wide reading comprehension test? Journal of Educational
Psychology, 108, 7, 925–942. StdntAchvmntGaps_Schulte_Stevens_ETAL
Patterns of Statewide Test Participation for Students With Significant Cognitive Disabilities.
Students with significant cognitive disabilities are eligible to participate in two statewide testing options for accountability: alternate assessments or general assessments with appropriate accommodations. Participation guidelines are generally quite vague, leading to students “switching” test participation between years. In this study, we tracked test participation for two cohorts of students with a documented disability over 3 years. Results suggested approximately 25% of students who initially took the alternate assessment switched test type at least once, although patterns of switching were not consistent across disabilities. Students on the performance “bubble” were more likely to switch test participation. Test switching poses challenges for monitoring students’ academic growth within accountability frameworks.
Saven, J. L., Anderson, D., Nese, J. F. T., Farley, D., & Tindal, G. (2016). Patterns of statewide test participation for students
with significant cognitive disabilities. The Journal of Special Education. 49, 209-220, doi: 10.1177/0022466915582213.
Sáez, L. (2015, March). The Hidden Reason Why Some Kids Can’t Follow Your Directions: UnlockingMulti-step directions. Available at: The Hidden Reason Why Some Kids Can’t Follow Your Directions_Saez.