Statistics Department 2007 Alumni Workshop

March 29-30^th, 2007

Time: 9 a.m to 4 p.m.

Place: Eisenhower Auditorium Conference Room

The goal of educational measurement is to draw inferences about the knowledge, skills and abilities of a potential learner or student. Evidence Centered Design (ECD) is a new way of thinking about assessment design that emphasizes the evidence each task provides for potential claims which can be made about the students’ proficiencies. Focusing on the evidence enables the assessment designer to craft performance situations – tasks – in which students can appropriately demonstrate their proficiency. This talk reviews the basic ECD models. Special attention is given to the roles of probability-based reasoning in accumulating evidence across task performances, in terms of belief about unobservable variables that characterize the knowledge, skills, and/or abilities of students.

This talk will demonstrate an ECD design using Bayesian network scoring for a simple example: a performance based language assessment. This example emphasizes the points of communication between test authors and psychometricians rather than looking at specific psychometric or test design issues. In particular, it looks at the important design decisions which must be made when constructing a Bayesian network model for an assessment. It then presents several examples on Bayesian network applications in educational assessment including a simple IRT model, a cognitive skill diagnosis assessment, and a new assessment under development.