The Florida State University
International Society for Design and Development in Education
The Florida State University
The Florida State University Graduate School
Professor (Mack and Effie Campbell Tyner Endowed Professor of Education), The Florida State University
Associate Professor, The Florida State University
Principal Research Scientist, Educational Testing Service
Senior Cognitive Scientist, Knowledge Planet
Director, R&D Laboratory, GKIS, Inc.
Senior Research Psychologist, United States Air Force Armstrong Laboratory
Postdoctoral Fellow, University of Pittsburgh – Learning Research and Development Center
Valerie J. Shute is Professor Emerita in the Department of Educational Psychology and Learning Systems at The Florida State University. She retired from FSU at the beginning of 2022 after serving as the Mack and Effie Campbell Tyner Endowed Professor of Education (2007–2022). Before joining FSU, she was a Principal Research Scientist at Educational Testing Service (2001–2007), worked in industry (1999–2001), and earlier served as a Senior Research Psychologist at the U.S. Air Force’s Armstrong Laboratory/Air Force Research Laboratory in San Antonio (1986–1999). She earned her PhD (1984) and MA (1981) in Educational/Cognitive Psychology from the University of California, Santa Barbara, and completed a two‑year postdoctoral fellowship at the Learning Research and Development Center, University of Pittsburgh. Her research focuses on assessment and learning in technology‑rich environments (e.g., games), including the development of stealth assessment grounded in evidence‑centered design, learning analytics, and support for hard‑to‑measure competencies (e.g., creativity, problem solving, persistence). Contact: [email protected]. citeturn2search1turn0search2turn2search5turn2search3
An embedded, unobtrusive approach to assessment in technology‑rich environments (e.g., games) that uses evidence‑centered design to define competency, evidence, and task models, and applies statistical modeling (e.g., Bayesian networks) to accumulate real‑time evidence of learners’ knowledge, skills, and dispositions to both measure and support learning.
Educational Technology Research and Development • Journal
Abstract: Research fields related to learning have typically focused on what works using traditional pre‑/post comparisons. This paper discusses stealth assessment—an evidence‑based methodology for technology‑rich environments (e.g., games) to assess and support hard‑to‑measure constructs as well as knowledge acquisition. It outlines how evidence‑centered design (ECD) underpins the design, embedding, and evaluation of stealth assessments and illustrates two examples (creativity and physics understanding) in the Physics Playground game. The goal is to provide sufficient detail to help researchers adopt stealth assessment to assess, foster, and investigate learning processes in technology‑rich environments.
Computers in Human Behavior • Journal
Abstract: Creativity is widely recognized as essential for success in a complex, interconnected world. This study embedded and validated a stealth assessment of creativity in the Physics Playground game with 167 8th–9th graders using a one‑group pretest–posttest design. The stealth assessment estimate correlated with external performance‑based creativity measures and significantly predicted in‑game performance and enjoyment; an indirect relation to learning via game performance was also observed.
Educational Technology Research and Development • Journal
Abstract: In a 2D physics game featuring game‑related, content‑related, and hybrid learning supports, the authors examined relationships among students’ prior knowledge, support usage, learning outcomes, and game performance (N=199, grades 9–11). Students tended to access game‑related supports most. More frequent use of hybrid supports predicted greater knowledge gains and more levels solved; content‑related and strictly game‑related supports showed no significant relations to learning or performance. High prior‑knowledge students used hybrid supports more often.
Computers & Education • Journal
Abstract: This study evaluated an incentive system designed to increase access to content‑related supports and reduce reliance on solution videos in a physics game (n=199). Students earned in‑game currency for viewing content supports and paid to watch solution videos. The incentive system increased content‑support use and decreased solution‑video use; greater use of content supports predicted higher posttest scores and more levels solved (controlling for pretest), while solution videos showed no effect. Log‑data analyses indicated students did not abuse solution videos.
Educational Research Review • Journal
Abstract: Reviewing definitions, interventions, assessments, and models, the authors propose a working definition of computational thinking (CT) as the conceptual foundation for solving problems effectively and efficiently, algorithmically, with or without computers. They categorize CT into six facets—decomposition, abstraction, algorithm design, debugging, iteration, and generalization—and call for more extensive research on CT across disciplines and educational levels.
Computers & Education • Journal
Abstract: In a randomized study (N=77), undergraduates played either Portal 2 or Lumosity for 8 hours. Pre/post testing showed Portal 2 players outperformed Lumosity players on composite measures of problem solving, spatial skill, and persistence, and showed significant spatial test gains, whereas the Lumosity group showed no pre/post differences. Findings suggest potential positive impacts of certain commercial video games on cognitive and noncognitive skills.
Review of Educational Research • Journal
Abstract: This integrative review synthesizes research on formative feedback—information provided to learners to modify thinking or behavior to improve learning. It outlines characteristics of effective feedback (e.g., specific, timely, supportive), types and timing (e.g., verification, elaborated, hints), and interacting variables (learner and task factors). The paper concludes with guidelines for generating effective formative feedback in technology‑rich contexts.
Digital Promise / International Society of the Learning Sciences (CIRCL Rapid Community Reports) • Report
Abstract: Traditional classroom assessments are usually summative, constrained in items and time, and can trigger anxiety. Stealth assessment is a type of formative assessment embedded within learning activities intended to address these problems. This primer briefly describes stealth assessment, shares lessons learned from research, and discusses issues and future directions, with resources for further reading.
The MIT Press (MacArthur Foundation Reports on Digital Media and Learning) • Book
Book description: Presents an approach to performance‑based assessment that embeds measurement within digital games to monitor learners’ progress toward targeted competencies (e.g., problem solving, systems thinking) and to adaptively support learning, grounded in evidence‑centered design and Bayesian modeling.