Rapid Online Assessment of Reading

Director of Psychometrics or Postdoc in Psychometrics

Advisors: Jason D. Yeatman, Ph.D. and Ben W. Domingue, Ph.D.

The Rapid Online Assessment of Reading (ROAR) is a bridge between the lab and the classroom. As an open-source, open-access assessment platform, we are currently supporting hundreds of schools around the United States (and abroad) with a more efficient and precise approach to assessment. Through this research practice partnership model we are also able to conduct research at an unprecedented scale. We have an opening for a Director of Psychometrics or a Postdoc in Psychometrics to lead innovations in measurement, improve the psychometrics of the ROAR assessments, direct new research priorities (both technical and applied), and to oversee the implementation of rigorous measurement methodologies at scale. This could either be a 2-3 year postdoc position for a candidate interested in an academic path or a permanent research staff position leading and overseeing the psychometrics of the ROAR platform.

ROAR is a dynamic and collaborative team science project consisting of graduate students, postdocs, faculty, research coordinators, professional web developers, and school partnership coordinators. This position is ideal for someone with exceptional technical skills who thrives in a collaborative team environment.

Required Qualifications:

• Highly motivated postdoctoral researcher with extensive experience with item response theory models, computer adaptive testing, and related measurement methods.
• Demonstrated ability to bridge research (innovative ideas, writing papers, etc.) and practice (implementation of methods at scale for use in the real world)
• Ph.D. in quantitative methods for social science (e.g., educational measurement, quantitative psychology) or related discipline.
• Interest in measurement of reading development and mechanisms of learning
• Substantial experience with latent variable models (especially factor analysis, item response theory, and growth modeling) and coding in R.
• Strong collaborative skills and ability to work well in a complex, multidisciplinary environment across multiple teams, with the ability to prioritize effectively.
• Eager to contribute to a vibrant group of faculty, post-docs, and students coalescing around psychometric and longitudinal modeling issues.

To apply, please submit the following to jyeatman@stanford.edu:

• Cover letter (1 page)
• CV
• Copies of two research papers that demonstrate research agenda
• Contact information for two letters of recommendation 

ROAR Vision and Mission: A bridge between the lab and the classroom

Assessments are typically time-consuming and resource-intensive to administer: Individually administering assessments to each student in a classroom means a substantial amount of lost instruction time and requires extensive training for teachers to accurately administer and score measures that are used for high-stakes decisions (e.g., access to intervention). Researchers face these same challenges creating a bottle-neck to research at scale. While education technology companies have built products that lower the demands on teachers, many of these products are expensive, grounded in opaque, proprietary technology, and lack a strong research backing. Hence, these products rarely get used in research, creating a disconnect between educational research and practice.

We launched ROAR envisioning a new model: an open-source, open-access assessment platform, grounded in ongoing academic research, and co-developed in collaboration with school-district stakeholders. Rather than a one-way street from the lab to society (often with a commercial intermediary), ROAR’s goal is to inculcate a virtuous cycle between research and practice. We aim to build a suite of completely automated, lightly gamified, online assessments that are grounded in ongoing cognitive neuroscience research and validated against the current “gold standard” of standardized, individually-administered assessments. Our approach is to partner with school districts and community based organizations at each stage of research and development to ensure that our research is grounded in real-world problems and inspired by the deep knowledge of educators who work with children and youth across a diversity of contexts. Through this “Research Practice Partnership” model, we endeavor towards a new assessment methodology that is more valid, precise, efficient, and informative. We aim to design this platform around the diversity of learners in the United States (and abroad). We prioritize transparency at every stage: whenever feasible, materials and technology are made public and each measure within ROAR is published in open-access, peer-reviewed journals with the goal of building more systemic connections between the lab, classroom, and society.