[Remote] Senior Manager, Psychometric Operations & Data Science

Note: The job is a remote job and is open to candidates in USA. The College Board is dedicated to expanding educational opportunities and evidence-based decision-making. They are seeking a Senior Manager in Psychometric Operations & Data Science to lead the operational execution and continuous improvement of psychometric data workflows for assessments, ensuring accuracy and efficiency in scoring and reporting processes.

Responsibilities

• Manage recurring scoring cycles, ensuring that scoring, analysis, validation, and reporting readiness activities meet defined timelines, accuracy expectations, and QA standards
• Execute and oversee psychometric data workflows related to item analysis, item calibration, scoring validation, equating support, reporting file readiness, and post-administration review
• Partner with psychometricians to operationalize technical guidance for scaling, equating, calibration, form rotation, and scoring rules
• Run, review, and improve statistical and data validation checks using tools such as SQL, R, Python, or similar platforms
• Conduct root-cause analysis when scoring irregularities, data discrepancies, or reporting issues arise
• Maintain strong documentation, version control, traceability, and reproducibility standards across scoring and psychometric workflows
• Translate psychometric requirements into clear operational steps, data specifications, QA checks, and handoffs across teams
• Ensure that scoring and psychometric processes are defensible, repeatable, and aligned with relevant professional expectations for educational assessment
• Identify and reduce manual, file-based, or error-prone processes across scoring, item analysis, calibration, validation, and reporting workflows
• Define business and operational requirements for improved psychometric data workflows, automated checks, dashboards, and pipeline enhancements
• Partner with Technology to implement data workflow improvements, while ensuring that technical solutions meet the operational needs of R&M, Scoring, and Reporting
• Develop or support scripts, queries, validation routines, and repeatable processes that improve speed, consistency, and transparency
• Establish practical standards for data inputs, outputs, file formats, naming conventions, documentation, and handoffs
• Represent R&M in cross-functional conversations related to scoring data, reporting data, psychometric workflows, and assessment data operations
• Promote consistent documentation, data definitions, process maps, and decision logs across psychometric and assessment data workflows
• Apply existing College Board data policies and responsible data practices within R&M workflows
• Help define access, retention, documentation, and data-use expectations for psychometric and operational assessment data, in partnership with appropriate governance and Technology partners
• Communicate complex operational, statistical, and data quality issues clearly to technical and non-technical audiences
• Mentor colleagues in reproducible data practices, workflow documentation, QA routines, and responsible use of automation
• Contribute to a culture of accuracy, accountability, continuous improvement, and thoughtful innovation

Skills

• Master's degree in Data Science, Statistics, Information Systems, Educational Measurement, Psychology, Psychometrics, Computer Science, or a related field
• 5+ years of hands-on experience in psychometrics, educational measurement, advanced statistical analysis, or similarly rigorous quantitative environments, with strong technical fluency in SQL and at least one programming language such as R or Python
• Experience supporting or managing scoring cycles, assessment data workflows, statistical analysis workflows, or reporting data processes
• Strong technical fluency in SQL and at least one programming language such as R or Python
• Experience running data validation checks, documenting workflows, troubleshooting data discrepancies, and improving repeatability of analytic processes
• Working knowledge of educational assessment concepts such as item analysis, classical test theory, item response theory, scaling, equating, calibration, scoring rules, or form development
• Ability to partner effectively with psychometricians, researchers, technology teams, scoring teams, and reporting teams
• Strong attention to detail and a track record of supporting processes where accuracy, documentation, and quality control are essential
• Ability to communicate clearly with both technical and non-technical stakeholders
• Bilingual proficiency in Spanish and English
• Ability to travel 1–2 times a year to College Board offices or on behalf of College Board business
• Experience with assessment programs, educational testing, credentialing, certification, admissions, large-scale assessment, or similar testing environments
• Experience improving or automating data workflows using R, Python, SQL, or related tools
• Familiarity with structured and semi-structured data formats, such as CSV, relational tables, JSON, or similar formats
• Familiarity with cloud-based data environments or modern data platforms
• Comfort using AI-enabled tools to improve QA, anomaly detection, documentation, workflow automation, or analytic review
• Experience working with Technology teams to define requirements for data tools, pipelines, dashboards, or workflow improvements

Benefits

• A meaningful career, a supportive team, and a comprehensive package designed to help you thrive
• We’re a self-sustaining nonprofit that believes in fair and competitive compensation grounded in your qualifications, experience, impact, and the market
• We aim to make our best offer upfront, rooted in fairness, transparency, and market data
• We adjust salaries by location to ensure fairness, no matter where you live
• You’ll have open, transparent conversations about compensation, benefits, and what it’s like to work at College Board throughout your hiring process

Company Overview