Head of Evaluation and Oversight Research
2 weeks ago
Overview Scale is the leading data and evaluation partner for frontier AI companies, playing an integral role in advancing the science of evaluating and characterizing large language models (LLMs). Our research focuses on tackling the hardest problems in scalable oversight and the evaluation of advanced AI capabilities. We collaborate broadly across industry and academia and regularly publish our findings. Responsibilities Lead a team of research scientists and engineers on foundational work in evaluation and oversight. Drive research initiatives on frameworks and benchmarks for frontier AI models, spanning reasoning, coding, multi-modal, and agentic behaviors. Design and advance scalable oversight methods, leveraging model-assisted evaluation, rubric-guided judgments, and recursive oversight. Collaborate with leading research labs across industry and academia. Publish research at top-tier venues and contribute to open-source benchmarking initiatives. Remain deeply engaged with the research community, both understanding trends and setting them. Ideal Qualifications Track record of impactful research in machine learning, especially in generative AI, evaluation, or oversight. Significant experience leading ML research in academia or industry. Strong written and verbal communication skills for cross-functional collaboration. Experience building and mentoring teams of research scientists and engineers. Publications at major ML/AI conferences (e.g. NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR) and/or journals. Interviews Our research interviews are crafted to assess candidates\' skills in practical ML prototyping and debugging, their grasp of research concepts, and their alignment with our organizational culture. We do not ask LeetCode-style questions. Compensation and Benefits Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The base salary range for this full-time position in the locations of San Francisco, New York, Seattle is $252,000 $315,000 USD. Eligible roles may include equity-based compensation, subject to Board of Directors approval. Benefits include comprehensive health, dental and vision coverage, retirement benefits, a learning and development stipend, generous PTO, and potential commuter stipends; specific details shared by your recruiter. About Us At Scale, our mission is to develop reliable AI systems for the world\'s most important decisions. Our products provide the high-quality data and full-stack technologies that power leading models, and help enterprises and governments build, deploy, and oversee AI applications that deliver real impact. We work closely with industry leaders and government partners and are expanding our team to accelerate the development of AI applications. Equal Opportunity and Accommodations We believe that everyone should be able to bring their whole selves to work and are proud to be an inclusive and equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability status, gender identity or Veteran status. We provide reasonable accommodations to applicants with disabilities. If you need assistance in the application or recruiting process, please contact us about accommodations. We comply with applicable pay transparency and privacy policies and encourage applicants to review relevant notices during the process. #J-18808-Ljbffr
-
Head of Evaluation and Oversight Research
1 week ago
Seattle, WA, United States Scale AI Full timeHead of Evaluation and Oversight Research Scale is the leading data and evaluation partner for frontier AI companies, playing an integral role in advancing the science of evaluating and characterizing large language models (LLMs). Our research focuses on tackling the hardest problems in scalable oversight and the evaluation of advanced AI capabilities. Our...
-
Head of Evaluation and Oversight Research
2 days ago
Seattle, United States Scale AI, Inc. Full timeScale is the leading data and evaluation partner for frontier AI companies, playing an integral role in advancing the science of evaluating and characterizing large language models (LLMs). Our research focuses on tackling the hardest problems in scalable oversight and the evaluation of advanced AI capabilities. We collaborate broadly across industry and...
-
Head of AI Evaluation
2 weeks ago
Seattle, WA, United States Scale AI, Inc. Full timeA leading AI research organization in Seattle is looking for a research leader to oversee groundbreaking evaluation projects in generative AI. The role involves leading a team, driving research initiatives, and publishing findings. Candidates should have a strong background in machine learning, experience leading research teams, and a record of impactful...
-
Seattle, United States University of Washington Full timeDescription The Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head. The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW...
-
Seattle, WA, United States University of Washington Full timeDescription The Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head. The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW...
-
Seattle, WA, United States University of Washington Full timeDescription The Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head. The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW...
-
Test And Evaluation Director-Jd
1 week ago
KEYPORT, WA US (Primary) McLaughlin Research Corporation Full timeDescription McLaughlin Research Corporation (MRC) is seeking an experience Test and Evaluation Director who will support the Naval Undersea Warfare Center. The Naval Undersea Warfare Center (NUWC) is the United States Navy's full-spectrum research, development, test and evaluation, engineering and fleet support center for submarines, autonomous underwater...
-
Seattle, Washington, United States University of Washington Full timeThe Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head.The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW departments,...
-
Seattle, United States University of Washington Full timeDescription The Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head. The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW...
-
Seattle, WA, United States University of Washington Full timeDescription The Fred Hutchinson Cancer Center (Fred Hutch) and the University of Washington (UW) invite nominations and applications for the Fred Hutch Palliative Care Program Head. The position will be dual appointed with a primary faculty appointment in the Clinical Research Division at Fred Hutch and a secondary appointment in one of the following UW...
