benchmark jobs in New York

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models.About the opportunity:We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each exchange,...

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models. About the opportunity: We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each...

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models. About the opportunity: We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each...

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models. About the opportunity: We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each...

Modern MedEd LLC is seeking licensed radiologists to participate in a paid study evaluating the accuracy and reliability of diagnostic imaging algorithms. You will review a series of de-identified cases, provide expert reads, and complete a structured scoring rubric. The workflow is fully remote and completed asynchronously, allowing you to work on your own...

Remotebridge is seeking a Procurement & Vendor Management Expert to design realistic, multi-step tasks for evaluating AI systems against real-world procurement workflows. This remote contractor role emphasizes practical judgment over lookup, with reference files and clear scoring criteria. You will craft scoring frameworks, supplier assessments, and...

Online Compensation Benchmarking Specialist Location: Remote Salary: USD 54,774 per year Job Description: We seek a detail-oriented and analytical Online Compensation Benchmarking Specialist to join our team. In this remote position, you will evaluate and compare compensation structures across various industries and geographic regions to ensure competitive...

Naukri Mitra is seeking an Online Compensation Benchmarking Specialist to work remotely. You will evaluate and compare compensation structures across regions to ensure competitive and equitable pay. This role supports strategic decisions in compensation planning and talent management. You will build models, gather data from surveys and proprietary sources,...

Google DeepMind is seeking a Forward Deployed Engineer for the Kaggle platform on a 12-month fixed-term contract. You will onboard external benchmarks to Kaggle, run joint evaluations, and synthesize signals to improve model performance.You will collaborate with engineering, product, and partner teams across time zones, producing high-quality docs and sample...

Verita AI is seeking Coding & QC Experts for a paid pilot evaluating the quality of AI benchmark tasks. Each task includes a question, a proposed gold answer and a rubric of claims describing what a correct answer should contain. Critically audit existing work, assess rubric claims, and verify the reference answer before treating it as ground truth. This...

Mercor is seeking an Applied Economics & Finance Benchmark Specialist for a remote contract role. The position focuses on creating rigorous economics and finance questions with clear solutions and supporting references. Ideal candidates hold a PhD or are doctoral candidates, with strong quantitative skills and excellent written English. CFA/CPA experience is...

Turing is seeking a Subject Matter Expert in Market Intelligence, Media Production, or Technical Communications to author and evaluate high-fidelity tasks and calibration rubrics. The role covers three disciplines and places you on a Mid-Level or Senior track based on depth of experience and peer-review background. Responsibilities include creating...

Burtch Works is seeking a mid-level Product Manager to own a Rate Benchmarking Simulator for a leading U.S. health care network. This remote (US) role will drive the product from kickoff to MVP, leading a small team of data scientists and a data engineer. You'll define the product vision, build the backlog, and refine requirements with analytics and...

Google DeepMind is seeking a Forward Deployed Engineer for a 12-month fixed-term contract to bridge partner engineering with Kaggle. You will onboard external benchmarks, run systematic evaluations, and deliver high-fidelity signals to model teams.The role requires strong software and ML experience, collaboration with global partners, and outstanding...

Mercor is seeking a Senior Character Animator for a remote, contract position. You will define standards for body mechanics, facial nuance, timing, and emotional intent, and review performances with precise notes. Candidates must have 10+ years’ experience with credits on feature films, VFX, or AAA game cinematics. Some in-person sessions may be required...

What We're Researching We're running a paid study on the accuracy and reliability of diagnostic imaging algorithms. High-quality reference evaluations are essential for establishing safe and effective medical benchmarks. Your clinical expertise will directly inform the baseline ground truth used to evaluate these emerging tools. How It Works You will review...

Mercor seeks an Applied Business & Commerce Benchmark Specialist for a remote contract role. You will craft rigorous business questions, rate difficulty, and supply correct answers with plausible distractors, focusing on deep conceptual understanding. The position offers 10+ hours per week, competitive hourly compensation ($77–$98), and a chance to work...

LILT is seeking experienced Government professionals to support a cutting-edge AI benchmarking project focused on Egyptian public sector environments. As a Subject Matter Expert, you will create and review realistic, high-quality scenarios that reflect operational, administrative, compliance, public safety, social service, and community engagement challenges...

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models. About the opportunity: We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each...

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models. About the opportunity: We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each...