remote ai evaluator

Job offers for remote ai evaluator. Find listings updated daily.


  • New York, United States Clera Full-time

    Clera Labs, Inc. is seeking a remote QA-focused role to design and evaluate assessments for AI model safety and capabilities. You will write test prompts, develop scoring rubrics, and create reference answers to challenge models across varied tasks. The role involves grading outputs, analyzing error patterns, and comparing models using identical prompts,...


  • San Francisco, CA, United States Planet Pharma Remote Full-time $120,000 - $180,000 Contract

    Planet Pharma is seeking experienced scientists and engineers to rigorously evaluate frontier AI models against real technical work, including data analysis, design verification, and experiment readouts.You'll craft realistic tasks from your practice, run them through frontier AI agents, and judge outputs to professional standards, with emphasis on clear...


  • New York, United States Clera Full-time

    Clera Labs, Inc. is seeking contributors for a project-based expert network focused on AI model evaluation and improvement. You will provide structured feedback to help make model responses more accurate, helpful, and safe, rating with rubrics and writing concise rationales. The role requires native-level English, strong attention to detail, and the ability...


  • New York, NY, United States Clera Full-time

    Clera Labs, Inc. offers project-based, fully remote opportunities to contribute to AI model evaluation and improvement. You will compare model responses, rate them on accuracy, helpfulness, tone, and safety, and provide concise rationales for your judgments. Ideal candidates bring strong written English, sharp attention to detail, and experience with...


  • United States Rise Data Labs Full-time

    Rise Data Labs is seeking advanced Biology and Life Sciences experts to support the training and evaluation of state-of-the-art AI systems. You will apply deep scientific knowledge to assess AI-generated reasoning and identify technically meaningful errors within your field. This remote contract position requires a Master's degree with PhD strongly...


  • United States Micro1 Full-time

    micro1 is a data lab that helps AI labs and enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM capabilities, as well as contextual evaluations to monitor and improve AI agents in enterprise settings. Role: AI Product Evaluator (Contractor). You will evaluate AI models and...


  • New York, NY, New York County, NY; New York State, United States Synthires Full-time

    Synthires is seeking evaluators to review AI-generated responses, judging quality and reasoning while maintaining high standards of accuracy. You will craft clear, well-supported rationales in a fully remote setting. The role emphasizes independent work, strict adherence to evaluation guidelines, and native English proficiency. Compensation is hourly and...


  • GA, United States Chenega Professional Services Strategic Business Unit Remote Full-time $102,000 - $120,000 Contract

    Chenega Services & FederalSolutions, LLC seeks an AI Evaluation Specialist to lead structured assessments of enterprise AI tools, including OpenAI and life sciences models like Rosalind. You’ll design evaluation frameworks, test real-world use cases, and compare solutions across quality, safety, privacy, and cost while collaborating with product, security,...


  • New York, United States AuraOne Human Data Full-time

    AuraOne, Inc. is seeking a Chinese (Singapore) Linguistic Expert — AI Evaluation to remotely review Chinese prompts and model outputs against AuraOne's quality rubric. You will compare paired responses, label edge cases, and write structured feedback to aid retraining. Responsibilities include applying rubrics, tagging issues, and documenting recurring...


  • New York, United States Remotebridge Full-time

    Remotebridge is seeking a Financial Expert (Contractor) to support a customer-facing project evaluating AI agents in financial domains. You'll help train models by designing realistic finance prompts and structured assessments. You will use Microsoft 365 tools to build prompts, rubrics, and documentation, and provide clear, written feedback in English. This...


  • Tallahassee, FL, United States Sedgwick Full-time

    Sedgwick is seeking an AI Evaluation Engineer to design, build, and operate evaluation systems for GenAI solutions. You will support multiple product pods, establish evaluation standards, and create automated pipelines to measure LLM, RAG, and agented applications for production readiness.You will work with product owners and SMEs, develop observability to...


  • , , United States OpenTrain AI, Inc. Full-time

    OpenTrain AI, Inc. seeks a contract, part-time Clinical Healthcare AI Evaluation reviewer in the United States. Remote work with at least 4 hours per day and 20+ hours per week is required, along with current clinical credentials and English fluency. You will review clinical charts, simulations, and AI outputs, explain your reasoning, and maintain consistent...

  • Federal Management

    5 days ago


    , , United States AuraOne Human Data Full-time

    Federal Management & Program Analyst (Remote AI Evaluation) is a remote review track for evaluating AI outputs across federal management program analyst specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it. Why...


  • New York, NY, New York County, NY; New York State, United States DataAnnotation Full-time

    AI Trainer Jobs is seeking Telugu linguists to evaluate AI translation, localization, and cultural nuance on real projects. This is remote, independent contractor work with flexible hours and project-based flow. You’ll review frontier models, identify errors, rate quality, and craft clearer, more accurate answers to improve AI systems. Clear written...


  • New York, NY, New York County, NY; New York State, United States Outlier Ai Full-time

    Outlier Ai is seeking detail-oriented, fluent English speakers to review AI-generated media on a flexible schedule. You will roleplay live conversations with an AI voice model, score exchanges, and compare AI-generated images and videos to determine which follows prompts better. Contributors provide written explanations of issues, rating outputs on...


  • New York, NY, New York County, NY; New York State, United States Crossing Hurdles Full-time

    Crossing Hurdles is seeking a Remote Personal Account Generalist Rater (Mobile) on a contract basis. The role involves reviewing AI-generated prompts and responses in English, evaluating accuracy and relevance, and providing structured feedback to support AI system improvements. The position offers a remote, flexible 10–40 hour weekly commitment with...


  • New York, NY, New York County, NY; New York State, United States Auraone Full-time

    AuraOne is seeking a remote Consumer Reviewer to evaluate prompts and responses against our quality rubric. You will compare outputs, label edge cases, and craft structured feedback to help retrain the model. This contractor role requires strong written reasoning, attention to detail, and the ability to work asynchronously for at least 10 hours per week....


  • New York, NY, New York County, NY; New York State, United States Crossing Hurdles Full-time

    Crossing Hurdles is seeking an English Speaking Generalist for remote contract work. You will craft prompts, run side-by-side evaluations using ChatGPT and Claude, and provide thorough verbal feedback in American English. The role emphasizes consistent evaluation standards and collaboration with AI model improvement efforts. Ideal candidates have strong...


  • New York, NY, United States prolificacademicltd Full-time

    Prolific Academic Ltd. is seeking reviewers for a fully remote contract role to evaluate AI-generated graphical abstracts from large language models against original research papers. As a remote reviewer, you will assess scientific accuracy, spot inaccuracies in methods and terminology, and provide clear written feedback within agreed timeframes. The role...

  • Remote Cardiologist

    3 days ago


    United States DataAnnotation Full-time

    DataAnnotation is seeking an experienced cardiologist to help train AI models by evaluating and rating cardiology-related responses. You will examine clinical reasoning, measure accuracy and safety, and document your rationale for each decision. We welcome board-certified cardiologists with 3+ years post-fellowship, or clinicians with cardiology experience,...